Patterns

The patterns actually used with FoxCore.Runtime 3.4.2: the batch, the large batch, progress with cancellation, handlers, the blackboard, events and the worker environment. And at the end, how this breaks.

1. The parallel batch - the most common case

Launch N scripts, pump until none is alive, and collect. The loop is written by the host: there is no call that blocks waiting, because that is exactly the frozen interface this product avoids.

A batch of 30 scripts: launch, pump and collect

* lcJwt es TU licencia, y GuionDe() es TU funcion: devuelve el texto
* del guion que va a correr cada tarea. Los dos son tuyos, no del
* producto.
LOCAL loRt, i, lnTotal, lnVivas
LOCAL ARRAY laIds[1]

loRt = CREATEOBJECT("FoxCore.Runtime")
loRt.Activate(lcJwt)
loRt.Attach(_VFP)
loRt.MaxWorkers = 8
loRt.DefaultTimeoutSeconds = 300

lnTotal = 30
DIMENSION laIds[lnTotal]
FOR i = 1 TO lnTotal
    laIds[i] = loRt.RunScript(GuionDe(i), "lote " + TRANSFORM(i))   && the script runs in ANOTHER process
ENDFOR

DO WHILE .T.
    lnVivas = 0
    FOR i = 1 TO lnTotal
        IF !loRt.IsCompleted(laIds[i])
            lnVivas = lnVivas + 1
        ENDIF
    ENDFOR
    IF lnVivas = 0
        EXIT
    ENDIF
    loRt.Pump()
    DOEVENTS
ENDDO

FOR i = 1 TO lnTotal
    ? laIds[i], loRt.GetState(laIds[i]), loRt.GetResult(laIds[i]), loRt.GetError(laIds[i])
ENDFOR

loRt.Shutdown()

2. The batch of hundreds or thousands

The loop above asks task by task. With 400 tasks that is 400 COM calls per round just to know who is still alive - at 0.08 ms each, about 32 ms per round spent asking. With batches of thousands it does not scale. The right way is PollCompleted(), which hands back the ids finished since the last call in ONE call, and Forget() to free each result as you read it.

Large batch with PollCompleted() and Forget()

LOCAL lcFeed, lnHechas, i, loStats
LOCAL ARRAY laTerm[1]

loRt.KeepCompletedTasks = 5000          && so the purge does not take what you have not read
lnHechas = 0

DO WHILE lnHechas < lnTotal
    loRt.Pump()
    lcFeed = loRt.PollCompleted()       && ONE call, whatever it hands back
    IF !EMPTY(lcFeed)
        FOR i = 1 TO ALINES(laTerm, lcFeed, 1, ";")
            * ... GetState / GetResult de laTerm[i] ...
            loRt.Forget(laTerm[i])      && explicit purge, no races
            lnHechas = lnHechas + 1
        ENDFOR
        * GetStats() devuelve JSON como cadena: se parsea con la libreria
        * que uses (aqui, jsonfox). honest progress comes from the LIFETIME counters
        loStats = loJson.Parse(loRt.GetStats())
        THISFORM.lblProgreso.Caption = "Procesando " + ;
            TRANSFORM(loStats.finishedEver) + " de " + TRANSFORM(lnTotal)
    ENDIF
    DOEVENTS
ENDDO

finishedEver, not completed. They are two different families and they look alike: the *Ever ones count what has HAPPENED since the runtime started and never go down; total, completed and friends are the SNAPSHOT of the registry right now, and Forget() and the purge subtract from them. With this loop - which calls Forget() every round - the snapshot stays at zero, and that is correct. Progress has to come from the history.

And two more traps this loop avoids: with KeepCompletedTasks at its default of 200 and a batch of 1,000 collected late, the first 800 results vanish before you read them and GetState() starts answering NotFound. And the completed channel is capped at 5,000 ids: if nobody drains it the oldest are lost, and GetStats() says so in completedDropped - it is not data loss, the task is still in the registry.

3. Progress and cancellation, in a form

The sink is not called from a .NET thread: it is called by Pump(), on YOUR thread. That is why a sink method can touch the screen safely. And in a form, the place for Pump() is a Timer.

A form with progress and a cancel button

* oRt y cTaskId son PROPIEDADES que le anades tu al formulario, y
* Terminado() un metodo tuyo. El boceto es del lado del host.
* Init del formulario
THIS.oRt = CREATEOBJECT("FoxCore.Runtime")
THIS.oRt.Activate(lcJwt)
THIS.oRt.Attach(_VFP)
THIS.oRt.SetEventSink(THISFORM)        && THISFORM has OnProgress and OnLog
THIS.cTaskId = ""

* Click del boton "Procesar"
THISFORM.cTaskId = THISFORM.oRt.RunScript(lcGuion, "cierre mensual")
THISFORM.timer1.Enabled = .T.

* Timer, Interval = 100
THISFORM.oRt.Pump()
IF !EMPTY(THISFORM.cTaskId) AND THISFORM.oRt.IsCompleted(THISFORM.cTaskId)
    THISFORM.timer1.Enabled = .F.
    THISFORM.Terminado()
ENDIF

* Click del boton "Cancelar"
THISFORM.oRt.Cancel(THISFORM.cTaskId)

* Destroy
THISFORM.oRt.ClearEventSink()          && IMPORTANT: while the runtime holds it, the form stays alive
THISFORM.oRt.Shutdown()

And in the script, so that Cancel() means something:

A script that honours Cancel() and reports progress

SCAN
    IF runtime.IsCancelled()
        EXIT
    ENDIF
    * ... trabajo ...
    IF MOD(RECNO(), 500) = 0
        runtime.Progress(100 * RECNO() / RECCOUNT(), "procesando...")
    ENDIF
ENDSCAN

A script that never checks IsCancelled() only stops with Kill() - which kills the process by PID - or when its deadline expires. That is the difference between cooperative and hard cancellation, and it has to be decided when the task is designed, not after.

4. The worker asks you for something - handlers

The classic case is a sequence number. You register an object of yours under a name, and the script calls it with runtime.Call().

CajaDeFolios: the handler registered in the host

loRt.RegisterHandler("SiguienteFolio", loCaja, "Siguiente")

DEFINE CLASS CajaDeFolios AS Custom
    nUltimo = 0
    FUNCTION Siguiente(tcSerie)          && not a single lock, and none needed
        THIS.nUltimo = THIS.nUltimo + 1
    RETURN TRANSFORM(tcSerie) + "-" + PADL(THIS.nUltimo, 4, "0")
    ENDFUNC
ENDDEFINE

The script asks for a number with runtime.Call()

lcFolio = runtime.Call('SiguienteFolio', 'FRA')

* O sin TRY/CATCH, que no lanza nunca:
lcJson = runtime.TryCallJson('SiguienteFolio', 'FRA')
IF 'true' $ LOWER(lcJson)
    * ... parsea el JSON como prefieras ...
ENDIF

And here is the property that matters: Pump() runs on your thread and serves the requests one at a time. The host object carries no lock and even so eight workers asking at once do not produce a single duplicate - verified with 400 tasks and 410 requests. The serialization comes from the Pump, not from the object.

There are three built-in handlers always available, to test the channel without writing anything: echo, upper and add. And RegisterHandler() returns .F. without throwing if the object does not expose that method, so the error surfaces in the form Init and not six hours later inside a worker.

5. Getting data to the worker - the blackboard

In order of preference: inside the script itself (it is text, build it with TEXTMERGE); the blackboard; a file, for large parameter sets; and Call() to a handler, which is the most expensive and is kept for what MUST be resolved in the host. Never through PUBLIC variables or cursors: they are not inherited.

Publishing a value on the blackboard and reading it in the script

* En el host, antes de lanzar:
loRt.Publish("cRutaDatos", "\\srv\datos")

* En el guion:
lcRuta = runtime.GetValue("cRutaDatos")      && no queue and no pump cost

6. Being told instead of asking - events

SetEventSink() is additional: GetProgress() and GetProgressMessage() work just the same without it. What changes is that you get told instead of having to ask.

EcoDeProgreso: an event sink

DEFINE CLASS EcoDeProgreso AS Custom
    nAvisos = 0
    cUltimo = ""

    *-- OnProgress y OnLog son OBLIGATORIOS: sin los dos, SetEventSink da .F.
    PROCEDURE OnProgress(tcTaskId, tnPercent, tcMessage)
        THIS.nAvisos = THIS.nAvisos + 1
        THIS.cUltimo = TRANSFORM(tnPercent) + "% " + TRANSFORM(tcMessage)
    ENDPROC

    PROCEDURE OnLog(tcTaskId, tcLevel, tcMessage)
    ENDPROC

    *-- Opcionales: se detectan si estan.
    PROCEDURE OnTaskCompleted(tcTaskId, tcState)
    ENDPROC
    PROCEDURE OnTaskFailed(tcTaskId, tcError)
    ENDPROC
ENDDEFINE

7. The worker environment

By default MaxTasksPerWorker is 1: one process per task, full isolation. Raising it, the worker is reused - 91% fewer processes and 70% less time, measured over 120 tasks - and then you need the two scripts: the init one builds the environment once, and the reset one cleans between tasks.

Reusing workers with init and reset scripts

loRt.MaxTasksPerWorker = 20
loRt.WorkerInitScript  = "SET PROCEDURE TO miapp.app ADDITIVE" + CHR(13) + CHR(10) + ;
                         "SET PATH TO " + lcRutaDatos      && once per worker, before its first task
loRt.WorkerResetScript = "CLOSE TABLES ALL"                && between tasks of a reused worker
loRt.WorkerInitFailurePolicy = "abort"

Nothing generic is cleaned between tasks, and that is not laziness: any cleanup broad enough to erase the previous task would also destroy the environment WorkerInitScript just built. That is what WorkerResetScript is for. And a worker returns to the pool only if its task ended Completed: anything else burns it, because a script that failed halfway may have left a cursor, a transaction or a lock behind.

How this breaks

The hard rules, with the consequence of ignoring them:

Rule	Consequence
Call Pump()	The workers' Call()s are not served and their deadline expires. Symptom: "the tasks do not move".
Call Shutdown()	Orphan vfp9.exe processes are left eating memory.
Pump() only from the Attach thread	From another thread it raises a controlled error.
No DOEVENTS inside a handler	It reenters Pump(), which returns 0 doing nothing. It does not throw, but it serves nobody.
ClearEventSink() in Destroy	While the runtime holds the sink, the form stays alive: a reference leak.
A VFP handler only fails outward with THROW	ERROR "..." and runtime errors do not cross to COM: the method carries on at the next line and returns as if nothing happened.
And even with THROW, the worker receives "Procedure canceled."	VFP does not marshal the exception Message across COM. If you need to explain why it failed, store it in a property of the host object or return it as a value.
VFP silently ignores extra arguments	Calling a one-parameter handler with three raises no error: the first one arrives.

Before parallelizing: look at the data

This is what most often turns an apparent candidate into a non-candidate:

The workers open the tables themselves. If the work writes to the same tables from eight processes there is contention and real locking - the same you would get with eight users. Parallelize reads and partitioned writes: each worker its range, its output file, its branch of data.
A transaction does not cross processes. There is no distributed transaction: if the work has to be atomic end to end, either it fits inside a single task, or it is not a candidate.
SET EXCLUSIVE, PACK, REINDEX and ZAP need exclusive use. They work as a single task in their own window, never as a parallel batch over the same table.
Careful with fixed-name temporary files. Eight workers writing c:\temp\out.txt stomp on each other. Use SYS(2015) or the TaskId in the name.
Sequence numbers and counters are exactly the host handler case, not "let each worker read and add".

Coming from the classic engine? You will not find RunAll or WaitFor here: they have no equivalent, and the reason is explained in "What no longer exists", at the end of the API Reference.

Next: API Reference ->