Reference · 08

External values and workers.

Results computed outside the database, and the worker loops that keep them current and queues moving.

External values

external from @flower-js/sdk keeps a result current for each input, computed by workers outside the database. The workers guide has a complete example.

APIContract
external(name, { input, result?, each? }): Externalinput(ctx, args) returns everything that affects the result, or null when there is nothing to compute. A result schema checks published values. With each, the arguments are that collection’s keys and pools can list stale ones. Add the value to define({ uses }).
External<A, I, R, Pool>A Derived<A, ExternalState<R> | null>: read it with ctx.get(value, args). Plus component, results (the stored results collection) and the methods below.
ExternalState<R>{ status: "pending" } or { status: "ready", value }. ctx.get returns null instead when there is no input.
ExternalWork<A, I>args, input, and key: the canonical JSON of both, which names this exact piece of work.
pending(ctx, args): ExternalWork | nullThe work for these arguments, or null when the stored result is current or there is no input.
publish(ctx, { args, key, value }): { accepted }Stores value only if key still names the current input. Racing workers keep the first result; stale ones get accepted: false.
next(ctx, { limit?, shard? }?): ExternalWork[]Pending work across the tracked collection, oldest first. Needs each.
http(prefix, { access? }?): ExternalHttpMethods to spread into define({ http }): `${prefix}.pending`, `${prefix}.publish` and, with each, `${prefix}.next`.
ExternalNextOptionsOptional limit (16; at most 1,024 over HTTP) and shard: [index, count], which keeps only keys whose hash falls in that shard.
ExternalHttp<Prefix, A, I, R, Pool>The types of the generated methods.
  • next finds a key only after its row in the each collection is written. An input that changes because of other records makes the value pending, and pending reports it, but next won’t list it until that row changes.
  • A write that leaves a tracked row without input also deletes its stored result.
  • If the input returns to an earlier value, the result stored for it counts again.
  • The helper creates `${name}.input`, `${name}.results` and, with each, `${name}.stale`. Don’t reuse those names.

Worker loops

@flower-js/sdk/worker runs the loops that feed queues and external values. They use only the client, so they run in Node or a browser.

APIContract
runQueueWorker<P, R>(client, options): Promise<void>Each lane waits for `${queue}.ready`, claims, runs work while renewing the lease, and completes or fails the job, retrying lost replies under one request ID. Resolves once the signal aborts and held jobs finish. Rejects on errors that aren’t transient, such as a missing method.
QueueWorkerOptions<P, R>Required queue (the http() prefix), work(job, signal) and signal. Optional owner (a random ID), lanes (1), leaseMs (30,000), renew (true; needs the renew method), marginMs (a fifth of leaseMs), scope (for scope: "argument"), retry and onEvent.
QueueWorkerEventclaimed (with job), completed, failed (with error), lost (the lease ended first), unreported (the outcome couldn’t be sent in time) and waiting (Flower is unreachable). Each has a lane.
reconcile<A, I, R>(client, options): Promise<void>Waits for pending work, runs compute, and publishes the result with its key. Stale results are rejected, never stored. Resolves once the signal aborts.
ReconcileOptions<A, I, R>Required external (the http() prefix), compute(input, work, signal) and signal. Optional args (keep one key current; omit to drain every stale key through next), shard, concurrency (1), batch (16), retry and onEvent.
ReconcileEventpublished (with key and accepted), failed (with key and error) and waiting.
  • The generic parameters type job.payload, results, arguments and inputs; they aren’t inferred from the app.
  • work can run again after a crash: give external services job.id as an idempotency key. Its signal aborts marginMs before the lease would end.
  • A work error fails the job with { message }, and the queue’s retry policy decides what happens next.
  • compute may run more than once for the same input. Failures back off per key.