Guide · 09

Hand out work. Let records expire.

Queue jobs for workers, with leases, retries and delays, and let old records disappear. Both are components written in plain TypeScript.

A queue gives each job to one worker at a time, retries failures with backoff, and rejects results from workers that lost their lease. An expiring collection hides records after a deadline. Both are components. For worker design, see External workers.

Queue jobs in the same commit

Enqueue in the mutation that creates the obligation. Here, a paid invoice always has its receipt job, because both commit together.

receipts.ts
import { collection, define, fail, mutation, v } from "@flower-js/sdk";
import { queue } from "@flower-js/sdk/temporal";

const invoices = collection("invoices", v.object({ email: v.string(), paid: v.boolean() }));
const receipts = queue("receipts", {
  payload: v.object({ invoice: v.string(), email: v.string() }),
  lease: { defaultMs: 10_000, maxMs: 60_000 },
  retry: { maxAttempts: 5, initialDelayMs: 1_000, maxDelayMs: 60_000 },
});

const pay = mutation("invoice.pay", { args: v.string({ min: 1 }) }, (ctx, id) => {
  const invoice = ctx.get(invoices, id) ?? fail("INVOICE_NOT_FOUND", `No invoice ${id}`);
  if (invoice.paid) return null;
  ctx.set(invoices, id, { ...invoice, paid: true });
  receipts.enqueue(ctx, `receipt:${id}`, { invoice: id, email: invoice.email });
  return null;
});

const app = define({
  uses: [receipts],
  http: { "invoice.pay": pay, ...receipts.http("receipts") },
});
export default app;
A worker process
await runQueueWorker<{ invoice: string; email: string }>(client, {
  queue: "receipts",
  signal: shutdown.signal,
  work: async (job, signal) => {
    await mailer.send(job.payload.email, { idempotencyKey: job.id, signal });
    return { sent: true };
  },
});
  • receipts.http("receipts") exposes what workers need: receipts.claim, .renew, .complete, .fail, .get, .ready and .stats. List methods to add enqueue, retry or cancel, and pass access to protect them.
  • runQueueWorker waits for work, claims, renews the lease while your code runs, and reports the outcome. Worker pools runs it for real.
  • A payload schema checks every enqueue. enqueue fails with JOB_EXISTS while a job with that ID is pending or leased; replace: true replaces a finished one.
  • delayMs or at hold a job back. Jobs are claimed oldest first.

Leases, retries and fencing

  • claim returns { scope, id, payload, owner, token, expiresAt, attempt, history? }, or null when nothing is ready. token increases with every claim in the queue.
  • renew, complete and fail need the current owner and token before the lease ends, or fail with LEASE_LOST. renew keeps the token.
  • An expired lease counts as a failed attempt, and the job can be claimed again at once.
  • fail requeues the job after 1 s, doubling up to 60 s, for up to five attempts in all. Pass retry: false to the queue, or to one fail call, to make failure final.
  • Failed jobs stay until retry requeues them with a fresh attempt count. Completed jobs stay too: clean them up yourself.
  • Claims use lease.defaultMs (30 s unless set), and leaseMs may ask for up to lease.maxMs (5 minutes unless set). Longer requests fail with LEASE_TOO_LONG.
  • receipts.scope("tenant-a") is the same queue API for one namespace, with its own tokens. http(prefix, { scope }) takes the scope from an argument or from the caller. Scopes are not access control.

Expiry does not stop the old worker. A paused worker can wake up after its lease and still act. The receiving service should take idempotency keys, such as the job ID, or check fencing tokens.

Let records expire

sessions.ts
import { define, mutation, v } from "@flower-js/sdk";
import { expiringCollection } from "@flower-js/sdk/temporal";

const sessions = expiringCollection("sessions", {
  expiration: { afterUpdateMs: 60_000 },
  value: v.object({ user: v.string() }),
});
const touch = mutation("session.touch", { args: v.string({ min: 1 }) }, (ctx, id) =>
  sessions.set(ctx, id, sessions.get(ctx, id) ?? { user: "guest" }));

export default define({ uses: [sessions], http: { "session.touch": touch } });

A record is hidden once ctx.now() >= expiresAt, and a maintenance task deletes it soon after. entry returns the value with its timestamps.

Expiration policies
Policy passed to setMeaning
{ afterCreationMs: 10_000 }Expires from its original live creation time.
{ afterUpdateMs: 60_000 }Restarts the lifetime on each update.
{ at: epochMilliseconds }Uses an absolute deadline chosen by your code.
nullDoes not expire.
  • Updates keep the creation time; writing over an expired record starts fresh.
  • Reads never extend a lifetime; rewrite in a mutation to refresh. For logic on expiry, use a timer.
  • Keep node clocks in sync. ctx.now() is fixed per evaluation, and commit time counts against leases and lifetimes.