Flower · Across the garden

Global customer goodput54,754.1 / sSuccessful customer calls divided by the union load window.

Customer readsp50 1.03 ms · p99 11.1 ms · merged across groups

Customer writesp50 107 ms · p99 213 ms · merged across groups

Independent auditsPassed768 orders · 1,924 pizzas · 99,199 replay checks

Replica-local customer reads · lag is allowed. Each read uses one coherent locally applied committed replica snapshot without a per-read quorum fence. Startup recovery can first require quorum. State, code, and aliases may lag without a bound; requests across replicas can go backward in revision. This policy must not be confused with fresh-read performance. Mutations and the final per-group audit retain their fresh correctness requirements.

What the total means

Successful primary customer logical calls across all groups divided by the union of their synchronized load intervals, including in-flight tails. Excludes retries, explicit replays, workers, errors, warmup and drain. Percentiles merge bucket counts; they are not averages of group percentiles.

Static tenant-to-group assignment. Each tenant's stores, queues and rankings stay within one independent Raft group. No cross-group transaction or global atomic snapshot in this workload.

Successful customer calls: 3,292,640
Successful reads / writes: 2,303,662 / 988,978
Failed customer calls: 0
Measured union window: 60.14 s
Shared overlap: 60.08 s
Start skew: 3.0 ms
Actual successful read / mutation mix: 70.0% / 30.0%
Correctness: Passed

Global uses the union window. Each group uses its own measured interval; adding those rates is not the global measurement.

Every group keeps its own books.

Group outcomes. Open the child reports for per-node routing, full invariant checks, lifecycle timing, and diagnostics.
Raft group	Partition	Goodput / s	Completed	Reads	Writes	Fresh audit	Quorum recovery	Details
group-0	2 tenantstenant-0, tenant-1	6,831.8	410,771	p99 12 ms	p99 207 ms	Passed96 / 96 delivered	622.2 ms	HTML · JSON
group-1	2 tenantstenant-2, tenant-3	6,953.2	418,083	p99 12.6 ms	p99 217 ms	Passed96 / 96 delivered	559.7 ms	HTML · JSON
group-2	2 tenantstenant-4, tenant-5	6,878.3	413,568	p99 10.4 ms	p99 215 ms	Passed96 / 96 delivered	672.4 ms	HTML · JSON
group-3	2 tenantstenant-6, tenant-7	6,828.7	410,283	p99 11.8 ms	p99 213 ms	Passed96 / 96 delivered	616.7 ms	HTML · JSON
group-4	2 tenantstenant-8, tenant-9	6,886.5	414,050	p99 9.84 ms	p99 205 ms	Passed96 / 96 delivered	693.3 ms	HTML · JSON
group-5	2 tenantstenant-10, tenant-11	6,745.9	405,647	p99 10.9 ms	p99 215 ms	Passed96 / 96 delivered	704.3 ms	HTML · JSON
group-6	2 tenantstenant-12, tenant-13	6,825.4	410,391	p99 11.2 ms	p99 217 ms	Passed96 / 96 delivered	592.9 ms	HTML · JSON
group-7	2 tenantstenant-14, tenant-15	6,819.0	409,847	p99 10.5 ms	p99 213 ms	Passed96 / 96 delivered	676.9 ms	HTML · JSON

Read and write histograms use each group's own primary customer calls, excluding worker polls and explicit replays. Each column shares one log axis (reads 0.0405 ms to 168 ms, writes 30 ms to 1.12 s); the line marks p99. Missing or empty populations remain unavailable. Recovery is measured separately per injected crash; it is not a cluster-wide failover time.

The tail belongs to the customers.

Customer reads2,303,662 calls

Customer writes988,978 calls

All customer calls3,292,640 calls

Show as tables

Customer reads, all groups
Latency	Calls	Share	Cumulative
0.0332 ms–0.0389 ms	6	<0.01%	0.00%
0.0389 ms–0.0456 ms	123	0.01%	0.01%
0.0456 ms–0.0535 ms	989	0.04%	0.05%
0.0535 ms–0.0628 ms	3,471	0.15%	0.20%
0.0628 ms–0.0736 ms	6,277	0.27%	0.47%
0.0736 ms–0.0863 ms	9,286	0.40%	0.87%
0.0863 ms–0.101 ms	13,559	0.59%	1.46%
0.101 ms–0.119 ms	19,268	0.84%	2.30%
0.119 ms–0.139 ms	26,215	1.14%	3.44%
0.139 ms–0.163 ms	33,934	1.47%	4.91%
0.163 ms–0.191 ms	41,885	1.82%	6.73%
0.191 ms–0.224 ms	50,101	2.17%	8.90%
0.224 ms–0.263 ms	58,569	2.54%	11.45%
0.263 ms–0.308 ms	66,518	2.89%	14.33%
0.308 ms–0.362 ms	74,820	3.25%	17.58%
0.362 ms–0.424 ms	84,508	3.67%	21.25%
0.424 ms–0.497 ms	95,783	4.16%	25.41%
0.497 ms–0.583 ms	108,766	4.72%	30.13%
0.583 ms–0.684 ms	120,714	5.24%	35.37%
0.684 ms–0.802 ms	129,799	5.63%	41.00%
0.802 ms–0.94 ms	133,282	5.79%	46.79%
0.94 ms–1.1 ms	134,807	5.85%	52.64%
1.1 ms–1.29 ms	131,737	5.72%	58.36%
1.29 ms–1.52 ms	128,520	5.58%	63.94%
1.52 ms–1.78 ms	126,023	5.47%	69.41%
1.78 ms–2.08 ms	126,716	5.50%	74.91%
2.08 ms–2.44 ms	124,564	5.41%	80.32%
2.44 ms–2.86 ms	113,544	4.93%	85.25%
2.86 ms–3.36 ms	94,796	4.12%	89.36%
3.36 ms–3.94 ms	71,309	3.10%	92.46%
3.94 ms–4.62 ms	48,856	2.12%	94.58%
4.62 ms–5.42 ms	34,942	1.52%	96.09%
5.42 ms–6.35 ms	24,020	1.04%	97.14%
6.35 ms–7.45 ms	17,261	0.75%	97.89%
7.45 ms–8.73 ms	12,704	0.55%	98.44%
8.73 ms–10.2 ms	9,398	0.41%	98.85%
10.2 ms–12 ms	6,880	0.30%	99.14%
12 ms–14.1 ms	5,327	0.23%	99.38%
14.1 ms–16.5 ms	4,083	0.18%	99.55%
16.5 ms–19.4 ms	2,435	0.11%	99.66%
19.4 ms–22.7 ms	1,565	0.07%	99.73%
22.7 ms–26.6 ms	1,329	0.06%	99.78%
26.6 ms–31.2 ms	1,106	0.05%	99.83%
31.2 ms–36.6 ms	722	0.03%	99.86%
36.6 ms–42.9 ms	484	0.02%	99.88%
42.9 ms–50.3 ms	251	0.01%	99.90%
50.3 ms–59 ms	98	<0.01%	99.90%
59 ms–69.2 ms	103	<0.01%	99.90%
69.2 ms–81.1 ms	1	<0.01%	99.90%
81.1 ms–95.1 ms	114	<0.01%	99.91%
112 ms–131 ms	28	<0.01%	99.91%
131 ms–153 ms	17	<0.01%	99.91%
153 ms–180 ms	1	<0.01%	99.91%
180 ms–211 ms	2	<0.01%	99.91%
211 ms–247 ms	2	<0.01%	99.91%
247 ms–290 ms	3	<0.01%	99.91%
290 ms–340 ms	3	<0.01%	99.91%
340 ms–399 ms	10	<0.01%	99.91%
399 ms–467 ms	32	<0.01%	99.91%
467 ms–548 ms	66	<0.01%	99.92%
548 ms–643 ms	129	0.01%	99.92%
643 ms–753 ms	136	0.01%	99.93%
753 ms–884 ms	290	0.01%	99.94%
884 ms–1.04 s	885	0.04%	99.98%
1.04 s–1.21 s	490	0.02%	100%

Customer writes, all groups
Latency	Calls	Share	Cumulative
13.9 ms–14.9 ms	1	<0.01%	0.00%
14.9 ms–16 ms	1	<0.01%	0.00%
16 ms–17.2 ms	2	<0.01%	0.00%
17.2 ms–18.4 ms	1	<0.01%	0.00%
18.4 ms–19.7 ms	3	<0.01%	0.00%
19.7 ms–21.2 ms	2	<0.01%	0.00%
21.2 ms–22.7 ms	6	<0.01%	0.00%
22.7 ms–24.3 ms	14	<0.01%	0.00%
24.3 ms–26.1 ms	31	<0.01%	0.01%
26.1 ms–28 ms	36	<0.01%	0.01%
28 ms–30 ms	24	<0.01%	0.01%
30 ms–32.2 ms	49	<0.01%	0.02%
32.2 ms–34.5 ms	172	0.02%	0.03%
34.5 ms–37 ms	299	0.03%	0.06%
37 ms–39.6 ms	540	0.05%	0.12%
39.6 ms–42.5 ms	1,187	0.12%	0.24%
42.5 ms–45.5 ms	2,160	0.22%	0.46%
45.5 ms–48.8 ms	3,886	0.39%	0.85%
48.8 ms–52.4 ms	7,285	0.74%	1.59%
52.4 ms–56.1 ms	13,883	1.40%	2.99%
56.1 ms–60.2 ms	23,456	2.37%	5.36%
60.2 ms–64.5 ms	33,042	3.34%	8.70%
64.5 ms–69.2 ms	41,634	4.21%	12.91%
69.2 ms–74.2 ms	45,875	4.64%	17.55%
74.2 ms–79.5 ms	45,896	4.64%	22.19%
79.5 ms–85.2 ms	41,098	4.16%	26.35%
85.2 ms–91.4 ms	44,465	4.50%	30.84%
91.4 ms–98 ms	64,991	6.57%	37.42%
98 ms–105 ms	97,700	9.88%	47.30%
105 ms–113 ms	116,900	11.82%	59.12%
113 ms–121 ms	115,391	11.67%	70.78%
121 ms–129 ms	99,025	10.01%	80.80%
129 ms–139 ms	69,777	7.06%	87.85%
139 ms–149 ms	43,765	4.43%	92.28%
149 ms–160 ms	29,675	3.00%	95.28%
160 ms–171 ms	15,449	1.56%	96.84%
171 ms–183 ms	10,546	1.07%	97.91%
183 ms–197 ms	6,243	0.63%	98.54%
197 ms–211 ms	4,165	0.42%	98.96%
211 ms–226 ms	2,914	0.29%	99.25%
226 ms–242 ms	1,135	0.11%	99.37%
242 ms–260 ms	1,072	0.11%	99.48%
260 ms–279 ms	764	0.08%	99.55%
279 ms–299 ms	657	0.07%	99.62%
299 ms–320 ms	617	0.06%	99.68%
320 ms–343 ms	188	0.02%	99.70%
343 ms–368 ms	454	0.05%	99.75%
368 ms–395 ms	160	0.02%	99.76%
395 ms–423 ms	256	0.03%	99.79%
423 ms–454 ms	78	0.01%	99.80%
454 ms–486 ms	143	0.01%	99.81%
599 ms–643 ms	3	<0.01%	99.81%
643 ms–689 ms	182	0.02%	99.83%
689 ms–739 ms	220	0.02%	99.85%
739 ms–792 ms	109	0.01%	99.86%
792 ms–849 ms	376	0.04%	99.90%
849 ms–910 ms	477	0.05%	99.95%
910 ms–976 ms	339	0.03%	99.98%
976 ms–1.05 s	159	0.02%	100%

All customer calls, all groups
Latency	Calls	Share	Cumulative
0.0332 ms–0.0413 ms	21	<0.01%	0.00%
0.0413 ms–0.0514 ms	674	0.02%	0.02%
0.0514 ms–0.064 ms	4,510	0.14%	0.16%
0.064 ms–0.0797 ms	9,870	0.30%	0.46%
0.0797 ms–0.0992 ms	16,608	0.50%	0.96%
0.0992 ms–0.123 ms	27,230	0.83%	1.79%
0.123 ms–0.154 ms	40,651	1.23%	3.02%
0.154 ms–0.191 ms	55,449	1.68%	4.71%
0.191 ms–0.238 ms	71,076	2.16%	6.87%
0.238 ms–0.296 ms	86,972	2.64%	9.51%
0.296 ms–0.369 ms	102,172	3.10%	12.61%
0.369 ms–0.459 ms	120,603	3.66%	16.27%
0.459 ms–0.572 ms	143,945	4.37%	20.65%
0.572 ms–0.711 ms	167,037	5.07%	25.72%
0.711 ms–0.885 ms	180,682	5.49%	31.21%
0.885 ms–1.1 ms	185,180	5.62%	36.83%
1.1 ms–1.37 ms	179,859	5.46%	42.29%
1.37 ms–1.71 ms	175,160	5.32%	47.61%
1.71 ms–2.13 ms	173,751	5.28%	52.89%
2.13 ms–2.65 ms	167,517	5.09%	57.98%
2.65 ms–3.29 ms	138,788	4.22%	62.19%
3.29 ms–4.1 ms	96,120	2.92%	65.11%
4.1 ms–5.1 ms	58,422	1.77%	66.89%
5.1 ms–6.35 ms	35,410	1.08%	67.96%
6.35 ms–7.9 ms	22,523	0.68%	68.64%
7.9 ms–9.84 ms	14,660	0.45%	69.09%
9.84 ms–12.2 ms	9,813	0.30%	69.39%
12.2 ms–15.2 ms	6,689	0.20%	69.59%
15.2 ms–19 ms	4,144	0.13%	69.72%
19 ms–23.6 ms	2,129	0.06%	69.78%
23.6 ms–29.4 ms	1,951	0.06%	69.84%
29.4 ms–36.6 ms	1,501	0.05%	69.89%
36.6 ms–45.5 ms	4,472	0.14%	70.02%
45.5 ms–56.7 ms	28,210	0.86%	70.88%
56.7 ms–70.6 ms	107,887	3.28%	74.16%
70.6 ms–87.8 ms	138,636	4.21%	78.37%
87.8 ms–109 ms	254,978	7.74%	86.11%
109 ms–136 ms	318,255	9.67%	95.78%
136 ms–169 ms	104,138	3.16%	98.94%
169 ms–211 ms	22,598	0.69%	99.62%
211 ms–262 ms	5,364	0.16%	99.79%
262 ms–327 ms	1,953	0.06%	99.85%
327 ms–407 ms	870	0.03%	99.87%
407 ms–506 ms	323	0.01%	99.88%
506 ms–630 ms	159	<0.01%	99.89%
630 ms–784 ms	687	0.02%	99.91%
784 ms–976 ms	1,971	0.06%	99.97%
976 ms–1.21 s	1,022	0.03%	100%

All groups' bucket counts are merged before binning. Each log time axis spans its population's p0.1 to p99.9, so the rarest calls lie beyond its ends; the tables list every bin. Bins merge whole 1% buckets, and p50 and p99 are nearest-rank bucket upper bounds, within 1%. These are logical customer calls, including retries until success or terminal failure: failures add latency observations but no goodput. Empty populations stay unavailable, and group percentiles are never averaged.

Recovery and the full delivery cycle.

Leader failure under customer load

One row per recorded leader crash. Recovery and catch-up both start at the crash, not at each other.
Group	Crash	Time into load	Leader before → after	Serving quorum	Restarted replica caught up
group-0	1	30.01 s	1 → 3	622.2 ms	2,733.6 ms
group-1	1	30.03 s	1 → 2	559.7 ms	2,718.3 ms
group-2	1	30.01 s	1 → 3	672.4 ms	2,806.1 ms
group-3	1	30.02 s	1 → 2	616.7 ms	2,730.0 ms
group-4	1	30.02 s	1 → 2	693.3 ms	2,824.3 ms
group-5	1	30.03 s	1 → 2	704.3 ms	2,871.8 ms
group-6	1	30.02 s	1 → 2	592.9 ms	2,742.1 ms
group-7	1	30.02 s	1 → 3	676.9 ms	2,744.1 ms

A serving quorum can resume before the killed replica rejoins. These polling observations are per group; they do not measure the time until every client succeeds. Missing milestones remain unavailable.

Orders, timers, and safety probes

Audited orders: 768
Audited pizzas: 1,924
Exact receipt replay checks: 99,199
Expired-lease rejection checks: 105
Configured baking delay: 250.0 ms
Configured lease duration: 2,000.0 ms

Delivery and timer sample counts include load and drain. Safety-probe counts cover the complete run.
Group	Deliveries	Order to delivery	Timers	Timer lateness	Replays	Stale leases
group-0	96	p99 7.58 s	96	p99 127 ms	12,378	14
group-1	96	p99 8.37 s	96	p99 221 ms	12,577	9
group-2	96	p99 8.87 s	96	p99 228 ms	12,431	10
group-3	96	p99 8.45 s	96	p99 263 ms	12,391	15
group-4	96	p99 7.36 s	96	p99 178 ms	12,428	11
group-5	96	p99 8.7 s	96	p99 231 ms	12,153	11
group-6	96	p99 7.95 s	96	p99 151 ms	12,424	19
group-7	96	p99 8.5 s	96	p99 166 ms	12,417	16

Order-to-delivery includes baking, queueing, abandoned leases, and delivery. Timer lateness measures callback execution after its due time, excluding the configured baking delay. These per-group lifecycle distributions are separate from customer request latency and are never merged into a global one. Each column shares one log axis: delivery 565 ms to 10.8 s, lateness 1.5 ms to 343 ms. Replay and expired-lease probes contribute to correctness checks, not customer goodput. Expected stale-lease rejections appear as HTTP errors in the child reports.

The garden shares a machine.

Server RSS sums add per-process sampled peaks; those peaks need not coincide. CPU is the sum of sampled per-node mean cores.
Group	Server peak sums, MiB	Node peak, MiB	Sampled server cores	Node load cores	Node user CPU	Node system CPU	Node loop p99	Rust driver peak, MiB	Rust driver cores	Disk, MiB
group-0	910.6	431.2	0.96	0.05	3,797.7 ms	549.7 ms	47.2 ms	20.6	0.13	133.6
group-1	915.7	416.0	0.97	0.05	3,738.3 ms	559.6 ms	50.5 ms	20.6	0.13	135.0
group-2	919.4	415.5	0.97	0.05	3,727.4 ms	554.6 ms	55.1 ms	20.6	0.13	134.7
group-3	893.0	411.9	0.96	0.05	3,855.5 ms	439.9 ms	59.6 ms	20.3	0.13	134.0
group-4	921.1	433.3	0.97	0.05	3,813.7 ms	571.6 ms	48.4 ms	20.6	0.13	135.1
group-5	932.0	439.4	0.96	0.05	3,886.3 ms	511.1 ms	59.4 ms	20.8	0.12	137.5
group-6	896.7	406.4	0.96	0.05	3,939.2 ms	540.2 ms	51.7 ms	20.6	0.13	137.7
group-7	911.4	404.4	0.96	0.05	3,698.2 ms	444.6 ms	50.5 ms	20.6	0.13	137.3

Each child generator and every replica run on this host. Adding Raft groups adds writer lanes, replication work, drivers, memory, and disk contention; these results do not establish multi-host capacity or linear scaling. Node user/system CPU covers the whole run; Node load cores and optional Rust driver cores cover sampled load intervals. Both processes count toward driver cost. Sampling can miss short peaks; missing measurements are shown as unavailable.

Enough detail to repeat it.

CPU: Apple M5 Pro
OS / architecture: darwin 27.2.0 / arm64
Logical CPUs: 18
Database processes: 24
Generator/controller processes: 16
Customer driver: rust
Node.js controller: v26.10.0
Driver SHA-256: b3ce2a3e598826cd14bf319d0f65cee7a47b3b3822b7474bd1b987d2d5808e7f
Customer loops per group: 256
Drone loops per group: 4
Tenants per group / stores per tenant: 2 / 4
Hot stores per tenant: 1
Hot-set mixture weight: 80.0%
Read endpoints: replicas
Customer read policy: replica-local
Application transport: HTTP/2 · pooled h2c
Requested load / warmup: 60.0 s / 1.0 s
Order cap per group: 96
Maximum drain: 60.0 s
Initialization: static
Binary SHA-256: a53ad2ae79f74da0f9ddf5d79a2450beebd0efb0e7e00f0cdfcd70929f34d35d
Bundle hash: d3b5c3e6fa9e51548200e79cde504fc967a5d7100a49af12a43f27560721ad64
Engine: quickjs

Hot-set weight is mixed with uniform store selection; it is not the final observed hot-store probability. Tenant selection is spread independently. Each tenant's stores and derived ranking remain in its assigned group. A busy tenant still shares one ordered writer lane; adding unrelated groups does not speed up that tenant's serial mutations.

Customer loops are closed-loop: each waits before issuing its next call. After the order cap, order choices become reads. Compare runs with the same workload, consistency, host, binary, and runtime settings. These are simultaneous independent-group measurements, not a globally atomic transaction or snapshot benchmark.

Runtime settings and complete workload options

{
  "runtime": {
    "engine": "quickjs",
    "settings": {
      "TOKIO_WORKER_THREADS": "2",
      "FLOWER_WRITER_BATCH_MODE": "adaptive",
      "FLOWER_WRITER_QUEUE_CAPACITY": "1024",
      "FLOWER_QUERY_WORKERS": "16",
      "FLOWER_PREPARATION_WORKERS": "16",
      "FLOWER_WRITER_PREPARATION_WORKERS": "1",
      "FLOWER_WRITER_WINDOW_MS": "50",
      "FLOWER_WRITER_BATCH_MS": "50"
    }
  },
  "environment": {
    "node": "v26.10.0",
    "os": "darwin 27.2.0",
    "arch": "arm64",
    "cpu": "Apple M5 Pro",
    "logicalCpus": 18,
    "colocatedNodes": 24,
    "replicasPerGroup": 3,
    "colocatedDrivers": 16,
    "colocatedControllers": 8,
    "colocatedNativeDrivers": 8
  },
  "options": {
    "offeredRate": 0,
    "groups": 8,
    "tenants": 2,
    "duration": 60,
    "warmup": 1,
    "drain": 60,
    "concurrency": 256,
    "workers": 4,
    "shops": 4,
    "hotShops": 1,
    "hotProbability": 0.8,
    "maxOrders": 96,
    "bakeMs": 250,
    "leaseMs": 2000,
    "duplicateRate": 0.1,
    "abandonRate": 0.1,
    "pollMs": 150,
    "requestTimeoutMs": 2000,
    "retryBudgetMs": 15000,
    "nodes": 3,
    "seed": "42",
    "initialization": "static",
    "queryRouting": "replicas",
    "readConsistency": "replica-local",
    "driver": "rust",
    "driverBinary": "/Users/pcarrier/src/flower/target/release/flower-bench-driver",
    "binary": "/Users/pcarrier/src/flower/target/release/flower",
    "json": "/Users/pcarrier/src/flower/bench/results/latest.json",
    "html": "/Users/pcarrier/src/flower/bench/results/latest.html",
    "baseline": null,
    "cpuProfile": null,
    "chaos": true,
    "http2": true,
    "keepData": false,
    "help": false
  }
}

Group failures and replica routing

[
  {
    "group": "group-0",
    "failures": [],
    "queryRouting": {
      "mode": "replicas",
      "consistency": "replica-local",
      "auditConsistency": "fresh",
      "nodes": [
        {
          "id": 1,
          "url": "http://127.0.0.1:62799",
          "attempts": 97231,
          "completed": 97026,
          "failures": 205
        },
        {
          "id": 2,
          "url": "http://127.0.0.1:62808",
          "attempts": 101336,
          "completed": 101336,
          "failures": 0
        },
        {
          "id": 3,
          "url": "http://127.0.0.1:62813",
          "attempts": 101336,
          "completed": 101336,
          "failures": 0
        }
      ]
    },
    "counters": {
      "issuedOrders": 96,
      "delivered": 96,
      "replayChecks": 12378,
      "claims": 110,
      "emptyClaims": 872,
      "reclaimed": 14,
      "abandoned": 14,
      "lostLeases": 0,
      "staleLeaseChecks": 14
    }
  },
  {
    "group": "group-1",
    "failures": [],
    "queryRouting": {
      "mode": "replicas",
      "consistency": "replica-local",
      "auditConsistency": "fresh",
      "nodes": [
        {
          "id": 1,
          "url": "http://127.0.0.1:62804",
          "attempts": 98016,
          "completed": 97831,
          "failures": 185
        },
        {
          "id": 2,
          "url": "http://127.0.0.1:62818",
          "attempts": 102759,
          "completed": 102759,
          "failures": 0
        },
        {
          "id": 3,
          "url": "http://127.0.0.1:62821",
          "attempts": 102759,
          "completed": 102759,
          "failures": 0
        }
      ]
    },
    "counters": {
      "issuedOrders": 96,
      "delivered": 96,
      "replayChecks": 12577,
      "claims": 105,
      "emptyClaims": 855,
      "reclaimed": 9,
      "abandoned": 9,
      "lostLeases": 0,
      "staleLeaseChecks": 9
    }
  },
  {
    "group": "group-2",
    "failures": [],
    "queryRouting": {
      "mode": "replicas",
      "consistency": "replica-local",
      "auditConsistency": "fresh",
      "nodes": [
        {
          "id": 1,
          "url": "http://127.0.0.1:62805",
          "attempts": 97205,
          "completed": 97061,
          "failures": 144
        },
        {
          "id": 2,
          "url": "http://127.0.0.1:62819",
          "attempts": 101704,
          "completed": 101704,
          "failures": 0
        },
        {
          "id": 3,
          "url": "http://127.0.0.1:62822",
          "attempts": 101704,
          "completed": 101704,
          "failures": 0
        }
      ]
    },
    "counters": {
      "issuedOrders": 96,
      "delivered": 96,
      "replayChecks": 12431,
      "claims": 106,
      "emptyClaims": 843,
      "reclaimed": 10,
      "abandoned": 10,
      "lostLeases": 0,
      "staleLeaseChecks": 10
    }
  },
  {
    "group": "group-3",
    "failures": [],
    "queryRouting": {
      "mode": "replicas",
      "consistency": "replica-local",
      "auditConsistency": "fresh",
      "nodes": [
        {
          "id": 1,
          "url": "http://127.0.0.1:62801",
          "attempts": 96694,
          "completed": 96505,
          "failures": 189
        },
        {
          "id": 2,
          "url": "http://127.0.0.1:62811",
          "attempts": 101384,
          "completed": 101384,
          "failures": 0
        },
        {
          "id": 3,
          "url": "http://127.0.0.1:62816",
          "attempts": 101383,
          "completed": 101383,
          "failures": 0
        }
      ]
    },
    "counters": {
      "issuedOrders": 96,
      "delivered": 96,
      "replayChecks": 12391,
      "claims": 111,
      "emptyClaims": 850,
      "reclaimed": 15,
      "abandoned": 15,
      "lostLeases": 0,
      "staleLeaseChecks": 15
    }
  },
  {
    "group": "group-4",
    "failures": [],
    "queryRouting": {
      "mode": "replicas",
      "consistency": "replica-local",
      "auditConsistency": "fresh",
      "nodes": [
        {
          "id": 1,
          "url": "http://127.0.0.1:62802",
          "attempts": 97646,
          "completed": 97555,
          "failures": 91
        },
        {
          "id": 2,
          "url": "http://127.0.0.1:62810",
          "attempts": 102068,
          "completed": 102068,
          "failures": 0
        },
        {
          "id": 3,
          "url": "http://127.0.0.1:62815",
          "attempts": 102068,
          "completed": 102068,
          "failures": 0
        }
      ]
    },
    "counters": {
      "issuedOrders": 96,
      "delivered": 96,
      "replayChecks": 12428,
      "claims": 107,
      "emptyClaims": 859,
      "reclaimed": 11,
      "abandoned": 11,
      "lostLeases": 0,
      "staleLeaseChecks": 11
    }
  },
  {
    "group": "group-5",
    "failures": [],
    "queryRouting": {
      "mode": "replicas",
      "consistency": "replica-local",
      "auditConsistency": "fresh",
      "nodes": [
        {
          "id": 1,
          "url": "http://127.0.0.1:62803",
          "attempts": 96030,
          "completed": 95839,
          "failures": 191
        },
        {
          "id": 2,
          "url": "http://127.0.0.1:62809",
          "attempts": 100019,
          "completed": 100019,
          "failures": 0
        },
        {
          "id": 3,
          "url": "http://127.0.0.1:62814",
          "attempts": 100018,
          "completed": 100018,
          "failures": 0
        }
      ]
    },
    "counters": {
      "issuedOrders": 96,
      "delivered": 96,
      "replayChecks": 12153,
      "claims": 107,
      "emptyClaims": 842,
      "reclaimed": 11,
      "abandoned": 11,
      "lostLeases": 0,
      "staleLeaseChecks": 11
    }
  },
  {
    "group": "group-6",
    "failures": [],
    "queryRouting": {
      "mode": "replicas",
      "consistency": "replica-local",
      "auditConsistency": "fresh",
      "nodes": [
        {
          "id": 1,
          "url": "http://127.0.0.1:62806",
          "attempts": 97194,
          "completed": 97062,
          "failures": 132
        },
        {
          "id": 2,
          "url": "http://127.0.0.1:62817",
          "attempts": 100781,
          "completed": 100781,
          "failures": 0
        },
        {
          "id": 3,
          "url": "http://127.0.0.1:62820",
          "attempts": 100780,
          "completed": 100780,
          "failures": 0
        }
      ]
    },
    "counters": {
      "issuedOrders": 96,
      "delivered": 96,
      "replayChecks": 12424,
      "claims": 115,
      "emptyClaims": 844,
      "reclaimed": 19,
      "abandoned": 19,
      "lostLeases": 0,
      "staleLeaseChecks": 19
    }
  },
  {
    "group": "group-7",
    "failures": [],
    "queryRouting": {
      "mode": "replicas",
      "consistency": "replica-local",
      "auditConsistency": "fresh",
      "nodes": [
        {
          "id": 1,
          "url": "http://127.0.0.1:62800",
          "attempts": 96081,
          "completed": 95869,
          "failures": 212
        },
        {
          "id": 2,
          "url": "http://127.0.0.1:62807",
          "attempts": 100601,
          "completed": 100601,
          "failures": 0
        },
        {
          "id": 3,
          "url": "http://127.0.0.1:62812",
          "attempts": 100600,
          "completed": 100600,
          "failures": 0
        }
      ]
    },
    "counters": {
      "issuedOrders": 96,
      "delivered": 96,
      "replayChecks": 12417,
      "claims": 112,
      "emptyClaims": 845,
      "reclaimed": 16,
      "abandoned": 16,
      "lostLeases": 0,
      "staleLeaseChecks": 16
    }
  }
]