Skip to content

perf(evm): optimistic parallel block execution in payload validation behind a flag - #9374

Draft
gakonst wants to merge 39 commits into
mainfrom
perf/parallel-block-execution
Draft

gakonst wants to merge 39 commits into
mainfrom
perf/parallel-block-execution

Conversation

@gakonst

@gakonst gakonst commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Adds optimistic parallel block execution to payload validation behind --execution.parallel-threads <N> (blocks with at least 128 transactions), using evm2's Block-STM executor with batched commit, streamed transaction input, receipts and state-root updates, and static scheduling hints derived from senders, fee payers, call targets and TIP-20 payment holders. Blocks run sequentially with prewarming until the prewarm cache is warm (miss rate under 2% for 4 blocks, 8 to 64 blocks). The batched committer applies expiring-nonce inserts from worker-computed ring slots, writes each transaction's batched storage ops with one lock per store shard, takes worker-built receipts without cloning and sends them in batches, and skips per-transaction bookkeeping that workers already did. The multi-version store skips shard locks for keys no transaction wrote and uses a seeded word-at-a-time slot hasher. Idle workers park instead of spinning once no task is left. When parallel execution is on, proof workers run at niceness 19 (--engine.proof-worker-niceness) so execution threads preempt them.

100 GiB e2e bench (validator newPayload, 5 pairs, 6 threads), parallel vs sequential on the same commit, before idle parking: Mgas/s +8.6% (2491 to 2705), P50 -10.2%, P90 -3.7%. Idle parking on top: Mgas/s +4.2% at 6 threads (P90 +2.4%), +1.5% at 8 threads (P90 -7.3%). A randomized differential test (including expiring-nonce ring wrap and full-set cases) found no divergence from sequential execution.

Depends on reth and evm2 branches pinned in Cargo.toml; scripts/foundry-patch.sh builds Forge against the evm2 revision Tempo patches to.

@codspeed

codspeed Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 26 untouched benchmarks
⏩ 25 skipped benchmarks1


Comparing perf/parallel-block-execution (a83337e) with main (b837d84)2

Open in CodSpeed

Footnotes

  1. 25 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on main (1200580) during the generation of this report, so b837d84 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@tempo-github-sts

tempo-github-sts Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

❌ Replay benchmark failed. View logs

@tempo-github-sts

tempo-github-sts Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: perf/parallel-block-execution vs perf/parallel-block-execution
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=perf/parallel-block-execution feature=perf/parallel-block-execution baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=3 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 8"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 3
  • Baseline blocks: 544
  • Feature blocks: 541

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 13553 12901 -4.81% ⚪ (+/-5.07/floor 0.55)
Gas Throughput [Mgas/s] 992.9 944.1 -4.91% ⚪ (+/-4.96/floor 0.50)
Block Time Mean [ms] 505.1 522.7 +3.48% ⚪ (+/-6.11/floor 0.40)
Block Time P50 [ms] 451.0 475.0 +5.32% ❌ (+/-2.14/floor 0.70)
Block Time P90 [ms] 578.0 626.0 +8.30% ❌ (+/-1.41/floor 0.70)
Block Time P99 [ms] 2850.0 3636.0 +27.58% ⚪ (+/-60.61/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2020.0 2038.7 +0.92% ⚪ (+/-0.64/floor 0.95)
P50 [ms] 244.1 237.7 -2.62% ✅ (+/-0.65/floor 0.45)
P90 [ms] 277.3 263.3 -5.05% ✅ (+/-2.49/floor 0.90)
P99 [ms] 323.7 301.8 -6.77% ⚪ (+/-22.54/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 28.4 26.6 -6.34%
Finish P90 [ms] 54.8 54.3 -0.91%
Finish P99 [ms] 98.4 99.6 +1.22%
Pool Fetch P50 [ms] 8.5 8.7 +2.35%
Pool Fetch P90 [ms] 15.8 16.0 +1.27%
Pool Fetch P99 [ms] 27.0 41.3 +52.96%
Stop Reason — Build Budget 542 542 +0.00%
Serialized Block Size P50 [KiB] 2035.3 2027.5 -0.38%
Serialized Block Size P90 [KiB] 2439.4 2336.2 -4.23%
Serialized Block Size P99 [KiB] 2820.1 2898.3 +2.77%
Serialized Block Size / Tx P50 [B/tx] 294.9 294.9 +0.00%
Serialized Block Size / Tx P90 [B/tx] 307.0 307.5 +0.16%
Serialized Block Size / Tx P99 [B/tx] 347.2 375.8 +8.24%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2484.5 2146.9 -13.59% ❌ (+/-1.95/floor 0.65)
P50 [ms] 194.6 220.9 +13.51% ❌ (+/-0.60/floor 1.55)
P90 [ms] 271.5 323.0 +18.97% ❌ (+/-4.60/floor 1.55)
P99 [ms] 454.8 454.6 -0.04% ⚪ (+/-46.21/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 368.3 352.5 -4.29%
Blocks Persisted [/s] 1.73 1.71 -1.16%
Persistence Busy [%] 85.8 86.1 +0.35%
Save Batch Mean [ms] 7212 7567 +4.92%
Max Unpersisted [blocks] 34 34 +0.00%
Backpressure Stall [s] 5.06 8.34 +64.82%

Observability

Warn/Error Logs

Run type Total lines
Baseline 37
Feature 47
Counts by message
Message Baseline Feature
context request for block `0x0f10d3957c0f145be469a2aaede89632adbfafabb7a5798fd96bdfc742419c91` with no consensus context 18 18
failed to read dealer log from block extraData header field 12 12
requested buffer capacity is too low, increasing it to floor 6 6
Parallel block execution is experimental 0 6
Execution cache still in use, executing without it 0 2
[failed delivering block `0x9690279a9122c01a070e183fe62b8c9f25daf5a7a0cba62e9f70141d6b2875e7` for verification (0, 420) 0x9690279a9122c01a070e183fe62b8c9f25daf5a7a0cba62e9f70141d6b2875e7, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
Blocked waiting for execution cache mutex 1 0
executor could not verify the block; abstaining 0 1
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 0 1

@tempo-github-sts

tempo-github-sts Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

✅ Replay benchmark complete! View job

Chain: mainnet
Warmup: 1250
Blocks: 5000
Run pairs: 2

Benchmark Results

Metric perf/parallel-block-execution perf/parallel-block-execution Change
newPayload P50 0.60ms 0.60ms +0.50% ⚪ (±1.34%)
newPayload P90 1.76ms 1.99ms +12.97% ❌ (±1.37%)
newPayload P99 2.69ms 3.24ms +20.38% ❌ (±4.84%)
newPayload Mgas/s 47.42 41.65 -12.16% ❌ (±1.07%)
Wall Clock 8.93s 9.69s +8.50% ❌ (±0.95%)
Persist Wait 0.03ms 0.03ms -3.40% ⚪ (±4.29%)
Wait Time Breakdown

Persistence Wait

Metric perf/parallel-block-execution perf/parallel-block-execution
Mean 0.03ms 0.03ms
P50 0.02ms 0.02ms
P95 0.08ms 0.07ms

Trie Cache Update Wait

Metric perf/parallel-block-execution perf/parallel-block-execution
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric perf/parallel-block-execution perf/parallel-block-execution
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms
## Observability

Warn/Error Logs

Run type Total lines
Baseline 13
Feature 16
Counts by message
Message Baseline Feature
Execution cache still in use, executing without it 11 12
Error updating fork choice: Invalid fork choice update ForkchoiceState { head_block_hash: 0xbe466097abfbb9cfbef9aa2dfec811476ce60ffcebdcc71713fea01855ac0d7b, safe_block_hash: 0xbe466097abfbb9cfbef9aa2dfec811476ce60ffcebdcc71713fea01855ac0d7b, finalized_block_hash: 0xbe466097abfbb9cfbef9aa2dfec811476ce60ffcebdcc71713fea01855ac0d7b }: ForkchoiceUpdated { payload_status: PayloadStatus { status: Syncing, latest_valid_hash: None }, payload_id: None } 2 2
Parallel block execution is experimental 0 2

Charts

Gas vs Latency

Gas vs Latency

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

@tempo-github-sts

tempo-github-sts Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

✅ Replay benchmark complete! View job

Chain: mainnet
Warmup: 1250
Blocks: 5000
Run pairs: 2

Benchmark Results

Metric perf/parallel-block-execution perf/parallel-block-execution Change
newPayload P50 0.60ms 0.59ms -0.67% ⚪ (±1.25%)
newPayload P90 1.78ms 1.80ms +1.41% ❌ (±1.10%)
newPayload P99 2.72ms 2.84ms +4.49% ❌ (±3.46%)
newPayload Mgas/s 47.23 46.89 -0.72% ⚪ (±0.93%)
Wall Clock 8.95s 9.01s +0.71% ⚪ (±0.89%)
Persist Wait 0.03ms 0.03ms +2.72% ⚪ (±4.94%)
Wait Time Breakdown

Persistence Wait

Metric perf/parallel-block-execution perf/parallel-block-execution
Mean 0.03ms 0.03ms
P50 0.02ms 0.02ms
P95 0.07ms 0.07ms

Trie Cache Update Wait

Metric perf/parallel-block-execution perf/parallel-block-execution
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms

Execution Cache Update Wait

Metric perf/parallel-block-execution perf/parallel-block-execution
Mean 0.00ms 0.00ms
P50 0.00ms 0.00ms
P95 0.00ms 0.00ms
## Observability

Warn/Error Logs

Run type Total lines
Baseline 17
Feature 8
Counts by message
Message Baseline Feature
Execution cache still in use, executing without it 15 4
Error updating fork choice: Invalid fork choice update ForkchoiceState { head_block_hash: 0xbe466097abfbb9cfbef9aa2dfec811476ce60ffcebdcc71713fea01855ac0d7b, safe_block_hash: 0xbe466097abfbb9cfbef9aa2dfec811476ce60ffcebdcc71713fea01855ac0d7b, finalized_block_hash: 0xbe466097abfbb9cfbef9aa2dfec811476ce60ffcebdcc71713fea01855ac0d7b }: ForkchoiceUpdated { payload_status: PayloadStatus { status: Syncing, latest_valid_hash: None }, payload_id: None } 2 2
Parallel block execution is experimental 0 2

Charts

Gas vs Latency

Gas vs Latency

Latency, Throughput & Diff

Latency, Throughput & Diff

Wait Time Breakdown

Wait Time Breakdown

@tempo-github-sts

tempo-github-sts Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: perf/parallel-block-execution vs perf/parallel-block-execution
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=perf/parallel-block-execution feature=perf/parallel-block-execution baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=3 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 8 --execution.parallel-in-order"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 3
  • Baseline blocks: 567
  • Feature blocks: 545

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 14031 12907 -8.01% ❌ (+/-5.70/floor 0.55)
Gas Throughput [Mgas/s] 1025.9 943.6 -8.02% ❌ (+/-6.04/floor 0.50)
Block Time Mean [ms] 494.8 497.8 +0.61% ⚪ (+/-7.34/floor 0.40)
Block Time P50 [ms] 453.0 481.0 +6.18% ❌ (+/-2.06/floor 0.70)
Block Time P90 [ms] 589.0 638.0 +8.32% ❌ (+/-3.03/floor 0.70)
Block Time P99 [ms] 2544.0 2541.0 -0.12% ⚪ (+/-29.43/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2035.0 2029.9 -0.25% ⚪ (+/-2.02/floor 0.95)
P50 [ms] 243.1 229.6 -5.55% ✅ (+/-1.73/floor 0.45)
P90 [ms] 281.5 259.1 -7.96% ✅ (+/-3.88/floor 0.90)
P99 [ms] 337.0 416.1 +23.47% ⚪ (+/-96.56/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 28.7 28.0 -2.44%
Finish P90 [ms] 52.4 59.8 +14.12%
Finish P99 [ms] 111.2 158.5 +42.54%
Pool Fetch P50 [ms] 8.6 9.0 +4.65%
Pool Fetch P90 [ms] 16.2 16.8 +3.70%
Pool Fetch P99 [ms] 29.2 27.9 -4.45%
Stop Reason — Build Budget 566 543 -4.06%
Serialized Block Size P50 [KiB] 2060.3 1910.8 -7.25%
Serialized Block Size P90 [KiB] 2491.0 2268.7 -8.93%
Serialized Block Size P99 [KiB] 2785.1 2571.4 -7.67%
Serialized Block Size / Tx P50 [B/tx] 295.2 294.7 -0.17%
Serialized Block Size / Tx P90 [B/tx] 308.2 307.0 -0.39%
Serialized Block Size / Tx P99 [B/tx] 399.7 337.7 -15.51%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2507.8 1864.8 -25.64% ❌ (+/-1.29/floor 0.65)
P50 [ms] 196.3 247.6 +26.13% ❌ (+/-2.15/floor 1.55)
P90 [ms] 276.2 338.5 +22.56% ❌ (+/-5.31/floor 1.55)
P99 [ms] 628.4 747.8 +19.00% ⚪ (+/-34.26/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 394.2 373.0 -5.38%
Blocks Persisted [/s] 1.83 1.77 -3.28%
Persistence Busy [%] 85.5 80.8 -5.50%
Save Batch Mean [ms] 6818 6247 -8.37%
Max Unpersisted [blocks] 34 34 +0.00%
Backpressure Stall [s] 3.92 1.69 -56.89%

Observability

Warn/Error Logs

Run type Total lines
Baseline 36
Feature 47
Counts by message
Message Baseline Feature
context request for block `0x0f10d3957c0f145be469a2aaede89632adbfafabb7a5798fd96bdfc742419c91` with no consensus context 18 18
failed to read dealer log from block extraData header field 12 12
requested buffer capacity is too low, increasing it to floor 6 6
Parallel block execution is experimental 0 6
Blocked waiting for execution cache mutex 0 3
Execution cache still in use, executing without it 0 2

@tempo-github-sts

tempo-github-sts Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: perf/parallel-block-execution vs perf/parallel-block-execution
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=perf/parallel-block-execution feature=perf/parallel-block-execution baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=3 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 6 --engine.prewarming-threads 2"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 3
  • Baseline blocks: 535
  • Feature blocks: 671

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 13530 8626 -36.25% ❌ (+/-7.29/floor 0.55)
Gas Throughput [Mgas/s] 976.5 631.6 -35.32% ❌ (+/-8.01/floor 0.50)
Block Time Mean [ms] 515.5 420.3 -18.47% ✅ (+/-9.72/floor 0.40)
Block Time P50 [ms] 450.0 427.0 -5.11% ✅ (+/-1.81/floor 0.70)
Block Time P90 [ms] 609.0 520.0 -14.61% ✅ (+/-8.84/floor 0.70)
Block Time P99 [ms] 2580.0 663.0 -74.30% ✅ (+/-70.03/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2043.4 875.6 -57.15% ❌ (+/-1.68/floor 0.95)
P50 [ms] 242.1 300.9 +24.29% ❌ (+/-2.29/floor 0.45)
P90 [ms] 281.3 340.6 +21.08% ❌ (+/-3.98/floor 0.90)
P99 [ms] 487.7 367.6 -24.63% ⚪ (+/-35.37/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 28.3 2.6 -90.81%
Finish P90 [ms] 55.0 5.1 -90.73%
Finish P99 [ms] 219.8 8.5 -96.13%
Pool Fetch P50 [ms] 8.6 6.4 -25.58%
Pool Fetch P90 [ms] 16.9 10.2 -39.64%
Pool Fetch P99 [ms] 34.1 14.7 -56.89%
Stop Reason — Build Budget 532 664 +24.81%
Serialized Block Size P50 [KiB] 2102.1 1055.3 -49.80%
Serialized Block Size P90 [KiB] 2504.3 1371.3 -45.24%
Serialized Block Size P99 [KiB] 2909.9 1447.2 -50.27%
Serialized Block Size / Tx P50 [B/tx] 293.6 294.3 +0.24%
Serialized Block Size / Tx P90 [B/tx] 305.0 305.1 +0.03%
Serialized Block Size / Tx P99 [B/tx] 375.8 323.2 -14.00%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2483.2 1818.0 -26.79% ❌ (+/-1.11/floor 0.65)
P50 [ms] 198.2 144.1 -27.30% ✅ (+/-3.78/floor 1.55)
P90 [ms] 281.7 204.4 -27.44% ✅ (+/-5.50/floor 1.55)
P99 [ms] 614.7 426.3 -30.65% ✅ (+/-15.57/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 364.5 336.6 -7.65%
Blocks Persisted [/s] 1.68 2.24 +33.33%
Persistence Busy [%] 85.6 63.7 -25.58%
Save Batch Mean [ms] 7615 3121 -59.02%
Max Unpersisted [blocks] 37 17 -54.05%
Backpressure Stall [s] 5.32 0.00 -100.00%

Observability

Warn/Error Logs

Run type Total lines
Baseline 36
Feature 47
Counts by message
Message Baseline Feature
context request for block `0x0f10d3957c0f145be469a2aaede89632adbfafabb7a5798fd96bdfc742419c91` with no consensus context 18 18
failed to read dealer log from block extraData header field 12 12
requested buffer capacity is too low, increasing it to floor 6 6
Parallel block execution is experimental 0 6
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 0 2
[failed delivering block `0xd47972673aab33ff92c28fc74ba1eefe7fa6da31173e586339aa42db596865df` for verification (0, 384) 0xd47972673aab33ff92c28fc74ba1eefe7fa6da31173e586339aa42db596865df, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
[failed delivering block `0xd5bfe67926b574856f46a85c13bb05a792d63fd5b579e1e575ddcd430dca4bc0` for verification (0, 561) 0xd5bfe67926b574856f46a85c13bb05a792d63fd5b579e1e575ddcd430dca4bc0, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
executor could not verify the block; abstaining 0 1

@tempo-github-sts

tempo-github-sts Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: 846e55f vs 846e55f
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=846e55fae82a0034e213d59a9122276e9432a97b feature=846e55fae82a0034e213d59a9122276e9432a97b baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=3 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 8"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 3
  • Baseline blocks: 480
  • Feature blocks: 496

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 11747 11760 +0.11% ⚪ (+/-3.09/floor 0.55)
Gas Throughput [Mgas/s] 852.4 860.7 +0.97% ⚪ (+/-3.45/floor 0.50)
Block Time Mean [ms] 599.3 573.1 -4.37% ✅ (+/-2.04/floor 0.40)
Block Time P50 [ms] 464.0 487.0 +4.96% ❌ (+/-1.08/floor 0.70)
Block Time P90 [ms] 624.0 621.0 -0.48% ⚪ (+/-12.42/floor 0.70)
Block Time P99 [ms] 5317.0 5144.0 -3.25% ⚪ (+/-18.98/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2055.2 2078.8 +1.15% ⚪ (+/-1.10/floor 0.95)
P50 [ms] 243.5 234.9 -3.53% ✅ (+/-2.43/floor 0.45)
P90 [ms] 280.6 254.6 -9.27% ✅ (+/-1.96/floor 0.90)
P99 [ms] 342.9 304.2 -11.29% ✅ (+/-9.76/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 26.6 27.3 +2.63%
Finish P90 [ms] 54.6 53.4 -2.20%
Finish P99 [ms] 115.7 108.1 -6.57%
Pool Fetch P50 [ms] 8.7 9.0 +3.45%
Pool Fetch P90 [ms] 16.1 15.7 -2.48%
Pool Fetch P99 [ms] 39.2 33.7 -14.03%
Invalid Tx Attempts P99 6103.0 0.0 -100.00%
Stop Reason — Build Budget 485 496 +2.27%
Serialized Block Size P50 [KiB] 2084.4 2004.0 -3.86%
Serialized Block Size P90 [KiB] 2489.4 2344.3 -5.83%
Serialized Block Size P99 [KiB] 2810.2 2782.2 -1.00%
Serialized Block Size / Tx P50 [B/tx] 293.9 294.8 +0.31%
Serialized Block Size / Tx P90 [B/tx] 304.1 306.8 +0.89%
Serialized Block Size / Tx P99 [B/tx] 351.9 347.3 -1.31%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2467.7 2005.3 -18.74% ❌ (+/-3.36/floor 0.65)
P50 [ms] 198.7 241.8 +21.69% ❌ (+/-4.80/floor 1.55)
P90 [ms] 292.3 324.0 +10.85% ❌ (+/-6.96/floor 1.55)
P99 [ms] 648.8 558.5 -13.92% ⚪ (+/-31.33/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 316.9 318.0 +0.35%
Blocks Persisted [/s] 1.48 1.58 +6.76%
Persistence Busy [%] 84.6 85.9 +1.54%
Save Batch Mean [ms] 8639 8080 -6.47%
Max Unpersisted [blocks] 34 34 +0.00%
Backpressure Stall [s] 13.20 8.58 -35.00%

Observability

Warn/Error Logs

Run type Total lines
Baseline 37
Feature 49
Counts by message
Message Baseline Feature
context request for block `0x0f10d3957c0f145be469a2aaede89632adbfafabb7a5798fd96bdfc742419c91` with no consensus context 18 18
failed to read dealer log from block extraData header field 12 12
requested buffer capacity is too low, increasing it to floor 6 6
Parallel block execution is experimental 0 6
[failed delivering build parent, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 2
[failed delivering finalized block `0x85a9209346582914eb1321b13496fa58f9737c135a3685b916c6f19ed9b160a5` at height `327`, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 2
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 0 2
[executor dropped the payload channel: the build failed (the executor logs the cause) or the executor shut down, oneshot canceled] 0 1
Execution cache still in use, executing without it 1 0

@tempo-github-sts

tempo-github-sts Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: 846e55f vs 846e55f
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=846e55fae82a0034e213d59a9122276e9432a97b feature=846e55fae82a0034e213d59a9122276e9432a97b baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=3 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 8 --engine.disable-prewarming"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 3
  • Baseline blocks: 552
  • Feature blocks: 543

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 13932 13571 -2.59% ⚪ (+/-6.79/floor 0.55)
Gas Throughput [Mgas/s] 1030.0 995.2 -3.38% ⚪ (+/-6.95/floor 0.50)
Block Time Mean [ms] 498.8 497.2 -0.32% ⚪ (+/-7.62/floor 0.40)
Block Time P50 [ms] 448.0 473.0 +5.58% ❌ (+/-1.90/floor 0.70)
Block Time P90 [ms] 607.0 655.0 +7.91% ❌ (+/-4.39/floor 0.70)
Block Time P99 [ms] 2275.0 1618.0 -28.88% ✅ (+/-18.32/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2031.2 2089.3 +2.86% ✅ (+/-0.99/floor 0.95)
P50 [ms] 246.9 234.4 -5.06% ✅ (+/-1.92/floor 0.45)
P90 [ms] 285.4 256.3 -10.20% ✅ (+/-2.08/floor 0.90)
P99 [ms] 379.0 309.4 -18.36% ⚪ (+/-67.74/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 29.2 27.1 -7.19%
Finish P90 [ms] 53.7 50.1 -6.70%
Finish P99 [ms] 127.5 96.3 -24.47%
Pool Fetch P50 [ms] 8.5 8.5 +0.00%
Pool Fetch P90 [ms] 16.3 13.9 -14.72%
Pool Fetch P99 [ms] 37.7 28.6 -24.14%
Stop Reason — Build Budget 552 540 -2.17%
Serialized Block Size P50 [KiB] 2073.2 1999.4 -3.56%
Serialized Block Size P90 [KiB] 2498.8 2386.9 -4.48%
Serialized Block Size P99 [KiB] 2904.3 2725.4 -6.16%
Serialized Block Size / Tx P50 [B/tx] 295.7 295.2 -0.17%
Serialized Block Size / Tx P90 [B/tx] 309.3 308.6 -0.23%
Serialized Block Size / Tx P99 [B/tx] 362.9 359.3 -0.99%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2524.5 2062.0 -18.32% ❌ (+/-2.18/floor 0.65)
P50 [ms] 200.1 234.8 +17.34% ❌ (+/-1.65/floor 1.55)
P90 [ms] 275.8 342.7 +24.26% ❌ (+/-3.44/floor 1.55)
P99 [ms] 552.7 524.4 -5.12% ⚪ (+/-59.95/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 380.1 371.8 -2.18%
Blocks Persisted [/s] 1.74 1.73 -0.57%
Persistence Busy [%] 84.7 81.4 -3.90%
Save Batch Mean [ms] 6943 6819 -1.79%
Max Unpersisted [blocks] 35 34 -2.86%
Backpressure Stall [s] 3.38 2.36 -30.18%

Observability

Warn/Error Logs

Run type Total lines
Baseline 40
Feature 46
Counts by message
Message Baseline Feature
context request for block `0x0f10d3957c0f145be469a2aaede89632adbfafabb7a5798fd96bdfc742419c91` with no consensus context 18 18
failed to read dealer log from block extraData header field 12 12
requested buffer capacity is too low, increasing it to floor 6 6
Parallel block execution is experimental 0 6
executor could not verify the block; abstaining 1 1
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 1 1
[failed delivering block `0x072b172cf03b7049131006ffa540a783d776f0dd18f098f6ac8096d00569cf85` for verification (0, 368) 0x072b172cf03b7049131006ffa540a783d776f0dd18f098f6ac8096d00569cf85, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
[failed delivering block `0x81e5ca284165fee47cceee9f4f12f41b7ef79b6abdfb0619090ae913589ab22a` for verification (0, 408) 0x81e5ca284165fee47cceee9f4f12f41b7ef79b6abdfb0619090ae913589ab22a, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
dropped our proposal 1 0
Execution cache still in use, executing without it 0 1

@tempo-github-sts

tempo-github-sts Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: 45353f2 vs 45353f2
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=45353f296974b0e303692f5707faee00798bfbed feature=45353f296974b0e303692f5707faee00798bfbed baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=3 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 8"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 3
  • Baseline blocks: 528
  • Feature blocks: 535

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 13430 12722 -5.27% ❌ (+/-3.42/floor 0.55)
Gas Throughput [Mgas/s] 982.3 935.6 -4.75% ❌ (+/-3.78/floor 0.50)
Block Time Mean [ms] 526.3 528.9 +0.49% ⚪ (+/-4.59/floor 0.40)
Block Time P50 [ms] 454.0 478.0 +5.29% ❌ (+/-1.17/floor 0.70)
Block Time P90 [ms] 592.0 674.0 +13.85% ❌ (+/-6.08/floor 0.70)
Block Time P99 [ms] 1129.0 2210.0 +95.75% ⚪ (+/-319.62/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2067.1 2099.2 +1.55% ⚪ (+/-2.39/floor 0.95)
P50 [ms] 244.7 233.7 -4.50% ✅ (+/-0.51/floor 0.45)
P90 [ms] 283.3 254.1 -10.31% ✅ (+/-2.29/floor 0.90)
P99 [ms] 349.2 294.2 -15.75% ⚪ (+/-30.72/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 27.1 26.1 -3.69%
Finish P90 [ms] 51.0 47.7 -6.47%
Finish P99 [ms] 116.1 83.8 -27.82%
Pool Fetch P50 [ms] 8.2 9.1 +10.98%
Pool Fetch P90 [ms] 14.9 15.4 +3.36%
Pool Fetch P99 [ms] 36.3 24.6 -32.23%
Stop Reason — Build Budget 524 534 +1.91%
Serialized Block Size P50 [KiB] 2091.6 1969.3 -5.85%
Serialized Block Size P90 [KiB] 2502.1 2369.2 -5.31%
Serialized Block Size P99 [KiB] 2725.3 2777.5 +1.91%
Serialized Block Size / Tx P50 [B/tx] 294.8 295.3 +0.17%
Serialized Block Size / Tx P90 [B/tx] 306.3 306.5 +0.07%
Serialized Block Size / Tx P99 [B/tx] 340.5 388.9 +14.21%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2559.8 2029.5 -20.71% ❌ (+/-0.68/floor 0.65)
P50 [ms] 196.3 237.3 +20.89% ❌ (+/-3.65/floor 1.55)
P90 [ms] 275.0 362.3 +31.75% ❌ (+/-4.44/floor 1.55)
P99 [ms] 368.3 549.1 +49.09% ❌ (+/-13.99/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 351.4 356.9 +1.57%
Blocks Persisted [/s] 1.65 1.68 +1.82%
Persistence Busy [%] 86.8 82.3 -5.18%
Save Batch Mean [ms] 7985 6964 -12.79%
Max Unpersisted [blocks] 36 35 -2.78%
Backpressure Stall [s] 11.08 5.05 -54.42%

Observability

Warn/Error Logs

Run type Total lines
Baseline 47
Feature 49
Counts by message
Message Baseline Feature
context request for block `0x57874769dbc59f1326797f6baa060c0ee1502b13a28f976f967e9cf801c925c0` with no consensus context 18 18
failed to read dealer log from block extraData header field 12 12
requested buffer capacity is too low, increasing it to floor 6 6
Parallel block execution is experimental 0 6
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 3 2
[failed delivering build parent, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 2 2
[executor dropped the payload channel: the build failed (the executor logs the cause) or the executor shut down, oneshot canceled] 1 1
[failed delivering finalized block `0xd4ae100e8a42fbfebe5ca228e4ff34f4c644faad2e384e667525b941dbc9a6ae` at height `377`, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 2 0
executor could not verify the block; abstaining 1 1
[failed delivering block `0x8f517b1306ae36f6f05089bfe49186c0d80d4c7d183fe08d748987268d78ea6b` for verification (0, 363) 0x8f517b1306ae36f6f05089bfe49186c0d80d4c7d183fe08d748987268d78ea6b, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
[failed delivering block `0xbca4074a147cf333c38127c49d2cebd3d2b07ce03a8cada4b35fa8ffb003c730` for verification (0, 340) 0xbca4074a147cf333c38127c49d2cebd3d2b07ce03a8cada4b35fa8ffb003c730, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
dropped our proposal 1 0

@tempo-github-sts

tempo-github-sts Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: 45353f2 vs 45353f2
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=45353f296974b0e303692f5707faee00798bfbed feature=45353f296974b0e303692f5707faee00798bfbed baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=3 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 12"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 3
  • Baseline blocks: 549
  • Feature blocks: 545

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 13941 13309 -4.53% ⚪ (+/-4.19/floor 0.55)
Gas Throughput [Mgas/s] 1016.7 977.6 -3.85% ⚪ (+/-5.05/floor 0.50)
Block Time Mean [ms] 496.0 493.8 -0.44% ⚪ (+/-3.47/floor 0.40)
Block Time P50 [ms] 449.0 485.0 +8.02% ❌ (+/-0.67/floor 0.70)
Block Time P90 [ms] 617.0 667.0 +8.10% ❌ (+/-4.11/floor 0.70)
Block Time P99 [ms] 2327.0 1089.0 -53.20% ✅ (+/-15.26/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2006.3 2043.6 +1.86% ⚪ (+/-2.18/floor 0.95)
P50 [ms] 245.8 231.9 -5.66% ✅ (+/-2.01/floor 0.45)
P90 [ms] 285.1 255.6 -10.35% ✅ (+/-2.02/floor 0.90)
P99 [ms] 458.6 302.2 -34.10% ✅ (+/-14.38/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 28.9 26.3 -9.00%
Finish P90 [ms] 51.6 50.8 -1.55%
Finish P99 [ms] 193.7 90.2 -53.43%
Pool Fetch P50 [ms] 9.2 8.8 -4.35%
Pool Fetch P90 [ms] 16.8 15.8 -5.95%
Pool Fetch P99 [ms] 34.9 29.1 -16.62%
Stop Reason — Build Budget 550 540 -1.82%
Serialized Block Size P50 [KiB] 2054.8 1886.3 -8.20%
Serialized Block Size P90 [KiB] 2483.0 2358.7 -5.00%
Serialized Block Size P99 [KiB] 2717.3 2644.9 -2.66%
Serialized Block Size / Tx P50 [B/tx] 295.0 295.7 +0.24%
Serialized Block Size / Tx P90 [B/tx] 306.5 305.6 -0.29%
Serialized Block Size / Tx P99 [B/tx] 354.4 341.5 -3.64%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2461.3 1877.4 -23.72% ❌ (+/-2.25/floor 0.65)
P50 [ms] 199.9 250.3 +25.21% ❌ (+/-1.23/floor 1.55)
P90 [ms] 283.8 355.9 +25.41% ❌ (+/-6.61/floor 1.55)
P99 [ms] 847.0 564.5 -33.35% ✅ (+/-26.73/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 384.7 376.9 -2.03%
Blocks Persisted [/s] 1.73 1.81 +4.62%
Persistence Busy [%] 82.3 83.3 +1.22%
Save Batch Mean [ms] 6557 6521 -0.55%
Max Unpersisted [blocks] 35 35 +0.00%
Backpressure Stall [s] 2.02 1.92 -4.95%

Observability

Warn/Error Logs

Run type Total lines
Baseline 40
Feature 43
Counts by message
Message Baseline Feature
context request for block `0x57874769dbc59f1326797f6baa060c0ee1502b13a28f976f967e9cf801c925c0` with no consensus context 18 18
failed to read dealer log from block extraData header field 12 12
requested buffer capacity is too low, increasing it to floor 6 6
Parallel block execution is experimental 0 6
Execution cache still in use, executing without it 1 1
[failed delivering block `0x48e01f450719ad7b70958267559c65113b30ef36ebadf2fa0c6631775247cf14` for verification (0, 391) 0x48e01f450719ad7b70958267559c65113b30ef36ebadf2fa0c6631775247cf14, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
executor could not verify the block; abstaining 1 0
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 1 0

@tempo-github-sts

tempo-github-sts Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚪ Benchmark complete: No Difference View job

⚪ Bench Comparison: No Difference

Refs: 45353f2 vs 45353f2
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=45353f296974b0e303692f5707faee00798bfbed feature=45353f296974b0e303692f5707faee00798bfbed baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=1 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 samply feature-args="--execution.parallel-threads 8"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 1
  • Baseline blocks: 186
  • Feature blocks: 200

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 13415 13519 +0.78% ⚪
Gas Throughput [Mgas/s] 990.7 995.9 +0.52% ⚪
Block Time Mean [ms] 483.2 448.6 -7.16% ⚪
Block Time P50 [ms] 450.0 449.0 -0.22% ⚪
Block Time P90 [ms] 619.0 616.0 -0.48% ⚪
Block Time P99 [ms] 3359.0 978.0 -70.88% ⚪

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 1893.8 1873.0 -1.10% ⚪
P50 [ms] 244.9 232.7 -4.98% ⚪
P90 [ms] 288.1 257.0 -10.79% ⚪
P99 [ms] 509.7 299.9 -41.16% ⚪
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 31.8 31.4 -1.26%
Finish P90 [ms] 63.0 54.6 -13.33%
Finish P99 [ms] 107.2 94.2 -12.13%
Pool Fetch P50 [ms] 8.8 8.9 +1.14%
Pool Fetch P90 [ms] 15.5 15.6 +0.65%
Pool Fetch P99 [ms] 38.9 43.2 +11.05%
Invalid Tx Attempts P99 0.0 5108.0 n/a
Stop Reason — Build Budget 186 197 +5.91%
Serialized Block Size P50 [KiB] 1907.4 1765.3 -7.45%
Serialized Block Size P90 [KiB] 2322.4 2154.9 -7.21%
Serialized Block Size P99 [KiB] 2523.0 2550.2 +1.08%
Serialized Block Size / Tx P50 [B/tx] 295.7 295.8 +0.03%
Serialized Block Size / Tx P90 [B/tx] 308.8 308.5 -0.10%
Serialized Block Size / Tx P99 [B/tx] 453.8 341.2 -24.81%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2361.0 1918.2 -18.76% ⚪
P50 [ms] 191.6 225.1 +17.48% ⚪
P90 [ms] 291.0 326.1 +12.06% ⚪
P99 [ms] 508.4 495.7 -2.50% ⚪

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 394.7 396.1 +0.35%
Blocks Persisted [/s] 1.82 2.03 +11.54%
Persistence Busy [%] 84.5 86.4 +2.25%
Save Batch Mean [ms] 6045 6353 +5.10%
Max Unpersisted [blocks] 34 35 +2.94%
Backpressure Stall [s] 0.55 0.59 +7.27%

Observability

Warn/Error Logs

Run type Total lines
Baseline 12
Feature 14
Counts by message
Message Baseline Feature
context request for block `0x0f10d3957c0f145be469a2aaede89632adbfafabb7a5798fd96bdfc742419c91` with no consensus context 6 6
failed to read dealer log from block extraData header field 4 4
requested buffer capacity is too low, increasing it to floor 2 2
Parallel block execution is experimental 0 2

Samply Profiles

@tempo-github-sts

tempo-github-sts Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: 45353f2 vs 45353f2
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=45353f296974b0e303692f5707faee00798bfbed feature=45353f296974b0e303692f5707faee00798bfbed baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=3 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 2"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 3
  • Baseline blocks: 552
  • Feature blocks: 554

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 13601 13510 -0.67% ⚪ (+/-4.74/floor 0.55)
Gas Throughput [Mgas/s] 990.9 984.1 -0.69% ⚪ (+/-4.83/floor 0.50)
Block Time Mean [ms] 518.3 479.5 -7.49% ✅ (+/-6.98/floor 0.40)
Block Time P50 [ms] 449.0 435.0 -3.12% ⚪ (+/-15.00/floor 0.70)
Block Time P90 [ms] 590.0 804.0 +36.27% ❌ (+/-5.48/floor 0.70)
Block Time P99 [ms] 2788.0 1344.0 -51.79% ✅ (+/-40.25/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2051.3 2109.5 +2.84% ✅ (+/-0.88/floor 0.95)
P50 [ms] 243.3 230.5 -5.26% ✅ (+/-3.42/floor 0.45)
P90 [ms] 285.2 250.4 -12.20% ✅ (+/-2.83/floor 0.90)
P99 [ms] 331.1 287.3 -13.23% ✅ (+/-2.61/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 27.0 25.2 -6.67%
Finish P90 [ms] 50.4 43.2 -14.29%
Finish P99 [ms] 92.8 86.8 -6.47%
Pool Fetch P50 [ms] 8.5 9.1 +7.06%
Pool Fetch P90 [ms] 16.0 14.0 -12.50%
Pool Fetch P99 [ms] 25.8 25.2 -2.33%
Stop Reason — Build Budget 547 549 +0.37%
Serialized Block Size P50 [KiB] 2105.1 1912.8 -9.13%
Serialized Block Size P90 [KiB] 2506.3 2389.5 -4.66%
Serialized Block Size P99 [KiB] 2779.4 2773.4 -0.22%
Serialized Block Size / Tx P50 [B/tx] 294.4 294.6 +0.07%
Serialized Block Size / Tx P90 [B/tx] 307.8 306.5 -0.42%
Serialized Block Size / Tx P99 [B/tx] 531.9 392.8 -26.15%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2508.3 2021.6 -19.40% ❌ (+/-3.72/floor 0.65)
P50 [ms] 199.5 217.4 +8.97% ❌ (+/-3.49/floor 1.55)
P90 [ms] 279.5 525.0 +87.84% ❌ (+/-5.71/floor 1.55)
P99 [ms] 421.4 784.7 +86.21% ❌ (+/-22.28/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 368.5 376.6 +2.20%
Blocks Persisted [/s] 1.67 1.82 +8.98%
Persistence Busy [%] 83.7 76.1 -9.08%
Save Batch Mean [ms] 7660 5402 -29.48%
Max Unpersisted [blocks] 35 35 +0.00%
Backpressure Stall [s] 7.42 0.03 -99.60%

Observability

Warn/Error Logs

Run type Total lines
Baseline 43
Feature 43
Counts by message
Message Baseline Feature
context request for block `0x57874769dbc59f1326797f6baa060c0ee1502b13a28f976f967e9cf801c925c0` with no consensus context 18 18
failed to read dealer log from block extraData header field 12 12
requested buffer capacity is too low, increasing it to floor 6 6
Parallel block execution is experimental 0 6
BatchSpanProcessor dropped a Span due to queue full. No further log will be emitted for further drops until Shutdown. During Shutdown time, a log will be emitted with exact count of total spans dropped. 2 1
Execution cache still in use, executing without it 2 0
[failed delivering block `0x788bb3efe98642a8beb6ae5678c6f92a7b5d1af5ca6bfe55511b878835867884` for verification (0, 384) 0x788bb3efe98642a8beb6ae5678c6f92a7b5d1af5ca6bfe55511b878835867884, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
executor could not verify the block; abstaining 1 0
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 1 0

@tempo-github-sts

tempo-github-sts Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: 45353f2 vs 45353f2
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=45353f296974b0e303692f5707faee00798bfbed feature=45353f296974b0e303692f5707faee00798bfbed baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=3 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 4"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 3
  • Baseline blocks: 578
  • Feature blocks: 571

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 13961 14047 +0.62% ⚪ (+/-2.79/floor 0.55)
Gas Throughput [Mgas/s] 1019.0 1036.2 +1.69% ⚪ (+/-2.70/floor 0.50)
Block Time Mean [ms] 481.0 465.6 -3.20% ✅ (+/-1.27/floor 0.40)
Block Time P50 [ms] 449.0 455.0 +1.34% ⚪ (+/-1.04/floor 0.70)
Block Time P90 [ms] 583.0 652.0 +11.84% ❌ (+/-1.49/floor 0.70)
Block Time P99 [ms] 2527.0 1224.0 -51.56% ✅ (+/-17.48/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 1963.5 2031.7 +3.47% ✅ (+/-1.09/floor 0.95)
P50 [ms] 243.2 236.2 -2.88% ✅ (+/-0.52/floor 0.45)
P90 [ms] 283.2 253.7 -10.42% ✅ (+/-3.10/floor 0.90)
P99 [ms] 640.1 286.6 -55.23% ⚪ (+/-61.65/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 28.8 26.3 -8.68%
Finish P90 [ms] 57.3 46.4 -19.02%
Finish P99 [ms] 359.6 75.3 -79.06%
Pool Fetch P50 [ms] 8.4 9.1 +8.33%
Pool Fetch P90 [ms] 16.3 14.5 -11.04%
Pool Fetch P99 [ms] 27.6 24.5 -11.23%
Stop Reason — Build Budget 579 567 -2.07%
Serialized Block Size P50 [KiB] 2004.7 1932.3 -3.61%
Serialized Block Size P90 [KiB] 2446.3 2318.6 -5.22%
Serialized Block Size P99 [KiB] 2753.4 2572.8 -6.56%
Serialized Block Size / Tx P50 [B/tx] 295.1 295.8 +0.24%
Serialized Block Size / Tx P90 [B/tx] 307.7 306.5 -0.39%
Serialized Block Size / Tx P99 [B/tx] 541.6 393.7 -27.31%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2435.3 2092.8 -14.06% ❌ (+/-1.87/floor 0.65)
P50 [ms] 194.8 223.3 +14.63% ❌ (+/-1.04/floor 1.55)
P90 [ms] 278.8 379.6 +36.15% ❌ (+/-3.34/floor 1.55)
P99 [ms] 495.1 620.6 +25.35% ⚪ (+/-47.89/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 391.4 389.7 -0.43%
Blocks Persisted [/s] 1.85 1.86 +0.54%
Persistence Busy [%] 85.2 81.3 -4.58%
Save Batch Mean [ms] 6737 6217 -7.72%
Max Unpersisted [blocks] 34 34 +0.00%
Backpressure Stall [s] 2.39 0.48 -79.92%

Observability

Warn/Error Logs

Run type Total lines
Baseline 45
Feature 50
Counts by message
Message Baseline Feature
context request for block `0x0f10d3957c0f145be469a2aaede89632adbfafabb7a5798fd96bdfc742419c91` with no consensus context 18 18
failed to read dealer log from block extraData header field 12 12
requested buffer capacity is too low, increasing it to floor 6 6
Parallel block execution is experimental 0 6
BatchSpanProcessor dropped a Span due to queue full. No further log will be emitted for further drops until Shutdown. During Shutdown time, a log will be emitted with exact count of total spans dropped. 2 2
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 2 2
[failed delivering build parent, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 2
executor could not verify the block; abstaining 2 0
[executor dropped the payload channel: the build failed (the executor logs the cause) or the executor shut down, oneshot canceled] 0 1
[failed delivering block `0x1fb5e37895fd52bbc1d266ca6d223cc9beaea996c3cf52730b0fc023a7217c9e` for verification (0, 418) 0x1fb5e37895fd52bbc1d266ca6d223cc9beaea996c3cf52730b0fc023a7217c9e, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
[failed delivering block `0xd55efa45149e6f31f96dbc1cef3c42a2204c1edc56c4c666af546090bc763520` for verification (0, 372) 0xd55efa45149e6f31f96dbc1cef3c42a2204c1edc56c4c666af546090bc763520, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
[failed delivering block `0xe39fba69c22483ef8f5da3482e3f4022e3f4cefec83a195eae02f4050992b2f2` for verification (0, 370) 0xe39fba69c22483ef8f5da3482e3f4022e3f4cefec83a195eae02f4050992b2f2, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
Execution cache still in use, executing without it 1 0

…s out of it

The parallel executor now starts each transaction as soon as the engine
decodes and recovers it, running pre-execution checks and lane
classification as transactions arrive, and hands each receipt to the
receipt root task as it commits. Section and lane accounting still runs as
each transaction commits, and a failed batched commit still executes the
block again one transaction at a time.
…e committer

Pins evm2 to a revision whose batched commit hands the state hook to a streamer thread, runs upcoming deferred storage operations on idle workers so the slots they read are cached before commit, reads the final state on several threads and frees the store in the background. The tip20_parallel bench gains TEMPO_PAR_HOOK and TEMPO_PAR_BASE_NS to model a node state hook and cold database reads.
@gakonst gakonst changed the title perf(evm): optimistic parallel block execution with batched commit behind a flag perf(evm): optimistic parallel block execution in payload validation behind a flag Nov 15, 2026
@tempo-github-sts

tempo-github-sts Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: d5a5e13 vs d5a5e13
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=d5a5e134c55f36421028d2b3f260bdf9188a9e8d feature=d5a5e134c55f36421028d2b3f260bdf9188a9e8d baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=2 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 4"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 2
  • Baseline blocks: 381
  • Feature blocks: 385

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 13870 13695 -1.26% ⚪ (+/-0.75/floor 0.55)
Gas Throughput [Mgas/s] 1012.1 1008.7 -0.34% ⚪ (+/-1.66/floor 0.50)
Block Time Mean [ms] 474.3 466.0 -1.75% ⚪ (+/-3.14/floor 0.40)
Block Time P50 [ms] 432.0 414.0 -4.17% ⚪ (+/-6.60/floor 0.70)
Block Time P90 [ms] 627.0 658.0 +4.94% ⚪ (+/-7.26/floor 0.70)
Block Time P99 [ms] 2522.0 2001.0 -20.66% ✅ (+/-7.47/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 1972.0 2048.0 +3.85% ✅ (+/-1.37/floor 0.95)
P50 [ms] 237.3 231.6 -2.40% ✅ (+/-0.36/floor 0.45)
P90 [ms] 279.8 255.1 -8.83% ✅ (+/-2.07/floor 0.90)
P99 [ms] 723.9 468.5 -35.28% ✅ (+/-22.82/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 30.5 26.6 -12.79%
Finish P90 [ms] 63.6 55.4 -12.89%
Finish P99 [ms] 523.2 205.0 -60.82%
Pool Fetch P50 [ms] 9.5 8.7 -8.42%
Pool Fetch P90 [ms] 17.2 16.0 -6.98%
Pool Fetch P99 [ms] 28.3 37.0 +30.74%
Invalid Tx Attempts P99 0.0 4560.0 n/a
Stop Reason — Build Budget 380 383 +0.79%
Serialized Block Size P50 [KiB] 1914.5 1848.9 -3.43%
Serialized Block Size P90 [KiB] 2383.8 2363.7 -0.84%
Serialized Block Size P99 [KiB] 2683.7 2635.8 -1.79%
Serialized Block Size / Tx P50 [B/tx] 294.6 295.3 +0.24%
Serialized Block Size / Tx P90 [B/tx] 307.6 307.8 +0.07%
Serialized Block Size / Tx P99 [B/tx] 481.4 345.2 -28.29%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2349.8 2399.2 +2.10% ✅ (+/-1.41/floor 0.65)
P50 [ms] 200.2 187.5 -6.34% ✅ (+/-0.72/floor 1.55)
P90 [ms] 274.0 369.0 +34.67% ❌ (+/-5.95/floor 1.55)
P99 [ms] 856.6 545.1 -36.36% ✅ (+/-27.49/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 404.8 399.0 -1.43%
Blocks Persisted [/s] 1.89 1.81 -4.23%
Persistence Busy [%] 85.7 79.5 -7.23%
Save Batch Mean [ms] 6281 5941 -5.41%
Max Unpersisted [blocks] 37 36 -2.70%
Backpressure Stall [s] 1.18 0.94 -20.34%

Observability

Warn/Error Logs

Run type Total lines
Baseline 32
Feature 34
Counts by message
Message Baseline Feature
context request for block `0x57874769dbc59f1326797f6baa060c0ee1502b13a28f976f967e9cf801c925c0` with no consensus context 12 12
failed to read dealer log from block extraData header field 8 8
requested buffer capacity is too low, increasing it to floor 4 4
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 2 2
Parallel block execution is experimental 0 4
[failed delivering build parent, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 2 0
Execution cache still in use, executing without it 2 0
executor could not verify the block; abstaining 0 2
[executor dropped the payload channel: the build failed (the executor logs the cause) or the executor shut down, oneshot canceled] 1 0
[failed delivering block `0x56f92be17b00806a2fa78bc6cbc7360cbcec96b61af6f9e9119d2f96d26579ac` for verification (0, 389) 0x56f92be17b00806a2fa78bc6cbc7360cbcec96b61af6f9e9119d2f96d26579ac, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
[failed delivering block `0x9fb381fc4b847f0c3520e16e6d733badb9c4931a7e4ca4dde9c1854e3c54af7f` for verification (0, 550) 0x9fb381fc4b847f0c3520e16e6d733badb9c4931a7e4ca4dde9c1854e3c54af7f, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
[failed delivering block `0xf4a93851b2e3ae0ecad15aa3be12fc2df81a3a99d57cd4bb88404993271078de` for verification (0, 479) 0xf4a93851b2e3ae0ecad15aa3be12fc2df81a3a99d57cd4bb88404993271078de, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0

@tempo-github-sts

tempo-github-sts Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: d5a5e13 vs d5a5e13
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=d5a5e134c55f36421028d2b3f260bdf9188a9e8d feature=d5a5e134c55f36421028d2b3f260bdf9188a9e8d baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=2 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 6"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 2
  • Baseline blocks: 374
  • Feature blocks: 377

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 14316 14359 +0.30% ⚪ (+/-1.25/floor 0.55)
Gas Throughput [Mgas/s] 1054.6 1048.4 -0.59% ⚪ (+/-2.15/floor 0.50)
Block Time Mean [ms] 478.7 477.6 -0.23% ⚪ (+/-1.14/floor 0.40)
Block Time P50 [ms] 444.0 436.0 -1.80% ✅ (+/-0.56/floor 0.70)
Block Time P90 [ms] 619.0 653.0 +5.49% ❌ (+/-4.36/floor 0.70)
Block Time P99 [ms] 1580.0 1687.0 +6.77% ⚪ (+/-28.13/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2010.8 2072.2 +3.05% ⚪ (+/-2.33/floor 0.95)
P50 [ms] 243.7 234.7 -3.69% ✅ (+/-1.37/floor 0.45)
P90 [ms] 283.1 263.7 -6.85% ✅ (+/-1.73/floor 0.90)
P99 [ms] 603.0 339.6 -43.68% ⚪ (+/-53.46/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 29.9 28.1 -6.02%
Finish P90 [ms] 59.8 52.3 -12.54%
Finish P99 [ms] 394.5 130.5 -66.92%
Pool Fetch P50 [ms] 9.2 8.5 -7.61%
Pool Fetch P90 [ms] 17.3 16.3 -5.78%
Pool Fetch P99 [ms] 26.3 36.9 +40.30%
Stop Reason — Build Budget 374 375 +0.27%
Serialized Block Size P50 [KiB] 2054.0 2001.9 -2.54%
Serialized Block Size P90 [KiB] 2477.1 2458.0 -0.77%
Serialized Block Size P99 [KiB] 2718.9 2711.0 -0.29%
Serialized Block Size / Tx P50 [B/tx] 295.8 294.6 -0.41%
Serialized Block Size / Tx P90 [B/tx] 309.2 309.6 +0.13%
Serialized Block Size / Tx P99 [B/tx] 432.4 399.0 -7.72%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2496.0 2467.1 -1.16% ⚪ (+/-4.55/floor 0.65)
P50 [ms] 201.2 197.8 -1.69% ⚪ (+/-0.52/floor 1.55)
P90 [ms] 288.7 312.9 +8.38% ❌ (+/-4.90/floor 1.55)
P99 [ms] 732.6 587.6 -19.79% ⚪ (+/-20.01/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 404.5 393.1 -2.82%
Blocks Persisted [/s] 1.82 1.80 -1.10%
Persistence Busy [%] 86.0 82.7 -3.84%
Save Batch Mean [ms] 6377 6630 +3.97%
Max Unpersisted [blocks] 34 34 +0.00%
Backpressure Stall [s] 0.50 2.42 +384.00%

Observability

Warn/Error Logs

Run type Total lines
Baseline 27
Feature 28
Counts by message
Message Baseline Feature
context request for block `0x0f10d3957c0f145be469a2aaede89632adbfafabb7a5798fd96bdfc742419c91` with no consensus context 12 12
failed to read dealer log from block extraData header field 8 8
requested buffer capacity is too low, increasing it to floor 4 4
Parallel block execution is experimental 0 4
[failed delivering block `0x290b793310f047b24bfd5fe02b48ec3891cab5682cedf68f3d2a3c343e2e19c9` for verification (0, 380) 0x290b793310f047b24bfd5fe02b48ec3891cab5682cedf68f3d2a3c343e2e19c9, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
executor could not verify the block; abstaining 1 0
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 1 0

@tempo-github-sts

tempo-github-sts Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: d5a5e13 vs d5a5e13
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=d5a5e134c55f36421028d2b3f260bdf9188a9e8d feature=d5a5e134c55f36421028d2b3f260bdf9188a9e8d baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=2 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 10"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 2
  • Baseline blocks: 408
  • Feature blocks: 411

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 14611 14923 +2.14% ⚪ (+/-2.01/floor 0.55)
Gas Throughput [Mgas/s] 1079.5 1093.7 +1.32% ⚪ (+/-2.14/floor 0.50)
Block Time Mean [ms] 477.5 442.6 -7.31% ✅ (+/-2.76/floor 0.40)
Block Time P50 [ms] 444.0 432.0 -2.70% ⚪ (+/-2.36/floor 0.70)
Block Time P90 [ms] 592.0 579.0 -2.20% ⚪ (+/-3.89/floor 0.70)
Block Time P99 [ms] 1072.0 1169.0 +9.05% ⚪ (+/-40.63/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2057.0 2031.8 -1.23% ⚪ (+/-0.89/floor 0.95)
P50 [ms] 246.5 234.3 -4.95% ✅ (+/-1.40/floor 0.45)
P90 [ms] 284.0 261.6 -7.89% ✅ (+/-0.56/floor 0.90)
P99 [ms] 332.4 299.9 -9.78% ✅ (+/-8.05/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 29.2 28.7 -1.71%
Finish P90 [ms] 51.6 47.1 -8.72%
Finish P99 [ms] 91.7 80.4 -12.32%
Pool Fetch P50 [ms] 8.8 8.4 -4.55%
Pool Fetch P90 [ms] 15.5 15.8 +1.94%
Pool Fetch P99 [ms] 29.4 26.1 -11.22%
Stop Reason — Build Budget 406 407 +0.25%
Serialized Block Size P50 [KiB] 2102.2 1961.2 -6.70%
Serialized Block Size P90 [KiB] 2504.3 2373.6 -5.22%
Serialized Block Size P99 [KiB] 2756.5 2710.8 -1.66%
Serialized Block Size / Tx P50 [B/tx] 295.9 295.0 -0.30%
Serialized Block Size / Tx P90 [B/tx] 308.6 308.5 -0.03%
Serialized Block Size / Tx P99 [B/tx] 358.6 338.0 -5.74%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2523.3 2379.0 -5.72% ❌ (+/-0.43/floor 0.65)
P50 [ms] 198.7 196.2 -1.26% ⚪ (+/-1.08/floor 1.55)
P90 [ms] 281.8 308.3 +9.40% ❌ (+/-6.17/floor 1.55)
P99 [ms] 375.3 430.2 +14.63% ❌ (+/-10.47/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 407.5 409.0 +0.37%
Blocks Persisted [/s] 1.89 2.01 +6.35%
Persistence Busy [%] 87.1 86.4 -0.80%
Save Batch Mean [ms] 7028 6784 -3.47%
Max Unpersisted [blocks] 36 37 +2.78%
Backpressure Stall [s] 4.75 2.20 -53.68%

Observability

Warn/Error Logs

Run type Total lines
Baseline 24
Feature 31
Counts by message
Message Baseline Feature
context request for block `0x57874769dbc59f1326797f6baa060c0ee1502b13a28f976f967e9cf801c925c0` with no consensus context 12 12
failed to read dealer log from block extraData header field 8 8
requested buffer capacity is too low, increasing it to floor 4 4
Parallel block execution is experimental 0 4
[failed delivering block `0x2a18e6dab5d2995b2cfb09ae156b0312513117383700ff9a73bf743c9b236d09` for verification (0, 612) 0x2a18e6dab5d2995b2cfb09ae156b0312513117383700ff9a73bf743c9b236d09, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
executor could not verify the block; abstaining 0 1
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 0 1

@tempo-github-sts

tempo-github-sts Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚪ Benchmark complete: No Difference View job

⚪ Bench Comparison: No Difference

Refs: e2aec68 vs e2aec68
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=e2aec682fe24bbed71811b7edf51976f8a3c0529 feature=e2aec682fe24bbed71811b7edf51976f8a3c0529 baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=5 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 6"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 5
  • Baseline blocks: 922
  • Feature blocks: 919

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 13852 13798 -0.39% ⚪ (+/-6.04/floor 0.55)
Gas Throughput [Mgas/s] 1013.7 1010.4 -0.33% ⚪ (+/-5.70/floor 0.50)
Block Time Mean [ms] 503.5 501.1 -0.48% ⚪ (+/-6.42/floor 0.40)
Block Time P50 [ms] 448.0 447.0 -0.22% ⚪ (+/-1.16/floor 0.70)
Block Time P90 [ms] 628.0 623.0 -0.80% ⚪ (+/-3.66/floor 0.70)
Block Time P99 [ms] 2545.0 2481.0 -2.51% ⚪ (+/-48.93/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2043.7 2032.5 -0.55% ⚪ (+/-1.85/floor 0.95)
P50 [ms] 242.7 241.3 -0.58% ⚪ (+/-1.72/floor 0.45)
P90 [ms] 285.2 282.7 -0.88% ⚪ (+/-2.22/floor 0.90)
P99 [ms] 440.2 405.3 -7.93% ⚪ (+/-43.83/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 29.7 30.1 +1.35%
Finish P90 [ms] 59.3 58.3 -1.69%
Finish P99 [ms] 212.6 161.4 -24.08%
Pool Fetch P50 [ms] 8.5 8.8 +3.53%
Pool Fetch P90 [ms] 16.3 16.9 +3.68%
Pool Fetch P99 [ms] 25.3 29.6 +17.00%
Invalid Tx Attempts P99 0.0 5962.0 n/a
Stop Reason — Build Budget 923 917 -0.65%
Serialized Block Size P50 [KiB] 2058.7 2033.0 -1.25%
Serialized Block Size P90 [KiB] 2511.3 2506.2 -0.21%
Serialized Block Size P99 [KiB] 2828.6 2774.1 -1.93%
Serialized Block Size / Tx P50 [B/tx] 295.2 294.9 -0.10%
Serialized Block Size / Tx P90 [B/tx] 307.5 308.6 +0.36%
Serialized Block Size / Tx P99 [B/tx] 362.6 373.5 +3.01%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2482.5 2514.2 +1.28% ⚪ (+/-1.66/floor 0.65)
P50 [ms] 199.2 199.3 +0.05% ⚪ (+/-2.09/floor 1.55)
P90 [ms] 296.8 284.8 -4.04% ⚪ (+/-2.80/floor 1.55)
P99 [ms] 697.8 722.0 +3.47% ⚪ (+/-32.88/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 378.9 378.3 -0.16%
Blocks Persisted [/s] 1.78 1.76 -1.12%
Persistence Busy [%] 85.8 85.8 +0.00%
Save Batch Mean [ms] 7057 6996 -0.86%
Max Unpersisted [blocks] 35 34 -2.86%
Backpressure Stall [s] 3.12 3.10 -0.64%

Observability

Warn/Error Logs

Run type Total lines
Baseline 72
Feature 80
Counts by message
Message Baseline Feature
context request for block `0x0f10d3957c0f145be469a2aaede89632adbfafabb7a5798fd96bdfc742419c91` with no consensus context 30 30
failed to read dealer log from block extraData header field 20 20
requested buffer capacity is too low, increasing it to floor 10 10
Parallel block execution is experimental 0 10
BatchSpanProcessor dropped a Span due to queue full. No further log will be emitted for further drops until Shutdown. During Shutdown time, a log will be emitted with exact count of total spans dropped. 4 4
Execution cache still in use, executing without it 2 2
executor could not verify the block; abstaining 2 1
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 2 1
[failed delivering block `0x2896841d8988c2250607a686733757b9ee30bfc7e0a103118535260c9d632d7b` for verification (0, 376) 0x2896841d8988c2250607a686733757b9ee30bfc7e0a103118535260c9d632d7b, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
[failed delivering block `0xc5d3c52f2cf6e28ed93f212610e76f0fd6a5ca9ccb777ff29460ae9424486ec9` for verification (0, 364) 0xc5d3c52f2cf6e28ed93f212610e76f0fd6a5ca9ccb777ff29460ae9424486ec9, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
[failed delivering block `0xf07921cf3daccbda42ffd8874f770ca38f5437880ab8d9cc6b4236cfc5c148e0` for verification (0, 370) 0xf07921cf3daccbda42ffd8874f770ca38f5437880ab8d9cc6b4236cfc5c148e0, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
Blocked waiting for execution cache mutex 0 1

@tempo-github-sts

tempo-github-sts Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

❌ Benchmark complete: Regression View job

❌ Bench Comparison: Regression

Refs: 71b1d30 vs 71b1d30
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=71b1d30e782201e6107a13915b63449550a93a1f feature=71b1d30e782201e6107a13915b63449550a93a1f baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=5 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 4"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 5
  • Baseline blocks: 1030
  • Feature blocks: 1033

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 14979 15002 +0.15% ⚪ (+/-3.25/floor 0.55)
Gas Throughput [Mgas/s] 1103.3 1100.8 -0.23% ⚪ (+/-3.47/floor 0.50)
Block Time Mean [ms] 459.1 456.7 -0.52% ⚪ (+/-2.84/floor 0.40)
Block Time P50 [ms] 442.0 437.0 -1.13% ⚪ (+/-1.15/floor 0.70)
Block Time P90 [ms] 578.0 594.0 +2.77% ❌ (+/-1.21/floor 0.70)
Block Time P99 [ms] 1028.0 1124.0 +9.34% ⚪ (+/-16.88/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2009.2 2014.5 +0.26% ⚪ (+/-1.16/floor 0.95)
P50 [ms] 247.2 245.0 -0.89% ⚪ (+/-1.42/floor 0.45)
P90 [ms] 283.3 282.0 -0.46% ⚪ (+/-1.97/floor 0.90)
P99 [ms] 337.1 330.1 -2.08% ⚪ (+/-6.36/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 28.6 29.0 +1.40%
Finish P90 [ms] 51.4 51.1 -0.58%
Finish P99 [ms] 98.9 98.5 -0.40%
Pool Fetch P50 [ms] 8.7 8.7 +0.00%
Pool Fetch P90 [ms] 16.3 15.5 -4.91%
Pool Fetch P99 [ms] 32.3 38.7 +19.81%
Stop Reason — Build Budget 1025 1027 +0.20%
Serialized Block Size P50 [KiB] 2041.4 2048.2 +0.33%
Serialized Block Size P90 [KiB] 2503.3 2466.6 -1.46%
Serialized Block Size P99 [KiB] 2746.1 2790.4 +1.61%
Serialized Block Size / Tx P50 [B/tx] 295.8 295.2 -0.20%
Serialized Block Size / Tx P90 [B/tx] 308.2 307.2 -0.32%
Serialized Block Size / Tx P99 [B/tx] 357.9 427.9 +19.56%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2520.6 2526.5 +0.23% ⚪ (+/-1.85/floor 0.65)
P50 [ms] 195.6 196.2 +0.31% ⚪ (+/-1.12/floor 1.55)
P90 [ms] 273.5 291.8 +6.69% ❌ (+/-4.10/floor 1.55)
P99 [ms] 373.7 456.4 +22.13% ❌ (+/-7.10/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 420.4 417.9 -0.59%
Blocks Persisted [/s] 1.96 1.98 +1.02%
Persistence Busy [%] 87.3 88.4 +1.26%
Save Batch Mean [ms] 7000 6900 -1.43%
Max Unpersisted [blocks] 36 36 +0.00%
Backpressure Stall [s] 2.79 2.20 -21.15%

Observability

Warn/Error Logs

Run type Total lines
Baseline 60
Feature 74
Counts by message
Message Baseline Feature
context request for block `0x57874769dbc59f1326797f6baa060c0ee1502b13a28f976f967e9cf801c925c0` with no consensus context 30 30
failed to read dealer log from block extraData header field 20 20
requested buffer capacity is too low, increasing it to floor 10 10
Parallel block execution is experimental 0 10
[failed delivering block `0xcc039dc07528d6620e163c6b9d3c812f5e784ed1c9e9ac9b1897df825453be52` for verification (0, 416) 0xcc039dc07528d6620e163c6b9d3c812f5e784ed1c9e9ac9b1897df825453be52, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
Blocked waiting for execution cache mutex 0 1
executor could not verify the block; abstaining 0 1
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 0 1

@tempo-github-sts

tempo-github-sts Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚪ Benchmark complete: No Difference View job

⚪ Bench Comparison: No Difference

Refs: 71b1d30 vs 71b1d30
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=71b1d30e782201e6107a13915b63449550a93a1f feature=71b1d30e782201e6107a13915b63449550a93a1f baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=5 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 10"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 5
  • Baseline blocks: 954
  • Feature blocks: 938

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 14394 14388 -0.04% ⚪ (+/-3.68/floor 0.55)
Gas Throughput [Mgas/s] 1056.5 1055.1 -0.13% ⚪ (+/-3.47/floor 0.50)
Block Time Mean [ms] 475.0 480.5 +1.16% ⚪ (+/-4.21/floor 0.40)
Block Time P50 [ms] 444.0 452.0 +1.80% ⚪ (+/-1.71/floor 0.70)
Block Time P90 [ms] 583.0 600.0 +2.92% ⚪ (+/-4.97/floor 0.70)
Block Time P99 [ms] 1742.0 2023.0 +16.13% ⚪ (+/-46.99/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2032.8 2026.5 -0.31% ⚪ (+/-1.35/floor 0.95)
P50 [ms] 240.3 242.2 +0.79% ⚪ (+/-1.12/floor 0.45)
P90 [ms] 277.4 280.7 +1.19% ⚪ (+/-2.28/floor 0.90)
P99 [ms] 367.6 585.9 +59.39% ⚪ (+/-66.34/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 29.9 29.8 -0.33%
Finish P90 [ms] 54.9 57.2 +4.19%
Finish P99 [ms] 139.5 363.6 +160.65%
Pool Fetch P50 [ms] 9.0 8.6 -4.44%
Pool Fetch P90 [ms] 16.5 17.1 +3.64%
Pool Fetch P99 [ms] 29.4 31.8 +8.16%
Stop Reason — Build Budget 952 937 -1.58%
Serialized Block Size P50 [KiB] 1974.2 2018.5 +2.24%
Serialized Block Size P90 [KiB] 2468.0 2477.6 +0.39%
Serialized Block Size P99 [KiB] 2763.2 2783.9 +0.75%
Serialized Block Size / Tx P50 [B/tx] 295.1 295.4 +0.10%
Serialized Block Size / Tx P90 [B/tx] 309.1 307.6 -0.49%
Serialized Block Size / Tx P99 [B/tx] 337.4 351.8 +4.27%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2486.6 2514.1 +1.11% ⚪ (+/-0.81/floor 0.65)
P50 [ms] 195.0 196.0 +0.51% ⚪ (+/-1.84/floor 1.55)
P90 [ms] 274.3 286.5 +4.45% ⚪ (+/-8.17/floor 1.55)
P99 [ms] 669.2 535.3 -20.01% ⚪ (+/-40.24/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 412.7 410.9 -0.44%
Blocks Persisted [/s] 1.86 1.87 +0.54%
Persistence Busy [%] 85.0 85.5 +0.59%
Save Batch Mean [ms] 6303 6420 +1.86%
Max Unpersisted [blocks] 34 34 +0.00%
Backpressure Stall [s] 1.08 1.28 +18.52%

Observability

Warn/Error Logs

Run type Total lines
Baseline 70
Feature 73
Counts by message
Message Baseline Feature
context request for block `0x0f10d3957c0f145be469a2aaede89632adbfafabb7a5798fd96bdfc742419c91` with no consensus context 30 30
failed to read dealer log from block extraData header field 20 20
requested buffer capacity is too low, increasing it to floor 10 10
Parallel block execution is experimental 0 10
Execution cache still in use, executing without it 3 0
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 2 1
[failed delivering build parent, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 2 0
[failed delivering finalized block `0xceb2ea3188e21c9f30988f6788a9d9bad77d37359fd692bef679d534c6807f47` at height `378`, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 2 0
[executor dropped the payload channel: the build failed (the executor logs the cause) or the executor shut down, oneshot canceled] 1 0
[failed delivering block `0xa075b56c52310ce504feb4911d384786018385f27c1a5fdaf4ec59a28d400bb3` for verification (0, 559) 0xa075b56c52310ce504feb4911d384786018385f27c1a5fdaf4ec59a28d400bb3, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
executor could not verify the block; abstaining 0 1

@tempo-github-sts

tempo-github-sts Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚪ Benchmark complete: No Difference View job

⚪ Bench Comparison: No Difference

Refs: 71b1d30 vs 71b1d30
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=71b1d30e782201e6107a13915b63449550a93a1f feature=71b1d30e782201e6107a13915b63449550a93a1f baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=5 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 8"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 5
  • Baseline blocks: 939
  • Feature blocks: 924

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 14490 14367 -0.85% ⚪ (+/-2.96/floor 0.55)
Gas Throughput [Mgas/s] 1058.0 1053.1 -0.46% ⚪ (+/-3.05/floor 0.50)
Block Time Mean [ms] 482.9 493.1 +2.11% ⚪ (+/-4.34/floor 0.40)
Block Time P50 [ms] 446.0 445.0 -0.22% ⚪ (+/-1.70/floor 0.70)
Block Time P90 [ms] 621.0 637.0 +2.58% ⚪ (+/-4.40/floor 0.70)
Block Time P99 [ms] 2033.0 2662.0 +30.94% ⚪ (+/-35.86/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2020.3 2048.3 +1.39% ⚪ (+/-1.49/floor 0.95)
P50 [ms] 244.7 246.9 +0.90% ⚪ (+/-2.01/floor 0.45)
P90 [ms] 289.2 287.4 -0.62% ⚪ (+/-2.14/floor 0.90)
P99 [ms] 354.5 394.9 +11.40% ⚪ (+/-26.75/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 29.4 29.8 +1.36%
Finish P90 [ms] 57.4 57.4 +0.00%
Finish P99 [ms] 119.0 193.5 +62.61%
Pool Fetch P50 [ms] 8.6 8.2 -4.65%
Pool Fetch P90 [ms] 16.8 16.4 -2.38%
Pool Fetch P99 [ms] 38.4 28.2 -26.56%
Stop Reason — Build Budget 939 927 -1.28%
Serialized Block Size P50 [KiB] 2066.1 2089.3 +1.12%
Serialized Block Size P90 [KiB] 2570.8 2606.5 +1.39%
Serialized Block Size P99 [KiB] 2831.5 2896.9 +2.31%
Serialized Block Size / Tx P50 [B/tx] 295.0 294.9 -0.03%
Serialized Block Size / Tx P90 [B/tx] 307.5 308.1 +0.20%
Serialized Block Size / Tx P99 [B/tx] 393.2 382.7 -2.67%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2511.6 2531.8 +0.80% ⚪ (+/-1.03/floor 0.65)
P50 [ms] 200.0 200.4 +0.20% ⚪ (+/-2.75/floor 1.55)
P90 [ms] 284.0 294.1 +3.56% ⚪ (+/-4.65/floor 1.55)
P99 [ms] 601.7 655.8 +8.99% ⚪ (+/-50.61/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 399.6 395.5 -1.03%
Blocks Persisted [/s] 1.81 1.74 -3.87%
Persistence Busy [%] 84.1 82.7 -1.66%
Save Batch Mean [ms] 6814 6740 -1.09%
Max Unpersisted [blocks] 35 34 -2.86%
Backpressure Stall [s] 1.59 1.21 -23.90%

Observability

Warn/Error Logs

Run type Total lines
Baseline 62
Feature 74
Counts by message
Message Baseline Feature
context request for block `0x0f10d3957c0f145be469a2aaede89632adbfafabb7a5798fd96bdfc742419c91` with no consensus context 30 30
failed to read dealer log from block extraData header field 20 20
requested buffer capacity is too low, increasing it to floor 10 10
Parallel block execution is experimental 0 10
Execution cache still in use, executing without it 1 1
[failed delivering block `0x317b7f7cb386d5084da67177f0f79f80934ad262b9e6f4ded7a1f7ea6fd3ef2d` for verification (0, 386) 0x317b7f7cb386d5084da67177f0f79f80934ad262b9e6f4ded7a1f7ea6fd3ef2d, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
dropped our proposal 1 0
executor could not verify the block; abstaining 0 1
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 0 1

…cution

Ports the seeded differential fuzz test and runs some seeds over an expiring-nonce ring about to wrap, with expired entries to evict or an unexpired one that fills it, so predicted ring slots, eviction and the full-ring fallback are compared against sequential execution.
…lock

With a batched parallel commit, each expiring nonce transaction checks the
ring slot sequential execution inserts it at, from its index among the
block's expiring nonce transactions and the pointer before the block, and
marks its hash seen during execution. The committer only checks the index,
and the ring entries, evictions and pointer are written once after the
block's transactions commit, instead of reading the ring and the evicted
entry for every transaction on the committer.
…ion commits

Batched parallel commits write each insert's ring slot and evicted entry as
its transaction commits, so they stream with the block's prefix, and only
advance the pointer once the block's transactions committed.
Parallel workers clone each speculative result's logs into a receipt copy,
and the committer only fills in its cumulative gas and type before handing
it to on_receipt, so the receipt root stream no longer clones receipts on
the committing thread. The tip20_parallel bench models the engine's receipt
root task with TEMPO_PAR_RECEIPTS=1.
@tempo-github-sts

tempo-github-sts Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

❌ Benchmark complete: Regression View job

❌ Bench Comparison: Regression

Refs: 6eb88aa vs 6eb88aa
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=6eb88aa25ccaa59b7b391cdad4e8250da798d3b4 feature=6eb88aa25ccaa59b7b391cdad4e8250da798d3b4 baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=5 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 6"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 5
  • Baseline blocks: 977
  • Feature blocks: 962

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 14919 14359 -3.75% ⚪ (+/-4.04/floor 0.55)
Gas Throughput [Mgas/s] 1096.1 1048.7 -4.32% ⚪ (+/-4.13/floor 0.50)
Block Time Mean [ms] 464.6 486.7 +4.76% ⚪ (+/-6.30/floor 0.40)
Block Time P50 [ms] 444.0 445.0 +0.23% ⚪ (+/-2.03/floor 0.70)
Block Time P90 [ms] 592.0 618.0 +4.39% ❌ (+/-2.97/floor 0.70)
Block Time P99 [ms] 1089.0 2973.0 +173.00% ❌ (+/-130.14/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2015.0 2023.6 +0.43% ⚪ (+/-1.73/floor 0.95)
P50 [ms] 247.9 249.0 +0.44% ⚪ (+/-1.04/floor 0.45)
P90 [ms] 288.1 289.5 +0.49% ⚪ (+/-1.76/floor 0.90)
P99 [ms] 367.0 346.5 -5.59% ⚪ (+/-27.13/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 28.7 28.4 -1.05%
Finish P90 [ms] 53.6 54.5 +1.68%
Finish P99 [ms] 108.4 120.4 +11.07%
Pool Fetch P50 [ms] 8.9 8.1 -8.99%
Pool Fetch P90 [ms] 16.9 17.1 +1.18%
Pool Fetch P99 [ms] 36.5 35.8 -1.92%
Stop Reason — Build Budget 971 954 -1.75%
Serialized Block Size P50 [KiB] 2075.5 2086.0 +0.51%
Serialized Block Size P90 [KiB] 2516.0 2545.9 +1.19%
Serialized Block Size P99 [KiB] 2809.3 2742.8 -2.36%
Serialized Block Size / Tx P50 [B/tx] 295.8 295.0 -0.27%
Serialized Block Size / Tx P90 [B/tx] 308.2 308.0 -0.06%
Serialized Block Size / Tx P99 [B/tx] 416.8 347.6 -16.60%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2524.4 2558.2 +1.34% ⚪ (+/-1.79/floor 0.65)
P50 [ms] 197.9 195.0 -1.47% ⚪ (+/-1.77/floor 1.55)
P90 [ms] 275.1 291.7 +6.03% ❌ (+/-4.42/floor 1.55)
P99 [ms] 406.1 451.6 +11.20% ⚪ (+/-28.10/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 407.3 397.6 -2.38%
Blocks Persisted [/s] 1.88 1.85 -1.60%
Persistence Busy [%] 85.6 86.0 +0.47%
Save Batch Mean [ms] 6884 7096 +3.08%
Max Unpersisted [blocks] 36 34 -5.56%
Backpressure Stall [s] 2.85 3.76 +31.93%

Observability

Warn/Error Logs

Run type Total lines
Baseline 65
Feature 75
Counts by message
Message Baseline Feature
context request for block `0x57874769dbc59f1326797f6baa060c0ee1502b13a28f976f967e9cf801c925c0` with no consensus context 30 30
failed to read dealer log from block extraData header field 20 20
requested buffer capacity is too low, increasing it to floor 10 10
Parallel block execution is experimental 0 10
BatchSpanProcessor dropped a Span due to queue full. No further log will be emitted for further drops until Shutdown. During Shutdown time, a log will be emitted with exact count of total spans dropped. 2 0
executor could not verify the block; abstaining 1 1
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 1 1
[failed delivering block `0xaa150da00eb0dcd899318d7c70f01ac0ef26a1bd5c83cdd822f835c7b4dfccd0` for verification (0, 390) 0xaa150da00eb0dcd899318d7c70f01ac0ef26a1bd5c83cdd822f835c7b4dfccd0, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
[failed delivering block `0xc6bc6cd6c2e8e3778a4d853e68e9488a9c1103c84d9d308431799598cbe25b27` for verification (0, 388) 0xc6bc6cd6c2e8e3778a4d853e68e9488a9c1103c84d9d308431799598cbe25b27, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
Blocked waiting for execution cache mutex 0 1
Execution cache still in use, executing without it 0 1

@tempo-github-sts

tempo-github-sts Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: 58e30da vs 58e30da
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=58e30da5d2ce19fbf7fecbf0ebe2eb387cd8b948 feature=58e30da5d2ce19fbf7fecbf0ebe2eb387cd8b948 baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=5 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 6"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 5
  • Baseline blocks: 948
  • Feature blocks: 949

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 13845 13438 -2.94% ⚪ (+/-4.63/floor 0.55)
Gas Throughput [Mgas/s] 1015.8 985.4 -2.99% ⚪ (+/-4.84/floor 0.50)
Block Time Mean [ms] 493.8 487.8 -1.22% ⚪ (+/-5.58/floor 0.40)
Block Time P50 [ms] 443.0 430.0 -2.93% ✅ (+/-2.14/floor 0.70)
Block Time P90 [ms] 587.0 597.0 +1.70% ⚪ (+/-2.59/floor 0.70)
Block Time P99 [ms] 3240.0 2446.0 -24.51% ⚪ (+/-46.68/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2024.9 1941.1 -4.14% ❌ (+/-1.82/floor 0.95)
P50 [ms] 243.1 240.9 -0.91% ⚪ (+/-2.39/floor 0.45)
P90 [ms] 278.2 294.3 +5.79% ❌ (+/-2.43/floor 0.90)
P99 [ms] 453.0 458.5 +1.21% ⚪ (+/-24.16/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 29.8 39.0 +30.87%
Finish P90 [ms] 56.4 89.8 +59.22%
Finish P99 [ms] 157.6 243.1 +54.25%
Pool Fetch P50 [ms] 8.6 8.6 +0.00%
Pool Fetch P90 [ms] 16.9 16.7 -1.18%
Pool Fetch P99 [ms] 28.8 36.9 +28.12%
Stop Reason — Build Budget 949 945 -0.42%
Serialized Block Size P50 [KiB] 2046.7 1955.9 -4.44%
Serialized Block Size P90 [KiB] 2441.5 2360.6 -3.31%
Serialized Block Size P99 [KiB] 2696.5 2590.5 -3.93%
Serialized Block Size / Tx P50 [B/tx] 295.1 295.2 +0.03%
Serialized Block Size / Tx P90 [B/tx] 309.3 308.2 -0.36%
Serialized Block Size / Tx P99 [B/tx] 380.2 368.1 -3.18%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2516.4 2620.5 +4.14% ✅ (+/-2.23/floor 0.65)
P50 [ms] 197.0 179.9 -8.68% ✅ (+/-1.71/floor 1.55)
P90 [ms] 276.6 272.5 -1.48% ⚪ (+/-4.43/floor 1.55)
P99 [ms] 506.3 609.6 +20.40% ⚪ (+/-50.34/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 389.2 379.9 -2.39%
Blocks Persisted [/s] 1.80 1.82 +1.11%
Persistence Busy [%] 85.8 85.7 -0.12%
Save Batch Mean [ms] 6949 6753 -2.82%
Max Unpersisted [blocks] 37 36 -2.70%
Backpressure Stall [s] 3.03 3.52 +16.17%

Observability

Warn/Error Logs

Run type Total lines
Baseline 64
Feature 77
Counts by message
Message Baseline Feature
context request for block `0x0f10d3957c0f145be469a2aaede89632adbfafabb7a5798fd96bdfc742419c91` with no consensus context 30 30
failed to read dealer log from block extraData header field 20 20
requested buffer capacity is too low, increasing it to floor 10 10
Parallel block execution is experimental 0 10
executor could not verify the block; abstaining 1 2
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 1 2
[failed delivering block `0xb5cd1d944ecb3abeea81ad36d24422ae2b79e44474f4ec9fffab9664964a1d07` for verification (0, 530) 0xb5cd1d944ecb3abeea81ad36d24422ae2b79e44474f4ec9fffab9664964a1d07, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
[failed delivering block `0xc0e485b0a45b3439e15a60eb2f3ac2d39fc7cfc334f89eb4b6f9415b4ee97f05` for verification (0, 397) 0xc0e485b0a45b3439e15a60eb2f3ac2d39fc7cfc334f89eb4b6f9415b4ee97f05, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 0 1
[failed delivering block `0xf8d053d355197741bc87356c2729836486ff11dcd925a68ba307651f5546abe4` for verification (0, 376) 0xf8d053d355197741bc87356c2729836486ff11dcd925a68ba307651f5546abe4, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
Blocked waiting for execution cache mutex 0 1
Execution cache still in use, executing without it 1 0

@tempo-github-sts

tempo-github-sts Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: 3fdce65 vs 3fdce65
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=3fdce658f6f6d808e3197a1828b453ec8ce2c52a feature=3fdce658f6f6d808e3197a1828b453ec8ce2c52a baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=5 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 6"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 5
  • Baseline blocks: 920
  • Feature blocks: 947

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 14243 13432 -5.69% ❌ (+/-4.58/floor 0.55)
Gas Throughput [Mgas/s] 1040.5 985.6 -5.28% ❌ (+/-4.69/floor 0.50)
Block Time Mean [ms] 495.1 482.8 -2.48% ⚪ (+/-4.93/floor 0.40)
Block Time P50 [ms] 452.0 420.0 -7.08% ✅ (+/-1.64/floor 0.70)
Block Time P90 [ms] 610.0 613.0 +0.49% ⚪ (+/-5.30/floor 0.70)
Block Time P99 [ms] 1914.0 2546.0 +33.02% ❌ (+/-30.21/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2046.5 1941.1 -5.15% ❌ (+/-1.02/floor 0.95)
P50 [ms] 245.2 238.6 -2.69% ⚪ (+/-2.25/floor 0.45)
P90 [ms] 283.7 289.6 +2.08% ⚪ (+/-3.42/floor 0.90)
P99 [ms] 339.4 439.5 +29.49% ⚪ (+/-76.44/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 28.8 39.3 +36.46%
Finish P90 [ms] 52.7 88.9 +68.69%
Finish P99 [ms] 107.9 237.2 +119.83%
Pool Fetch P50 [ms] 8.9 8.4 -5.62%
Pool Fetch P90 [ms] 16.1 16.5 +2.48%
Pool Fetch P99 [ms] 27.5 29.3 +6.55%
Stop Reason — Build Budget 917 952 +3.82%
Serialized Block Size P50 [KiB] 2094.3 1899.5 -9.31%
Serialized Block Size P90 [KiB] 2520.5 2337.1 -7.28%
Serialized Block Size P99 [KiB] 2783.6 2629.7 -5.53%
Serialized Block Size / Tx P50 [B/tx] 295.2 295.6 +0.14%
Serialized Block Size / Tx P90 [B/tx] 306.9 308.2 +0.42%
Serialized Block Size / Tx P99 [B/tx] 352.0 395.4 +12.33%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2527.3 2685.3 +6.25% ✅ (+/-3.61/floor 0.65)
P50 [ms] 196.4 174.1 -11.35% ✅ (+/-3.84/floor 1.55)
P90 [ms] 281.3 263.6 -6.29% ✅ (+/-3.97/floor 1.55)
P99 [ms] 503.6 757.9 +50.50% ⚪ (+/-48.60/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 394.6 385.7 -2.26%
Blocks Persisted [/s] 1.80 1.83 +1.67%
Persistence Busy [%] 84.7 82.8 -2.24%
Save Batch Mean [ms] 6921 6185 -10.63%
Max Unpersisted [blocks] 34 34 +0.00%
Backpressure Stall [s] 2.97 2.73 -8.08%

Observability

Warn/Error Logs

Run type Total lines
Baseline 64
Feature 73
Counts by message
Message Baseline Feature
context request for block `0x0f10d3957c0f145be469a2aaede89632adbfafabb7a5798fd96bdfc742419c91` with no consensus context 30 30
failed to read dealer log from block extraData header field 20 20
requested buffer capacity is too low, increasing it to floor 10 10
Parallel block execution is experimental 0 10
BatchSpanProcessor dropped a Span due to queue full. No further log will be emitted for further drops until Shutdown. During Shutdown time, a log will be emitted with exact count of total spans dropped. 1 2
[failed delivering block `0x7af48c7b74eadc00f8cdf8bcb595123ebcaacb8451bf4c5822a5a54e15dfc686` for verification (0, 369) 0x7af48c7b74eadc00f8cdf8bcb595123ebcaacb8451bf4c5822a5a54e15dfc686, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
Execution cache still in use, executing without it 0 1
executor could not verify the block; abstaining 1 0
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 1 0

@tempo-github-sts

tempo-github-sts Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

✅ Benchmark complete: Improvement View job

✅ Bench Comparison: Improvement

Refs: 129282a vs 129282a
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=129282a848e07eaa69086571c140f15a7cd18b43 feature=129282a848e07eaa69086571c140f15a7cd18b43 baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=5 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 baseline-args="--execution.parallel-threads 6" feature-args="--execution.parallel-threads 8"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 5
  • Baseline blocks: 1023
  • Feature blocks: 1029

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 14647 15191 +3.71% ✅ (+/-2.69/floor 0.55)
Gas Throughput [Mgas/s] 1073.6 1112.6 +3.63% ✅ (+/-2.73/floor 0.50)
Block Time Mean [ms] 452.3 438.7 -3.01% ✅ (+/-1.61/floor 0.40)
Block Time P50 [ms] 404.0 409.0 +1.24% ⚪ (+/-2.08/floor 0.70)
Block Time P90 [ms] 602.0 585.0 -2.82% ⚪ (+/-4.93/floor 0.70)
Block Time P99 [ms] 1494.0 1104.0 -26.10% ⚪ (+/-52.62/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 1956.3 1946.6 -0.50% ⚪ (+/-0.93/floor 0.95)
P50 [ms] 241.5 246.1 +1.90% ⚪ (+/-3.38/floor 0.45)
P90 [ms] 295.0 298.1 +1.05% ⚪ (+/-3.28/floor 0.90)
P99 [ms] 408.1 404.7 -0.83% ⚪ (+/-19.51/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 36.6 36.1 -1.37%
Finish P90 [ms] 85.2 84.3 -1.06%
Finish P99 [ms] 186.8 188.9 +1.12%
Pool Fetch P50 [ms] 8.4 8.1 -3.57%
Pool Fetch P90 [ms] 16.8 16.6 -1.19%
Pool Fetch P99 [ms] 31.5 31.1 -1.27%
Invalid Tx Attempts P99 3560.0 0.0 -100.00%
Stop Reason — Build Budget 1016 1021 +0.49%
Serialized Block Size P50 [KiB] 1949.9 1950.8 +0.04%
Serialized Block Size P90 [KiB] 2428.2 2452.3 +0.99%
Serialized Block Size P99 [KiB] 2712.7 2689.8 -0.84%
Serialized Block Size / Tx P50 [B/tx] 295.2 295.1 -0.03%
Serialized Block Size / Tx P90 [B/tx] 308.7 308.7 +0.00%
Serialized Block Size / Tx P99 [B/tx] 347.3 370.1 +6.56%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2771.2 2786.8 +0.56% ⚪ (+/-2.26/floor 0.65)
P50 [ms] 170.9 169.4 -0.88% ⚪ (+/-1.32/floor 1.55)
P90 [ms] 277.8 263.3 -5.22% ⚪ (+/-7.52/floor 1.55)
P99 [ms] 481.0 435.6 -9.44% ⚪ (+/-17.88/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 416.1 417.4 +0.31%
Blocks Persisted [/s] 2.01 2.00 -0.50%
Persistence Busy [%] 87.3 85.7 -1.83%
Save Batch Mean [ms] 6421 6471 +0.78%
Max Unpersisted [blocks] 35 36 +2.86%
Backpressure Stall [s] 2.46 1.54 -37.40%

Observability

Warn/Error Logs

Run type Total lines
Baseline 73
Feature 73
Counts by message
Message Baseline Feature
context request for block `0x57874769dbc59f1326797f6baa060c0ee1502b13a28f976f967e9cf801c925c0` with no consensus context 30 30
failed to read dealer log from block extraData header field 20 20
Parallel block execution is experimental 10 10
requested buffer capacity is too low, increasing it to floor 10 10
BatchSpanProcessor dropped a Span due to queue full. No further log will be emitted for further drops until Shutdown. During Shutdown time, a log will be emitted with exact count of total spans dropped. 0 2
[failed delivering block `0xfe578521e310904167301881ca9d96e710c4e90b551455de4058e8bc0c5e2884` for verification (0, 391) 0xfe578521e310904167301881ca9d96e710c4e90b551455de4058e8bc0c5e2884, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
Execution cache still in use, executing without it 0 1
executor could not verify the block; abstaining 1 0
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 1 0

@tempo-github-sts

tempo-github-sts Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

cc @gakonst

⚠️ Benchmark complete: Mixed Results View job

⚠️ Bench Comparison: Mixed Results

Refs: 2ab1b3d vs 2ab1b3d
Criteria: 95% run-bootstrap CI must clear floor; cells show delta (+/-CI/floor).

Configuration

  • Derek command: derek bench mode=e2e preset=default duration=90 bloat=100 token-count=4 tps=50000 accounts=1000 max-concurrent-requests=100 baseline=2ab1b3d0eabedc6a28faf04e8213b490970f1b72 feature=2ab1b3d0eabedc6a28faf04e8213b490970f1b72 baseline-hardfork="" feature-hardfork="" gas-limit=1000000000000 run-pairs=5 run-side=comparison otlp=true metrics=false no-cache=false force-bloat=false general-gas-limit=1500000000 feature-args="--execution.parallel-threads 6"
  • Bloat: 100000 MiB
  • Token count: 4
  • Preset: default
  • Target TPS: 50000
  • Duration: 90s
  • Run pairs: 5
  • Baseline blocks: 962
  • Feature blocks: 965

Tempo Metrics

Metric Baseline Feature Delta
TPS Mean 13956 13443 -3.68% ⚪ (+/-4.08/floor 0.55)
Gas Throughput [Mgas/s] 1022.8 986.9 -3.51% ⚪ (+/-4.39/floor 0.50)
Block Time Mean [ms] 485.7 482.2 -0.72% ⚪ (+/-3.89/floor 0.40)
Block Time P50 [ms] 445.0 418.0 -6.07% ✅ (+/-1.71/floor 0.70)
Block Time P90 [ms] 576.0 597.0 +3.65% ❌ (+/-2.15/floor 0.70)
Block Time P99 [ms] 1800.0 2525.0 +40.28% ⚪ (+/-47.44/floor 1.60)

Builder

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 1983.8 1905.5 -3.95% ❌ (+/-1.79/floor 0.95)
P50 [ms] 246.9 243.6 -1.34% ⚪ (+/-2.02/floor 0.45)
P90 [ms] 281.7 294.3 +4.47% ❌ (+/-1.70/floor 0.90)
P99 [ms] 326.8 430.7 +31.79% ❌ (+/-20.00/floor 1.25)
Builder details
Metric Baseline Feature Delta
Finish P50 [ms] 30.6 37.3 +21.90%
Finish P90 [ms] 60.4 92.4 +52.98%
Finish P99 [ms] 108.9 209.5 +92.38%
Pool Fetch P50 [ms] 8.7 8.2 -5.75%
Pool Fetch P90 [ms] 16.1 16.1 +0.00%
Pool Fetch P99 [ms] 35.5 38.1 +7.32%
Invalid Tx Attempts P99 0.0 1733.0 n/a
Stop Reason — Build Budget 953 961 +0.84%
Serialized Block Size P50 [KiB] 2013.3 1917.0 -4.78%
Serialized Block Size P90 [KiB] 2413.6 2329.8 -3.47%
Serialized Block Size P99 [KiB] 2638.0 2626.0 -0.46%
Serialized Block Size / Tx P50 [B/tx] 295.4 295.2 -0.07%
Serialized Block Size / Tx P90 [B/tx] 306.5 307.4 +0.29%
Serialized Block Size / Tx P99 [B/tx] 338.5 350.1 +3.43%
Fill Idle P50 [ms] 0.0 0.0 0.00%
Fill Idle P90 [ms] 0.0 0.0 0.00%
Fill Idle P99 [ms] 0.0 0.0 0.00%

Validator

Metric Baseline Feature Delta
Gas Throughput [Mgas/s] 2490.7 2705.3 +8.62% ✅ (+/-2.68/floor 0.65)
P50 [ms] 192.3 172.7 -10.19% ✅ (+/-2.35/floor 1.55)
P90 [ms] 276.1 265.9 -3.69% ⚪ (+/-5.74/floor 1.55)
P99 [ms] 381.5 476.5 +24.90% ⚪ (+/-40.78/floor 2.05)

Persistence

Metric Baseline Feature Delta
Durable Writes [MB/s] 387.4 382.0 -1.39%
Blocks Persisted [/s] 1.89 1.88 -0.53%
Persistence Busy [%] 88.4 86.2 -2.49%
Save Batch Mean [ms] 7315 6757 -7.63%
Max Unpersisted [blocks] 35 34 -2.86%
Backpressure Stall [s] 5.94 4.98 -16.16%

Observability

Warn/Error Logs

Run type Total lines
Baseline 67
Feature 71
Counts by message
Message Baseline Feature
context request for block `0x0f10d3957c0f145be469a2aaede89632adbfafabb7a5798fd96bdfc742419c91` with no consensus context 30 30
failed to read dealer log from block extraData header field 20 20
requested buffer capacity is too low, increasing it to floor 10 10
Parallel block execution is experimental 0 10
executor could not verify the block; abstaining 2 0
executor encountered fatal execution-layer update error; shutting down to prevent consensus-execution divergence 2 0
[failed delivering block `0x3b066d4cca432741196400c633e1ff68199b37c3495ff9c5907235260f1731bc` for verification (0, 401) 0x3b066d4cca432741196400c633e1ff68199b37c3495ff9c5907235260f1731bc, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
[failed delivering block `0x7c72113599fa14def6a4c52def2444308aabd1ed0b8d8684dde4008760b785f3` for verification (0, 380) 0x7c72113599fa14def6a4c52def2444308aabd1ed0b8d8684dde4008760b785f3, failed sending new-payload request to execution layer, beacon consensus engine task stopped] 1 0
Blocked waiting for execution cache mutex 0 1
dropped our proposal 1 0

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant