Skip to content

Run 1771812258

Trial run

Warning — trial run. This benchmark was executed with fewer than 10 instances per fuzzer and/or a time budget shorter than 24h. Results from trial runs are meant for debugging purposes and are not valid for extracting conclusions across different fuzzers.

Charts

Bugs Over TimeTime To KFinal DistributionPlateau And Late ShareInvariant Overlap (UpSet)CPU Usage Over TimeMemory Usage Over Time

Report

Fuzzer Benchmark Report (from bug-count CSV)

  • Time budget: 6.00h

Executive summary

This report is derived solely from cumulative bugs-found over time across repeated runs per fuzzer. It emphasizes robust, distribution-based metrics (median/IQR, success rates, time-to-k) and shape-based behavior (plateau time, late discovery share) instead of single-run time-to-first-bug.

Bugs found at fixed time budgets (median [IQR])

FuzzerRuns1h4h
foundry46 [6,6]6 [6,6]
medusa46 [6,6]6 [6,7]
echidna44 [4,4]6 [5,6]

Overall metrics

FuzzerAUC (norm)Plateau timeLate discovery shareFinal medianFinal IQR
foundry0.6650.10h0.00060.00
medusa0.6460.50h0.00060.75
echidna0.5484.40h0.30861.75

Milestones: time-to-k and success rates

Fuzzertime-to-1 (p50)time-to-3 (p50)time-to-5 (p50)reach-1 ratereach-3 ratereach-5 rate
foundry0.00h0.00h0.00h100.0%100.0%100.0%
medusa0.10h0.10h0.35h100.0%100.0%100.0%
echidna0.10h0.10h2.20h100.0%100.0%75.0%

Shape-based interpretation (rules of thumb)

  • Fast-start / early-plateau: high early checkpoint median + early plateau time + low late discovery share.
  • Steady: moderate AUC, later plateau, consistent improvements across checkpoints, moderate variance.
  • Slow-burn / late-surge: low early checkpoints but high late discovery share and later plateau time; often higher final median.

Limitations

  • Core metrics in this section are count-based; use broken_invariants.md / broken_invariants.csv for invariant identities.
  • Severity, exploitability, and root-cause uniqueness cannot be measured directly without richer per-bug metadata.
  • Harness design still affects results; mitigate by keeping harness identical across fuzzers and reporting many runs.

Broken invariants

  • Budget filter: 6.00h
  • Events considered: 76 / 76
  • Unique invariants: 17

Per-fuzzer totals

FuzzerInvariants
echidna9
foundry6
medusa9

High-level overlap

  • Shared by all fuzzers: 1
  • Exclusive to echidna: 3
  • Exclusive to foundry: 5
  • Exclusive to medusa: 3

Grouped invariants

Exclusive to echidna (3)
  • doomsday_fulfillDoesntOverRedeemMultipleActors
  • doomsday_maxRedeemResetsAfterFullRedemption
  • doomsday_maxWithdrawResetsAfterFullWithdrawal
Exclusive to foundry (5)
  • invariant_assertion_failure_CANARY
  • invariant_erc7540_1
  • invariant_erc7540_2
  • invariant_erc7540_3
  • invariant_maxRedeemMaxWithdrawSymmetry
Exclusive to medusa (3)
  • global_comparePreviewMintAndConvertToAssets
  • global_previewEquivalenceFromAssets
  • invariant_accumulatorCostBasisIncrease
Shared by all fuzzers (1)
  • invariant_canary

Top shared subsets (top 1 by size):

echidna, medusa (5)
  • assert_canary
  • doomsday_depositWithdrawSymmetrical
  • doomsday_mintRedeemSymmetrical
  • global_previewEquivalenceFromShares
  • invariant_accumulatorSharesDecreaseOnFulfill_exact

Runner resource usage

  • Budget filter: 6.00h
  • Instances with metrics: 12
  • Total samples: 51431

Per-fuzzer medians (across instances)

FuzzerInstancesCPU active avg (%)CPU active peak (%)Memory used avg (GiB)Memory used peak (GiB)Memory used avg (%)Memory used peak (%)
echidna485.0988.193.774.6112.3115.06
foundry499.39100.002.803.009.149.77
medusa498.5399.982.724.028.8913.11

Instance stats

InstanceFuzzerSamplesDuration (h)CPU active avg (%)CPU active peak (%)Memory avg (GiB)Memory peak (GiB)Memory avg (%)Memory peak (%)
i-00965c9f9795b13b8-echidna-v2.3.1echidna42926.0085.2688.563.744.5612.2114.88
i-07a29d7d39100b4fa-echidna-v2.3.1echidna42946.0084.6487.863.784.6712.3315.24
i-08f5a34bcc30a9549-echidna-v2.3.1echidna42926.0085.0888.343.784.5912.3414.97
i-0dfb6d7bebd36709f-echidna-v2.3.1echidna42926.0085.1088.053.774.6412.2915.15
i-0161ad399d4adf2cb-foundry-git-366c5f9foundry42826.0099.39100.002.893.059.439.96
i-033be0b3c5862600b-foundry-git-366c5f9foundry42826.0099.41100.002.883.139.4010.23
i-0791b01030b001965-foundry-git-366c5f9foundry42826.0099.39100.002.442.907.969.47
i-0a76c07d9185086a0-foundry-git-366c5f9foundry42826.0099.39100.002.722.948.889.58
i-05495c9b67a089458-medusa-v1.4.1medusa42836.0098.55100.002.743.988.9613.00
i-07c5f28e720c9a554-medusa-v1.4.1medusa42836.0098.7399.992.723.948.8712.86
i-0db808cd3fe860ab5-medusa-v1.4.1medusa42846.0097.6199.912.734.108.9013.39
i-0ea0912f40247d18e-medusa-v1.4.1medusa42836.0098.5199.972.704.058.8013.22

Manifest

  • scfuzzbench_commit: 58b11959c0f5d01cf74f9869e08ca7baf2bbeafb
  • target_repo_url: https://github.com/Recon-Fuzz/superform-v2-periphery-scfuzzbench
  • target_commit: dev-recon
  • benchmark_type: property
  • instance_type: c6a.4xlarge
  • instances_per_fuzzer: 4
  • timeout_hours: 6
  • aws_region: us-east-1
  • ubuntu_ami_id: ami-0071174ad8cbb9e17
  • foundry_version: v1.6.0-rc1
  • foundry_git_repo: https://github.com/aviggiano/foundry
  • foundry_git_ref: master
  • echidna_version: 2.3.1
  • medusa_version: 1.4.1
  • fuzzer_keys: echidna, foundry, medusa

Artifacts

Fully static. Generated in CI from S3 run artifacts.