Run 1771794816
- Date (UTC):
2026-02-22 21:13:36Z - Benchmark:
911bd544dadb96194ba616e91483567d - Timeout:
1h
Trial run
Warning — trial run. This benchmark was executed with fewer than 10 instances per fuzzer and/or a time budget shorter than 24h. Results from trial runs are meant for debugging purposes and are not valid for extracting conclusions across different fuzzers.
Charts
Report
Fuzzer Benchmark Report (from bug-count CSV)
- Time budget: 1.00h
Executive summary
This report is derived solely from cumulative bugs-found over time across repeated runs per fuzzer. It emphasizes robust, distribution-based metrics (median/IQR, success rates, time-to-k) and shape-based behavior (plateau time, late discovery share) instead of single-run time-to-first-bug.
Bugs found at fixed time budgets (median [IQR])
| Fuzzer | Runs | 1h | 4h | 8h | 24h |
|---|---|---|---|---|---|
| medusa | 2 | 7 [6,8] | 7 [6,8] | 7 [6,8] | 7 [6,8] |
| foundry | 2 | 6 [6,6] | 6 [6,6] | 6 [6,6] | 6 [6,6] |
| echidna | 2 | 4 [4,4] | 4 [4,4] | 4 [4,4] | 4 [4,4] |
Overall metrics
| Fuzzer | AUC (norm) | Plateau time | Late discovery share | Final median | Final IQR |
|---|---|---|---|---|---|
| medusa | 0.663 | 0.80h | 0.143 | 7 | 1.00 |
| foundry | 0.738 | 0.10h | 0.000 | 6 | 0.00 |
| echidna | 0.450 | 0.10h | 0.000 | 4 | 0.00 |
Milestones: time-to-k and success rates
| Fuzzer | time-to-1 (p50) | time-to-3 (p50) | time-to-5 (p50) | reach-1 rate | reach-3 rate | reach-5 rate |
|---|---|---|---|---|---|---|
| medusa | 0.10h | 0.10h | 0.25h | 100.0% | 100.0% | 100.0% |
| foundry | 0.00h | 0.00h | 0.00h | 100.0% | 100.0% | 100.0% |
| echidna | 0.10h | 0.10h | inf | 100.0% | 100.0% | 0.0% |
Shape-based interpretation (rules of thumb)
- Fast-start / early-plateau: high early checkpoint median + early plateau time + low late discovery share.
- Steady: moderate AUC, later plateau, consistent improvements across checkpoints, moderate variance.
- Slow-burn / late-surge: low early checkpoints but high late discovery share and later plateau time; often higher final median.
Limitations
- This dataset does not identify which specific bugs were found. It measures only counts.
- Bug depth/complexity cannot be measured directly without per-bug metadata (e.g., call-sequence length, coverage, or state metrics).
- Harness design still affects results; mitigate by keeping harness identical across fuzzers and reporting many runs.
Broken invariants
- Budget filter: 1.00h
- Events considered: 34 / 34
- Unique invariants: 13
Per-fuzzer totals
| Fuzzer | Invariants |
|---|---|
| echidna | 4 |
| foundry | 6 |
| medusa | 8 |
High-level overlap
- Shared by all fuzzers: 1
- Exclusive to
echidna: 0 - Exclusive to
foundry: 5 - Exclusive to
medusa: 4
Grouped invariants
Exclusive to echidna (0)
None.
Exclusive to foundry (5)
invariant_assertion_failure_CANARYinvariant_erc7540_1invariant_erc7540_2invariant_erc7540_3invariant_maxRedeemMaxWithdrawSymmetry
Exclusive to medusa (4)
doomsday_depositWithdrawSymmetricalglobal_comparePreviewMintAndConvertToAssetsglobal_previewEquivalenceFromAssetsinvariant_accumulatorSharesDecreaseOnFulfill_exact
Shared by all fuzzers (1)
invariant_canary
Top shared subsets (top 1 by size):
echidna, medusa (3)
assert_canarydoomsday_mintRedeemSymmetricalglobal_previewEquivalenceFromShares
Manifest
- scfuzzbench_commit:
4b43a0224d049cd0530fb21114fd60d57cd29839 - target_repo_url: https://github.com/Recon-Fuzz/superform-v2-periphery-scfuzzbench
- target_commit:
issue-74-superform-invariant-naming - benchmark_type:
property - instance_type:
c6a.4xlarge - instances_per_fuzzer:
2 - timeout_hours:
1 - aws_region:
us-east-1 - ubuntu_ami_id:
ami-0071174ad8cbb9e17 - foundry_version:
v1.6.0-rc1 - foundry_git_repo:
https://github.com/aviggiano/foundry - foundry_git_ref:
master - echidna_version:
2.3.1 - medusa_version:
1.4.1 - fuzzer_keys:
echidna, foundry, medusa
Artifacts
Manifest (index): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/runs/1771794816/911bd544dadb96194ba616e91483567d/manifest.json
Report prefix: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/911bd544dadb96194ba616e91483567d/1771794816/
Broken invariants (Markdown): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/911bd544dadb96194ba616e91483567d/1771794816/broken_invariants.md
Broken invariants (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/911bd544dadb96194ba616e91483567d/1771794816/broken_invariants.csv
Analysis bundle: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/911bd544dadb96194ba616e91483567d/1771794816/bundles/analysis.zip
Corpus bundle: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/911bd544dadb96194ba616e91483567d/1771794816/bundles/corpus.zip
Raw logs prefix: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/logs/1771794816/911bd544dadb96194ba616e91483567d/
Raw corpus prefix: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/corpus/1771794816/911bd544dadb96194ba616e91483567d/