Run 1784176445

• Date (UTC): 2026-07-16 04:34:05Z
• Benchmark: 1d808539eed5207a8ce4c6380bffd089
• Timeout: 4h
• Target: scfuzzbench/drips-fuzzing-scfuzzbench @ c50d160ead
• Benchmark type: property
• Instances: 2 × c6a.4xlarge per fuzzer
• Harness: scfuzzbench/scfuzzbench@3ea47a58c1

Trial run

Warning — trial run. This benchmark was executed with fewer than 10 instances per fuzzer and/or a time budget shorter than 24h. Results from trial runs are meant for debugging purposes and are not valid for extracting conclusions across different fuzzers.

Charts

Report

Fuzzer Benchmark Report (from bug-count CSV)

• Time budget: 4.00h

Warning — trial run. This benchmark was executed with fewer than 10 instances per fuzzer and/or a time budget shorter than 24h. Results from trial runs are meant for debugging purposes and are not valid for extracting conclusions across different fuzzers.

Executive summary

This report is derived solely from cumulative bugs-found over time across repeated runs per fuzzer. It emphasizes robust, distribution-based metrics (median/IQR, success rates, time-to-k) and shape-based behavior (plateau time, late discovery share) instead of single-run time-to-first-bug.

Bugs found at fixed time budgets (median [IQR])

Fuzzer	Runs	1h	4h
echidna	2	13 [12,14]	16 [15,16]
medusa	2	14 [14,15]	14 [14,15]
foundry	2	0 [0,0]	11 [10,12]
recon-fuzzer	2	6 [6,7]	6 [6,7]

Overall metrics

Fuzzer	AUC (norm)	Plateau time	Late discovery share	Final median	Final IQR
echidna	0.830	3.90h	0.097	16	0.50
medusa	0.872	0.60h	0.000	14	0.50
foundry	0.245	3.30h	0.545	11	1.00
recon-fuzzer	0.395	0.50h	0.000	6	0.50

Milestones: time-to-k and success rates

Fuzzer	time-to-1 (p50)	time-to-3 (p50)	time-to-5 (p50)	reach-1 rate	reach-3 rate	reach-5 rate
echidna	0.10h	0.10h	0.10h	100.0%	100.0%	100.0%
medusa	0.00h	0.10h	0.10h	100.0%	100.0%	100.0%
foundry	2.50h	2.50h	2.50h	100.0%	100.0%	100.0%
recon-fuzzer	0.00h	0.10h	0.10h	100.0%	100.0%	100.0%

Throughput metrics (if supported by log format)

Values are run-level rates aggregated per fuzzer; n/a indicates the parser could not recover that metric from logs.

Fuzzer	Runs	Tx/s runs	Tx/s p50 [p25,p75]	Gas/s runs	Gas/s p50 [p25,p75]
echidna	2	2	777.96 [767.27,788.65]	2	8414652567.00 [8113156162.50,8716148971.50]
medusa	2	2	4673.00 [4615.00,4731.00]	2	1453235975.50 [1447487201.25,1458984749.75]
recon-fuzzer	2	2	595.98 [584.42,607.54]	2	81991172.50 [77271186.25,86711158.75]

Progress metrics from logs (fuzzer-specific proxies)

Coverage/corpus values are parsed from each fuzzer's native progress output and are useful for trend context, not strict cross-fuzzer equivalence.

Fuzzer	Runs	Seq/s runs	Seq/s p50 [p25,p75]	Coverage runs	Coverage p50 [p25,p75]	Corpus runs	Corpus p50 [p25,p75]
echidna	2	0	n/a	2	44533.00 [44532.50,44533.50]	2	61.50 [59.75,63.25]
medusa	2	2	46.00 [45.50,46.50]	2	5206.00 [5205.50,5206.50]	2	107.00 [107.00,107.00]
recon-fuzzer	2	0	n/a	2	45193.50 [45156.25,45230.75]	2	84.00 [73.50,94.50]

Statistical comparison (Mann-Whitney U test)

Pairwise Mann-Whitney U tests on end-of-budget bug counts (two-sided). Bonferroni correction applied for 6 comparison(s). Significance level: alpha = 0.05.

Warnings:

• One or more fuzzers have fewer than 5 runs. Statistical power may be limited.

Fuzzer A	Fuzzer B	Median A	Median B	U statistic	p-value	p (corrected)	Significant
echidna	medusa	15.5	14.5	3.5	0.4142	1.0000	no
echidna	foundry	15.5	11	4.0	0.3333	1.0000	no
echidna	recon-fuzzer	15.5	6.5	4.0	0.3333	1.0000	no
medusa	foundry	14.5	11	4.0	0.3333	1.0000	no
medusa	recon-fuzzer	14.5	6.5	4.0	0.3333	1.0000	no
foundry	recon-fuzzer	11	6.5	4.0	0.3333	1.0000	no

No pairwise comparison reached significance after Bonferroni correction.

Note: The Mann-Whitney U test assesses whether one fuzzer tends to find more bugs than another across runs. It does not measure effect size or practical importance. Small sample sizes (fewer than 5 runs) reduce statistical power.

Shape-based interpretation (rules of thumb)

• Fast-start / early-plateau: high early checkpoint median + early plateau time + low late discovery share.
• Steady: moderate AUC, later plateau, consistent improvements across checkpoints, moderate variance.
• Slow-burn / late-surge: low early checkpoints but high late discovery share and later plateau time; often higher final median.

Limitations

• Core metrics in this section are count-based; use broken_invariants.md / broken_invariants.csv for invariant identities.
• Severity, exploitability, and root-cause uniqueness cannot be measured directly without richer per-bug metadata.
• Harness design still affects results; mitigate by keeping harness identical across fuzzers and reporting many runs.

Broken invariants

• Budget filter: 4.00h
• Events considered: 95 / 95
• Unique invariants: 16

Warning — trial run. This benchmark was executed with fewer than 10 instances per fuzzer and/or a time budget shorter than 24h. Results from trial runs are meant for debugging purposes and are not valid for extracting conclusions across different fuzzers.

Per-fuzzer totals

Fuzzer	Invariants
echidna	16
foundry	12
medusa	15
recon-fuzzer	7

High-level overlap

• Shared by all active fuzzers: 7
• Exclusive to echidna: 1
• Exclusive to foundry: 0
• Exclusive to medusa: 0
• Exclusive to recon-fuzzer: 0

Grouped invariants

Exclusive to echidna (1)

• AssertionFailed

Exclusive to foundry (0)

None.

Exclusive to medusa (0)

None.

Exclusive to recon-fuzzer (0)

None.

Shared by all active fuzzers (7)

• assert_canary
• invariantAccountingVsTokenBalance
• invariantWithdrawAllTokensShouldNotRevert
• invariant_canary
• testReceiveStreamsShouldNotRevert
• testSqueezableAmountCantBeUndone
• testSqueezableAmountCantBeWithdrawn

Top shared subsets (top 2 by size):

echidna, foundry, medusa (5)

• testBalanceAtInFuture
• testSetStreamBalanceWithdrawAllShouldNotRevert
• testSqueezableVsReceived
• testSqueezeWithDefaultHistoryShouldNotRevert
• testSqueezeWithFuzzedHistoryShouldNotRevert

echidna, medusa (3)

• testAddStreamShouldNotRevert
• testSetStreamBalanceShouldNotRevert
• testSetStreamsShouldNotRevert

Runner resource usage

• Budget filter: 4.00h
• Instances with metrics: 8
• Total samples: 22848

Per-fuzzer medians (across instances)

Fuzzer	Instances	CPU active avg (%)	CPU active peak (%)	Memory used avg (GiB)	Memory used peak (GiB)	Memory used avg (%)	Memory used peak (%)
echidna	2	96.29	98.17	20.88	23.12	68.12	75.45
foundry	2	6.28	20.00	0.95	1.34	3.09	4.38
medusa	2	98.36	99.42	1.25	1.56	4.07	5.09
recon-fuzzer	2	96.22	98.72	3.01	3.47	9.84	11.32

Instance stats

Instance	Fuzzer	Samples	Duration (h)	CPU active avg (%)	CPU active peak (%)	Memory avg (GiB)	Memory peak (GiB)	Memory avg (%)	Memory peak (%)
i-072f1ee3a594bc502-echidna-v2.3.1	echidna	2846	4.00	96.35	98.12	20.84	23.03	68.01	75.15
i-0ac7d4fde6343153c-echidna-v2.3.1	echidna	2847	4.00	96.24	98.22	20.91	23.21	68.24	75.75
i-02e3879ef9c633e9a-foundry-git-7254e01	foundry	2860	3.98	6.28	20.00	0.95	1.30	3.10	4.24
i-0c064f365aa4fa6ce-foundry-git-7254e01	foundry	2860	3.98	6.28	20.00	0.95	1.39	3.08	4.52
i-02daf67e353d84400-medusa-v1.4.1	medusa	2859	4.00	98.35	99.42	1.29	1.66	4.20	5.41
i-0cd32332b1821c80a-medusa-v1.4.1	medusa	2860	4.00	98.37	99.43	1.20	1.46	3.93	4.77
i-070ae9e45ba8dd29a-recon-v0.4.6	recon-fuzzer	2858	4.00	96.67	98.97	3.04	3.43	9.91	11.18
i-0c16da2afaaef5ee8-recon-v0.4.6	recon-fuzzer	2858	4.00	95.77	98.46	2.99	3.51	9.76	11.47

Manifest

• scfuzzbench_commit: 3ea47a58c13bdfac9f7021cc1253da5e15795314
• target_repo_url: https://github.com/scfuzzbench/drips-fuzzing-scfuzzbench
• target_commit: c50d160ead6bf82b1d1071e853dc74da0b23f595
• benchmark_type: property
• instance_type: c6a.4xlarge
• instances_per_fuzzer: 2
• timeout_hours: 4
• aws_region: us-east-1
• ubuntu_ami_id: ami-052355af2a014bd2c
• foundry_version: v1.7.1
• foundry_git_repo: https://github.com/foundry-rs/foundry
• foundry_git_ref: 7254e0107a696d72b40eea3bbb04d67e7bd1b847
• echidna_version: 2.3.1
• medusa_version: 1.4.1
• recon_version: 0.4.6
• fuzzer_keys: echidna, foundry, medusa, recon-fuzzer

Artifacts

• Manifest (index): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/runs/1784176445/1d808539eed5207a8ce4c6380bffd089/manifest.json

• Report prefix: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/1d808539eed5207a8ce4c6380bffd089/1784176445/

• Broken invariants (Markdown): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/1d808539eed5207a8ce4c6380bffd089/1784176445/broken_invariants.md

• Broken invariants (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/1d808539eed5207a8ce4c6380bffd089/1784176445/broken_invariants.csv

• Throughput summary (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/1d808539eed5207a8ce4c6380bffd089/1784176445/throughput_summary.csv

• Progress metrics summary (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/1d808539eed5207a8ce4c6380bffd089/1784176445/progress_metrics_summary.csv

• Runner resource usage (Markdown): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/1d808539eed5207a8ce4c6380bffd089/1784176445/runner_resource_usage.md

• Runner resource summary (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/1d808539eed5207a8ce4c6380bffd089/1784176445/runner_resource_summary.csv

• Runner resource timeseries (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/1d808539eed5207a8ce4c6380bffd089/1784176445/runner_resource_timeseries.csv

• Analysis bundle: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/1d808539eed5207a8ce4c6380bffd089/1784176445/bundles/analysis.zip

• Logs bundle: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/1d808539eed5207a8ce4c6380bffd089/1784176445/bundles/logs.zip

• Corpus bundle: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/1d808539eed5207a8ce4c6380bffd089/1784176445/bundles/corpus.zip

• Raw logs prefix: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/logs/1784176445/1d808539eed5207a8ce4c6380bffd089/

• Raw corpus prefix: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/corpus/1784176445/1d808539eed5207a8ce4c6380bffd089/