Run gh-30146496505-1

• Date (UTC): 2026-07-25 05:52:12Z
• Benchmark: ebfa6b2cf86704a76a30b80525dda7b9
• Timeout: 4h
• Target: scfuzzbench/drips-fuzzing-scfuzzbench @ 0ae8bd881e
• Benchmark type: property
• Instances: 2 × c6a.4xlarge per fuzzer
• Harness: scfuzzbench/scfuzzbench@3d834b12c5

Trial run

Warning — trial run. This benchmark was executed with fewer than 10 instances per fuzzer and/or a time budget shorter than 24h. Results from trial runs are meant for debugging purposes and are not valid for extracting conclusions across different fuzzers.

Charts

Report

Fuzzer Benchmark Report (from bug-count CSV)

• Time budget: 4.00h

Warning — trial run. This benchmark was executed with fewer than 10 instances per fuzzer and/or a time budget shorter than 24h. Results from trial runs are meant for debugging purposes and are not valid for extracting conclusions across different fuzzers.

Executive summary

This report is derived solely from cumulative bugs-found over time across repeated runs per fuzzer. It emphasizes robust, distribution-based metrics (median/IQR, success rates, time-to-k) and shape-based behavior (plateau time, late discovery share) instead of single-run time-to-first-bug.

Bugs found at fixed time budgets (median [IQR])

Fuzzer	Runs	1h	4h
recon-fuzzer	2	14 [13,14]	15 [15,15]
medusa	2	14 [14,15]	14 [14,15]
foundry	2	13 [13,13]	14 [14,15]
echidna	2	10 [9,11]	13 [12,14]

Overall metrics

Fuzzer	AUC (norm)	Plateau time	Late discovery share	Final median	Final IQR
recon-fuzzer	0.914	3.50h	0.033	15	0.00
medusa	0.945	0.40h	0.000	14	0.50
foundry	0.873	3.10h	0.034	14	0.50
echidna	0.763	2.00h	0.000	13	1.00

Milestones: time-to-k and success rates

Fuzzer	time-to-1 (p50)	time-to-3 (p50)	time-to-5 (p50)	reach-1 rate	reach-3 rate	reach-5 rate
recon-fuzzer	0.00h	0.10h	0.10h	100.0%	100.0%	100.0%
medusa	0.00h	0.00h	0.10h	100.0%	100.0%	100.0%
foundry	0.00h	0.10h	0.10h	100.0%	100.0%	100.0%
echidna	0.10h	0.10h	0.10h	100.0%	100.0%	100.0%

Throughput metrics (if supported by log format)

Values are run-level rates aggregated per fuzzer; n/a indicates the parser could not recover that metric from logs.

Fuzzer	Runs	Tx/s runs	Tx/s p50 [p25,p75]	Gas/s runs	Gas/s p50 [p25,p75]
recon-fuzzer	2	2	2246.17 [2196.42,2295.92]	2	2377684751.50 [2357361316.75,2398008186.25]
medusa	2	2	5134.00 [5113.00,5155.00]	2	1467660424.00 [1461080946.50,1474239901.50]
foundry	2	2	14739.50 [13570.08,15908.93]	2	5942870567.53 [5754299146.47,6131441988.60]
echidna	2	2	1161.42 [1120.59,1202.25]	2	9812304278.50 [9755860579.75,9868747977.25]

Progress metrics from logs (fuzzer-specific proxies)

Sequence-rate and corpus values are parsed from native progress output and are useful for within-tool trend context.

Fuzzer	Runs	Seq/s runs	Seq/s p50 [p25,p75]	Corpus runs	Corpus p50 [p25,p75]
recon-fuzzer	2	0	n/a	2	105.50 [104.75,106.25]
medusa	2	2	51.00 [51.00,51.00]	2	146.00 [145.50,146.50]
foundry	2	0	n/a	2	0.00 [0.00,0.00]
echidna	2	0	n/a	2	62.00 [61.50,62.50]

Function selector sanity checks

Counts are occurrences in unique saved corpus sequences, not runtime execution frequencies The complete per-instance distribution is in selector_distribution.csv; selector_summary.json preserves statuses and provenance.

• Expected-set status: available (67 selector(s))
• Expected-set provenance: peer-consensus heuristic: observed in every available corpus for at least two independent evidence families (Echidna and Recon count as one related typed-corpus family); not benchmark ground truth

Fuzzer	Status	Instances with telemetry	Saved-corpus calls	Unique selectors
echidna	available	2/2	12295	68
foundry	unavailable	0/2	0	0
medusa	available	2/2	7086	67
recon-fuzzer	available	2/2	20928	67

Most common saved-corpus selectors

Fuzzer	Selector	Function	Calls	Share
echidna	0x3ba851ca	setStreamsWithClamping(uint8,uint8,uint160,uint32,uint32,int128)	233	1.90%
echidna	0x13a600d0	addStream(uint8,uint8,uint160,uint32,uint32,int128)	227	1.85%
echidna	0x662fe867	invariantAmtPerSecVsMinAmtPerSec(uint8,uint256)	223	1.81%
echidna	0x1a042ae7	invariant_canary()	217	1.76%
echidna	0xf56082ce	setStreamBalanceWithClamping(uint8,int128)	213	1.73%
echidna	0x19ca05b3	testSplittableAfterSplit(uint8)	212	1.72%
echidna	0x3b0786f0	receiveStreamsAllCycles(uint8)	208	1.69%
echidna	0x1a3ed582	setStreams(uint8,uint8,uint160,uint32,uint32,int128)	207	1.68%
echidna	0x45b9b0ba	testReceiveStreamsViewVsActual(uint8,uint32)	206	1.68%
echidna	0x4e0ad60b	testSqueezableAmountCantBeWithdrawn(uint8,uint8)	204	1.66%
medusa	0x13a600d0	addStream(uint8,uint8,uint160,uint32,uint32,int128)	196	2.77%
medusa	0x0e8df0d4	testSqueeze(uint8,uint8)	171	2.41%
medusa	0xdf65cad7	testSetSplitsShouldNotRevert(uint8,uint8,uint32)	156	2.20%
medusa	0xf89367f3	testReceiversReceivedSplit(uint8)	151	2.13%
medusa	0x441581e7	setSplitsWithClamping(uint8,uint8,uint32)	146	2.06%
medusa	0x4e0ad60b	testSqueezableAmountCantBeWithdrawn(uint8,uint8)	146	2.06%
medusa	0xfa1227cd	testSetStreamBalance(uint8,int128)	142	2.00%
medusa	0x462664e4	testRemoveStreamShouldNotRevert(uint8,uint256)	137	1.93%
medusa	0x662fe867	invariantAmtPerSecVsMinAmtPerSec(uint8,uint256)	133	1.88%
medusa	0x1ba3a23c	testReceiveStreamsViewConsistency(uint8,uint32)	130	1.83%
recon-fuzzer	0xed0c06d6	removeAllSplits(uint8)	481	2.30%
recon-fuzzer	0x8319b8db	receiveStreamsSplitAndCollectToSelf(uint8)	422	2.02%
recon-fuzzer	0x6cc8862a	testGiveShouldNotRevert(uint8,uint8,uint128)	412	1.97%
recon-fuzzer	0x82ce0f6d	testReceiveStreamsShouldNotRevert(uint8)	404	1.93%
recon-fuzzer	0xf89367f3	testReceiversReceivedSplit(uint8)	383	1.83%
recon-fuzzer	0x30387686	testSplitShouldNotRevert(uint8)	380	1.82%
recon-fuzzer	0x3ba851ca	setStreamsWithClamping(uint8,uint8,uint160,uint32,uint32,int128)	374	1.79%
recon-fuzzer	0xed2ce2d6	squeezeWithFuzzedHistory(uint8,uint8,uint256,bytes32)	368	1.76%
recon-fuzzer	0x218be192	invariantSumAmtDeltaIsZero(uint8)	364	1.74%
recon-fuzzer	0x32dc4442	testSqueezeWithFullyHashedHistory(uint8,uint8)	354	1.69%

Selector diagnostic findings

• INFO: i-0d48a81d5d8532c0e-recon-v0.4.18: peer-heuristic gap (diagnostic, not ground truth): absent 1 selector(s): 0x21d76db7
• INFO: i-0eeac9664aa2ffaf4-recon-v0.4.18: peer-heuristic gap (diagnostic, not ground truth): absent 1 selector(s): 0x21d76db7

Selector telemetry limitations

• i-081cdf71adeaf688e-foundry-git-02c05d9: Foundry selector distribution is unavailable because no persisted corpus was present; failure-event selectors are intentionally not used as a distribution
• i-0e51c013e3ab1511f-foundry-git-02c05d9: Foundry selector distribution is unavailable because no persisted corpus was present; failure-event selectors are intentionally not used as a distribution

Statistical comparison (Mann-Whitney U and Vargha-Delaney A12)

Pairwise Mann-Whitney U tests and Vargha-Delaney A12 effect sizes on end-of-budget bug counts. The tests are two-sided, with Bonferroni correction applied for 6 comparison(s). Significance level: alpha = 0.05.

Warnings:

• One or more fuzzers have fewer than 5 runs. Statistical power may be limited.

Fuzzer A	Fuzzer B	Median A	Median B	U statistic	A12 (A over B)	Effect magnitude	p-value	p (corrected)	Significant
recon-fuzzer	medusa	15	14.5	3.0	0.750	large	0.6171	1.0000	no
recon-fuzzer	foundry	15	14.5	3.0	0.750	large	0.6171	1.0000	no
recon-fuzzer	echidna	15	13	4.0	1.000	large	0.2207	1.0000	no
medusa	foundry	14.5	14.5	2.0	0.500	negligible	1.0000	1.0000	no
medusa	echidna	14.5	13	3.5	0.875	large	0.4142	1.0000	no
foundry	echidna	14.5	13	3.5	0.875	large	0.4142	1.0000	no

No pairwise comparison reached significance after Bonferroni correction.

Note: A12 is the probability that a randomly selected Fuzzer A run has a higher final bug count than a randomly selected Fuzzer B run, counting ties as half. Values above 0.5 favor A, values below 0.5 favor B, and 0.5 means equal tendency. Magnitude thresholds by distance from 0.5 are negligible (<0.06), small (<0.14), medium (<0.21), and large (>=0.21). Small sample sizes (fewer than 5 runs) reduce statistical power. A12 and its magnitude label are descriptive; the thresholds are rules of thumb. Neither the effect size nor statistical significance establishes practical importance, causation, or performance beyond the observed runs.

Shape-based interpretation (rules of thumb)

• Fast-start / early-plateau: high early checkpoint median + early plateau time + low late discovery share.
• Steady: moderate AUC, later plateau, consistent improvements across checkpoints, moderate variance.
• Slow-burn / late-surge: low early checkpoints but high late discovery share and later plateau time; often higher final median.

Limitations

• Core count-based charts use normalized event identities stored under the legacy bugs_found column. They do not count crash inputs, but they are not confirmed root-cause bug counts.
• Use known_bug_report.md, known_bug_summary.csv, and known_bug_findings.csv for evidence-backed known-bug hit rates; use broken_invariants.md / broken_invariants.csv for raw invariant identities.
• Severity, exploitability, and root-cause uniqueness cannot be measured directly without richer per-bug metadata.
• Harness design still affects results; mitigate by keeping harness identical across fuzzers and reporting many runs.

Broken invariants

• Budget filter: 4.00h
• Events considered: 114 / 114
• Unique invariants: 15

Warning — trial run. This benchmark was executed with fewer than 10 instances per fuzzer and/or a time budget shorter than 24h. Results from trial runs are meant for debugging purposes and are not valid for extracting conclusions across different fuzzers.

Per-fuzzer totals

Fuzzer	Invariants
echidna	15
foundry	15
medusa	15
recon-fuzzer	15

High-level overlap

• Shared by all active fuzzers: 15
• Exclusive to echidna: 0
• Exclusive to foundry: 0
• Exclusive to medusa: 0
• Exclusive to recon-fuzzer: 0

Grouped invariants

Exclusive to echidna (0)

None.

Exclusive to foundry (0)

None.

Exclusive to medusa (0)

None.

Exclusive to recon-fuzzer (0)

None.

Shared by all active fuzzers (15)

• assert_canary
• invariantAccountingVsTokenBalance
• invariantWithdrawAllTokensShouldNotRevert
• invariant_canary
• testAddStreamShouldNotRevert
• testBalanceAtInFuture
• testReceiveStreamsShouldNotRevert
• testSetStreamBalanceShouldNotRevert
• testSetStreamBalanceWithdrawAllShouldNotRevert
• testSetStreamsShouldNotRevert
• testSqueezableAmountCantBeUndone
• testSqueezableAmountCantBeWithdrawn
• testSqueezableVsReceived
• testSqueezeWithDefaultHistoryShouldNotRevert
• testSqueezeWithFuzzedHistoryShouldNotRevert

Ground-truth known-bug mapping

Ground-truth mapping was not applied.

• Reason: run commit 0ae8bd881e3c4f4a254f3cf92091e9997c5e88be is not the evidence-pinned catalog revision c50d160ead6bf82b1d1071e853dc74da0b23f595
• Catalog: benchmarks/known_bugs.json

No event was classified as a known bug. Raw event analysis remains available, but it must not be interpreted as a confirmed bug count.

Runner resource usage

• Budget filter: 4.00h
• Instances with metrics: 8
• Total samples: 22367

Per-fuzzer medians (across instances)

Fuzzer	Instances	CPU active avg (%)	CPU active peak (%)	Memory used avg (GiB)	Memory used peak (GiB)	Memory used avg (%)	Memory used peak (%)
echidna	2	96.06	98.16	21.16	23.54	69.05	76.82
foundry	2	99.91	100.00	1.00	1.37	3.27	4.46
medusa	2	98.36	99.28	1.33	1.73	4.34	5.64
recon-fuzzer	2	68.65	98.84	7.50	10.38	24.48	33.87

Instance stats

Instance	Fuzzer	Samples	Duration (h)	CPU active avg (%)	CPU active peak (%)	Memory avg (GiB)	Memory peak (GiB)	Memory avg (%)	Memory peak (%)
i-0537465ab58fd927d-echidna-v2.3.2	echidna	2775	4.00	96.16	98.18	21.05	23.39	68.69	76.33
i-0a6a8a9d574c3af9e-echidna-v2.3.2	echidna	2775	4.00	95.96	98.15	21.27	23.69	69.41	77.32
i-081cdf71adeaf688e-foundry-git-02c05d9	foundry	2799	3.98	99.91	100.00	1.00	1.44	3.26	4.68
i-0e51c013e3ab1511f-foundry-git-02c05d9	foundry	2797	3.98	99.90	100.00	1.01	1.30	3.29	4.24
i-0392b7426b8d0e6f3-medusa-v1.5.1	medusa	2810	4.00	98.32	99.44	1.32	1.79	4.31	5.84
i-0f36cf811fb0f5fda-medusa-v1.5.1	medusa	2811	4.00	98.41	99.11	1.34	1.67	4.36	5.45
i-0d48a81d5d8532c0e-recon-v0.4.18	recon-fuzzer	2805	4.00	60.89	97.81	6.61	9.36	21.58	30.55
i-0eeac9664aa2ffaf4-recon-v0.4.18	recon-fuzzer	2795	4.00	76.42	99.88	8.39	11.40	27.37	37.18

Manifest

• scfuzzbench_commit: 3d834b12c5d851968933acddd10ef5429228fb5d
• target_repo_url: https://github.com/scfuzzbench/drips-fuzzing-scfuzzbench
• target_commit: 0ae8bd881e3c4f4a254f3cf92091e9997c5e88be
• benchmark_type: property
• instance_type: c6a.4xlarge
• instances_per_fuzzer: 2
• timeout_hours: 4
• aws_region: us-east-1
• ubuntu_ami_id: ami-052355af2a014bd2c
• foundry_version: 1.7.2-dev
• foundry_git_repo: https://github.com/foundry-rs/foundry
• foundry_git_ref: 02c05d970d2801da0aef8b82486ce84b01ede36d
• foundry_source_patch: scfuzzbench-throughput-progress-v1@sha256:2ee9e69b77c8007c78c816eb9ca791684aa5ecede0651b63f86cdd2e055eb17e
• echidna_version: 2.3.2
• medusa_version: 1.5.1
• recon_version: 0.4.18
• fuzzer_keys: echidna, foundry, medusa, recon-fuzzer

Artifacts

• Manifest (index): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/runs/gh-30146496505-1/ebfa6b2cf86704a76a30b80525dda7b9/manifest.json

• Report prefix: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/

• Broken invariants (Markdown): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/broken_invariants.md

• Broken invariants (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/broken_invariants.csv

• Ground-truth known bugs (Markdown): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/known_bug_report.md

• Ground-truth summary (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/known_bug_summary.csv

• Ground-truth findings (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/known_bug_findings.csv

• Throughput summary (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/throughput_summary.csv

• Progress metrics summary (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/progress_metrics_summary.csv

• Function selector distribution (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/selector_distribution.csv

• Function selector summary and health (JSON): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/selector_summary.json

• Runner resource usage (Markdown): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/runner_resource_usage.md

• Runner resource summary (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/runner_resource_summary.csv

• Runner resource timeseries (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/runner_resource_timeseries.csv

• Analysis bundle: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/bundles/analysis.zip

• Logs bundle: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/bundles/logs.zip

• Corpus bundle: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/ebfa6b2cf86704a76a30b80525dda7b9/gh-30146496505-1/bundles/corpus.zip

• Raw logs prefix: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/logs/gh-30146496505-1/ebfa6b2cf86704a76a30b80525dda7b9/

• Raw corpus prefix: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/corpus/gh-30146496505-1/ebfa6b2cf86704a76a30b80525dda7b9/