Run gh-29726518626-1

• Date (UTC): 2026-07-20 08:02:46Z
• Benchmark: 9d3d0bf18b48ba76634016ca00b8e9d3
• Timeout: 0.25h
• Target: scfuzzbench/origin-dollar-scfuzzbench @ 299ec6b6bf
• Benchmark type: property
• Instances: 1 × c6a.xlarge per fuzzer
• Harness: scfuzzbench/scfuzzbench@7c2f667291

Trial run

Warning — trial run. This benchmark was executed with fewer than 10 instances per fuzzer and/or a time budget shorter than 24h. Results from trial runs are meant for debugging purposes and are not valid for extracting conclusions across different fuzzers.

Charts

Report

Fuzzer Benchmark Report (from bug-count CSV)

• Time budget: 0.25h

Warning — trial run. This benchmark was executed with fewer than 10 instances per fuzzer and/or a time budget shorter than 24h. Results from trial runs are meant for debugging purposes and are not valid for extracting conclusions across different fuzzers.

Executive summary

This report is derived solely from cumulative bugs-found over time across repeated runs per fuzzer. It emphasizes robust, distribution-based metrics (median/IQR, success rates, time-to-k) and shape-based behavior (plateau time, late discovery share) instead of single-run time-to-first-bug.

Bugs found at fixed time budgets (median [IQR])

Fuzzer	Runs	0.25h
echidna	1	13 [13,13]

Overall metrics

Fuzzer	AUC (norm)	Plateau time	Late discovery share	Final median	Final IQR
echidna	0.400	0.10h	0.000	13	0.00

Milestones: time-to-k and success rates

Fuzzer	time-to-1 (p50)	time-to-3 (p50)	time-to-5 (p50)	reach-1 rate	reach-3 rate	reach-5 rate
echidna	0.10h	0.10h	0.10h	100.0%	100.0%	100.0%

Throughput metrics (if supported by log format)

Values are run-level rates aggregated per fuzzer; n/a indicates the parser could not recover that metric from logs.

Fuzzer	Runs	Tx/s runs	Tx/s p50 [p25,p75]	Gas/s runs	Gas/s p50 [p25,p75]
echidna	1	1	3776.58 [3776.58,3776.58]	1	34771245176.00 [34771245176.00,34771245176.00]

Progress metrics from logs (fuzzer-specific proxies)

Sequence-rate and corpus values are parsed from native progress output and are useful for within-tool trend context.

Fuzzer	Runs	Seq/s runs	Seq/s p50 [p25,p75]	Corpus runs	Corpus p50 [p25,p75]
echidna	1	0	n/a	1	21.00 [21.00,21.00]

Function selector sanity checks

Counts are occurrences in unique saved corpus sequences, not runtime execution frequencies The complete per-instance distribution is in selector_distribution.csv; selector_summary.json preserves statuses and provenance.

• Expected-set status: unavailable (0 selector(s))
• Expected-set provenance: peer-consensus heuristic: observed in every available corpus for at least two independent evidence families (Echidna and Recon count as one related typed-corpus family); not benchmark ground truth

Fuzzer	Status	Instances with telemetry	Saved-corpus calls	Unique selectors
echidna	available	1/1	2100	47

Most common saved-corpus selectors

Fuzzer	Selector	Function	Calls	Share
echidna	0xa04961e8	testTransferToZeroAddress(uint8,uint256)	70	3.33%
echidna	0x6a91b27a	testRebasingCreditsPerTokenAboveZero()	69	3.29%
echidna	0x26610da8	testBurnAllBalanceToZero(uint8)	68	3.24%
echidna	0x1e7cfc16	testTransferSelf(uint8,uint256)	65	3.10%
echidna	0xe4840d4f	testTransferBalanceReceivedMore(uint8,uint8,uint256)	64	3.05%
echidna	0x1f3efb39	testTotalNonRebasingSupplyLessThanTotalBalance()	61	2.90%
echidna	0xbfb72b06	testMintBalanceRounding(uint8,uint256)	61	2.90%
echidna	0x324b65f5	testApprove(uint8,uint8,uint256)	60	2.86%
echidna	0x6396b18d	testAutoConvertBalance(uint8)	58	2.76%
echidna	0xdf4d5977	getTotalNonRebasingBalance()	57	2.71%

Selector telemetry limitations

• No expected selector set was available: the peer heuristic requires corroboration from every available instance in at least two fuzzer families

Statistical comparison (Mann-Whitney U and Vargha-Delaney A12)

Pairwise Mann-Whitney U tests and Vargha-Delaney A12 effect sizes on end-of-budget bug counts. The tests are two-sided, with Bonferroni correction applied for 0 comparison(s). Significance level: alpha = 0.05.

Warnings:

• Only one fuzzer present; skipping pairwise statistical tests.

Note: A12 is the probability that a randomly selected Fuzzer A run has a higher final bug count than a randomly selected Fuzzer B run, counting ties as half. Values above 0.5 favor A, values below 0.5 favor B, and 0.5 means equal tendency. Magnitude thresholds by distance from 0.5 are negligible (<0.06), small (<0.14), medium (<0.21), and large (>=0.21). Small sample sizes (fewer than 5 runs) reduce statistical power. A12 and its magnitude label are descriptive; the thresholds are rules of thumb. Neither the effect size nor statistical significance establishes practical importance, causation, or performance beyond the observed runs.

Shape-based interpretation (rules of thumb)

• Fast-start / early-plateau: high early checkpoint median + early plateau time + low late discovery share.
• Steady: moderate AUC, later plateau, consistent improvements across checkpoints, moderate variance.
• Slow-burn / late-surge: low early checkpoints but high late discovery share and later plateau time; often higher final median.

Limitations

• Core count-based charts use normalized event identities stored under the legacy bugs_found column. They do not count crash inputs, but they are not confirmed root-cause bug counts.
• Use known_bug_report.md, known_bug_summary.csv, and known_bug_findings.csv for evidence-backed known-bug hit rates; use broken_invariants.md / broken_invariants.csv for raw invariant identities.
• Severity, exploitability, and root-cause uniqueness cannot be measured directly without richer per-bug metadata.
• Harness design still affects results; mitigate by keeping harness identical across fuzzers and reporting many runs.

Broken invariants

• Budget filter: 0.25h
• Events considered: 14 / 14
• Unique invariants: 14

Warning — trial run. This benchmark was executed with fewer than 10 instances per fuzzer and/or a time budget shorter than 24h. Results from trial runs are meant for debugging purposes and are not valid for extracting conclusions across different fuzzers.

Per-fuzzer totals

Fuzzer	Invariants
echidna	14

High-level overlap

• Shared by all active fuzzers: 14
• Exclusive to echidna: 14

Grouped invariants

Exclusive to echidna (14)

• AssertionFailed
• assert_canary
• invariant_canary
• testBurnBalance
• testChangeSupply
• testMintBalanceRounding
• testOptInBalance
• testOptInBalanceRounding
• testRebasingCreditsPerTokenNotIncreased
• testTotalSupplyLessThanTotalBalance
• testTransferBalanceReceivedLessRounding
• testTransferBalanceReceivedMore
• testTransferBalanceSentLess
• testTransferSelf

Ground-truth known-bug mapping

• Target: origin-dollar
• Evidence-pinned commit: 299ec6b6bf5401b43efc3e9f7ee5cab3e76167c4
• Resolution: matched run repository and commit
• Catalog: benchmarks/known_bugs.json

Known bugs are limited to handlers explicitly annotated with hasKnownIssue in the pinned target. Each handler remains a separate catalog entry because the target does not provide evidence that any pair is the same root cause.

Canaries are reported separately as harness-health checks and are never included in the known-bug hit-rate denominator.

Per-fuzzer results

Fuzzer	Replicates	Known-bug replicate hits	Catalog coverage	Canary replicate hits	Unmapped event findings
echidna	1	5/12 (41.7%)	5/12 (41.7%)	2/2 (100.0%)	7

Known-bug catalog

Ground-truth ID	Kind	Title	Evidence
origin-dollar/change-supply-mismatch	documented-known-issue	changeSupply does not exactly reach the requested total supply	source
origin-dollar/total-supply-below-balances	documented-known-issue	Total supply can fall below the sum of account balances	source
origin-dollar/non-rebasing-supply-above-total	documented-known-issue	Non-rebasing supply can exceed total supply	source
origin-dollar/rebasing-credits-per-token-increase	documented-known-issue	Rebasing credits per token can increase	source
origin-dollar/non-rebasing-supply-below-balances	documented-known-issue	Non-rebasing supply can fall below non-rebasing balances	source
origin-dollar/mint-balance-shortfall	documented-known-issue	Minting can increase an account balance by less than the minted amount	source
origin-dollar/burn-balance-shortfall	documented-known-issue	Burning can decrease an account balance by less than the burned amount	source
origin-dollar/burn-all-leaves-balance	documented-known-issue	Burning an account's full balance can leave a nonzero balance	source
origin-dollar/burn-all-reverts	documented-known-issue	Burning an account's full balance can revert	source
origin-dollar/transfer-recipient-shortfall	documented-known-issue	A transfer recipient can receive less than the transferred amount	source
origin-dollar/transfer-sender-shortfall	documented-known-issue	A transfer sender's balance can decrease by less than the transferred amount	source
origin-dollar/transfer-within-balance-reverts	documented-known-issue	A transfer within the sender's balance can revert	source

Unmapped event identities

Fuzzer	Event identity	Replicates observed
echidna	AssertionFailed	1
echidna	testMintBalanceRounding	1
echidna	testOptInBalance	1
echidna	testOptInBalanceRounding	1
echidna	testTransferBalanceReceivedLessRounding	1
echidna	testTransferBalanceReceivedMore	1
echidna	testTransferSelf	1

Counting semantics

• A canonical known-bug ID counts at most once per replicate, even when several event aliases or counterexamples reach it.
• A replicate is identified by (run_id, instance_id, fuzzer).
• Known-bug hit rate is canonical bug/replicate hits ÷ (cataloged bugs × replicates).
• Unmapped rows are distinct normalized event identities per replicate. They are triage candidates, not claimed bugs.
• Crash inputs and corpus files are not used as bug identities.

Runner resource usage

• Budget filter: 0.25h
• Instances with metrics: 1
• Total samples: 175

Per-fuzzer medians (across instances)

Fuzzer	Instances	CPU active avg (%)	CPU active peak (%)	Memory used avg (GiB)	Memory used peak (GiB)	Memory used avg (%)	Memory used peak (%)
echidna	1	97.20	99.42	5.97	6.30	79.06	83.38

Instance stats

Instance	Fuzzer	Samples	Duration (h)	CPU active avg (%)	CPU active peak (%)	Memory avg (GiB)	Memory peak (GiB)	Memory avg (%)	Memory peak (%)
i-0a45e7f0e5442db19-echidna-v2.3.1	echidna	175	0.25	97.20	99.42	5.97	6.30	79.06	83.38

Manifest

• scfuzzbench_commit: 7c2f6672915338c559f7f68f4a910e2263d729a8
• target_repo_url: https://github.com/scfuzzbench/origin-dollar-scfuzzbench
• target_commit: 299ec6b6bf5401b43efc3e9f7ee5cab3e76167c4
• benchmark_type: property
• instance_type: c6a.xlarge
• instances_per_fuzzer: 1
• timeout_hours: 0.25
• aws_region: us-east-1
• ubuntu_ami_id: ami-052355af2a014bd2c
• foundry_version: 1.7.2-dev
• foundry_git_repo: https://github.com/foundry-rs/foundry
• foundry_git_ref: 02c05d970d2801da0aef8b82486ce84b01ede36d
• foundry_source_patch: scfuzzbench-throughput-progress-v1@sha256:2ee9e69b77c8007c78c816eb9ca791684aa5ecede0651b63f86cdd2e055eb17e
• echidna_version: 2.3.1
• medusa_version: 1.4.1
• recon_version: 0.4.6
• fuzzer_keys: echidna

Artifacts

• Manifest (index): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/runs/gh-29726518626-1/9d3d0bf18b48ba76634016ca00b8e9d3/manifest.json

• Report prefix: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/

• Broken invariants (Markdown): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/broken_invariants.md

• Broken invariants (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/broken_invariants.csv

• Ground-truth known bugs (Markdown): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/known_bug_report.md

• Ground-truth summary (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/known_bug_summary.csv

• Ground-truth findings (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/known_bug_findings.csv

• Throughput summary (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/throughput_summary.csv

• Progress metrics summary (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/progress_metrics_summary.csv

• Function selector distribution (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/selector_distribution.csv

• Function selector summary and health (JSON): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/selector_summary.json

• Runner resource usage (Markdown): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/runner_resource_usage.md

• Runner resource summary (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/runner_resource_summary.csv

• Runner resource timeseries (CSV): https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/runner_resource_timeseries.csv

• Analysis bundle: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/bundles/analysis.zip

• Logs bundle: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/bundles/logs.zip

• Corpus bundle: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/analysis/9d3d0bf18b48ba76634016ca00b8e9d3/gh-29726518626-1/bundles/corpus.zip

• Raw logs prefix: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/logs/gh-29726518626-1/9d3d0bf18b48ba76634016ca00b8e9d3/

• Raw corpus prefix: https://scfuzzbench-logs-185f44d6.s3.us-east-1.amazonaws.com/corpus/gh-29726518626-1/9d3d0bf18b48ba76634016ca00b8e9d3/