Skip to content

chore[ci]: run fuzzing with both codegen backends - #5270

Draft
banteg wants to merge 1 commit into
vyperlang:masterfrom
banteg:chore/venom-fuzzing-ci
Draft

banteg wants to merge 1 commit into
vyperlang:masterfrom
banteg:chore/venom-fuzzing-ci

Conversation

@banteg

@banteg banteg commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

What I did

Addresses the fuzzing-lane portion of #5116. Run the existing 120 fuzzing shards on both legacy and Venom codegen, with distinct job names and coverage artifacts.

Merge prerequisite: #5126. Enabling the lane exposes the existing negative integer-to-decimal clamp defect from #5110. For example, converting -18707220957835557353007165858768422651596 to decimal incorrectly succeeds on current master. The tests remain strict; this PR does not skip those failures. Kept as a draft until the compiler fix lands.

How I did it

The workflow matrix adds the backend dimension and passes --experimental-codegen to the Venom shards. The existing success gate covers both backends. This doubles the fuzzing matrix from 120 to 240 jobs.

Adjust the conversion harness for Venom's stronger constant analysis: a literal conversion can be rejected by either constant folding or SCCP. The constant-local runtime case uses disable_static_exceptions=True so the failing assertion stays in bytecode and the test still requires a runtime revert. The calldata-input case keeps ordinary compiler settings.

How to verify it

  • Reproduced the original SCCP/harness mismatch before the adjustment.
  • The 48 affected negative-boundary cases pass with refactor[lang]: share conversion rules and bounds #5126 applied; 49 selected negative-boundary/address cases also pass under legacy codegen with that fix.
  • All 70,708 passing conversion cases pass on current master with this PR.
  • All 15,750 reverting conversion cases pass on current master with this PR and refactor[lang]: share conversion rules and bounds #5126's source patch applied.
  • A complete CI-style Venom shard using the published duration map passes (group 1; one long-running test).
  • make lint and git diff --check pass. YAML validation confirms 240 backend/shard combinations, the Venom flag and the shared success gate.

The complete conversion matrices above and one CI-style shard were run locally; all 240 remote shards have not been run. This draft does not claim a green full Venom fuzzing matrix on unmodified master.

Commit message

The fuzzing matrix only runs legacy codegen, leaving Venom's numeric
clamps without the existing boundary-value coverage. Run every fuzzing
shard with both backends while retaining the shared success gate and
separate coverage artifacts.

Adapt conversion rejection tests to Venom's stronger constant analysis.
Literal conversions must still fail compilation, whether constant
folding or SCCP detects the failure. For the constant-local runtime
case, retain the assertion in bytecode by disabling static exceptions.
The calldata-input case continues to use ordinary compiler settings.

Description for the changelog

Run the fuzzing CI matrix with both codegen backends and retain runtime conversion-clamp coverage under Venom.

The fuzzing matrix only runs legacy codegen, leaving Venom's numeric
clamps without the existing boundary-value coverage. Run every fuzzing
shard with both backends while retaining the shared success gate and
separate coverage artifacts.

Adapt conversion rejection tests to Venom's stronger constant analysis.
Literal conversions must still fail compilation, whether constant
folding or SCCP detects the failure. For the constant-local runtime
case, retain the assertion in bytecode by disabling static exceptions.
The calldata-input case continues to use ordinary compiler settings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant