Skip to content

bugc: Add behavioral test helper for end-to-end bytecode validation#183

Merged
gnidan merged 3 commits intomainfrom
compiler-behavioral-tests
Mar 11, 2026
Merged

bugc: Add behavioral test helper for end-to-end bytecode validation#183
gnidan merged 3 commits intomainfrom
compiler-behavioral-tests

Conversation

@gnidan
Copy link
Member

@gnidan gnidan commented Mar 11, 2026

Summary

Adds a lightweight executeProgram(source, opts) test helper that compiles BUG source, deploys bytecode via @ethdebug/evm Executor, optionally calls with calldata, and returns execution results for direct assertion — no pointer dereferencing needed.

This decouples bytecode correctness testing from debug info correctness, making it easy to write focused behavioral tests.

Helper (test/evm/behavioral.ts):

  • executeProgram(source, { calldata?, value?, optimizationLevel? }){ deployed, callSuccess, returnValue, getStorage(), executor }
  • readUint256(returnValue, wordOffset) for parsing return data

Tests (src/evmgen/behavioral.test.ts) — 8 passing, 1 skipped:

  • Constructor storage initialization (single + multiple slots with computed values)
  • Runtime storage modification via call
  • Multiple sequential calls accumulating state
  • For loop execution
  • Conditional branching
  • Bare return (STOP) with no return data
  • Compilation failure error path
  • Internal function calls (skipped — known stack underflow issue)

Add executeProgram() helper that compiles BUG source, deploys
bytecode via @ethdebug/evm Executor, optionally calls with
calldata, and returns results for direct assertion on storage
and execution state — no pointer dereferencing needed.

Tests cover: constructor storage init, runtime storage
modification, multiple calls, for loops, conditionals,
bare return (STOP), and compilation failure. Internal
function calls are skipped (known stack underflow issue).
@gnidan
Copy link
Member Author

gnidan commented Mar 11, 2026

Self-review

Design decisions:

  • The helper lives in test/evm/behavioral.ts alongside the existing executor re-export, accessible via #test/evm/behavioral.
  • executeProgram() throws on compilation failure rather than returning an error result — keeps test code concise since most tests expect compilation to succeed.
  • The executor instance is exposed on the result for advanced use (e.g., multiple sequential calls, direct storage queries).
  • readUint256() utility provided for future return value testing (BUG's code block is currently void, but internal functions with return types could eventually support this).

Known limitation discovered:

  • Internal function calls (via define { function ... }) fail at runtime with stack underflow. This is a pre-existing bug, not introduced by this PR. The test is included but skipped to document it.

Test patterns established:

  • Constructor-only tests: compile + deploy, assert storage
  • Runtime tests: compile + deploy + call, assert storage changes
  • Multi-call tests: reuse executor for sequential calls
  • Error tests: expect compilation to throw

Copy link
Member Author

@gnidan gnidan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style review — overall this is clean and well-structured. A few notes:

Source string indentation: The existing tests in this package (e.g., integration.test.ts) use flush-left source strings inside template literals:

const source = `name Minimal;

storage {
  [0] value: uint256;
}

code {
  value = 1;
}`;

The new file indents every source string by 8 spaces inside the test body. Not a hard rule, but matching the existing convention keeps things consistent. (The skipped test also has 10-space indentation instead of 8 — inconsistent within the file itself.)

JSDoc on the helper: The file-level docblock and per-function docblocks (/** Compile BUG source, deploy... */) are heavier than the rest of the codebase for test helpers. The function signature executeProgram(source, options) is self-documenting. The interface field comments (/** Whether deployment succeeded */) are similarly obvious from the field names. Consider trimming to just the file-level comment or removing entirely.

readUint256 is exported but unused — none of the tests use it. If it's anticipatory, that's fine for a test helper, but worth noting.

Looks good otherwise: functional style (no class), clean interfaces, good test coverage across behavioral categories, double quotes, appropriate use of @ethdebug/evm Executor.

gnidan added 2 commits March 11, 2026 00:01
Copy returnValue via `new Uint8Array()` to avoid
Uint8Array<ArrayBufferLike> vs Uint8Array<ArrayBuffer>
mismatch in TS 5.7+ strict mode.
@github-actions
Copy link
Contributor

github-actions bot commented Mar 11, 2026

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-03-11 04:13 UTC

@gnidan gnidan merged commit 6c5842d into main Mar 11, 2026
4 checks passed
@gnidan gnidan deleted the compiler-behavioral-tests branch March 11, 2026 04:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant