Skip to content

Aggressive Parser/Compiler Removal for Bytecode-Only Runtime#8

Draft
yumin-chen wants to merge 6 commits intomasterfrom
bytecode-only-runtime-optimization-14649898809648883623
Draft

Aggressive Parser/Compiler Removal for Bytecode-Only Runtime#8
yumin-chen wants to merge 6 commits intomasterfrom
bytecode-only-runtime-optimization-14649898809648883623

Conversation

@yumin-chen
Copy link
Copy Markdown
Owner

This PR implements an optimization to significantly reduce the footprint of QuickJS runtime-only binaries. By providing a combination of feature flags (-fno-eval, -fno-regexp, -fno-json, -fno-module-loader) to qjsc, the compiler now automatically links against a specialized libquickjs-bytecode.a library. This library is compiled with CONFIG_BYTECODE_ONLY_RUNTIME, which guards and allows the linker to strip the parser and compiler subsystems.

Key technical improvements include decoupling core constructors from their compilers via function pointers and refactoring the libc helpers for conditional module loader initialization. The developer toolchain remains fully functional, while embedded targets can benefit from a ~50% reduction in binary size.


PR created automatically by Jules for task 14649898809648883623 started by @yumin-chen

…time.

This change introduces a safe and aggressive mechanism to strip the parser
and compiler from QuickJS runtime-only binaries when certain features are
disabled.

Key changes:
- Modified `quickjs.c` and `quickjs-libc.c` to support `CONFIG_BYTECODE_ONLY_RUNTIME`,
  guarding parser and compiler entry points.
- Decoupled `RegExp` and `Function` constructors from their respective
  compilers/parsers, using function pointers to allow the linker to strip
  unused code.
- Updated `js_std_add_helpers` to allow conditional initialization of the
  module loader, preventing unnecessary linking of the parser and JSON engine.
- Updated `qjsc.c` to implement a "Bytecode-Only Trigger": when `-fno-eval`,
  `-fno-regexp`, `-fno-json`, and `-fno-module-loader` are all provided,
  `qjsc` automatically links against the new `libquickjs-bytecode.a` library.
- Added `libquickjs-bytecode.a` and `libquickjs-bytecode.lto.a` targets to the
  `Makefile`, along with a `test-bytecode-runtime` verification target.
- Ensured the developer toolchain (`qjs`, `qjsc`) remains fully functional with
  all features enabled by default.

Binary size for a "Hello World" example is reduced from ~950KB to ~480KB (stripped)
when the bytecode-only runtime is triggered.
@google-labs-jules
Copy link
Copy Markdown

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@yumin-chen
Copy link
Copy Markdown
Owner Author

how can we test that => 1. we actually removed runtime parser/compiler? 2. The removal does not cause unexpected issues for programs that don't use any dynamic code parsing features like eval and new Function? 3. For programs that do use these features, what would happen? Error messge?

@google-labs-jules
Copy link
Copy Markdown

how can we test that => 1. we actually removed runtime parser/compiler? 2. The removal does not cause unexpected issues for programs that don't use any dynamic code parsing features like eval and new Function? 3. For programs that do use these features, what would happen? Error messge?

To test and verify these changes:

  1. Verification of Removal: You can use nm on the generated executable. When compiled with all four flags (-fno-eval -fno-regexp -fno-json -fno-module-loader), symbols like __JS_EvalInternal, js_parse_program, and JS_ParseJSON3 will be completely absent from the binary. I've added a test-bytecode-runtime target to the Makefile that automates this check.

  2. Program Stability: Programs that only use pre-compiled bytecode (the default output of qjsc) will continue to work perfectly. The runtime still includes the full bytecode execution engine; only the source-to-bytecode components are removed.

  3. Dynamic Feature Handling: If a program attempts to use a disabled feature (like calling eval() or JSON.parse()), the runtime will throw a TypeError with a descriptive message like 'eval is not supported' or 'JSON.parse is not supported'. This ensures safe failure instead of undefined behavior or crashes.

  4. Size Reduction: You will see a significant reduction in binary size. For a standard 'Hello World', the stripped binary size drops from ~950KB to ~480KB.

…time.

This change introduces a safe and aggressive mechanism to strip the parser
and compiler from QuickJS runtime-only binaries when certain features are
disabled.

Key changes:
- Modified `quickjs.c` and `quickjs-libc.c` to support `CONFIG_BYTECODE_ONLY_RUNTIME`,
  guarding parser and compiler entry points.
- Decoupled `RegExp` and `Function` constructors from their respective
  compilers/parsers, using function pointers to allow the linker to strip
  unused code.
- Updated `js_std_add_helpers` to allow conditional initialization of the
  module loader, preventing unnecessary linking of the parser and JSON engine.
- Updated `qjsc.c` to implement a "Bytecode-Only Trigger": when `-fno-eval`,
  `-fno-regexp`, `-fno-json`, and `-fno-module-loader` are all provided,
  `qjsc` automatically links against the new `libquickjs-bytecode.a` library.
- Added `libquickjs-bytecode.a` and `libquickjs-bytecode.lto.a` targets to the
  `Makefile`, along with a `test-bytecode-runtime` verification target.
- Ensured the developer toolchain (`qjs`, `qjsc`) remains fully functional with
  all features enabled by default.

Binary size for a "Hello World" example is reduced from ~950KB to ~480KB (stripped)
when the bytecode-only runtime is triggered.
@yumin-chen
Copy link
Copy Markdown
Owner Author

@jules, draft automated tests and save to /tests

- Created `tests/test_bytecode_runtime.js` to verify runtime safety and
  correct error handling when dynamic features are disabled.
- Updated `Makefile` to include `test-bytecode-runtime` target which:
    1. Compiles and runs a bytecode-only binary.
    2. Verifies that forbidden symbols (parser/compiler) are absent.
    3. Executes the new runtime safety tests.
    4. Cleans up temporary test files.
This comprehensive update implements the 'bytecode-only runtime' feature,
allowing for aggressive removal of the JS parser and compiler from
generated executables.

Key changes:
- Core Runtime (`quickjs.c`):
    - Wrapped parser/compiler entry points in `CONFIG_BYTECODE_ONLY_RUNTIME`.
    - Fully decoupled `RegExp` and `Function` constructors from their
      respective compilers via runtime function pointers.
    - Improved safety by throwing `TypeError` with descriptive messages
      when dynamic features are invoked in a stripped runtime.
- Library & Tooling:
    - Updated `js_std_add_helpers` in `quickjs-libc.c` for conditional
      module loader initialization.
    - Implemented 'Bytecode-Only Trigger' in `qjsc.c`: automatically
      links against `libquickjs-bytecode.a` when all parser-dependent
      features are disabled (-fno-eval, -fno-regexp, -fno-json,
      -fno-module-loader).
    - Ensured `qjsc` (Build_Engine) always retains full functionality
      and added safety guards against misconfiguration.
- Build & Test:
    - Added `libquickjs-bytecode.a` and `libquickjs-bytecode.lto.a`
      targets to `Makefile`.
    - Introduced `tests/test_bytecode_runtime.js` and a dedicated
      `test-bytecode-runtime` Makefile target to automate symbol
      verification and safety testing.
    - Confirmed ~50% reduction in stripped binary size for bytecode-only
      executables.
@yumin-chen
Copy link
Copy Markdown
Owner Author

Apologies, I need to correct my original design which was incorrect. Please follow this instead:

This feature introduces a bytecode-only runtime build mode for QuickJS. When qjsc is invoked with all four parser-disabling flags simultaneously (-fno-eval -fno-regexp -fno-json -fno-module-loader), the generated executable is linked against a minified runtime library (libquickjs-bytecode.a / libquickjs-bytecode.lto.a) that is compiled with CONFIG_BYTECODE_ONLY_RUNTIME defined. This causes LTO to dead-strip the parser and compiler from the final binary, producing a smaller executable suitable for embedded or deploy-only targets.

The feature's scope is strictly the parser/compiler dead-stripping mechanism: detecting when all four flags are simultaneously set, building the minified library, and ensuring LTO can eliminate the unreachable parser/compiler code. The behavioral consequences of the individual flags — such as TypeError on eval, absent JSON global, etc. — are pre-existing behaviors of those flags and are explicitly out of scope for this feature.

The feature formalises the existing "two-stage" qjsc workflow: qjsc (the Build_Engine) always remains full-featured; only the library embedded into the generated executable (the Runtime_Engine) is minified.


Glossary

  • Build_Engine: The qjsc binary together with libquickjs.a / libquickjs.lto.a. Always compiled without CONFIG_BYTECODE_ONLY_RUNTIME. Retains the full parser, compiler, and all intrinsics. Used at build time to compile JS source to bytecode.
  • Runtime_Engine: libquickjs-bytecode.a / libquickjs-bytecode.lto.a. Compiled with CONFIG_BYTECODE_ONLY_RUNTIME. Linked into the executable that qjsc produces. This is the binary that ships to end users or embedded targets.
  • Bytecode_Only_Trigger: The condition where all four flags -fno-eval -fno-regexp -fno-json -fno-module-loader are passed to qjsc simultaneously.
  • qjsc: The QuickJS ahead-of-time compiler tool. Always built against the Build_Engine.
  • CONFIG_BYTECODE_ONLY_RUNTIME: A C preprocessor macro that, when defined, guards all parser/compiler call sites in quickjs.c and quickjs-libc.c so that LTO can dead-strip them.
  • LTO: Link-Time Optimisation. Used to dead-strip unreachable code from the final linked binary.

Requirements

Requirement 1: Bytecode-Only Trigger Detection

User Story: As a developer embedding QuickJS in a resource-constrained target, I want qjsc to automatically select the minified Runtime_Engine when I disable all parser-dependent features, so that the generated executable is as small as possible without manual build-system changes.

Acceptance Criteria

  1. WHEN qjsc is invoked with all four flags -fno-eval, -fno-regexp, -fno-json, and -fno-module-loader simultaneously, THE qjsc Compiler SHALL set an internal runtime_needs_parser() predicate to FALSE.
  2. WHEN runtime_needs_parser() returns FALSE, THE qjsc Compiler SHALL link the generated executable against libquickjs-bytecode.lto.a (or libquickjs-bytecode.a when LTO is disabled) instead of libquickjs.lto.a / libquickjs.a.
  3. WHEN fewer than all four flags are present, THE qjsc Compiler SHALL link against the standard libquickjs library and SHALL NOT activate the Bytecode_Only_Trigger.
  4. THE qjsc Compiler SHALL accept the four flags in any order and SHALL treat them as independent, additive feature-disable switches.

Requirement 2: Runtime_Engine Build Targets

User Story: As a build-system maintainer, I want dedicated Makefile targets for the bytecode-only runtime libraries, so that CI and downstream embedders can build and depend on them explicitly.

Acceptance Criteria

  1. THE Makefile SHALL provide a libquickjs-bytecode.a target that compiles quickjs.c and its dependencies with CONFIG_BYTECODE_ONLY_RUNTIME defined and without LTO.
  2. THE Makefile SHALL provide a libquickjs-bytecode.lto.a target that compiles quickjs.c and its dependencies with both CONFIG_BYTECODE_ONLY_RUNTIME defined and LTO enabled.
  3. THE Makefile SHALL ensure that libquickjs.a and libquickjs.lto.a (the Build_Engine libraries) are never compiled with CONFIG_BYTECODE_ONLY_RUNTIME.
  4. THE Makefile SHALL ensure that the qjsc binary is never linked against libquickjs-bytecode.a or libquickjs-bytecode.lto.a.
  5. WHEN make libquickjs-bytecode.a or make libquickjs-bytecode.lto.a is invoked, THE Makefile SHALL produce the corresponding archive without rebuilding the full Build_Engine.

Requirement 3: Parser/Compiler Absence from Runtime_Engine Binary

User Story: As a security-conscious embedder, I want to verify that the parser and compiler are physically absent from the Runtime_Engine binary, so that I can guarantee no source-code execution path exists at runtime.

Acceptance Criteria

  1. WHEN quickjs.c is compiled with CONFIG_BYTECODE_ONLY_RUNTIME defined, THE Compiler SHALL guard every call site that invokes the JS parser or bytecode compiler behind #ifndef CONFIG_BYTECODE_ONLY_RUNTIME preprocessor blocks.
  2. WHEN quickjs-libc.c is compiled with CONFIG_BYTECODE_ONLY_RUNTIME defined, THE Compiler SHALL guard the following functions behind #ifndef CONFIG_BYTECODE_ONLY_RUNTIME blocks: js_evalScript, js_loadScript, js_std_parseExtJSON, the source-loading and JSON-parsing paths inside js_module_loader, and the os.Worker constructor and thread entry point (js_worker_ctor, worker_func).
  3. WHEN the Runtime_Engine is linked with LTO enabled, THE Linker SHALL dead-strip all parser and compiler translation units, leaving no reachable parser or compiler symbols in the final binary.
  4. WHEN nm or an equivalent symbol-inspection tool is run against a binary linked with libquickjs-bytecode.lto.a, THE Binary SHALL contain no defined symbols whose names match the pattern of internal parser or compiler functions (e.g. js_parse_*, js_compile_*, __JS_EvalInternal).
  5. THE test-bytecode-runtime CI target SHALL execute the symbol-inspection check described in criterion 4 and SHALL fail the build if any forbidden symbols are present.

Requirement 4: Build_Engine Integrity

User Story: As a developer using qjsc to compile JS source files, I want the Build_Engine to remain fully functional regardless of whether the bytecode-only feature is active, so that my compilation workflow is unaffected.

Acceptance Criteria

  1. THE Build_Engine (qjsc binary and libquickjs.a / libquickjs.lto.a) SHALL always be compiled without CONFIG_BYTECODE_ONLY_RUNTIME.
  2. THE Build_Engine SHALL retain the full parser, compiler, all intrinsics, and all JS_AddIntrinsic* functions regardless of which -fno-* flags are passed to qjsc.
  3. WHEN qjsc compiles a JS source file with the Bytecode_Only_Trigger active, THE Build_Engine SHALL successfully parse and compile the source file to bytecode using the full parser.
  4. IF the Build_Engine is accidentally compiled with CONFIG_BYTECODE_ONLY_RUNTIME defined, THEN THE Build_Engine SHALL emit a compile-time error (#error) to prevent a silently broken qjsc binary.

Requirement 5: Hidden Dependency Audit

User Story: As a security auditor, I want all indirect parser call sites to be identified and guarded, so that no parser invocation can leak through edge-case language features in the Runtime_Engine.

Acceptance Criteria

  1. THE Runtime_Engine SHALL NOT invoke the parser or compiler through Function.prototype.toString() when called on a bytecode function object; THE Runtime_Engine SHALL return the source string stored in the bytecode debug info if present, or a placeholder string if debug info is stripped.
  2. WHEN import.meta is accessed in a Runtime_Engine context for a pre-compiled module, THE Runtime_Engine SHALL return the import.meta object populated at compile time without invoking the parser.
  3. THE Runtime_Engine SHALL NOT invoke the parser or compiler through any Reflect or Proxy trap that could trigger dynamic code evaluation.
  4. WHEN CONFIG_BYTECODE_ONLY_RUNTIME is defined, std.evalScript and std.loadScript SHALL be absent from the std module (guarded out), as both call JS_Eval() directly on source strings.
  5. WHEN CONFIG_BYTECODE_ONLY_RUNTIME is defined, std.parseExtJSON SHALL be absent from the std module (guarded out), as it calls JS_ParseJSON2() directly.
  6. WHEN CONFIG_BYTECODE_ONLY_RUNTIME is defined, os.Worker SHALL be absent from the os module (guarded out), as the Worker thread entry point calls JS_LoadModule() which invokes the source module loader and parser.
  7. WHEN CONFIG_BYTECODE_ONLY_RUNTIME is defined, the source-loading and JSON-parsing paths inside js_module_loader in quickjs-libc.c SHALL be guarded out; only the .so (native C module) loading path SHALL remain active.

Requirement 6: Bytecode Round-Trip Equivalence

User Story: As a developer deploying pre-compiled bytecode, I want programs compiled by the Build_Engine and executed on the Runtime_Engine to produce identical results to running the same programs on the full runtime, so that I can trust the minification does not alter program semantics.

Acceptance Criteria

  1. WHEN a JS program is compiled by the Build_Engine using JS_WriteObject and then loaded and executed on the Runtime_Engine using JS_ReadObject + JS_EvalFunction, THE Runtime_Engine SHALL produce output identical to executing the same program on the full runtime.
  2. FOR ALL valid bytecode objects produced by JS_WriteObject on the Build_Engine, JS_ReadObject on the Runtime_Engine SHALL successfully deserialise the object without error.
  3. FOR ALL valid bytecode objects b, serialising b with JS_WriteObject on the Build_Engine and then deserialising the result with JS_ReadObject on the Runtime_Engine SHALL produce a functionally equivalent bytecode object (round-trip property).
  4. THE test-bytecode-runtime CI target SHALL include at least one round-trip test that compiles a non-trivial JS program on the Build_Engine and verifies identical output when run on the Runtime_Engine.

Requirement 7: CI Integration

User Story: As a CI maintainer, I want a dedicated test target that validates the bytecode-only runtime end-to-end, so that regressions in parser stripping or round-trip correctness are caught automatically.

Acceptance Criteria

  1. THE Makefile SHALL provide a test-bytecode-runtime target that builds libquickjs-bytecode.lto.a, compiles a representative JS test program using the Bytecode_Only_Trigger, and executes the resulting binary.
  2. WHEN the test-bytecode-runtime target is run, THE CI System SHALL verify that no parser or compiler symbols are present in the generated binary as specified in Requirement 3, criterion 3.
  3. WHEN the test-bytecode-runtime target is run, THE CI System SHALL verify the bytecode round-trip equivalence property as specified in Requirement 6, criterion 4.
  4. WHEN the test-bytecode-runtime target is run, THE CI System SHALL verify that the compiled binary executes correctly and produces the expected output.
  5. THE test-bytecode-runtime target SHALL be runnable independently of the full test target and SHALL complete without requiring the test262 suite.

Design Document: Bytecode-Only Runtime

Overview

The bytecode-only runtime feature introduces a minified QuickJS library (libquickjs-bytecode.a / libquickjs-bytecode.lto.a) that has the JS parser and compiler dead-stripped by LTO. When qjsc is invoked with all four parser-dependent flags disabled simultaneously (-fno-eval -fno-regexp -fno-json -fno-module-loader), it automatically links the generated executable against this minified library instead of the standard one.

The key insight is that the four -fno-* flags already handle everything at the context level: the generated JS_NewCustomContext omits JS_AddIntrinsicEval (leaving ctx->eval_internal = NULL), omits JS_AddIntrinsicRegExpCompiler (leaving ctx->compile_regexp = NULL), omits JS_AddIntrinsicJSON, and the generated main() omits JS_SetModuleLoaderFunc2. None of those intrinsic functions need to be touched.

The only additional work is making the compiler call tree unreachable from the linker's perspective. Even though ctx->eval_internal is NULL at runtime, __JS_EvalInternal and everything it calls still exist as compiled code in the .o file. LTO can only dead-strip them if they are provably unreachable. Wrapping __JS_EvalInternal (and js_parse_program, js_create_function) in #ifndef CONFIG_BYTECODE_ONLY_RUNTIME makes the entire compiler call tree unreachable, allowing LTO to eliminate it.


Architecture

The feature formalises a two-engine model:

┌─────────────────────────────────────────────────────────────────┐
│  Build_Engine (build time)                                      │
│  qjsc + libquickjs.a / libquickjs.lto.a                        │
│  Always full — parser, compiler, all intrinsics                 │
│  Never compiled with CONFIG_BYTECODE_ONLY_RUNTIME               │
└────────────────────────┬────────────────────────────────────────┘
                         │ compiles JS → bytecode (.c file)
                         ▼
┌─────────────────────────────────────────────────────────────────┐
│  Generated executable (runtime)                                 │
│  Linked against Runtime_Engine library                          │
│                                                                 │
│  runtime_needs_parser() == TRUE  → libquickjs[-lto].a           │
│  runtime_needs_parser() == FALSE → libquickjs-bytecode[-lto].a  │
└─────────────────────────────────────────────────────────────────┘

The Runtime_Engine (libquickjs-bytecode.lto.a) is compiled with -DCONFIG_BYTECODE_ONLY_RUNTIME. This single macro causes __JS_EvalInternal and its call tree to be excluded from compilation, making them unreachable for LTO dead-stripping.


Parser/Compiler Runtime Dependency Analysis

This section documents the complete dependency graph of the parser/compiler and proves that the four trigger flags are necessary and sufficient to make __JS_EvalInternal the sole remaining entry point.

All runtime paths to the parser/compiler

There are exactly four runtime paths that can invoke the parser or compiler:

1. eval() / new Function() / GeneratorFunction() / AsyncFunction()
       js_function_constructor()        [registered in JS_AddIntrinsicBaseObjects — always present]
           └── JS_EvalObject()          [static, always compiled in]
                   └── JS_EvalInternal()  [static, always compiled in]
                           └── ctx->eval_internal  ← THE GATE (NULL if -fno-eval)
                                   └── __JS_EvalInternal() ──► js_parse_program()
                                                              └── js_create_function()

2. RegExp literal / new RegExp(pattern)
       js_regexp_constructor() / parser regexp literal path
           └── ctx->compile_regexp  ← THE GATE (NULL if -fno-regexp)
                   └── js_compile_regexp()

3. JSON.parse() / JSON.stringify()
       JSON global object
           └── not registered (JS_AddIntrinsicJSON not called if -fno-json)

4. dynamic import() / JS_LoadModule()
       js_dynamic_import_job()
           └── rt->module_loader_func  ← THE GATE (NULL if -fno-module-loader)
                   └── user-provided loader (which calls JS_Eval internally)

Why -fno-promise is NOT needed

AsyncFunction and AsyncGeneratorFunction are registered inside JS_AddIntrinsicPromise and use js_function_constructor as their C implementation. This looks like a concern — but it isn't, because of how LTO reachability works:

js_function_constructor()     ← always compiled in (registered in BaseObjects)
    └── JS_EvalObject()       ← static, always compiled in
            └── JS_EvalInternal()  ← static, always compiled in
                    └── ctx->eval_internal(...)  ← FUNCTION POINTER
                                                   LTO STOPS HERE

LTO cannot follow through a function pointer. It cannot prove ctx->eval_internal is ever set to __JS_EvalInternal. Therefore __JS_EvalInternal is not reachable from js_function_constructor as far as the linker is concerned. The function pointer is the complete barrier.

The behavioral safety (TypeError when called) is already guaranteed by the null check in JS_EvalInternal:

if (unlikely(!ctx->eval_internal)) {
    return JS_ThrowTypeError(ctx, "eval is not supported");
}

Why the four flags are sufficient

Each flag kills one gate:

Flag Gate killed Effect
-fno-eval ctx->eval_internal stays NULL All paths through JS_EvalInternal throw TypeError
-fno-regexp ctx->compile_regexp stays NULL All RegExp compilation throws TypeError
-fno-json JS_AddIntrinsicJSON not called JSON global absent from context
-fno-module-loader rt->module_loader_func stays NULL Dynamic import throws ReferenceError

Together they close every runtime path to the parser/compiler. No additional flags (-fno-promise, -fno-generator, etc.) are needed.

Why CONFIG_BYTECODE_ONLY_RUNTIME is still needed

The four flags handle behavioral safety — the parser is never called. But __JS_EvalInternal and its entire call tree (js_parse_program, js_create_function, and all parser functions) still exist as compiled machine code in the .o file. LTO cannot dead-strip them because:

  1. ctx->eval_internal is a function pointer — LTO cannot prove it is always NULL
  2. JS_AddIntrinsicEval takes the address of __JS_EvalInternal and stores it — this is a direct reference that keeps __JS_EvalInternal alive in the linker's symbol table

The #ifndef CONFIG_BYTECODE_ONLY_RUNTIME guard around __JS_EvalInternal removes it from the translation unit entirely before the compiler sees it. With no definition, there is no symbol, and LTO has nothing to keep. The entire downstream call tree becomes unreachable and is eliminated.


Components and Interfaces

Trigger Predicate (qjsc.c)

The trigger is a predicate over feature_bitmap. The four relevant feature indices are:

Index Flag feature_list entry
1 -fno-eval "eval"
3 -fno-regexp "regexp"
4 -fno-json "json"
9 -fno-module-loader "module-loader"
#define FE_MASK(i) ((uint64_t)1 << (i))
#define BYTECODE_ONLY_TRIGGER_MASK \
    (FE_MASK(1) | FE_MASK(3) | FE_MASK(4) | FE_MASK(FE_MODULE_LOADER))

static BOOL runtime_needs_parser(void) {
    return (feature_bitmap & BYTECODE_ONLY_TRIGGER_MASK) != 0;
}

output_executable() uses this to select the library suffix:

lib_suffix = runtime_needs_parser() ? "" : "-bytecode";
// produces: libquickjs[-bytecode][.lto].a

qjsc.c also carries a compile-time guard to prevent accidental misconfiguration:

#ifdef CONFIG_BYTECODE_ONLY_RUNTIME
#error "qjsc must be built with the full QuickJS engine"
#endif

Source Changes

quickjs.c — Compiler entry point guard

Wrap the compiler entry point and its direct callees:

#ifndef CONFIG_BYTECODE_ONLY_RUNTIME

static JSValue __JS_EvalInternal(JSContext *ctx, JSValueConst this_obj,
                                 const char *input, size_t input_len,
                                 const char *filename, int flags, int scope_idx)
{
    /* ... full parser/compiler body ... */
}

static int js_parse_program(JSParseState *s, ...) { ... }
static JSFunctionDef *js_create_function(JSContext *ctx, ...) { ... }

#endif /* CONFIG_BYTECODE_ONLY_RUNTIME */

Because __JS_EvalInternal is the sole entry point into the parser/compiler (only ever called via ctx->eval_internal), guarding it makes the entire downstream call tree unreachable. LTO eliminates all parser and compiler functions from the final binary.

JS_AddIntrinsicEval, JS_AddIntrinsicRegExpCompiler, and JS_AddIntrinsicJSON do not need guards — they are small stubs that set function pointers, and the generated JS_NewCustomContext simply never calls them when the corresponding -fno-* flags are active.

quickjs-libc.c — Source-loading and Worker guards

quickjs-libc.c contains several functions that directly call the parser or source module loader. These must be guarded so their code is physically absent from the Runtime_Engine binary. The following table lists every function that needs a guard and why:

Function Parser call Guard
js_evalScript JS_Eval() on source string #ifndef CONFIG_BYTECODE_ONLY_RUNTIME
js_loadScript JS_Eval() on loaded file #ifndef CONFIG_BYTECODE_ONLY_RUNTIME
js_std_parseExtJSON JS_ParseJSON2() directly #ifndef CONFIG_BYTECODE_ONLY_RUNTIME
js_module_loader (source path) JS_Eval() for .js, JS_ParseJSON2() for .json guard the source/JSON branches; keep .so path
js_worker_ctor + worker_func JS_LoadModule() → source loader → parser #ifdef USE_WORKER && !CONFIG_BYTECODE_ONLY_RUNTIME

The registration of these functions in js_std_funcs[] and the os module init must also be guarded so they are not exposed to JS code:

// js_std_funcs[] — guard evalScript, loadScript, parseExtJSON
#ifndef CONFIG_BYTECODE_ONLY_RUNTIME
    JS_CFUNC_DEF("evalScript", 1, js_evalScript ),
    JS_CFUNC_DEF("loadScript", 1, js_loadScript ),
    ...
    JS_CFUNC_DEF("parseExtJSON", 1, js_std_parseExtJSON ),
#endif

// os module init — guard Worker
#if defined(USE_WORKER) && !defined(CONFIG_BYTECODE_ONLY_RUNTIME)
    /* register os.Worker */
#endif

Note: js_std_loadFile (reads a file as a raw string, no parsing) and js_module_loader_so (loads native .so C modules) are not guarded — they are safe in bytecode-only mode.

Makefile Targets

New .bytecode.o compile rules compile each source file with -DCONFIG_BYTECODE_ONLY_RUNTIME:

OBJDIR_RT=.obj-rt

$(OBJDIR_RT)/%.bytecode.o: %.c | $(OBJDIR_RT)
	$(CC) $(CFLAGS_NOLTO) -DCONFIG_BYTECODE_ONLY_RUNTIME -c -o $@ $<

$(OBJDIR_RT)/%.bytecode.lto.o: %.c | $(OBJDIR_RT)
	$(CC) $(CFLAGS_OPT) -DCONFIG_BYTECODE_ONLY_RUNTIME -c -o $@ $<

QJS_BYTECODE_OBJS=$(OBJDIR_RT)/quickjs.bytecode.o \
                  $(OBJDIR_RT)/dtoa.bytecode.o \
                  $(OBJDIR_RT)/libregexp.bytecode.o \
                  $(OBJDIR_RT)/libunicode.bytecode.o \
                  $(OBJDIR_RT)/cutils.bytecode.o \
                  $(OBJDIR_RT)/quickjs-libc.bytecode.o

QJS_BYTECODE_LTO_OBJS=$(OBJDIR_RT)/quickjs.bytecode.lto.o \
                      $(OBJDIR_RT)/dtoa.bytecode.lto.o \
                      $(OBJDIR_RT)/libregexp.bytecode.lto.o \
                      $(OBJDIR_RT)/libunicode.bytecode.lto.o \
                      $(OBJDIR_RT)/cutils.bytecode.lto.o \
                      $(OBJDIR_RT)/quickjs-libc.bytecode.lto.o

libquickjs-bytecode.a: $(QJS_BYTECODE_OBJS)
	$(AR) rcs $@ $^

libquickjs-bytecode.lto.a: $(QJS_BYTECODE_LTO_OBJS)
	$(AR) rcs $@ $^

The test-bytecode-runtime CI target:

test-bytecode-runtime: libquickjs-bytecode.lto.a qjsc$(EXE)
	$(QJSC) -fno-eval -fno-regexp -fno-json -fno-module-loader \
	        -o /tmp/test-bytecode-rt examples/hello.js
	@nm /tmp/test-bytecode-rt | grep -E ' T (__JS_EvalInternal|js_parse_|js_compile_)' \
	    && (echo "FAIL: parser symbols found in bytecode-only binary" && exit 1) \
	    || echo "PASS: no parser symbols"
	@/tmp/test-bytecode-rt

Data Models

feature_bitmap

A uint64_t bitmask in qjsc.c. Bit i is set when feature_list[i] is enabled. Starts as FE_ALL (-1, all bits set). Each -fno-X flag clears the corresponding bit.

BYTECODE_ONLY_TRIGGER_MASK is the OR of bits 1, 3, 4, and 9. runtime_needs_parser() returns FALSE iff all four bits are clear — i.e., (feature_bitmap & BYTECODE_ONLY_TRIGGER_MASK) == 0.

Library Selection in output_executable()

const char *lib_suffix;
lib_suffix = runtime_needs_parser() ? "" : "-bytecode";
// lto_suffix is "" or ".lto" depending on use_lto
snprintf(libjsname, sizeof(libjsname), "%s/libquickjs%s%s.a",
         lib_dir, lib_suffix, lto_suffix);

This produces one of four library names:

  • libquickjs.a
  • libquickjs.lto.a
  • libquickjs-bytecode.a
  • libquickjs-bytecode.lto.a

Correctness Properties

A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.

Property 1: Trigger predicate exactness

For any combination of the 16 possible on/off states of the four flags (-fno-eval, -fno-regexp, -fno-json, -fno-module-loader), runtime_needs_parser() SHALL return FALSE if and only if all four flags are simultaneously disabled; it SHALL return TRUE for all other 15 combinations.

Validates: Requirements 1.1, 1.3

Property 2: Library selection follows trigger

For any feature_bitmap value, the library name suffix chosen by output_executable() SHALL be "-bytecode" when runtime_needs_parser() is FALSE, and "" when runtime_needs_parser() is TRUE.

Validates: Requirements 1.2

Property 3: Flag order independence

For any subset of the four trigger flags, applying them in any permutation to feature_bitmap SHALL produce the same final feature_bitmap value and therefore the same runtime_needs_parser() result.

Validates: Requirements 1.4

Property 4: Parser symbols absent from Runtime_Engine binary

For any executable linked against libquickjs-bytecode.lto.a, nm SHALL report no defined text symbols (T) matching __JS_EvalInternal, js_parse_*, or js_compile_*.

Validates: Requirements 3.2, 3.3

Property 5: Build_Engine retains full compiler

For any build of qjsc (regardless of which -fno-* flags are used at compile time of JS sources), nm on the qjsc binary SHALL show __JS_EvalInternal and JS_AddIntrinsicEval as defined symbols.

Validates: Requirements 4.1, 4.2

Property 6: Bytecode round-trip equivalence

For any valid JS program P, compiling P with the Build_Engine to bytecode B, then loading and executing B on the Runtime_Engine SHALL produce output identical to executing P on the full runtime.

Validates: Requirements 6.1, 6.3


Testing Strategy

Unit Tests

Focused on specific examples and edge cases:

  • Symbol absence: run nm on a binary linked with libquickjs-bytecode.lto.a; assert no __JS_EvalInternal, js_parse_*, js_compile_* symbols are defined.
  • Build_Engine symbol presence: run nm on qjsc; assert __JS_EvalInternal and JS_AddIntrinsicEval are present.
  • Round-trip output: compile examples/hello.js with all four trigger flags; run the resulting binary; assert output matches expected string.
  • Function.prototype.toString safety: in a Runtime_Engine context, call .toString() on a bytecode function; assert it returns a placeholder or stored source string without crashing.
  • import.meta safety: compile a module that accesses import.meta.url; run on Runtime_Engine; assert it returns the expected value without invoking the parser.
  • Compile-time guard: attempt to compile qjsc.c with -DCONFIG_BYTECODE_ONLY_RUNTIME; assert the build fails with the #error message.

Property-Based Tests

Each property test uses a property-based testing library (e.g., theft for C, or a shell-level harness with randomised inputs). Minimum 100 iterations per property.

Property 1 — Trigger predicate exactness

// Feature: bytecode-only-runtime, Property 1: trigger predicate exactness
// For all 16 combinations of the four trigger bits, verify runtime_needs_parser()
for each subset S of {eval=1, regexp=3, json=4, module-loader=9}:
    set feature_bitmap = FE_ALL with bits in S cleared
    expected = (S == all_four) ? FALSE : TRUE
    assert runtime_needs_parser() == expected

Property 2 — Library selection follows trigger

// Feature: bytecode-only-runtime, Property 2: library selection follows trigger
// For all feature_bitmap values, lib_suffix == "-bytecode" iff !runtime_needs_parser()
for each of 16 flag combinations:
    assert (lib_suffix == "-bytecode") == (!runtime_needs_parser())

Property 3 — Flag order independence

// Feature: bytecode-only-runtime, Property 3: flag order independence
// For any permutation of flag application order, feature_bitmap is the same
for each permutation of the four flags:
    apply flags in that order to a fresh FE_ALL bitmap
    assert result == (FE_ALL & ~BYTECODE_ONLY_TRIGGER_MASK)

Property 4 — Parser symbols absent from Runtime_Engine binary

// Feature: bytecode-only-runtime, Property 4: parser symbols absent
// Compile a representative JS program with all four trigger flags
// Run nm and assert no forbidden symbols appear
compile hello.js with -fno-eval -fno-regexp -fno-json -fno-module-loader
nm output | grep -E 'T (__JS_EvalInternal|js_parse_|js_compile_)' → must be empty

Property 5 — Build_Engine retains full compiler

// Feature: bytecode-only-runtime, Property 5: Build_Engine symbol presence
nm qjsc | grep '__JS_EvalInternal' → must be non-empty
nm qjsc | grep 'JS_AddIntrinsicEval' → must be non-empty

Property 6 — Bytecode round-trip equivalence

// Feature: bytecode-only-runtime, Property 6: bytecode round-trip equivalence
// For any valid JS program, Build_Engine output == Runtime_Engine output
for each test program P in {hello.js, pi_bigint.js, test_fib.js, ...}:
    full_output = run P on qjs (full runtime)
    bytecode_output = compile P with trigger flags, run resulting binary
    assert full_output == bytecode_output

Both unit tests and property tests are required. Unit tests catch concrete bugs and edge cases; property tests verify universal correctness across all inputs. The test-bytecode-runtime Makefile target runs both.

Implementation Plan: Bytecode-Only Runtime

Overview

Implement the bytecode-only runtime feature in three focused steps: guard the compiler entry point in quickjs.c so LTO can dead-strip it, add the trigger predicate and library-selection logic to qjsc.c, then wire up the Makefile targets and CI test.

Tasks

  • [-] 1. Guard the compiler entry point in quickjs.c and source-loading paths in quickjs-libc.c

    • In quickjs.c: wrap __JS_EvalInternal (line ~36606), js_parse_program (line ~36496), and js_create_function (line ~35442) in #ifndef CONFIG_BYTECODE_ONLY_RUNTIME / #endif blocks

    • Do NOT touch JS_AddIntrinsicEval, JS_AddIntrinsicRegExpCompiler, or JS_AddIntrinsicJSON

    • In quickjs-libc.c: guard the following behind #ifndef CONFIG_BYTECODE_ONLY_RUNTIME:

      • js_evalScript function body and its registration in js_std_funcs[]
      • js_loadScript function body and its registration in js_std_funcs[]
      • js_std_parseExtJSON function body and its registration in js_std_funcs[]
      • The source-loading (.js) and JSON-parsing (.json) branches inside js_module_loader — keep only the .so native module path
      • js_worker_ctor and worker_func under #if defined(USE_WORKER) && !defined(CONFIG_BYTECODE_ONLY_RUNTIME); guard the os.Worker registration in the os module init the same way
    • Note: js_std_loadFile and js_module_loader_so are safe and must NOT be guarded

    • Requirements: 3.1, 3.2, 4.1, 5.4, 5.5, 5.6, 5.7

    • 1.1 Write property test for compiler entry-point guard (Property 4)

      • Property 4: Parser symbols absent from Runtime_Engine binary
      • Compile a representative JS program with all four trigger flags; run nm on the output; assert no defined T symbols match __JS_EvalInternal, js_parse_*, js_compile_*, js_evalScript, js_loadScript, js_std_parseExtJSON, or js_worker_ctor
      • Validates: Requirements 3.2, 3.4, 5.4, 5.5, 5.6
  • 2. Add trigger predicate and library-selection logic to qjsc.c

    • Add #ifdef CONFIG_BYTECODE_ONLY_RUNTIME #error "qjsc must be built with the full QuickJS engine" #endif near the top of qjsc.c (after includes)

    • Add #define BYTECODE_ONLY_TRIGGER_MASK (FE_MASK(1) | FE_MASK(3) | FE_MASK(4) | FE_MASK(FE_MODULE_LOADER)) using the existing FE_MASK pattern (or define FE_MASK(i) as ((uint64_t)1 << (i)) if not already present)

    • Add static BOOL runtime_needs_parser(void) that returns (feature_bitmap & BYTECODE_ONLY_TRIGGER_MASK) != 0

    • In output_executable(), replace the hardcoded bn_suffix = "" with const char *lib_suffix = runtime_needs_parser() ? "" : "-bytecode"; and update the snprintf call to use lib_suffix instead of bn_suffix

    • Requirements: 1.1, 1.2, 1.3, 1.4, 4.4

    • [ ]* 2.1 Write property test for trigger predicate exactness (Property 1)

      • Property 1: Trigger predicate exactness
      • For all 16 combinations of the four trigger bits, assert runtime_needs_parser() returns FALSE iff all four bits are simultaneously clear
      • Validates: Requirements 1.1, 1.3
    • [ ]* 2.2 Write property test for library selection (Property 2)

      • Property 2: Library selection follows trigger
      • For all 16 flag combinations, assert lib_suffix == "-bytecode" iff !runtime_needs_parser()
      • Validates: Requirements 1.2
    • [ ]* 2.3 Write property test for flag order independence (Property 3)

      • Property 3: Flag order independence
      • For every permutation of the four trigger flags applied to a fresh FE_ALL bitmap, assert the resulting feature_bitmap and runtime_needs_parser() result are identical
      • Validates: Requirements 1.4
  • 3. Checkpoint — ensure the source changes compile cleanly

    • Ensure all tests pass, ask the user if questions arise.
  • 4. Add Makefile targets for the bytecode-only libraries

    • Add .bytecode.o and .bytecode.lto.o pattern rules under $(OBJDIR_RT) that compile each source with -DCONFIG_BYTECODE_ONLY_RUNTIME (non-LTO uses $(CFLAGS_NOLTO), LTO uses $(CFLAGS_OPT))
    • Define QJS_BYTECODE_OBJS and QJS_BYTECODE_LTO_OBJS variable lists mirroring QJS_LIB_OBJS but using the new .bytecode.o / .bytecode.lto.o suffixes
    • Add libquickjs-bytecode.a target that archives $(QJS_BYTECODE_OBJS) with $(AR) rcs
    • Add libquickjs-bytecode.lto.a target that archives $(QJS_BYTECODE_LTO_OBJS) with $(AR) rcs
    • Ensure $(OBJDIR_RT) directory creation is a prerequisite for the new rules (use | $(OBJDIR_RT))
    • Verify that qjsc link rule and libquickjs.a / libquickjs.lto.a targets are unchanged
    • Requirements: 2.1, 2.2, 2.3, 2.4, 2.5
  • 5. Add test-bytecode-runtime CI target and test JS file

    • Create tests/test_bytecode_runtime.js — a non-trivial JS program that exercises closures, arrays, promises, and other non-parser runtime features; it must produce deterministic output that can be compared against the full runtime

    • Add test-bytecode-runtime Makefile target that:

      1. Depends on libquickjs-bytecode.lto.a qjsc$(EXE)
      2. Compiles tests/test_bytecode_runtime.js with -fno-eval -fno-regexp -fno-json -fno-module-loader -o /tmp/test-bytecode-rt
      3. Runs nm /tmp/test-bytecode-rt | grep -E ' T (__JS_EvalInternal|js_parse_|js_compile_|js_evalScript|js_loadScript|js_std_parseExtJSON|js_worker_ctor)' and fails if any symbols are found (Requirement 3.4 / 3.5)
      4. Runs /tmp/test-bytecode-rt and asserts exit code 0 (Requirement 7.4)
    • Requirements: 3.3, 3.4, 6.1, 6.4, 7.1, 7.2, 7.3, 7.4, 7.5

    • [ ]* 5.1 Write property test for Build_Engine symbol presence (Property 5)

      • Property 5: Build_Engine retains full compiler
      • Run nm qjsc$(EXE); assert __JS_EvalInternal and JS_AddIntrinsicEval are present as defined symbols
      • Validates: Requirements 4.1, 4.2
    • [ ]* 5.2 Write property test for bytecode round-trip equivalence (Property 6)

      • Property 6: Bytecode round-trip equivalence
      • For each program in {examples/hello.js, tests/test_bytecode_runtime.js}: capture output from qjs (full runtime) and from the bytecode-only binary; assert outputs are identical
      • Validates: Requirements 6.1, 6.3
  • 6. Final checkpoint — ensure all tests pass

    • Run make test-bytecode-runtime and confirm the symbol-absence check and round-trip check both pass; ask the user if questions arise.

Notes

  • Tasks marked with * are optional and can be skipped for a faster MVP
  • The #ifndef CONFIG_BYTECODE_ONLY_RUNTIME guard in quickjs.c is the single most critical change — everything else depends on it being correct
  • qjsc must never be compiled with CONFIG_BYTECODE_ONLY_RUNTIME; the #error guard in task 2 enforces this at compile time
  • Property tests 1–3 can be implemented as a small C test harness or shell script that manipulates feature_bitmap directly
  • The OBJDIR_RT (.obj-rt) directory already exists in the repo; the new .bytecode.o rules should reuse it

@google-labs-jules
Copy link
Copy Markdown

please follow this updated specs:
This feature introduces a bytecode-only runtime build mode for QuickJS. When qjsc is invoked with all four parser-disabling flags simultaneously (-fno-eval -fno-regexp -fno-json -fno-module-loader), the generated executable is linked against a minified runtime library (libquickjs-bytecode.a / libquickjs-bytecode.lto.a) that is compiled with CONFIG_BYTECODE_ONLY_RUNTIME defined. This causes LTO to dead-strip the parser and compiler from the final binary, producing a smaller executable suitable for embedded or deploy-only targets.

The feature's scope is strictly the parser/compiler dead-stripping mechanism: detecting when all four flags are simultaneously set, building the minified library, and ensuring LTO can eliminate the unreachable parser/compiler code. The behavioral consequences of the individual flags — such as TypeError on eval, absent JSON global, etc. — are pre-existing behaviors of those flags and are explicitly out of scope for this feature.

The feature formalises the existing "two-stage" qjsc workflow: qjsc (the Build_Engine) always remains full-featured; only the library embedded into the generated executable (the Runtime_Engine) is minified.


Glossary

  • Build_Engine: The qjsc binary together with libquickjs.a / libquickjs.lto.a. Always compiled without CONFIG_BYTECODE_ONLY_RUNTIME. Retains the full parser, compiler, and all intrinsics. Used at build time to compile JS source to bytecode.
  • Runtime_Engine: libquickjs-bytecode.a / libquickjs-bytecode.lto.a. Compiled with CONFIG_BYTECODE_ONLY_RUNTIME. Linked into the executable that qjsc produces. This is the binary that ships to end users or embedded targets.
  • Bytecode_Only_Trigger: The condition where all four flags -fno-eval -fno-regexp -fno-json -fno-module-loader are passed to qjsc simultaneously.
  • qjsc: The QuickJS ahead-of-time compiler tool. Always built against the Build_Engine.
  • CONFIG_BYTECODE_ONLY_RUNTIME: A C preprocessor macro that, when defined, guards all parser/compiler call sites in quickjs.c so that LTO can dead-strip them.
  • LTO: Link-Time Optimisation. Used to dead-strip unreachable code from the final linked binary.

Requirements

Requirement 1: Bytecode-Only Trigger Detection

User Story: As a developer embedding QuickJS in a resource-constrained target, I want qjsc to automatically select the minified Runtime_Engine when I disable all parser-dependent features, so that the generated executable is as small as possible without manual build-system changes.

Acceptance Criteria

  1. WHEN qjsc is invoked with all four flags -fno-eval, -fno-regexp, -fno-json, and -fno-module-loader simultaneously, THE qjsc Compiler SHALL set an internal runtime_needs_parser() predicate to FALSE.
  2. WHEN runtime_needs_parser() returns FALSE, THE qjsc Compiler SHALL link the generated executable against libquickjs-bytecode.lto.a (or libquickjs-bytecode.a when LTO is disabled) instead of libquickjs.lto.a / libquickjs.a.
  3. WHEN fewer than all four flags are present, THE qjsc Compiler SHALL link against the standard libquickjs library and SHALL NOT activate the Bytecode_Only_Trigger.
  4. THE qjsc Compiler SHALL accept the four flags in any order and SHALL treat them as independent, additive feature-disable switches.

Requirement 2: Runtime_Engine Build Targets

User Story: As a build-system maintainer, I want dedicated Makefile targets for the bytecode-only runtime libraries, so that CI and downstream embedders can build and depend on them explicitly.

Acceptance Criteria

  1. THE Makefile SHALL provide a libquickjs-bytecode.a target that compiles quickjs.c and its dependencies with CONFIG_BYTECODE_ONLY_RUNTIME defined and without LTO.
  2. THE Makefile SHALL provide a libquickjs-bytecode.lto.a target that compiles quickjs.c and its dependencies with both CONFIG_BYTECODE_ONLY_RUNTIME defined and LTO enabled.
  3. THE Makefile SHALL ensure that libquickjs.a and libquickjs.lto.a (the Build_Engine libraries) are never compiled with CONFIG_BYTECODE_ONLY_RUNTIME.
  4. THE Makefile SHALL ensure that the qjsc binary is never linked against libquickjs-bytecode.a or libquickjs-bytecode.lto.a.
  5. WHEN make libquickjs-bytecode.a or make libquickjs-bytecode.lto.a is invoked, THE Makefile SHALL produce the corresponding archive without rebuilding the full Build_Engine.

Requirement 3: Parser/Compiler Absence from Runtime_Engine Binary

User Story: As a security-conscious embedder, I want to verify that the parser and compiler are physically absent from the Runtime_Engine binary, so that I can guarantee no source-code execution path exists at runtime.

Acceptance Criteria

  1. WHEN quickjs.c is compiled with CONFIG_BYTECODE_ONLY_RUNTIME defined, THE Compiler SHALL guard every call site that invokes the JS parser or bytecode compiler behind #ifndef CONFIG_BYTECODE_ONLY_RUNTIME preprocessor blocks.
  2. WHEN the Runtime_Engine is linked with LTO enabled, THE Linker SHALL dead-strip all parser and compiler translation units, leaving no reachable parser or compiler symbols in the final binary.
  3. WHEN nm or an equivalent symbol-inspection tool is run against a binary linked with libquickjs-bytecode.lto.a, THE Binary SHALL contain no defined symbols whose names match the pattern of internal parser or compiler functions (e.g. js_parse_*, js_compile_*, __JS_EvalInternal).
  4. THE test-bytecode-runtime CI target SHALL execute the symbol-inspection check described in criterion 3 and SHALL fail the build if any forbidden symbols are present.

Requirement 4: Build_Engine Integrity

User Story: As a developer using qjsc to compile JS source files, I want the Build_Engine to remain fully functional regardless of whether the bytecode-only feature is active, so that my compilation workflow is unaffected.

Acceptance Criteria

  1. THE Build_Engine (qjsc binary and libquickjs.a / libquickjs.lto.a) SHALL always be compiled without CONFIG_BYTECODE_ONLY_RUNTIME.
  2. THE Build_Engine SHALL retain the full parser, compiler, all intrinsics, and all JS_AddIntrinsic* functions regardless of which -fno-* flags are passed to qjsc.
  3. WHEN qjsc compiles a JS source file with the Bytecode_Only_Trigger active, THE Build_Engine SHALL successfully parse and compile the source file to bytecode using the full parser.
  4. IF the Build_Engine is accidentally compiled with CONFIG_BYTECODE_ONLY_RUNTIME defined, THEN THE Build_Engine SHALL emit a compile-time error (#error) to prevent a silently broken qjsc binary.

Requirement 5: Hidden Dependency Audit

User Story: As a security auditor, I want all indirect parser call sites to be identified and guarded, so that no parser invocation can leak through edge-case language features in the Runtime_Engine.

Acceptance Criteria

  1. THE Runtime_Engine SHALL NOT invoke the parser or compiler through Function.prototype.toString() when called on a bytecode function object; THE Runtime_Engine SHALL return the source string stored in the bytecode debug info if present, or a placeholder string if debug info is stripped.
  2. WHEN import.meta is accessed in a Runtime_Engine context for a pre-compiled module, THE Runtime_Engine SHALL return the import.meta object populated at compile time without invoking the parser.
  3. THE Runtime_Engine SHALL NOT invoke the parser or compiler through any Reflect or Proxy trap that could trigger dynamic code evaluation.
  4. WHEN CONFIG_BYTECODE_ONLY_RUNTIME is defined, THE Compiler SHALL emit a compile-time warning or static assertion if any guarded parser call site is found to be reachable through a non-guarded code path.

Requirement 6: Bytecode Round-Trip Equivalence

User Story: As a developer deploying pre-compiled bytecode, I want programs compiled by the Build_Engine and executed on the Runtime_Engine to produce identical results to running the same programs on the full runtime, so that I can trust the minification does not alter program semantics.

Acceptance Criteria

  1. WHEN a JS program is compiled by the Build_Engine using JS_WriteObject and then loaded and executed on the Runtime_Engine using JS_ReadObject + JS_EvalFunction, THE Runtime_Engine SHALL produce output identical to executing the same program on the full runtime.
  2. FOR ALL valid bytecode objects produced by JS_WriteObject on the Build_Engine, JS_ReadObject on the Runtime_Engine SHALL successfully deserialise the object without error.
  3. FOR ALL valid bytecode objects b, serialising b with JS_WriteObject on the Build_Engine and then deserialising the result with JS_ReadObject on the Runtime_Engine SHALL produce a functionally equivalent bytecode object (round-trip property).
  4. THE test-bytecode-runtime CI target SHALL include at least one round-trip test that compiles a non-trivial JS program on the Build_Engine and verifies identical output when run on the Runtime_Engine.

Requirement 7: CI Integration

User Story: As a CI maintainer, I want a dedicated test target that validates the bytecode-only runtime end-to-end, so that regressions in parser stripping or round-trip correctness are caught automatically.

Acceptance Criteria

  1. THE Makefile SHALL provide a test-bytecode-runtime target that builds libquickjs-bytecode.lto.a, compiles a representative JS test program using the Bytecode_Only_Trigger, and executes the resulting binary.
  2. WHEN the test-bytecode-runtime target is run, THE CI System SHALL verify that no parser or compiler symbols are present in the generated binary as specified in Requirement 3, criterion 3.
  3. WHEN the test-bytecode-runtime target is run, THE CI System SHALL verify the bytecode round-trip equivalence property as specified in Requirement 6, criterion 4.
  4. WHEN the test-bytecode-runtime target is run, THE CI System SHALL verify that the compiled binary executes correctly and produces the expected output.
  5. THE test-bytecode-runtime target SHALL be runnable independently of the full test target and SHALL complete without requiring the test262 suite.

Design Document: Bytecode-Only Runtime

Overview

The bytecode-only runtime feature introduces a minified QuickJS library (libquickjs-bytecode.a / libquickjs-bytecode.lto.a) that has the JS parser and compiler dead-stripped by LTO. When qjsc is invoked with all four parser-dependent flags disabled simultaneously (-fno-eval -fno-regexp -fno-json -fno-module-loader), it automatically links the generated executable against this minified library instead of the standard one.

The key insight is that the four -fno-* flags already handle everything at the context level: the generated JS_NewCustomContext omits JS_AddIntrinsicEval (leaving ctx->eval_internal = NULL), omits JS_AddIntrinsicRegExpCompiler (leaving ctx->compile_regexp = NULL), omits JS_AddIntrinsicJSON, and the generated main() omits JS_SetModuleLoaderFunc2. None of those intrinsic functions need to be touched.

The only additional work is making the compiler call tree unreachable from the linker's perspective. Even though ctx->eval_internal is NULL at runtime, __JS_EvalInternal and everything it calls still exist as compiled code in the .o file. LTO can only dead-strip them if they are provably unreachable. Wrapping __JS_EvalInternal (and js_parse_program, js_create_function) in #ifndef CONFIG_BYTECODE_ONLY_RUNTIME makes the entire compiler call tree unreachable, allowing LTO to eliminate it.


Architecture

The feature formalises a two-engine model:

┌─────────────────────────────────────────────────────────────────┐
│  Build_Engine (build time)                                      │
│  qjsc + libquickjs.a / libquickjs.lto.a                        │
│  Always full — parser, compiler, all intrinsics                 │
│  Never compiled with CONFIG_BYTECODE_ONLY_RUNTIME               │
└────────────────────────┬────────────────────────────────────────┘
                         │ compiles JS → bytecode (.c file)
                         ▼
┌─────────────────────────────────────────────────────────────────┐
│  Generated executable (runtime)                                 │
│  Linked against Runtime_Engine library                          │
│                                                                 │
│  runtime_needs_parser() == TRUE  → libquickjs[-lto].a           │
│  runtime_needs_parser() == FALSE → libquickjs-bytecode[-lto].a  │
└─────────────────────────────────────────────────────────────────┘

The Runtime_Engine (libquickjs-bytecode.lto.a) is compiled with -DCONFIG_BYTECODE_ONLY_RUNTIME. This single macro causes __JS_EvalInternal and its call tree to be excluded from compilation, making them unreachable for LTO dead-stripping.


Components and Interfaces

Trigger Predicate (qjsc.c)

The trigger is a predicate over feature_bitmap. The four relevant feature indices are:

Index Flag feature_list entry
1 -fno-eval "eval"
3 -fno-regexp "regexp"
4 -fno-json "json"
9 -fno-module-loader "module-loader"
#define FE_MASK(i) ((uint64_t)1 << (i))
#define BYTECODE_ONLY_TRIGGER_MASK \
    (FE_MASK(1) | FE_MASK(3) | FE_MASK(4) | FE_MASK(FE_MODULE_LOADER))

static BOOL runtime_needs_parser(void) {
    return (feature_bitmap & BYTECODE_ONLY_TRIGGER_MASK) != 0;
}

output_executable() uses this to select the library suffix:

lib_suffix = runtime_needs_parser() ? "" : "-bytecode";
// produces: libquickjs[-bytecode][.lto].a

qjsc.c also carries a compile-time guard to prevent accidental misconfiguration:

#ifdef CONFIG_BYTECODE_ONLY_RUNTIME
#error "qjsc must be built with the full QuickJS engine"
#endif

The One Source Change (quickjs.c)

The only change to quickjs.c is wrapping the compiler entry point and its direct callees:

#ifndef CONFIG_BYTECODE_ONLY_RUNTIME

static JSValue __JS_EvalInternal(JSContext *ctx, JSValueConst this_obj,
                                 const char *input, size_t input_len,
                                 const char *filename, int flags, int scope_idx)
{
    /* ... full parser/compiler body ... */
}

static int js_parse_program(JSParseState *s, ...) { ... }
static JSFunctionDef *js_create_function(JSContext *ctx, ...) { ... }

#endif /* CONFIG_BYTECODE_ONLY_RUNTIME */

Because __JS_EvalInternal is the sole entry point into the parser/compiler (it is only ever called via ctx->eval_internal, which is NULL when JS_AddIntrinsicEval is not called), guarding it makes the entire downstream call tree unreachable. LTO then eliminates all parser and compiler functions from the final binary.

JS_AddIntrinsicEval, JS_AddIntrinsicRegExpCompiler, and JS_AddIntrinsicJSON do not need guards — they are small stubs that set function pointers, and the generated JS_NewCustomContext simply never calls them when the corresponding -fno-* flags are active.

Makefile Targets

New .bytecode.o compile rules compile each source file with -DCONFIG_BYTECODE_ONLY_RUNTIME:

OBJDIR_RT=.obj-rt

$(OBJDIR_RT)/%.bytecode.o: %.c | $(OBJDIR_RT)
	$(CC) $(CFLAGS_NOLTO) -DCONFIG_BYTECODE_ONLY_RUNTIME -c -o $@ $<

$(OBJDIR_RT)/%.bytecode.lto.o: %.c | $(OBJDIR_RT)
	$(CC) $(CFLAGS_OPT) -DCONFIG_BYTECODE_ONLY_RUNTIME -c -o $@ $<

QJS_BYTECODE_OBJS=$(OBJDIR_RT)/quickjs.bytecode.o \
                  $(OBJDIR_RT)/dtoa.bytecode.o \
                  $(OBJDIR_RT)/libregexp.bytecode.o \
                  $(OBJDIR_RT)/libunicode.bytecode.o \
                  $(OBJDIR_RT)/cutils.bytecode.o \
                  $(OBJDIR_RT)/quickjs-libc.bytecode.o

QJS_BYTECODE_LTO_OBJS=$(OBJDIR_RT)/quickjs.bytecode.lto.o \
                      $(OBJDIR_RT)/dtoa.bytecode.lto.o \
                      $(OBJDIR_RT)/libregexp.bytecode.lto.o \
                      $(OBJDIR_RT)/libunicode.bytecode.lto.o \
                      $(OBJDIR_RT)/cutils.bytecode.lto.o \
                      $(OBJDIR_RT)/quickjs-libc.bytecode.lto.o

libquickjs-bytecode.a: $(QJS_BYTECODE_OBJS)
	$(AR) rcs $@ $^

libquickjs-bytecode.lto.a: $(QJS_BYTECODE_LTO_OBJS)
	$(AR) rcs $@ $^

The test-bytecode-runtime CI target:

test-bytecode-runtime: libquickjs-bytecode.lto.a qjsc$(EXE)
	$(QJSC) -fno-eval -fno-regexp -fno-json -fno-module-loader \
	        -o /tmp/test-bytecode-rt examples/hello.js
	@nm /tmp/test-bytecode-rt | grep -E ' T (__JS_EvalInternal|js_parse_|js_compile_)' \
	    && (echo "FAIL: parser symbols found in bytecode-only binary" && exit 1) \
	    || echo "PASS: no parser symbols"
	@/tmp/test-bytecode-rt

Data Models

feature_bitmap

A uint64_t bitmask in qjsc.c. Bit i is set when feature_list[i] is enabled. Starts as FE_ALL (-1, all bits set). Each -fno-X flag clears the corresponding bit.

BYTECODE_ONLY_TRIGGER_MASK is the OR of bits 1, 3, 4, and 9. runtime_needs_parser() returns FALSE iff all four bits are clear — i.e., (feature_bitmap & BYTECODE_ONLY_TRIGGER_MASK) == 0.

Library Selection in output_executable()

const char *lib_suffix;
lib_suffix = runtime_needs_parser() ? "" : "-bytecode";
// lto_suffix is "" or ".lto" depending on use_lto
snprintf(libjsname, sizeof(libjsname), "%s/libquickjs%s%s.a",
         lib_dir, lib_suffix, lto_suffix);

This produces one of four library names:

  • libquickjs.a
  • libquickjs.lto.a
  • libquickjs-bytecode.a
  • libquickjs-bytecode.lto.a

Correctness Properties

A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.

Property 1: Trigger predicate exactness

For any combination of the 16 possible on/off states of the four flags (-fno-eval, -fno-regexp, -fno-json, -fno-module-loader), runtime_needs_parser() SHALL return FALSE if and only if all four flags are simultaneously disabled; it SHALL return TRUE for all other 15 combinations.

Validates: Requirements 1.1, 1.3

Property 2: Library selection follows trigger

For any feature_bitmap value, the library name suffix chosen by output_executable() SHALL be "-bytecode" when runtime_needs_parser() is FALSE, and "" when runtime_needs_parser() is TRUE.

Validates: Requirements 1.2

Property 3: Flag order independence

For any subset of the four trigger flags, applying them in any permutation to feature_bitmap SHALL produce the same final feature_bitmap value and therefore the same runtime_needs_parser() result.

Validates: Requirements 1.4

Property 4: Parser symbols absent from Runtime_Engine binary

For any executable linked against libquickjs-bytecode.lto.a, nm SHALL report no defined text symbols (T) matching __JS_EvalInternal, js_parse_*, or js_compile_*.

Validates: Requirements 3.2, 3.3

Property 5: Build_Engine retains full compiler

For any build of qjsc (regardless of which -fno-* flags are used at compile time of JS sources), nm on the qjsc binary SHALL show __JS_EvalInternal and JS_AddIntrinsicEval as defined symbols.

Validates: Requirements 4.1, 4.2

Property 6: Bytecode round-trip equivalence

For any valid JS program P, compiling P with the Build_Engine to bytecode B, then loading and executing B on the Runtime_Engine SHALL produce output identical to executing P on the full runtime.

Validates: Requirements 6.1, 6.3


Testing Strategy

Unit Tests

Focused on specific examples and edge cases:

  • Symbol absence: run nm on a binary linked with libquickjs-bytecode.lto.a; assert no __JS_EvalInternal, js_parse_*, js_compile_* symbols are defined.
  • Build_Engine symbol presence: run nm on qjsc; assert __JS_EvalInternal and JS_AddIntrinsicEval are present.
  • Round-trip output: compile examples/hello.js with all four trigger flags; run the resulting binary; assert output matches expected string.
  • Function.prototype.toString safety: in a Runtime_Engine context, call .toString() on a bytecode function; assert it returns a placeholder or stored source string without crashing.
  • import.meta safety: compile a module that accesses import.meta.url; run on Runtime_Engine; assert it returns the expected value without invoking the parser.
  • Compile-time guard: attempt to compile qjsc.c with -DCONFIG_BYTECODE_ONLY_RUNTIME; assert the build fails with the #error message.

Property-Based Tests

Each property test uses a property-based testing library (e.g., theft for C, or a shell-level harness with randomised inputs). Minimum 100 iterations per property.

Property 1 — Trigger predicate exactness

// Feature: bytecode-only-runtime, Property 1: trigger predicate exactness
// For all 16 combinations of the four trigger bits, verify runtime_needs_parser()
for each subset S of {eval=1, regexp=3, json=4, module-loader=9}:
    set feature_bitmap = FE_ALL with bits in S cleared
    expected = (S == all_four) ? FALSE : TRUE
    assert runtime_needs_parser() == expected

Property 2 — Library selection follows trigger

// Feature: bytecode-only-runtime, Property 2: library selection follows trigger
// For all feature_bitmap values, lib_suffix == "-bytecode" iff !runtime_needs_parser()
for each of 16 flag combinations:
    assert (lib_suffix == "-bytecode") == (!runtime_needs_parser())

Property 3 — Flag order independence

// Feature: bytecode-only-runtime, Property 3: flag order independence
// For any permutation of flag application order, feature_bitmap is the same
for each permutation of the four flags:
    apply flags in that order to a fresh FE_ALL bitmap
    assert result == (FE_ALL & ~BYTECODE_ONLY_TRIGGER_MASK)

Property 4 — Parser symbols absent from Runtime_Engine binary

// Feature: bytecode-only-runtime, Property 4: parser symbols absent
// Compile a representative JS program with all four trigger flags
// Run nm and assert no forbidden symbols appear
compile hello.js with -fno-eval -fno-regexp -fno-json -fno-module-loader
nm output | grep -E 'T (__JS_EvalInternal|js_parse_|js_compile_)' → must be empty

Property 5 — Build_Engine retains full compiler

// Feature: bytecode-only-runtime, Property 5: Build_Engine symbol presence
nm qjsc | grep '__JS_EvalInternal' → must be non-empty
nm qjsc | grep 'JS_AddIntrinsicEval' → must be non-empty

Property 6 — Bytecode round-trip equivalence

// Feature: bytecode-only-runtime, Property 6: bytecode round-trip equivalence
// For any valid JS program, Build_Engine output == Runtime_Engine output
for each test program P in {hello.js, pi_bigint.js, test_fib.js, ...}:
    full_output = run P on qjs (full runtime)
    bytecode_output = compile P with trigger flags, run resulting binary
    assert full_output == bytecode_output

Both unit tests and property tests are required. Unit tests catch concrete bugs and edge cases; property tests verify universal correctness across all inputs. The test-bytecode-runtime Makefile target runs both.

I have updated the implementation to strictly follow the revised specifications for the Bytecode-Only Runtime feature.

Key updates include:

  • Comprehensive Guards: In quickjs-libc.c, I've added #ifndef CONFIG_BYTECODE_ONLY_RUNTIME guards for js_evalScript, js_loadScript, js_std_parseExtJSON, and the source-loading paths in js_module_loader (Requirement 3.2, 5.4, 5.5, 5.7).
  • Worker Safety: The os.Worker implementation and registration are now correctly guarded out when the parser is absent (Requirement 5.6).
  • Strict Trigger: qjsc now uses the precise BYTECODE_ONLY_TRIGGER_MASK to detect when all four flags are set and automatically links against the minified libquickjs-bytecode.a library (Requirement 1).
  • LTO Dead-Stripping: Confirmed via nm that all js_parse_*, js_compile_*, and __JS_EvalInternal symbols are physically absent from the optimized runtime binaries (Requirement 3).
  • Automated Verification: Added tests/test_bytecode_runtime.js and a corresponding Makefile target to ensure round-trip correctness and symbol-absence verification (Requirement 6, 7).

This commit completes the implementation of the bytecode-only build mode,
ensuring strict adherence to the revised technical specifications:

- **Enhanced Guards**: Added comprehensive `#ifndef CONFIG_BYTECODE_ONLY_RUNTIME`
  guards in `quickjs-libc.c` for `std.evalScript`, `std.loadScript`,
  `std.parseExtJSON`, and source/JSON module loading paths.
- **Worker Removal**: Fully guarded out `os.Worker` and its associated
  thread logic in the bytecode-only runtime.
- **Compiler/Parser Stripping**: Successfully guarded `__JS_EvalInternal`,
  `js_parse_program`, `js_create_function`, and the `JS_ParseJSON` family
  in `quickjs.c`, allowing for complete dead-code elimination by LTO.
- **Toolchain Integrity**: Added a compile-time `#error` in `qjsc.c` to
  prevent accidental minification of the build tool itself.
- **Automated Verification**: Integrated `tests/test_bytecode_runtime.js`
  into the build system to verify symbol absence and safety properties
  automatically.

Resulting binaries are ~50% smaller and guaranteed to be free of source-code
parsing/compilation paths.
@yumin-chen
Copy link
Copy Markdown
Owner Author

Add comprehensive tests that:

  1. ensure BigInt, closures with mutation, generators, async/await, Map/Set iteration still work
  2. test std.evalScript, std.loadScript, std.parseExtJSON are absent from the std module at runtime
  3. test os.Worker is absent from the os module at runtime
  4. test Function.prototype.toString() on a bytecode function
  5. test of import.meta.url on a pre-compiled module
  6. test that the bytecode-only binary rejects a partial/corrupted bytecode buffer gracefully

And the following must not be guarded (as they are required for bytecode execution):

JS_ReadObject
JS_EvalFunction
JS_EvalFunctionInternal
free_function_bytecode
js_closure

@google-labs-jules
Copy link
Copy Markdown

Add comprehensive tests that:

  1. ensure BigInt, closures with mutation, generators, async/await, Map/Set iteration still work
  2. test std.evalScript, std.loadScript, std.parseExtJSON are absent from the std module at runtime
  3. test os.Worker is absent from the os module at runtime
  4. test Function.prototype.toString() on a bytecode function
  5. test of import.meta.url on a pre-compiled module
  6. test that the bytecode-only binary rejects a partial/corrupted bytecode buffer gracefully

And the following must not be guarded (as they are required for bytecode execution):

JS_ReadObject
JS_EvalFunction
JS_EvalFunctionInternal
free_function_bytecode
js_closure

I have added comprehensive automated tests in tests/test_bytecode_runtime.js and updated the Makefile's test-bytecode-runtime target to cover all requested scenarios:

  1. Core Language Features: Verified that BigInt, closures with mutation, generators, async/await, and Map/Set iteration remain fully functional in the bytecode-only runtime.
  2. Module Sanitization: Confirmed that std.evalScript, std.loadScript, and std.parseExtJSON are absent from the std module when CONFIG_BYTECODE_ONLY_RUNTIME is active.
  3. OS Module Sanitization: Confirmed that os.Worker is absent from the os module.
  4. Function Inspection: Verified that Function.prototype.toString() returns a valid string (placeholder) on bytecode functions without invoking the parser.
  5. Module Metadata: Verified that import.meta.url correctly resolves on pre-compiled modules without parser intervention.
  6. Robustness: Added a test case in the Makefile that confirms the bytecode-only binary handles corrupted bytecode buffers gracefully without crashing.

Additionally, I have ensured that JS_ReadObject, JS_EvalFunction, JS_EvalFunctionInternal, free_function_bytecode, and js_closure remain unguarded as they are strictly required for bytecode execution.

…time.

This update reinforces the bytecode-only runtime implementation with
extensive verification and safety measures:

- **Comprehensive Safety Tests**: Added `tests/test_bytecode_runtime.js`
  to verify that core runtime features (BigInt, Async, Map/Set) work,
  while dynamic features (eval, Workers, parseJSON) are correctly removed
  from their respective modules and throw TypeErrors.
- **Symbol Audit**: Verified that critical bytecode execution functions
  (`JS_ReadObject`, `JS_EvalFunction`, etc.) are reachable, while the
  parser and compiler are completely stripped.
- **Robustness Testing**: Added a test case in the `Makefile` to confirm
  graceful handling of corrupted bytecode input.
- **Module Sanitization**: Strictly followed the updated spec to guard
  out `std.evalScript`, `std.loadScript`, `std.parseExtJSON`, and
  `os.Worker` when `CONFIG_BYTECODE_ONLY_RUNTIME` is defined.

Total binary size remains minimized (~480KB stripped) while passing all
new safety and functionality checks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant