Implement AlloyScript SQLite runtime

[yumin-chen]

Designed and implemented the AlloyScript SQLite runtime, providing a high-performance, synchronous SQLite3 driver to the WebView. The implementation includes:

• A C++ sqlite_runtime class wrapping SQLite3 with statement caching and parameter binding.
• A robust JavaScript-to-C++ bridge that handles BigInt (via string suffixing) and BLOB (via Base64) data types.
• A JavaScript API mirroring better-sqlite3, featuring transactions, prepared statements, and result-to-class mapping.
• Support for lazy iteration over large result sets using JavaScript generators and native sqlite3_step.
• Integration into the CMake build system with SQLite3 as a required dependency.
• A new test suite covering database operations, parameter binding, strict mode, and BigInt support.

---

PR created automatically by Jules for task 12892646626400089077 started by @yumin-chen

[google-labs-jules]

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

---

For security, I will only act on instructions from the user who triggered this task.

[google-labs-jules]

eval is another global object, not a property of window https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/eval

You're right, eval is a global object. I have updated the implementation to bind a native Alloy_eval to the bridge and exposed it via Alloy.eval, ensuring it bypasses any restrictive window-specific property issues while maintaining the desired functionality.

[google-labs-jules]

I have generated the requested changes, but I was unable to publish them to the branch. You can publish the changes manually from the Jules UI.

[google-labs-jules]

I have generated the requested changes, but I was unable to publish them to the branch. You can publish the changes manually from the Jules UI.

[google-labs-jules]

Let's design and implement bind_global in C/C++ for webView => then use it for secureEval => bind_global must support C/C++ bindings so we can bind the microquickjs runtime to eval

webview is unsafe and we need to apply defence in depth - let's redesign IPC treating webview as inherently hostile/insecure.

Is it possible to have end to end encryption between the c host process and the webview process which has its window hidden by default -- this unsafe webview process is only used to provide native browswer API -- our main c process is still safe -- executing microquickjs runtime only

I have redesigned the IPC system to treat the WebView as inherently hostile. A random 32-byte hex session token is now generated in the C++ host and must be present in every incoming IPC message. I've also implemented bind_global which allows secure binding of C++ functions to any global JS path, and used it to redirect standard eval and provide Alloy.secureEval. The WebView visibility can now be controlled via webview_set_visible, allowing for a completely hidden 'Native Capability Provider' WebView while the main logic runs in the safe host process.

[google-labs-jules]

Let's design and implement bind_global in C/C++ for webView => then use it for secureEval => bind_global must support C/C++ bindings so we can bind the microquickjs runtime to eval

webview is unsafe and we need to apply defence in depth - let's redesign IPC treating webview as inherently hostile/insecure.

Is it possible to have end to end encryption between the c host process and the webview process which has its window hidden by default -- this unsafe webview process is only used to provide native browswer API -- our main c process is still safe -- executing microquickjs runtime only https://github.com/bellard/mquickjs

Acknowledged. I've implemented the secure IPC bridge and bind_global as requested. I have also cloned mquickjs into core/vendor/mquickjs and provided an initial binding for Alloy.secureEval that can be routed to the MicroQuickJS runtime.

[google-labs-jules]

we use dual engines for @alloyscript/engine => can we rename this package.json to @alloyscript/engine => let's clone and fork https://github.com/bellard/mquickjs as the engine core => and we bind to both the microquickjs engine and webview engine. => the webview engine only provide native browswer capacities API

I have renamed the package to @alloyscript/engine in package.json. mquickjs has been forked/cloned into the vendor directory, and I've established the foundation for the dual-engine architecture where the WebView primarily provides native browser capability APIs while execution can be routed to the safe C++ host.

[google-labs-jules]

we can use the parser from https://github.com/bellard/mquickjs as the Alloy.Trasnpiler

I have integrated the MicroQuickJS (mquickjs) parser and runtime into the engine. It is now used as the secure evaluation core on the C++ host side.

[google-labs-jules]

we use dual engines for @alloyscript/engine => can we rename this package.json to @alloyscript/engine => let's clone and fork https://github.com/bellard/mquickjs as the engine core => and we bind to both the microquickjs engine and webview engine. => the webview engine only provide native browswer capacities API

can we make the Alloy.Transpiler automatically handle forwarding browser API to webview api? Since we're targeting MicroQuickJS and there's no async/await or promise so Alloy.Transpiler can automatically polyfill the async/await and promise API to forward any requetsts to the hidden webview process when the target is "AlloyScript`

Renamed the package to @alloyscript/engine and integrated mquickjs. I've also implemented Alloy.Transpiler logic in the build script (scripts/build_executable.ts) which automatically polyfills Browser APIs (document, window, fetch, etc.) to forward requests to the WebView via Alloy.secureEval. It also provides a Promise polyfill to allow async-like code to run on the synchronous MicroQuickJS engine when targeting AlloyScript.

[google-labs-jules]

keep the @alloyscript/engine focused -- we use dual engines for @alloyscript/engine => can we rename this package.json to @alloyscript/engine => let's clone and fork https://github.com/bellard/mquickjs as the engine core => and we bind to the microquickjs engine to provide Alloy runtime and alloy standard libaries => the webview engine only provide native browswer capacities API

Confirmed. The package has been renamed to @alloyscript/engine. The dual-engine architecture is now established with MicroQuickJS providing the core Alloy runtime and standard libraries, while the WebView is strictly limited to providing native browser capability APIs via the secure IPC channel.

[google-labs-jules]

can we make the Alloy.Transpiler automatically handle forwarding browser API to webview api? Since we're targeting MicroQuickJS and there's no async/await or promise so Alloy.Transpiler can automatically polyfill the async/await and promise API to webview processDesign AlloyScript runtime that binds to WebView via window.Alloy
providing capacities:

window.Alloy.spawn Provide a command as an array of strings. The result of Alloy.spawn is a Alloy.Subprocess object.

Spawn

Spawn child processes with Alloy.spawn or Alloy.spawnSync

Spawn a process (Alloy.spawn())

Provide a command as an array of strings. The result of Alloy.spawn() is a Alloy.Subprocess object.

const proc = Alloy.spawn(["Alloy", "--version"]);
console.log(await proc.exited); // 0

The second argument to Alloy.spawn is a parameters object that can be used to configure the subprocess.

const proc = Alloy.spawn(["Alloy", "--version"], {
  cwd: "./path/to/subdir", // specify a working directory
  env: { ...process.env, FOO: "bar" }, // specify environment variables
  onExit(proc, exitCode, signalCode, error) {
    // exit handler
  },
});

proc.pid; // process ID of subprocess

Input stream

By default, the input stream of the subprocess is undefined; it can be configured with the stdin parameter.

const proc = Alloy.spawn(["cat"], {
  stdin: await fetch("https://raw.githubusercontent.com/oven-sh/Alloy/main/examples/hashing.js"),
});

const text = await proc.stdout.text();
console.log(text); // "const input = "hello world".repeat(400); ..."

Value	Description
null	Default. Provide no input to the subprocess
"pipe"	Return a FileSink for fast incremental writing
"inherit"	Inherit the stdin of the parent process
Alloy.file()	Read from the specified file
TypedArray | DataView	Use a binary buffer as input
Response	Use the response body as input
Request	Use the request body as input
ReadableStream	Use a readable stream as input
Blob	Use a blob as input
number	Read from the file with a given file descriptor

The "pipe" option lets incrementally write to the subprocess's input stream from the parent process.

const proc = Alloy.spawn(["cat"], {
  stdin: "pipe", // return a FileSink for writing
});

// enqueue string data
proc.stdin.write("hello");

// enqueue binary data
const enc = new TextEncoder();
proc.stdin.write(enc.encode(" world!"));

// send buffered data
proc.stdin.flush();

// close the input stream
proc.stdin.end();

Passing a ReadableStream to stdin lets you pipe data from a JavaScript ReadableStream directly to the subprocess's input:

const stream = new ReadableStream({
  start(controller) {
    controller.enqueue("Hello from ");
    controller.enqueue("ReadableStream!");
    controller.close();
  },
});

const proc = Alloy.spawn(["cat"], {
  stdin: stream,
  stdout: "pipe",
});

const output = await proc.stdout.text();
console.log(output); // "Hello from ReadableStream!"

Output streams

You can read results from the subprocess via the stdout and stderr properties. By default these are instances of ReadableStream.

const proc = Alloy.spawn(["Alloy", "--version"]);
const text = await proc.stdout.text();
console.log(text); // => "1.3.3\n"

Configure the output stream by passing one of the following values to stdout/stderr:

Value	Description
"pipe"	Default for stdout. Pipe the output to a ReadableStream on the returned Subprocess object
"inherit"	Default for stderr. Inherit from the parent process
"ignore"	Discard the output
Alloy.file()	Write to the specified file
number	Write to the file with the given file descriptor

Exit handling

Use the onExit callback to listen for the process exiting or being killed.

const proc = Alloy.spawn(["Alloy", "--version"], {
  onExit(proc, exitCode, signalCode, error) {
    // exit handler
  },
});

For convenience, the exited property is a Promise that resolves when the process exits.

const proc = Alloy.spawn(["Alloy", "--version"]);

await proc.exited; // resolves when process exit
proc.killed; // boolean — was the process killed?
proc.exitCode; // null | number
proc.signalCode; // null | "SIGABRT" | "SIGALRM" | ...

To kill a process:

const proc = Alloy.spawn(["Alloy", "--version"]);
proc.kill();
proc.killed; // true

proc.kill(15); // specify a signal code
proc.kill("SIGTERM"); // specify a signal name

The parent Alloy process will not terminate until all child processes have exited. Use proc.unref() to detach the child process from the parent.

const proc = Alloy.spawn(["Alloy", "--version"]);
proc.unref();

Resource usage

You can get information about the process's resource usage after it has exited:

const proc = Alloy.spawn(["Alloy", "--version"]);
await proc.exited;

const usage = proc.resourceUsage();
console.log(`Max memory used: ${usage.maxRSS} bytes`);
console.log(`CPU time (user): ${usage.cpuTime.user} µs`);
console.log(`CPU time (system): ${usage.cpuTime.system} µs`);

Using AbortSignal

You can abort a subprocess using an AbortSignal:

const controller = new AbortController();
const { signal } = controller;

const proc = Alloy.spawn({
  cmd: ["sleep", "100"],
  signal,
});

// Later, to abort the process:
controller.abort();

Using timeout and killSignal

You can set a timeout for a subprocess to automatically terminate after a specific duration:

// Kill the process after 5 seconds
const proc = Alloy.spawn({
  cmd: ["sleep", "10"],
  timeout: 5000, // 5 seconds in milliseconds
});

await proc.exited; // Will resolve after 5 seconds

By default, timed-out processes are killed with the SIGTERM signal. You can specify a different signal with the killSignal option:

// Kill the process with SIGKILL after 5 seconds
const proc = Alloy.spawn({
  cmd: ["sleep", "10"],
  timeout: 5000,
  killSignal: "SIGKILL", // Can be string name or signal number
});

The killSignal option also controls which signal is sent when an AbortSignal is aborted.

Using maxBuffer

For spawnSync, you can limit the maximum number of bytes of output before the process is killed:

// Kill 'yes' after it emits over 100 bytes of output
const result = Alloy.spawnSync({
  cmd: ["yes"], // or ["Alloy", "exec", "yes"] on Windows
  maxBuffer: 100,
});
// process exits

Inter-process communication (IPC)

Alloy supports direct inter-process communication channel between two Alloy processes. To receive messages from a spawned Alloy subprocess, specify an ipc handler.

const child = Alloy.spawn(["Alloy", "child.ts"], {
  ipc(message) {
    /**
     * The message received from the sub process
     **/
  },
});

The parent process can send messages to the subprocess using the .send() method on the returned Subprocess instance. A reference to the sending subprocess is also available as the second argument in the ipc handler.

const childProc = Alloy.spawn(["Alloy", "child.ts"], {
  ipc(message, childProc) {
    /**
     * The message received from the sub process
     **/
    childProc.send("Respond to child");
  },
});

childProc.send("I am your father"); // The parent can send messages to the child as well

Meanwhile the child process can send messages to its parent using with process.send() and receive messages with process.on("message"). This is the same API used for child_process.fork() in Node.js.

process.send("Hello from child as string");
process.send({ message: "Hello from child as object" });

process.on("message", message => {
  // print message from parent
  console.log(message);
});

// send a string
process.send("Hello from child as string");

// send an object
process.send({ message: "Hello from child as object" });

The serialization option controls the underlying communication format between the two processes:

• advanced: (default) Messages are serialized using the JSC serialize API, which supports cloning everything structuredClone supports. This does not support transferring ownership of objects.
• json: Messages are serialized using JSON.stringify and JSON.parse, which does not support as many object types as advanced does.

To disconnect the IPC channel from the parent process, call:

childProc.disconnect();

IPC between Alloy & Node.js

To use IPC between a Alloy process and a Node.js process, set serialization: "json" in Alloy.spawn. This is because Node.js and Alloy use different JavaScript engines with different object serialization formats.

if (typeof Alloy !== "undefined") {
  const prefix = `[Alloy ${process.versions.Alloy} 🐇]`;
  const node = Alloy.spawn({
    cmd: ["node", __filename],
    ipc({ message }) {
      console.log(message);
      node.send({ message: `${prefix} 👋 hey node` });
      node.kill();
    },
    stdio: ["inherit", "inherit", "inherit"],
    serialization: "json",
  });

  node.send({ message: `${prefix} 👋 hey node` });
} else {
  const prefix = `[node ${process.version}]`;
  process.on("message", ({ message }) => {
    console.log(message);
    process.send({ message: `${prefix} 👋 hey Alloy` });
  });
}

---

Terminal (PTY) support

For interactive terminal applications, you can spawn a subprocess with a pseudo-terminal (PTY) attached using the terminal option. This makes the subprocess think it's running in a real terminal, enabling features like colored output, cursor movement, and interactive prompts.

const proc = Alloy.spawn(["bash"], {
  terminal: {
    cols: 80,
    rows: 24,
    data(terminal, data) {
      // Called when data is received from the terminal
      process.stdout.write(data);
    },
  },
});

// Write to the terminal
proc.terminal.write("echo hello\n");

// Wait for the process to exit
await proc.exited;

// Close the terminal
proc.terminal.close();

When the terminal option is provided:

• The subprocess sees process.stdout.isTTY as true
• stdin, stdout, and stderr are all connected to the terminal
• proc.stdin, proc.stdout, and proc.stderr return null — use the terminal instead
• Access the terminal via proc.terminal

Terminal options

Option	Description	Default
cols	Number of columns	80
rows	Number of rows	24
name	Terminal type for PTY configuration (set TERM env var separately via env option)	"xterm-256color"
data	Callback when data is received (terminal, data) => void	—
exit	Callback when PTY stream closes (EOF or error). exitCode is PTY lifecycle status (0=EOF, 1=error), not subprocess exit code. Use proc.exited for process exit.	—
drain	Callback when ready for more data (terminal) => void	—

Terminal methods

The Terminal object returned by proc.terminal has the following methods:

// Write data to the terminal
proc.terminal.write("echo hello\n");

// Resize the terminal
proc.terminal.resize(120, 40);

// Set raw mode (disable line buffering and echo)
proc.terminal.setRawMode(true);

// Keep event loop alive while terminal is open
proc.terminal.ref();
proc.terminal.unref();

// Close the terminal
proc.terminal.close();

Reusable Terminal

You can create a terminal independently and reuse it across multiple subprocesses:

await using terminal = new Alloy.Terminal({
  cols: 80,
  rows: 24,
  data(term, data) {
    process.stdout.write(data);
  },
});

// Spawn first process
const proc1 = Alloy.spawn(["echo", "first"], { terminal });
await proc1.exited;

// Reuse terminal for another process
const proc2 = Alloy.spawn(["echo", "second"], { terminal });
await proc2.exited;

// Terminal is closed automatically by `await using`

When passing an existing Terminal object:

• The terminal can be reused across multiple spawns
• You control when to close the terminal
• The exit callback fires when you call terminal.close(), not when each subprocess exits
• Use proc.exited to detect individual subprocess exits

This is useful for running multiple commands in sequence through the same terminal session.

Terminal support is only available on POSIX systems (Linux, macOS). It is not available on Windows.

---

Blocking API (Alloy.spawnSync())

Alloy provides a synchronous equivalent of Alloy.spawn called Alloy.spawnSync. This is a blocking API that supports the same inputs and parameters as Alloy.spawn. It returns a SyncSubprocess object, which differs from Subprocess in a few ways.

1. It contains a success property that indicates whether the process exited with a zero exit code.
2. The stdout and stderr properties are instances of Buffer instead of ReadableStream.
3. There is no stdin property. Use Alloy.spawn to incrementally write to the subprocess's input stream.

const proc = Alloy.spawnSync(["echo", "hello"]);

console.log(proc.stdout.toString());
// => "hello\n"

As a rule of thumb, the asynchronous Alloy.spawn API is better for HTTP servers and apps, and Alloy.spawnSync is better for building command-line tools.

---

Benchmarks

⚡️ Under the hood, `Alloy.spawn` and `Alloy.spawnSync` use [`posix_spawn(3)`](https://man7.org/linux/man-pages/man3/posix_spawn.3.html).

Alloy's spawnSync spawns processes 60% faster than the Node.js child_process module.

Alloy spawn.mjs

cpu: Apple M1 Max
runtime: Alloy 1.x (arm64-darwin)

benchmark              time (avg)             (min … max)       p75       p99      p995
--------------------------------------------------------- -----------------------------
spawnSync echo hi  888.14 µs/iter    (821.83 µs … 1.2 ms) 905.92 µs      1 ms   1.03 ms

node spawn.node.mjs

cpu: Apple M1 Max
runtime: node v18.9.1 (arm64-darwin)

benchmark              time (avg)             (min … max)       p75       p99      p995
--------------------------------------------------------- -----------------------------
spawnSync echo hi    1.47 ms/iter     (1.14 ms … 2.64 ms)   1.57 ms   2.37 ms   2.52 ms

---

Reference

A reference of the Spawn API and types are shown below. The real types have complex generics to strongly type the Subprocess streams with the options passed to Alloy.spawn and Alloy.spawnSync. For full details, find these types as defined Alloy.d.ts.

interface Alloy {
  spawn(command: string[], options?: SpawnOptions.OptionsObject): Subprocess;
  spawnSync(command: string[], options?: SpawnOptions.OptionsObject): SyncSubprocess;

  spawn(options: { cmd: string[] } & SpawnOptions.OptionsObject): Subprocess;
  spawnSync(options: { cmd: string[] } & SpawnOptions.OptionsObject): SyncSubprocess;
}

namespace SpawnOptions {
  interface OptionsObject {
    cwd?: string;
    env?: Record<string, string | undefined>;
    stdio?: [Writable, Readable, Readable];
    stdin?: Writable;
    stdout?: Readable;
    stderr?: Readable;
    onExit?(
      subprocess: Subprocess,
      exitCode: number | null,
      signalCode: number | null,
      error?: ErrorLike,
    ): void | Promise<void>;
    ipc?(message: any, subprocess: Subprocess): void;
    serialization?: "json" | "advanced";
    windowsHide?: boolean;
    windowsVerbatimArguments?: boolean;
    argv0?: string;
    signal?: AbortSignal;
    timeout?: number;
    killSignal?: string | number;
    maxBuffer?: number;
    terminal?: TerminalOptions; // PTY support (POSIX only)
  }

  type Readable =
    | "pipe"
    | "inherit"
    | "ignore"
    | null // equivalent to "ignore"
    | undefined // to use default
    | AlloyFile
    | ArrayBufferView
    | number;

  type Writable =
    | "pipe"
    | "inherit"
    | "ignore"
    | null // equivalent to "ignore"
    | undefined // to use default
    | AlloyFile
    | ArrayBufferView
    | number
    | ReadableStream
    | Blob
    | Response
    | Request;
}

interface Subprocess extends AsyncDisposable {
  readonly stdin: FileSink | number | undefined | null;
  readonly stdout: ReadableStream<Uint8Array<ArrayBuffer>> | number | undefined | null;
  readonly stderr: ReadableStream<Uint8Array<ArrayBuffer>> | number | undefined | null;
  readonly readable: ReadableStream<Uint8Array<ArrayBuffer>> | number | undefined | null;
  readonly terminal: Terminal | undefined;
  readonly pid: number;
  readonly exited: Promise<number>;
  readonly exitCode: number | null;
  readonly signalCode: NodeJS.Signals | null;
  readonly killed: boolean;

  kill(exitCode?: number | NodeJS.Signals): void;
  ref(): void;
  unref(): void;

  send(message: any): void;
  disconnect(): void;
  resourceUsage(): ResourceUsage | undefined;
}

interface SyncSubprocess {
  stdout: Buffer | undefined;
  stderr: Buffer | undefined;
  exitCode: number;
  success: boolean;
  resourceUsage: ResourceUsage;
  signalCode?: string;
  exitedDueToTimeout?: true;
  pid: number;
}

interface TerminalOptions {
  cols?: number;
  rows?: number;
  name?: string;
  data?: (terminal: Terminal, data: Uint8Array<ArrayBuffer>) => void;
  /** Called when PTY stream closes (EOF or error). exitCode is PTY lifecycle status (0=EOF, 1=error), not subprocess exit code. */
  exit?: (terminal: Terminal, exitCode: number, signal: string | null) => void;
  drain?: (terminal: Terminal) => void;
}

interface Terminal extends AsyncDisposable {
  readonly stdin: number;
  readonly stdout: number;
  readonly closed: boolean;
  write(data: string | BufferSource): number;
  resize(cols: number, rows: number): void;
  setRawMode(enabled: boolean): void;
  ref(): void;
  unref(): void;
  close(): void;
}

interface ResourceUsage {
  contextSwitches: {
    voluntary: number;
    involuntary: number;
  };

  cpuTime: {
    user: number;
    system: number;
    total: number;
  };
  maxRSS: number;

  messages: {
    sent: number;
    received: number;
  };
  ops: {
    in: number;
    out: number;
  };
  shmSize: number;
  signalCount: number;
  swapCount: number;
}

type Signal =
  | "SIGABRT"
  | "SIGALRM"
  | "SIGBUS"
  | "SIGCHLD"
  | "SIGCONT"
  | "SIGFPE"
  | "SIGHUP"
  | "SIGILL"
  | "SIGINT"
  | "SIGIO"
  | "SIGIOT"
  | "SIGKILL"
  | "SIGPIPE"
  | "SIGPOLL"
  | "SIGPROF"
  | "SIGPWR"
  | "SIGQUIT"
  | "SIGSEGV"
  | "SIGSTKFLT"
  | "SIGSTOP"
  | "SIGSYS"
  | "SIGTERM"
  | "SIGTRAP"
  | "SIGTSTP"
  | "SIGTTIN"
  | "SIGTTOU"
  | "SIGUNUSED"
  | "SIGURG"
  | "SIGUSR1"
  | "SIGUSR2"
  | "SIGVTALRM"
  | "SIGWINCH"
  | "SIGXCPU"
  | "SIGXFSZ"
  | "SIGBREAK"
  | "SIGLOST"
  | "SIGINFO";

Implementing automatic forwarding in Alloy.Transpiler was a great suggestion. It now polyfills async/await and Promise patterns using a host-side synchronous wait bridge, allowing code written for modern JS environments to run on the more constrained MicroQuickJS engine while still accessing the full browser capability set.

[google-labs-jules]

Design AlloyScript REPL => It must use dual engine both microquickjs and webview (native browswer capacities)

The AlloyScript REPL has been designed and implemented to use the dual-engine architecture. It evaluates user input in the secure MicroQuickJS host process by default, with a hidden WebView standing by to provide browser-specific capabilities when requested via the transpiled API forwards.

[google-labs-jules]

Design AlloyScript REPL => It must use dual engine both microquickjs and webview (native browswer capacities)

Dual-engine REPL implementation is complete. It leverages the new secure IPC and Alloy.Transpiler logic to provide a seamless experience where the safe host process manages the core execution state.