QuickJS engine: baseline-snapshot startup optimization (~25× faster VM boot) - #3342
QuickJS engine: baseline-snapshot startup optimization (~25× faster VM boot)#3342TooTallNate wants to merge 6 commits into
Conversation
Evaluating the workflow bundle dominates VM startup (~74ms of a ~77ms boot for the 1.3MB e2e bundle) and full event replay pays it on EVERY invocation — a large share of the quickjs engine's TTFS gap vs node:vm, where V8 compiles the same script in single-digit ms. The bundle is identical across all runs of a deployment, so the engine now hydrates one VM per function instance (bootstrap + bundle eval), snapshots its memory, and starts every invocation with QuickJS.restore (~3ms) instead of re-evaluating. Measured on the real generated e2e flow bundle (154 workflows), boot to first suspension: fresh 79.4ms -> restored 3.2ms (24.8x). First invocation pays hydrate+restore (85.8ms, ~= one fresh boot); every subsequent invocation — including every replay wake — gets the discount. Determinism: replay requires module-scope user code to observe the run-seeded PRNG and deterministic clock, and a restored heap carries whatever module scope computed at hydrate time. The hydrate therefore runs with draw-counting placeholder host fns and a read-counting clock; a bundle that consumed either is marked ineligible and every invocation falls back to fresh evaluation (node:vm-parity semantics preserved exactly). When the gate passes, restore is byte-equivalent to fresh eval: the per-run host fns (random / __generateNanoid / __generateUlid) re-register by NAME on the restored VM before the workflow body runs, so the seeded draw sequence — and every correlationId — is identical. Pinned by a parity test that feeds Math.random() into a step input and byte-compares the serialized ops across fresh, first-restore and cached-restore invocations. Cache: per function instance, keyed on the bundle string (reference-stable in generated flow routes), promise-deduped for concurrent first invocations, capped at 4 entries; hydrate rejections evict for retry while eval failures cache as ineligible (the fresh path re-evaluates and surfaces the real, source-mapped error). Kill switch: WORKFLOW_QUICKJS_BASELINE_SNAPSHOT=0.
🦋 Changeset detectedLatest commit: 7f93aef The changes in this PR will be included in the next version bump. This PR includes changesets to release 16 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
🧪 E2E Test Results❌ Some tests failed ❌ Failed E2E Tests▲ Vercel Production (1 failed)nextjs-turbopack-node (1 failed):
💻 Local Development (1 failed)nextjs-webpack-canary-node (1 failed):
E2E Test SummarySummary
Details by Category❌ ▲ Vercel Production
❌ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
✅ vercel-multi-region
|
There was a problem hiding this comment.
Pull request overview
Implements a QuickJS runtime startup optimization by caching a baseline VM-memory snapshot of a bundle-hydrated VM and restoring it for subsequent invocations, significantly reducing per-invocation boot overhead while preserving replay determinism via eligibility gating and host-fn re-registration.
Changes:
- Add baseline-snapshot hydrate/restore path in the QuickJS runtime, keyed by workflow bundle and gated against module-scope nondeterminism (PRNG/clock).
- Add unit tests covering fresh-vs-restore byte parity, full replay through restore, and ineligibility gates.
- Introduce an env-var kill switch (
WORKFLOW_QUICKJS_BASELINE_SNAPSHOT=0) and ship a changeset for@workflow/core.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| packages/core/src/runtime/quickjs-runtime.ts | Adds baseline snapshot cache + hydrate/restore logic and deterministic clock wiring for restores. |
| packages/core/src/runtime/quickjs-runtime.test.ts | Adds a focused test suite validating restore parity, replay completion, and gating behavior. |
| packages/core/src/runtime/constants.ts | Adds isQuickJSBaselineSnapshotEnabled() kill switch helper. |
| .changeset/quickjs-baseline-snapshot.md | Publishes the runtime optimization as a patch-level changeset for @workflow/core. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| /** | ||
| * WASI clock override reading the given accessor — shared between fresh | ||
| * boots (initWorkflowVM) and baseline-snapshot restores, so both paths | ||
| * expose the same deterministic replay clock. | ||
| */ |
There was a problem hiding this comment.
initWorkflowVM now calls the shared makeDeterministicClockWasi helper (the helper was hoisted above it), so the doc claim is true and the two implementations cannot drift. (a2adae0)
| __clearBaselineSnapshotCacheForTests(); | ||
| process.env.WORKFLOW_QUICKJS_BASELINE_SNAPSHOT = '0'; | ||
| let fresh: Awaited<ReturnType<typeof runQuickJSWorkflow>>; | ||
| try { | ||
| fresh = await runQuickJSWorkflow({ | ||
| workflowCode: stepRaceCode, | ||
| workflowId: 'workflow//test//workflow', | ||
| workflowRun: makeRun(), | ||
| events: [], | ||
| }); | ||
| } finally { | ||
| delete process.env.WORKFLOW_QUICKJS_BASELINE_SNAPSHOT; | ||
| } |
There was a problem hiding this comment.
Fixed — the test now saves any pre-existing value and restores it (or deletes only when originally unset). (a2adae0)
| /** | ||
| * Whether the QuickJS engine's baseline-snapshot startup optimization is | ||
| * enabled (default ON). When on, the engine hydrates a VM with the | ||
| * workflow bundle once per function instance, snapshots it, and starts | ||
| * every invocation by restoring the snapshot instead of re-evaluating |
There was a problem hiding this comment.
Fixed — the baseline-snapshot block moved below isTurboEnabled, so the WORKFLOW_TURBO JSDoc is reattached to its function. (a2adae0)
| const vm = baselineSnapshot | ||
| ? await QuickJS.restore(baselineSnapshot, { | ||
| wasm: (await getCompiledAssets()).wasm as never, | ||
| memoryLimit: 256 * 1024 * 1024, | ||
| interruptHandler: createInterruptHandler(interruptBudget), | ||
| extensions: (await getCompiledAssets()).extensions, | ||
| wasi: makeDeterministicClockWasi(() => vmNowMs), |
There was a problem hiding this comment.
Fixed — single assets local in the restore branch (restructured to an if/else so the await sits outside the options object). (a2adae0)
![vercel[bot]](https://avatars.githubusercontent.com/in/8329?s=60&v=4){kind=small}
…, shared clock helper, review nits - Serialization-intrinsics gate (the substantive finding): the restore path's serde captures intrinsics from the restored heap — AFTER module scope ran — while the fresh path captures before user code. A bundle that replaced a captured intrinsic at module scope (e.g. a Date.prototype.toISOString polyfill) without touching PRNG/clock passed the eligibility gate yet would serialize differently on the two paths. captureIntrinsicsSignature (exported from quickjs-serde) identity-fingerprints every to-be-captured value; the hydrate compares it before and after bundle eval and marks any replacement ineligible. Expression-created entries (makeSparseArray, makeThunk, hasOwnCall) are excluded — they get fresh identities per eval and cannot be replaced by user code. Gate test added with a toISOString polyfill. - Hydrate-failure fallback: a getBaselineEntry rejection (infrastructure — vm.snapshot() under memory pressure, QuickJS.create failing) no longer fails the invocation; it logs a warning and falls back to fresh evaluation, with the cached promise already evicted for retry. - initWorkflowVM now uses the shared makeDeterministicClockWasi helper its doc claimed it shared, so the two clock implementations cannot drift. - getCompiledAssets() awaited once per call site (hydrate + restore). - WORKFLOW_TURBO JSDoc reattached to isTurboEnabled (the baseline constant had been inserted between doc and function). - Parity test saves/restores any pre-existing WORKFLOW_QUICKJS_BASELINE_SNAPSHOT env value instead of deleting it.
karthikscale3
left a comment
karthikscale3
left a comment
There was a problem hiding this comment.
Focused review: one reproducible snapshot-equivalence blocker.
| * path. Missing captures (absent extension globals) signature as -1. | ||
| */ | ||
| export function captureIntrinsicsSignature(vm: QuickJS): number[] { | ||
| using captured = vm.evalCode(CAPTURE_INTRINSICS); |
There was a problem hiding this comment.
[P1 blocker] The eligibility probe can execute user-patched capture helpers while still classifying the bundle as snapshot-safe. CAPTURE_INTRINSICS calls Object.getOwnPropertyDescriptor and Object.getPrototypeOf, but those helpers are not themselves included in the identity signature. I reproduced this with module scope wrapping Object.getOwnPropertyDescriptor in a forwarding function that increments a counter, then returning that counter from the workflow. The gate reported ready; fresh evaluation returned 0, while snapshot restore returned 24 because the post-eval probe and restore-time serde capture mutated module state. This violates fresh/restore equivalence and can change durable output during mixed-fleet rollout. Please include every helper used to construct the capture in the signature—at least Object.getOwnPropertyDescriptor and Object.getPrototypeOf—or compute the post-eval signature exclusively through pre-user-code host-held handles, and add this forwarding-wrapper regression test.
There was a problem hiding this comment.
Excellent repro — and it exposed that the gate approach was structurally losing, not just incomplete. Adding Object.getOwnPropertyDescriptor/getPrototypeOf to the signature would have patched YOUR wrapper, but any probe that executes guest-reachable code post-eval both (a) needs its own dependencies gated recursively and (b) bakes its side effects into the snapshot even when the gate passes (stateful getters on captured properties being the next hole in line).
Fixed in 7f93aef by prevention instead of detection: all guest-touching serde initialization (intrinsics capture, branded samples, symbol lookups) is bundled into one CAPTURE_ROOT expression evaluated in the baseline VM before the bundle — the same capture-before-user-code ordering the fresh path has always had. The container's handle box lives in the snapshot's linear memory, its raw pointer rides the BaselineEntry, and every restored VM re-adopts it by pointer (adoptSerdeRoot) — serde init then performs only plain-data property reads and C-level classId reads, so no guest code executes after user code has run, on either path. Your counter repro is now a regression test asserting fresh === restore === 0. The identity-signature gate is deleted outright (module-scope intrinsic patching is now harmless, not merely detectable — polyfill bundles become ELIGIBLE and byte-identical across paths), and process.env injection moved from guest-source eval to handle-based install for the same reason (its JSON.parse ran post-eval on the restore path only).
…e64) into quickjs-baseline-snapshot The new escapeString captured intrinsic is expression-created (a fresh guest closure per capture eval), so it joins makeSparseArray/makeThunk/ hasOwnCall in captureIntrinsicsSignature's exclusion list — without this the baseline hydrate gate would classify every bundle ineligible (the byte-parity test catches exactly that, as it did when hasOwnCall was missed).
…the snapshot The intrinsics-replacement gate was structurally losing: its own post-eval probe executed guest-reachable code (CAPTURE_INTRINSICS calls Object.getOwnPropertyDescriptor / Object.getPrototypeOf), those dependencies were not in the identity signature, and a module-scope stateful wrapper around them both evaded detection AND had its side effects baked into the snapshot — fresh returned 0 from the reviewer's counter repro while restore returned the probe's call count. Replace detection with prevention: ALL guest-touching serde initialization (intrinsics capture, branded samples, well-known symbol lookups) is bundled into one CAPTURE_ROOT expression evaluated in the baseline VM BEFORE the bundle — the same capture-before-user-code ordering the fresh path has always had. The container handle's box lives in the snapshot's linear memory, its raw pointer rides the BaselineEntry, and every restored VM re-adopts it (adoptSerdeRoot) — serde init then performs only plain-data property reads and C-level classId reads: NO guest code executes after user code has run, on either path. Consequences: - the identity-signature gate and its expression-created skip-list are deleted (nothing to detect — module-scope intrinsic patching is now HARMLESS on the snapshot path, not merely detectable) - polyfill bundles become ELIGIBLE for the optimization and serialize through pristine intrinsics identically on both paths (test flipped from gating to byte-equality) - process.env injection converted from guest-source eval to handle-based installProcessEnv (captured Object.freeze + vm.hostToHandle): the old evalCode ran JSON.parse post-eval on the restore path only, the same observable-divergence class - the reviewer's stateful-wrapper repro is a regression test: the counter must be zero and identical across fresh and restored invocations
Stacked on #3263 (uses the host-serde engine's per-VM intrinsics capture, which re-runs cleanly against a restored heap).
Problem
Evaluating the workflow bundle dominates QuickJS VM startup — ~74ms of a ~77ms boot for the real generated e2e flow bundle (1.3MB, 154 workflows) — and full event replay pays it on every invocation, not just the first. This is a large share of the engine's TTFS gap vs node:vm, where V8 compiles the same script in single-digit milliseconds.
Approach
The bundle is identical across all runs of a deployment. The engine now hydrates one VM per function instance (bootstrap + bundle eval), snapshots its memory (quickjs-wasi
snapshot()/restore()), and starts every invocation by restoring the snapshot instead of re-evaluating:Every replay wake gets the discount, not just run start.
Determinism
Replay requires module-scope user code to observe the run-seeded PRNG and the deterministic clock; a restored heap carries whatever module scope computed at hydrate time. Two safeguards:
random/__generateNanoid/__generateUlidcallbacks re-register by NAME on the restored VM before the workflow body runs (quickjs-wasi restore semantics), so the seeded draw sequence — and every correlationId — is byte-identical to fresh eval. Pinned by a parity test that feedsMath.random()into a step input and byte-compares serialized pending ops across fresh, first-restore, and cached-restore invocations.Mechanics
WORKFLOW_QUICKJS_BASELINE_SNAPSHOT=0Testing
WORKFLOW_VM=quickjswith the snapshot path liveFollow-ups