test(vendored): real-production e2e for scan --mode vendored across every package manager - #169
Merged
Mikola Lysenko (mikolalysenko) merged 1 commit intoAug 13, 2026
Conversation
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.
Bugbot Autofix is ON. A cloud agent has been kicked off to fix the reported issues.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 4a2f836. Configure here.
… every package manager Adds `crates/socket-patch-cli/tests/e2e_vendored_production.rs`, the vendored-mode counterpart to `e2e_hosted_production.rs`. It drives `scan --mode vendored` against the REAL Socket production service and REAL registries on the anonymous free public proxy (no token, `SOCKET_NO_CONFIG=true`), and proves the full vendored loop per package manager: install pristine -> vendor the real free-tier patch into `.socket/vendor/<eco>/<uuid>/` -> cold reinstall from ONLY the committable files (empty caches, offline) -> the installed bytes are the vendored (patched) bytes, not the pristine registry bytes -> idempotent re-run -> revert byte-restores. Byte delivery, not CVE efficacy (matches the capstones). `#[ignore]`-gated; soft-skips absent toolchains unless `SOCKET_PATCH_VENDORED_E2E_STRICT=1`. Companion doc in docs/testing/. Legs: npm, pnpm, yarn classic, yarn berry (node-modules), bun, pip (requirements.txt), uv (uv.lock), cargo; preflight patch canary; golang zero-patch, deno unsupported, and maven/nuget/composer canary negatives. Two real CLI gaps surfaced and marked xfail (tests-only PR, not fixed here): - pnpm >= 11 ignores package.json `pnpm.overrides` (moved to pnpm-workspace.yaml), so a frozen install refuses ERR_PNPM_LOCKFILE_CONFIG_MISMATCH even though the vendored artifact is correct; the leg proves delivery via the documented workaround and fails loudly on any other error. - gem vendoring refuses the platform-qualified purl (`?platform=ruby`) with `platform_gem_unsupported`; the leg asserts redirect+download and tolerates the vendor failure, auto-retiring once fixed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Mikola Lysenko (mikolalysenko)
force-pushed
the
test/e2e-vendored-production
branch
from
August 13, 2026 18:44
4a2f836 to
7c8a3f2
Compare
Wenxin Jiang (Wenxin-Jiang)
approved these changes
Aug 13, 2026
Mikola Lysenko (mikolalysenko)
enabled auto-merge (squash)
August 13, 2026 21:39
Mikola Lysenko (mikolalysenko)
deleted the
test/e2e-vendored-production
branch
August 13, 2026 21:39
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

What
Adds
crates/socket-patch-cli/tests/e2e_vendored_production.rs— the vendored-modecounterpart to
e2e_hosted_production.rs. The synthetice2e_vendor_*_build.rscapstones prove the vendoring mechanism with a hand-staged blob and
vendor --offline; they never contact production. This suite drivesscan --mode vendoredagainst the real Socket production service and realupstream registries, with no mocking anywhere, on the anonymous free public
proxy (no token;
SOCKET_NO_CONFIG=true, everySOCKET_*scrubbed with hostileseeds so a dropped scrub reddens the suite).
Per package manager it proves the full loop: install pristine (asserted pristine —
anti-vacuity) →
scan --mode vendoredmaterializes the real free-tier patchinto
.socket/vendor/<eco>/<uuid>/and rewires the lock/manifest → cold reinstallfrom ONLY the committable files (empty caches, offline) → the installed bytes are
the vendored (patched) bytes, not the pristine registry bytes → idempotent
re-run (
already_vendored, byte-stable lock) →vendor --revertbyte-restores.It proves byte delivery, not CVE efficacy — matching how the capstones assert.
#[ignore]-gated (stays out of thetest/e2ejobs), soft-skips absent toolchainsunless
SOCKET_PATCH_VENDORED_E2E_STRICT=1, cache-isolated viacommon/cache_env.rs. Reuses the hosted suite's pinned patch catalog. Companion doc:docs/testing/vendored-production-e2e.md.Coverage
npm cipnpm install --frozen-lockfile --offlineyarn install --frozen-lockfile --offlineyarn install --immutable --check-cachebun install --frozen-lockfilepip install --no-index -ruv sync --frozen --offline[patch.crates-io])cargo fetch --offline --lockedPlus
preflight_required_patches_are_published(catalog canary againstpatches-api.socket.dev, mirrors the hosted preflight).cargo uses
cargo fetch --offlinerather thancargo build: the productiontraitobject patch injects a
compile_error!unless theallow-unmaintainedfeatureis set (the patch's intent), so
buildfails by design — that's patch content, not avendoring defect. The leg proves the whole graph resolves from committable files with
zero registry downloads and byte-checks the vendored directory.
How to run
Knobs:
SOCKET_PATCH_VENDORED_E2E_GEM_STRICT=1(promote the gem gap to a hardfailure),
SOCKET_PATCH_VENDORED_E2E_CANARY_STRICT=1(fail when maven/nuget/composergain a free patch).
Notes
cargo test -p socket-patch-cli --test e2e_vendored_production --no-runcompilescleanly (the primary gate — the suite is
#[ignore]). Ran the full--ignoredsetlocally against production, all 13 tests green (npm 11.6.2, pnpm 11.18.0, yarn
4.6.0 / corepack yarn@1.22.22, bun 1.3.14, uv, python3/pip, cargo, ruby+bundle all
present; composer + deno absent but their legs need no toolchain):
preflight, golang (zero-patch), deno (unsupported), canary.
package.jsonpnpm.overrides(moved topnpm-workspace.yaml), so a frozen install refusesERR_PNPM_LOCKFILE_CONFIG_MISMATCHeven though the vendored tarball + lock arecorrect (the lockfile passes pnpm's supply-chain policy). The leg reproduces the
documented workaround (mirror the override into
pnpm-workspace.yaml) to provethe artifact IS installable, and fails loudly on any other error. Fix belongs in
the CLI: emit
pnpm-workspace.yaml overrideson pnpm ≥ 11.(
pkg:gem/activestorage@7.0.2.2?platform=ruby) withplatform_gem_unsupported(
status: partial_failure); download succeeds. The leg asserts redirect+downloadand tolerates the vendor failure, failing loudly on any other error and
auto-upgrading to a full
bundle installdelivery proof once the CLI vendorsplatform gems.
🤖 Generated with Claude Code
Note
Low Risk
Tests-only change with no CLI or runtime behavior modifications; live production tests are ignored by default and isolated from normal CI.
Overview
Adds
e2e_vendored_production.rs, a production counterpart to the hosted suite and the synthetice2e_vendor_*_buildcapstones:scan --mode vendoredagainst realpatches-api.socket.devand upstream registries on the free public proxy (scrubbedSOCKET_*,SOCKET_NO_CONFIG=true).Each leg runs install → pristine-byte check → vendoring → cold offline reinstall from committable files only (patched bytes, not registry) → idempotent re-scan →
vendor --revert. Coverage includes npm (npm/pnpm/yarn classic/berry/bun), PyPI (pip/uv), Cargo, plus preflight patch catalog checks, golang/deno guards, and a maven/nuget/composer canary.docs/testing/vendored-production-e2e.mddocuments patches, PM matrix, known CLI gaps (pnpm ≥11pnpm-workspace.yamloverrides, gemplatform_gem_unsupported), and strict env knobs. Tests are#[ignore]with optionalSOCKET_PATCH_VENDORED_E2E_STRICThard-fail on soft-skips.Reviewed by Cursor Bugbot for commit 4a2f836. Configure here.