test: hermetic package-manager caches — squash of #142/#143/#144 + review fixes + hosted-e2e unblock

[Mikola Lysenko (mikolalysenko)]

One reviewable unit combining John-David Dalton (@jdalton)'s cache-isolation stack (#142, #143, #144) with the fixes from a 13-agent adversarial review, plus the change that turns the hosted-e2e required check green again. If we merge this, the three original PRs can be closed as superseded — their commits are credited via Co-authored-by. If the team prefers merging jdalton's stack as-is instead, the review-fix commits here can be rebased into a small follow-up; either path works.

What's in here

The stack, squashed (unchanged in substance):

• test(setup-matrix): give each case its own home directory #142 — setup-matrix cases get their own HOME under WORKDIR (host-mode runs no longer install into the developer's real home).
• test(cli): give the Rust test suite its own package-manager caches #143 — tests/common/cache_env.rs + isolate() across ~24 CLI integration suites (3,601 leaked files → 0, per jdalton's measurements, which we independently spot-checked).
• test(hosted): isolate package-manager caches in the production e2e suite #144 — the same one-liner for e2e_hosted_production.rs.

Review fixes on top (each from a finding that survived adversarial verification):

1. hosted-e2e unblock — the required check has been red on every branch since 2026-08-02, including branches with none of these changes. Root cause is server-side: depscan#23630 (merged 07-31, deployed in the 08-01→08-02 window) made the gem patch-registry's compact-index routes fail closed (404 {"error":"not_built"}) until a re-queued per-package rebuild completes — and that rebuild hasn't completed for the activestorage patch. Bundler then falls back to /api/v1/dependencies, which serves HTTP 200 with a zero-byte body, and unmarshalling dies. The gem leg's known-defect matcher only accepted the old APIResponseMismatchError signature, so it flagged this as a new regression. This PR widens the matcher with a tight conjunction (Retrying dependency api due to error and the unmarshal error) covering both bundler generations: marshal data too short (bundler 2.x classic Marshal — what CI runs) and bytes' for nil (SafeMarshal, ruby 3.4+/bundler 4). Verified live: the leg passes against the current production registry with this change (took the known-defect branch; all redirect hard-assertions still enforced; SOCKET_PATCH_HOSTED_E2E_GEM_STRICT=1 still promotes to hard failure).
2. e2e_npm.rs macOS auto-discovery leak — the test spawns the CLI with scan -g; the binary's yarn global dir/pnpm root -g probes downloaded ~900 corepack files into the real home. Fixed with the same partial COREPACK_HOME/npm_config_cache pin global_packages_e2e.rs uses (full isolate() would defeat real-prefix resolution, which is the test's point).
3. Maven escaped test(setup-matrix): give each case its own home directory #142's sandbox entirely — the JVM derives user.home from passwd, not $HOME, so mvn kept writing the real ~/.m2 while resolve_target's maven arm scanned the always-empty fake home (silent assertion weakening; if maven apply ever starts working the matrix could never observe it). Fixed with MAVEN_OPTS=-Duser.home=$WORKDIR/home.
4. run-case.sh toolchain roots — test(setup-matrix): give each case its own home directory #142 preserved only RUSTUP_HOME; on a --host machine using rbenv/pyenv/nvm/volta/asdf/sdkman the redirected HOME made those shims unable to find their roots. Now seeded from the real home (only-if-unset + dir-exists, mirroring cache_env.rs's TOOLCHAIN_ROOTS), plus the ~/.tool-versions copy.
5. Probe/install environment agreement — bundler_version(), the ruby API probe, and find_python/find_uv were the only spawns in the stack not isolated; a version-manager setup could make the probe and the real install disagree (silent SKIP or spurious run). Now isolated, matching has_command everywhere else — including in e2e_hosted_production.rs, which test(hosted): isolate package-manager caches in the production e2e suite #144 missed.
6. GEM_HOME/GEM_PATH invariant documented in cache_env.rs — deliberately not pinned (rvm/chruby resolve bundle itself through them); safe today because every gem/bundle install in the suites passes --install-dir/BUNDLE_PATH.

Review provenance

Five parallel reviewers (helper internals, both halves of the call sites, the shell change, CI-interplay) + live forensics on the red check, with adversarial verification of every critical/major finding. Verdict on the stack itself: no breakage of existing testing infrastructure — Docker e2e/coverage suites untouched, CI setup-matrix containers unaffected, the e2e --ignored matrix legs re-pin their env after isolate() (last-write-wins, self-tested), Windows sandbox paths stay within MAX_PATH, and no CI job restores caches into locations isolate() bypasses (no actions/cache into $HOME package-manager paths). Notable refuted candidates (checked, not real): the shared persistent yarn cache masking the berry tampered-tarball negative control; container legs losing pre-baked corepack pins.

Verified

• cargo check -p socket-patch-cli --tests, cargo clippy --tests (no new warnings; the few pre-existing ones reproduce on main), bash -n + touched-file rustfmt on the pinned 1.93.1 toolchain.
• cache_env self-tests pass (5/5).
• The hosted gem leg run live against production: passes with the widened matcher (48s), fails identically on main.

Known residuals (deliberately out of scope, same as jdalton's deferrals)

• in_process_cargo_apply / in_process_remote_ecosystems_apply still leak (~900 files each): in-process apply shells out from the test process itself; fixing means mutating process env, which this repo has had races with (test: fix hostile-decoy env race flaking the setup-matrix and e2e suites #135/test: exclude crawler-e2e siblings from #[serial] env swaps via #[parallel] #136).
• The hosted-e2e CI job's corepack prepare --activate warm step now warms the wrong (real) COREPACK_HOME — dead weight, ~seconds of re-download per run; workflow-side follow-up.
• The real fix for the gem registry lives in depscan: unstick the re-queued gem package rebuild so package_gem_index_deps populates, and make /api/v1/dependencies stop serving zero-byte 200s. Once the index rebuilds with deps, the leg's success branch prints its loud NOTE to delete the tolerance entirely.

Supersedes #142, #143, #144 (author credited).

🤖 Generated with Claude Code

---

Note

Low Risk
Changes are confined to test harnesses and setup-matrix shell env; production CLI behavior is unchanged aside from hosted-e2e tolerance for a known registry defect.

Overview
Adds tests/common/cache_env.rs with isolate() to redirect package-manager caches (npm, pnpm, yarn, corepack, Go GOCACHE/GOMODCACHE, cargo, pip/uv, gem, composer, nuget, etc.) into a shared temp sandbox, preserve version-manager roots when needed, and ship self-tests so critical env vars stay pinned.

Wires cache_env::isolate through shared helpers (has_command, run_toolchain, cargo_run) and ~24 CLI integration suites so real installs and corepack probes no longer write to the developer’s home. Tests that must see real global prefixes (global_packages_e2e, parts of e2e_npm) only pin COREPACK_HOME / npm_config_cache. Several yarn helpers fix scrub-then-isolate ordering so private cache overrides are not wiped.

tests/setup_matrix/run-case.sh redirects each case’s HOME and tool caches under WORKDIR, seeds RUSTUP_HOME and other VM roots from the real home, copies .tool-versions, and sets MAVEN_OPTS=-Duser.home so Maven’s ~/.m2 follows the case home.

e2e_hosted_production: isolates toolchain probes/installs; widens the gem leg’s known production defect matcher for post–depscan#23630 bundler failures (empty dependency API + marshal errors) so the required check can pass while redirect assertions stay strict.

^{Reviewed by Cursor Bugbot for commit f8cc6e5. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{Bugbot Autofix is ON. A cloud agent has been kicked off to fix the reported issue.}

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit f8cc6e5. Configure here.}