Skip to content

test: fix pre-existing unit-test failures (green the suite) - #179

Draft
07souravkunda wants to merge 1 commit into
masterfrom
chore/green-unit-tests
Draft

test: fix pre-existing unit-test failures (green the suite)#179
07souravkunda wants to merge 1 commit into
masterfrom
chore/green-unit-tests

Conversation

@07souravkunda

Copy link
Copy Markdown
Collaborator

Why

Running npm test on master aborted the whole Mocha process partway
through (an uncaught exception thrown from inside an asynchronous callback),
so most of the suite never ran and its results were hidden. Several other
specs had also rotted against changes in lib/ (method signatures and the
download flow) and were failing or hanging. This makes it hard to tell
whether a red run is caused by a change under test or by the pre-existing
baseline.

This PR greens the deterministic unit tests and makes the run complete
cleanly, so a failing test can be trusted to mean something. Only
test/local.js changes; no production code is touched.

What was failing

Establishing a clean baseline on master (npm test, mocha over test/):

  • The run crashed at Local > should stop local: the test's assertion
    runs inside the tree-kill callback fired by Local.stop, i.e. outside
    Mocha's synchronous try/catch, so the throw became an uncaught exception
    that killed the Node process. Every test after it (Start sync,
    LocalBinary > *) was silently skipped.
  • Once the crash was worked around, the following were also red/broken:
    • LocalBinary > Download Path (5 specs): TypeError: Cannot read properties of undefined (reading 'proxyHost').
    • LocalBinary > Retries: the before() hook hung until its 10-minute
      timeout.

What I fixed (test rot / harness only)

  • Harness — uncaught throw aborts the whole run. Added a small
    check(done, assertions) helper. Assertions that execute inside async
    callbacks (the stop tree-kill callback, the binary-download callbacks)
    now run in a try/catch and route failures through done, so a failing
    assertion is a normal single-test failure and the run continues to the end.
  • Retries hang — stale method signature. LocalBinary.binaryPath gained
    a bsHost parameter — (conf, bsHost, key, parentRetries, callback) — but
    the specs still called the old 4-arg form, so the callback landed in the
    wrong slot and was never invoked, hanging before(). Passed the missing
    bsHost argument.
  • Retries stale value. A spec asserted download() was invoked with a
    retry count of 5; the default budget is now 9 (baseRetries). Updated
    the expectation.
  • Download Path — stale API. These specs called getDownloadPath() with
    the old synchronous no-arg signature and asserted a hard-coded public URL.
    getDownloadPath is now asynchronous and prefixes a dynamically fetched
    source URL; the OS/arch → filename mapping they were actually validating now
    lives in getBinaryFilename(). Rewrote them to assert getBinaryFilename()
    directly (renamed the block to Binary filename).

What I deliberately left red (real product behaviour — NOT masked)

  • Local > should stop local. It asserts isRunning() === false
    immediately after stop()'s callback fires. stop() sends SIGTERM via
    tree-kill and calls back as soon as the signal is dispatched — it does not
    wait for the process to exit, and it never clears its own
    pid / isProcessRunning state. So isRunning() keeps returning true
    for a few seconds after stop() "completes" (it flips to false only once
    the OS process actually dies). This is a genuine race in stop()'s
    completion semantics, so the test is left failing rather than weakened,
    sleep-padded, or deleted. It should be addressed in lib/Local.js
    (e.g. clear the running state in stop, and/or resolve only after exit).

Not addressed here (credentialed integration tests)

LocalBinary > Download (3 specs) download a real ~36 MB binary over the
network from an authenticated endpoint. They require a valid
BROWSERSTACK_ACCESS_KEY and network access, and currently never call back
without one (the download helper only logs on a source-URL fetch error and
never invokes its callback, so the test waits out its 10-minute timeout).
These are integration tests, not offline unit tests, so they are left
untouched here rather than force-mocked.

How to run

npm ci        # or: npm install
npm test      # eslint (lib/) then mocha over test/

The start/stop and download tests need a valid BROWSERSTACK_ACCESS_KEY in
the environment and network access. With a key present, the suite now runs to
completion: the unit tests are green, should stop local is the one
remaining product-behaviour failure described above, and the Download
integration specs need network + credentials as noted.

Repair test rot and a harness bug in test/local.js so the suite runs to
completion and reports per-test results instead of aborting mid-run.

Harness fix:
- Assertions inside asynchronous callbacks (the tree-kill callback from
  Local.stop, and the binary-download callbacks) run outside Mocha's
  synchronous try/catch. A throw there escaped as an uncaught exception and
  aborted the whole Mocha process, hiding every test that had not run yet.
  Added a small `check(done, assertions)` helper that runs the assertions in
  a try/catch and routes any failure through `done`, so a failing assertion
  is reported as a normal test failure and the run continues.

Test rot fixed:
- LocalBinary.binaryPath gained a `bsHost` parameter
  (conf, bsHost, key, parentRetries, callback), but the Retries specs still
  called the old 4-arg form, so the callback landed in the wrong slot and was
  never invoked -> the before() hook hung until its 10-minute timeout. Passed
  the missing bsHost argument.
- The Retries spec asserted download() was called with a retry count of 5;
  the default retry budget is now 9 (baseRetries). Updated the expectation.
- The "Download Path" specs called getDownloadPath() with the old synchronous
  no-arg signature and asserted a hard-coded public URL. getDownloadPath is
  now async and prefixes a dynamically fetched source URL; the OS/arch ->
  filename mapping they were really validating now lives in
  getBinaryFilename(). Rewrote them to assert getBinaryFilename() directly.

Left red on purpose (real product behaviour, not masked):
- "should stop local" asserts isRunning() === false immediately after
  stop()'s callback fires. stop() now sends SIGTERM via tree-kill and calls
  back as soon as the signal is dispatched, without waiting for the process
  to exit or clearing its own pid/isProcessRunning state, so isRunning() stays
  true for a few seconds. This is a genuine product race and is deliberately
  left failing rather than weakened.

No production code changed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant