Skip to content

fix(server-utils): Don't capture AI client errors as unhandled at the instrumentation level - #23024

Open
lux-in-tenebris-lucet wants to merge 1 commit into
getsentry:developfrom
lux-in-tenebris-lucet:fix/ai-instrumentation-error-capture
Open

fix(server-utils): Don't capture AI client errors as unhandled at the instrumentation level#23024
lux-in-tenebris-lucet wants to merge 1 commit into
getsentry:developfrom
lux-in-tenebris-lucet:fix/ai-instrumentation-error-capture

Conversation

@lux-in-tenebris-lucet

@lux-in-tenebris-lucet lux-in-tenebris-lucet commented Aug 4, 2026

Copy link
Copy Markdown

The exported instrumentOpenAiClient, instrumentAnthropicAiClient and instrumentGoogleGenAIClient wrappers called captureException(..., { mechanism: { handled: false } }) and then rethrew. The SDK decided the error was an unhandled crash before the application's retry or fallback logic ran. A call that succeeded on retry still reported a crash, and every retry reported another one.

This applies the convention already established for the channel-based OpenAI integration in #21877, letting the error bubble to the boundary that knows the outcome, to the manual client instrumentation. On the edge and serverless runtimes that is the only available path.

Fixes #23023

Why this matters beyond the flag

handled feeds release health, so the effect lands on numbers teams act on rather than on a label.

Crash-free rate stops describing reality. An unhandled error marks the session crashed, so a 429 the app retried successfully costs the same crash-free rate as a real crash. Deploy during a provider blip and release health flags the new version as a regression. Rolling back a good deploy is the expensive part.

On-call gets paged for failures the app already absorbed. Volume-threshold alerts fire on retry storms that never reached a user. Unhandled issues are auto-prioritized, so recovered transient failures outrank real bugs in triage.

Provider outages are when this hurts most. One retried request bills 3 to 5 events instead of 0. During an incident that spike is large, and orgs near their quota start dropping real errors, so an OpenAI outage costs teams visibility into unrelated production bugs.

The same failure is reported twice with conflicting metadata. The wrapper captures and rethrows, so when the error really is fatal the application boundary captures it again. dedupeIntegration is not in the Node defaults, so both land: one handled: false, one with the caller's actual status.

AI apps hit this harder than other integrations because the multipliers are standard practice here. Retry with backoff against providers that rate-limit aggressively. Fallback chains across models and vendors. Agent loops making 10 to 30 calls per user action. Errors that are ordinary control flow rather than faults, like context_length_exceeded handled by truncating and retrying, or a content-policy refusal the app renders as a normal message.

Teams who cannot fix the noise tend to remove its source. Muting the issue is one thing, but dropping the AI integration to stop false crash reports gives up the gen-AI spans, token accounting and latency data with it.

Reasoning

Removing the capture rather than adding an option. An opt-out (captureErrors: false, as @sentry/cloudflare has) would work, but #21877 chose removal for the channel-based path and framed it as matching the DB/cache channel subscribers. An option here would leave the two OpenAI paths behaving differently and add API surface the v11 direction is shedding. Error span status is unchanged, so the failure is still visible in tracing.

Scoped to errors that are rethrown to the caller. The captures that remain are the ones where a provider reports an error as data on an otherwise successful call: Anthropic's error stream events and error-shaped responses (isErrorEvent, handleResponseError), and Google GenAI's blocked-content signal. The caller never observes those as a thrown error, so removing them would silently drop the signal. The distinction is load-bearing rather than a judgement call. Removing the .ignore('event') masks left exactly those three tests emitting events, and they are the three that keep the mask with a comment explaining why.

mechanismType threading removed. It existed only to label the capture in createWithResponseWrapper, so it is dropped from wrapPromiseWithMethods and its four call sites. That exposed the helper's .catch() as a rethrow into a promise nothing observes, which is not a handler: awaiting the two promises in sequence orphans the second whenever the first rejects. They are awaited together now.

Integration tests now assert the absence. The AI suites previously dropped these events with .ignore('event'), which is why the behavior went unnoticed. Those masks are removed so the suites fail if instrumentation-level capture comes back. The anthropic suite also asserted the divergence directly, expecting the model-error event only while orchestrion was disabled; both paths agree now, so that branch is gone.

Root cause

The capture sat inside the wrapped method, upstream of every application-level decision about the error. Rethrowing after capturing means a fatal failure is recorded twice and a retried one is recorded once per attempt.

@lux-in-tenebris-lucet
lux-in-tenebris-lucet marked this pull request as ready for review August 4, 2026 14:35
@lux-in-tenebris-lucet
lux-in-tenebris-lucet requested review from a team as code owners August 4, 2026 14:35
@lux-in-tenebris-lucet
lux-in-tenebris-lucet requested review from isaacs and mydea and removed request for a team August 4, 2026 14:35
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

👋 @isaacs, @mydea, @nicohrubec, @andreiborza, @getsentry/team-javascript-sdks — Please review this PR when you get a chance!

… instrumentation level

The exported instrumentOpenAiClient, instrumentAnthropicAiClient and
instrumentGoogleGenAIClient wrappers captured provider errors with
mechanism.handled = false and then rethrew, so the SDK classified the
error as an unhandled crash before the application's retry or fallback
logic ran. A call that succeeded on retry still produced an unhandled
event, and each retry produced another one.

Applies the convention established for the channel-based OpenAI
integration in getsentry#21877 to the manual client instrumentation, which is the
only available path on the edge and serverless runtimes. Error span
status and the original error identity are unchanged.

The anthropic suite asserted this divergence directly, expecting the
model-error event only while orchestrion was disabled. Both paths agree
now, so that branch and its expectation are gone.

Captures are kept where a provider reports an error as data on an
otherwise successful call, since the caller never sees those as a thrown
error. The AI integration suites no longer mask these events with
.ignore('event'), so they fail if the capture returns.

Dropping the capture also left the .catch() in createWithResponseWrapper
rethrowing into a promise nothing observes, which is not a handler at
all: awaiting the two promises in sequence orphans the second whenever
the first rejects. They are awaited together now.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@lux-in-tenebris-lucet
lux-in-tenebris-lucet force-pushed the fix/ai-instrumentation-error-capture branch from c7facfc to 4c52da8 Compare August 8, 2026 13:01
Comment on lines +194 to +197
return originalResult.then(result => {
addResponseAttributes(span, result, options.recordOutputs);
return result;
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The original promise from the AI SDK (originalResult) lacks a rejection handler, which will cause Node.js to report an unhandled promise rejection if the API call fails.
Severity: HIGH

Suggested Fix

Attach a rejection handler to the originalResult promise to prevent the unhandled rejection warning. This can be done by adding a no-op .catch(() => {}) or by passing a rejection handler to the .then() call, like originalResult.then(onFulfilled, (err) => { throw err; }), to ensure the rejection is considered handled while still propagating it down the promise chain.

Prompt for AI Agent
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: packages/server-utils/src/ai/openai/index.ts#L194-L197

Potential issue: In the non-streaming instrumentation for OpenAI and Anthropic, the
promise returned by the SDK, `originalResult`, is only chained with a
`.then(onFulfilled)` handler, not a rejection handler. If the underlying API call fails,
`originalResult` will reject. Because no `.catch()` or `onRejected` handler is attached
directly to this promise before the next event loop tick, Node.js will detect and report
an unhandled promise rejection. While the error is eventually handled in the
instrumented promise chain, the initial unhandled rejection on `originalResult` can
cause process warnings or termination depending on the Node.js environment
configuration.

Also affects:

  • packages/server-utils/src/ai/anthropic-ai/index.ts:316~319

Did we get this right? 👍 / 👎 to inform future reviews.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

AI client instrumentation reports caller-handled provider errors as unhandled, once per retry

2 participants