fix(replay): Don't let a wedged video encoder freeze the app by romtsn · Pull Request #5842 · getsentry/sentry-java

romtsn · 2026-07-27T15:37:19Z

📜 Description

Fixes an ANR where Android apps freeze on foreground/background transitions until the system kills the process.

A reporter pulled the full deobfuscated 131-thread dump from Play Console (the Sentry ANR event only carries main, which is why the lock holder was invisible), giving us the exact chain:

main ──waits──▶ lifecycleLock ──held by──▶ timer thread
                                              │ waits
                                              ▼
                                          encoderLock ──held by──▶ SentryReplayIntegration-0
                                                                       │ parked forever
                                                                       ▼
                                                          MediaCodec.dequeueOutputBuffer

Two cooperating defects, so two bounds — both are needed:

SimpleVideoEncoder.drainCodec() had a while (true) with no exit on the endOfStream = true path. Some hardware encoders never emit BUFFER_FLAG_END_OF_STREAM after signalEndOfInputStream(), so the loop spun forever while holding encoderLock. It now gives up after 10 consecutive no-progress iterations (~1s at the existing 100ms poll) and drops the remaining frames.
ReplayCache.close() used an unbounded encoderLock.acquire(). It runs inline on the caller's thread from BaseCaptureStrategy.stop(), which ReplayIntegration.stop() invokes while holding lifecycleLock — that nesting is what promotes a stuck encoder into a frozen main thread. It now waits at most 2s and skips the release otherwise.

The loop bound alone isn't sufficient: the native frames show MediaCodec::dequeueOutputBuffer → AMessage::postAndAwaitResponse → ALooper::awaitResponse with MediaCodec_looper idle. TIMEOUT_USEC is handed to the codec looper, which is supposed to reply once it elapses — a dead looper never replies, and awaitResponse has no deadline of its own. So a single call can never return, and only bounding the lock wait keeps the lifecycle path free.

On the timeout path the (already-dead) codec is not released, leaking a native handle. That's the deliberate trade: leaking a handle in a process whose encoder is wedged beats freezing the app. isClosed.set(true) still runs on every path, since persistSegmentValues gates on it.

The other four encoderLock.acquire() sites in ReplayCache are unchanged — they all run on the replay worker, the thread that would be stuck, so a deadline there buys nothing.

Adds AutoClosableReentrantLock.tryAcquire(timeout, unit) (@ApiStatus.Internal, additive) for the second bound. Returns the token on success and null on timeout, so callers must branch explicitly rather than silently no-op via ?.use {}.

Net effect: a wedged codec degrades to "replay stops working for this process" instead of "the app freezes."

Fixes getsentry/sentry-dart#3556
Fixes #5870

💚 How did you test it?

ReplayShadowMediaCodec gained two hooks, since it ignores timeoutUs and always eventually produces EOS: neverSignalEos and blockOnDequeue. New tests cover both bounds in ReplayCacheTest, plus 4 for tryAcquire in AutoClosableReentrantLockTest.

Both ReplayCacheTest cases were mutation-checked — reverting each fix in turn makes the matching test fail with its assertion message rather than hang. Full :sentry-android-replay:testReleaseUnitTest passes; apiDump adds exactly the one tryAcquire line.

No device repro is available (Redmi Note 14 Pro 5G / Pro+ 5G, Android 16, MediaTek), so the shadow-based tests are the verification of record.

📝 Checklist

I reviewed the submitted code.
I added tests to verify the changes.
No new PII added or SDK only sends newly added PII if sendDefaultPii is enabled.
I updated the docs if needed.
No breaking change or entry added to the changelog.

🤖 Generated with Claude Code

runningcode · 2026-07-27T15:41:00Z

+ * [MediaCodec.signalEndOfInputStream], which used to spin the drain loop forever while holding the
+ * encoder lock, wedging the whole replay pipeline (and with it the app's lifecycle callbacks).
+ */
+private const val MAX_EOS_STALL_ITERATIONS = 10


how did we determine 10 here? Just curious. How fast do these iterations happen?

honestly just gut feeling (10 x 100ms = 1s wait at most). Unfortunately, I don't have a device at hand to reproduce it, but I will order one to actually check it.

On my two devices that I have (HMD Pulse and Pixel 2XL) each iteration takes exactly around 100ms (the full TIMEOUT_USEC), so we're well under a second here.

sentry · 2026-07-27T15:42:20Z

📲 Install Builds

Android

🔗 App Name	App ID	Version	Configuration
SDK Size	io.sentry.tests.size	8.52.0 (1)	release

⚙️ sentry-android Build Distribution Settings

github-actions · 2026-07-27T15:46:27Z

Performance metrics 🚀

	Plain	With Sentry	Diff
Startup time	292.27 ms	374.31 ms	82.04 ms
Size	0 B	0 B	0 B

Baseline results on branch: main

Startup times

Revision	Plain	With Sentry	Diff
`6b019b7`	319.84 ms	333.15 ms	13.31 ms
`d15471f`	322.58 ms	396.08 ms	73.50 ms
`b750b96`	421.25 ms	444.09 ms	22.84 ms
`d217708`	411.22 ms	430.86 ms	19.63 ms
`604a261`	380.65 ms	451.27 ms	70.62 ms
`5b1a06b`	352.27 ms	413.70 ms	61.43 ms
`fcec2f2`	357.47 ms	447.32 ms	89.85 ms
`7414e9b`	370.39 ms	422.18 ms	51.79 ms
`fcec2f2`	311.35 ms	384.94 ms	73.59 ms
`f6cdbf0`	314.19 ms	357.59 ms	43.40 ms

App size

Revision	Plain	With Sentry	Diff
`6b019b7`	0 B	0 B	0 B
`d15471f`	1.58 MiB	2.13 MiB	559.54 KiB
`b750b96`	1.58 MiB	2.10 MiB	533.20 KiB
`d217708`	1.58 MiB	2.10 MiB	532.97 KiB
`604a261`	1.58 MiB	2.10 MiB	533.42 KiB
`5b1a06b`	0 B	0 B	0 B
`fcec2f2`	1.58 MiB	2.12 MiB	551.50 KiB
`7414e9b`	0 B	0 B	0 B
`fcec2f2`	1.58 MiB	2.12 MiB	551.51 KiB
`f6cdbf0`	0 B	0 B	0 B

Previous results on branch: rz/fix/replay-encoder-wedge-anr

Startup times

Revision	Plain	With Sentry	Diff
`5c3296f`	373.74 ms	470.27 ms	96.53 ms
`22606d4`	321.23 ms	374.37 ms	53.13 ms

App size

Revision	Plain	With Sentry	Diff
`5c3296f`	0 B	0 B	0 B
`22606d4`	0 B	0 B	0 B

runningcode

looks good, i left a question for a discussion!

runningcode · 2026-07-27T15:46:38Z

+      val token = encoderLock.tryAcquire(ENCODER_RELEASE_TIMEOUT_MS, MILLISECONDS)
+      if (token == null) {
+        options.logger.log(
+          WARNING,


ok so just to check my understanding, we try to get the lock only to make sure nobody else is holding it before calling release but if we can't acquire the lock, what state are we in? is this safe to skip the release? won't we cause a memory leak?

yup, it could cause a memory leak, but that's the trade-off - if the encoder is wedged in the native layer, calling release would also hang in the native layer anyway. When the process dies the ref should get reclaimed still, so I guess we pick a lesser evil here.

0xadam-brown

A few quick comments for your consideration (no blockers); otherwise lgtm

0xadam-brown · 2026-07-28T08:21:14Z

-    encoderLock.acquire().use {
-      encoder?.release()
-      encoder = null
+    // close() is called inline from the lifecycle path (ReplayIntegration.stop/close), which holds


m: Thoughts about lifting this decision to the call site? That'd let us reduce how much we peg ReplayCache's implementation to its current caller.

Eg, we could:

create a new ReplayCache.tryClose() method and have ReplayIntegration call it instead; or

add a new ShutdownMode param to close() and have ReplayIntegration choose BEST_EFFORT.

Long-term side note: If we had a general pattern where, say, the Integration owned the execution context / threading decisions, we could leave everything beneath it thread _un_safe on the assumption that our Integrations would serialize execution via thread confinement by default. That'd make the bulk of our integration code much easier to implement and reason about.

I haven't looked in detail, but seems like that sort of pattern should be possible, given the very task-oriented nature of our processing + the fact that the host app needs very little from the SDK in real-time.

Some hardware encoders never emit BUFFER_FLAG_END_OF_STREAM after signalEndOfInputStream(), so SimpleVideoEncoder.drainCodec() spun forever while holding encoderLock. ReplayCache.close() then blocked on that lock, and since it runs inline under ReplayIntegration's lifecycleLock, the main thread froze until the system killed the process. Two bounds, both needed: the drain loop now gives up after 10 consecutive no-progress iterations (~1s), and close() only waits 2s for the encoder lock before skipping the release. The loop bound alone isn't enough -- a native dequeueOutputBuffer call can itself never return, since ALooper::awaitResponse has no deadline of its own. Adds AutoClosableReentrantLock.tryAcquire(timeout, unit) for the latter. Fixes getsentry/sentry-dart#3556 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

It's an abnormal condition that drops frames, and it fires at most once per segment, so it's worth surfacing without debug logging enabled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Move isClosed.set(true) into a finally block so it runs even on unchecked exceptions. Add a released flag to ReplayShadowMediaCodec and assert that close() releases the encoder when the lock is available, and skips it when timed out. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 228b6e9. Configure here.}

cursor · 2026-08-07T16:48:36Z

+    assertWithMessage("encoder should be released even when EOS was never signalled")
+      .that(ReplayShadowMediaCodec.released)
+      .isTrue()
+  }


Test named for close never invokes close

Low Severity

This test never calls close(); it only exercises createVideoOf, which releases the encoder internally and then nulls the field. Since the encoder reference is already cleared by the time the test ends, a subsequent close() would be a no-op, so the release-on-lock-acquired branch of close() remains uncovered despite the test name asserting otherwise.

^{Reviewed by Cursor Bugbot for commit 228b6e9. Configure here.}

cursor · 2026-08-07T16:48:36Z

+
+      assertWithMessage("encoder should not be released when the lock times out")
+        .that(ReplayShadowMediaCodec.released)
+        .isFalse()


Assertion cannot fail: encoder still null when wedged

Low Severity

While the codec is parked inside dequeueOutputBuffer, createVideoOf has not yet assigned the encoder field, so it is still null. Whether close() times out or acquires the lock, encoder?.release() is a no-op and released stays false, making this assertion pass unconditionally and giving false confidence that release is genuinely skipped on timeout.

^{Reviewed by Cursor Bugbot for commit 228b6e9. Configure here.}

romtsn requested review from 0xadam-brown, adinauer, markushi and runningcode as code owners July 27, 2026 15:37

runningcode reviewed Jul 27, 2026

View reviewed changes

0xadam-brown approved these changes Jul 28, 2026

View reviewed changes

runningcode approved these changes Jul 29, 2026

View reviewed changes

romtsn mentioned this pull request Jul 31, 2026

Permanent app freeze on foreground: main thread blocks in ReplayIntegration.start while replay executor hangs in MediaCodec.dequeueOutputBuffer holding the replay lock (HyperOS 3 / Android 16) #5870

Open

romtsn and others added 3 commits August 7, 2026 18:39

fix(replay): Log the encoder give-up at WARNING

7b44166

It's an abnormal condition that drops frames, and it fires at most once per segment, so it's worth surfacing without debug logging enabled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

romtsn force-pushed the rz/fix/replay-encoder-wedge-anr branch from 6d7152c to 228b6e9 Compare August 7, 2026 16:39

cursor Bot reviewed Aug 7, 2026

View reviewed changes

Uh oh!

Uh oh!

Conversation

romtsn commented Jul 27, 2026 • edited by 0xadam-brown Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📜 Description

💚 How did you test it?

📝 Checklist

Uh oh!

runningcode Jul 27, 2026

Choose a reason for hiding this comment

Uh oh!

romtsn Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sentry Bot commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📲 Install Builds

Android

Uh oh!

github-actions Bot commented Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance metrics 🚀

Baseline results on branch: main

Startup times

App size

Previous results on branch: rz/fix/replay-encoder-wedge-anr

Startup times

App size

Uh oh!

runningcode left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

runningcode Jul 27, 2026

Choose a reason for hiding this comment

Uh oh!

romtsn Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

0xadam-brown left a comment

Choose a reason for hiding this comment

Uh oh!

0xadam-brown Jul 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Aug 7, 2026

Choose a reason for hiding this comment

Test named for close never invokes close

Uh oh!

cursor Bot Aug 7, 2026

Choose a reason for hiding this comment

Assertion cannot fail: encoder still null when wedged

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

romtsn commented Jul 27, 2026 •

edited by 0xadam-brown

Loading

romtsn Aug 7, 2026 •

edited

Loading

sentry Bot commented Jul 27, 2026 •

edited

Loading

github-actions Bot commented Jul 27, 2026 •

edited

Loading

romtsn Aug 7, 2026 •

edited

Loading