Playing an audio file is a solved problem. Any framework hands you a play() and a pause(), and a demo works on the first try. Then you ship to real users — on trains, in basements, on metered connections, with the app backgrounded behind a dozen others — and you discover that a media player is one of the hardest reliability problems in mobile engineering.
This is the story of rewriting a mature app's audio playback engine from a single monolithic player into roughly fifteen small, single-responsibility modules, built offline-first from the ground up. It's anonymized, and deliberately about engineering judgment rather than any one product.
Why a player is so hard
The naive mental model is "fetch bytes, decode, output sound." The real model has to survive everything that interrupts those bytes:
- The network drops mid-chapter and comes back thirty seconds later.
- The user backgrounds the app, and the OS — Android especially — reclaims the process to free memory.
- A lock-screen "next" button fires while the app's JS thread is asleep.
- Buffering stalls forever instead of failing cleanly.
- The user navigates to another screen, and you must not interrupt the audio.
A monolithic player tangles all of these concerns into one stateful object, and every new edge case makes it more fragile. The fix isn't a cleverer player — it's decomposition.
Decomposing the monolith
The rewrite split the engine into modules that each own exactly one responsibility, each independently testable. The ones worth naming:
- State manager — the single source of truth for what's playing. The mini-player, the full player, the lock screen, and navigation all read from one place, so they can never disagree about the current track or position.
- Network monitor — tracks real connectivity, not merely whether a network interface exists. A device can be "connected" to Wi-Fi with no actual route to the internet; the engine has to know the difference.
- Media source selection — chooses between streaming the remote source and playing a validated local file, driven by the network monitor.
- Sync engine — queues playback progress locally and reconciles it with the backend.
- Retry manager + error classifier — categorizes failures and applies the right backoff, instead of blindly retrying everything.
- Buffering watchdog — detects and recovers from stalled playback.
- Prefetch / batch manager — warms upcoming tracks in a queue while avoiding concurrent-write races.
- Background-kill recovery — restores the session after the OS terminates the background process.
- Telemetry — custom instrumentation for the bugs that only happen in the field.
The architectural payoff isn't elegance for its own sake. It's that each failure mode now has a place to live. When background audio breaks on one platform, you fix the background-kill module, not a 2,000-line player you're afraid to touch.
Offline-first means choosing the source, not catching the error
The single most important decision in an offline-first player is when to use a local file. The wrong answer is "try the network, and fall back on error." That produces the worst user experience there is: every chapter starts with a failed request, a retry, a timeout, and only then the local copy that was sitting on disk the whole time.
The right answer is to make source selection a first-class, proactive decision driven by connectivity:
- When the network monitor reports the device is genuinely online, stream the remote source.
- The moment it reports offline, switch to the validated local file immediately — no failed request, no retry loop.
- When the network returns, reconnect cleanly rather than tearing down playback.
This kills the retry/reconnect death spiral that drains battery and stalls audio. The cost is more explicit logic — you're now reasoning about connectivity state transitions rather than just catching exceptions — but that's exactly the trade you want for reliability users can feel.
Syncing progress without clobbering it
Progress sync sounds trivial until you add intermittent connectivity. If you naively push "latest position" to the backend whenever you can, a brief offline scrub backward, or a stale write that lands after a fresh one, can overwrite real progress and send the user back twenty minutes.
The fix is twofold. First, queue progress locally so it survives offline windows and app restarts — the network is an eventual-consistency target, not a dependency. Second, reconcile the furthest-achieved position rather than the most recent value. Sending "the furthest point this user has reached" is monotonic and order-independent: out-of-order writes and reconnect races can't move it backward. It's a small modeling decision that eliminates a whole class of "it forgot where I was" bugs.
Retries are a classification problem
Blind retries are an anti-pattern dressed up as resilience. A 404 will never succeed no matter how many times you retry it; a transient timeout almost certainly will. Treating them identically means you either give up too early on recoverable errors or hammer pointlessly on unrecoverable ones.
So the engine classifies before it retries. Transient and network errors get exponential backoff up to a bounded number of attempts. Permanent client errors fail fast, with no retry, surfacing a real error to the user instead of a spinner that never resolves. The retry manager and the error classifier are separate modules precisely because "should I retry this?" and "how should I retry it?" are different questions.
Watchdogs for the failures that don't throw
The cruelest playback bugs don't raise an error — they just stop. Buffering that never resolves. A stream that's technically "playing" but producing no sound. Nothing throws, so nothing catches.
That's what the buffering watchdog is for: it watches for playback that should be progressing and isn't, and recovers — re-buffering, re-selecting the source, or surfacing a real failure — instead of leaving the user staring at a frozen scrubber. Reliability isn't only about handling errors; it's about noticing the absence of progress.
De-risking the rewrite itself
Rewriting the subsystem that every user touches, every session, is terrifying — which is the point of doing it carefully. The rewrite shipped as a parallel V2, running alongside the legacy engine so the two could be validated side-by-side. Short-term duplication is a fair price for never betting the whole product on a single big-bang swap. You migrate when the new engine has earned it, not when you hope it works.
And none of this would be debuggable without telemetry. Playback bugs are field bugs — they happen on a specific phone, on a specific network, in the background, and they almost never reproduce at your desk. Custom playback instrumentation, added early, is what turns "users say audio sometimes cuts out" into an actual, fixable signal.
The one invariant
Strip away the modules and the trade-offs, and the whole engine exists to protect a single invariant: the queue keeps playing. Audio shouldn't stutter because the user opened another screen, the train went into a tunnel, or the OS reclaimed memory. Every module — state, source selection, sync, retries, watchdogs, recovery, telemetry — is there to defend that one promise.
That's the real lesson of building a media engine. It isn't about audio. It's about designing for the world where everything that can interrupt your bytes eventually will.