feat(youtube): SABR extractor support

[Priveetee]

First real integration of the SABR extractor (supersedes the PoC #68). Pairs with the client SABR PR (coming) and the media3 migration InfinityLoop1308/PipePipeClient#45. Tracking: InfinityLoop1308/PipePipeClient#42.

Docs (how SABR works end to end): https://priveetee.github.io/Docs-PipePipe/developer-guide/introduction

Related:

• feat(player): migrate from ExoPlayer 2.18.7 to media3 1.10.1 PipePipeClient#45 (media3, the client SABR builds on it)
• [YouTube] Content Not Yet Suppoted (SABR) after 5.1.0 PipePipe#2330 ([YouTube] Content Not Yet Supported (SABR) after 5.1.0)
• [YouTube] Coordinating efforts on SABR implementation TeamNewPipe/NewPipe#12248 (coordinating SABR)

Summary

Adds the YouTube SABR (Server ABR) extraction path end to end: UMP wire reading, the proto request/response codecs, the typed UMP parts (media, policy, context, onesie...), response decoding + mp4/webm segment index parsing, and the session/state model the client pump drives. A new DeliveryMethod.SABR exposes it.

Changes

• DeliveryMethod.SABR delivery method.
• UMP reader + proto codec, response decoder, typed parts (media/policy/context/onesie/misc).
• media segment + mp4/webm index parsers, request builder + client profile, info/formats/probe + PO token provider interface.
• session, state and hardware-aware format selection; SABR wired into YoutubeStreamExtractor.
• reader-driven contiguous buffered-range reporting + play-head-aware byte-bounded eviction + reload budget, plus backward-seek re-request (rewindBufferedTo / prepareForRewind): the buffered head was one-way (assumeBufferedUntil only ever extends), so a rewind onto an already-sent segment left the request claiming we still had it and the server sent nothing back; this shrinks the head (contiguous + observed-timing) so the server re-sends from the target.
• SabrLiveMetadata getters (live foundation).
• YoutubeParsingHelper: Safari player request now uses the live web client version (the hardcoded one started returning page needs to be reloaded).

Why

YouTube serves some videos SABR-only; the old paths return "Content Not Yet Supported". This adds a real extractor path, with the client PR as the runnable counterpart.

Impact / Compatibility

The new sabr/ package and DeliveryMethod.SABR are additive. The one behavioural change is in YoutubeStreamExtractor: a FORCE_SABR_FOR_TESTING flag (currently true) routes every YouTube video through the SABR pipeline, so HLS/DASH/progressive are bypassed on purpose to stress-test SABR on everything. With the flag false (production default), HLS/DASH stay untouched and SABR only fills the SABR-only / no-HLS gap that upstream currently throws ContentNotSupportedException on. The Safari client-version fix also helps the existing logged-in path.

Validation

• ./gradlew :extractor:compileJava -x checkstyleMain
• Exercised end to end via the client on Pixel 8, Galaxy S25 and emulator (Android 14/16): probe -> session -> segments -> playback, including forward and backward seek.

Notes

• First real integration, not a PoC, but still WIP, residuals tracked in the client PR.
• Right now FORCE_SABR_FOR_TESTING = true on purpose: all YouTube playback goes through SABR with no HLS fallback, so we exercise the SABR path on every video. It flips back to false before merge (then HLS stays as fallback and SABR only covers the SABR-only gap).
• Not ready to merge yet, more testing coming.
• checkstyleMain excluded, fails repo-wide unrelated to this change.

[Priveetee]

bit of context since this isn't a fresh idea: the PoC #68 (and the client PoC InfinityLoop1308/PipePipeClient#43) were exactly that, throwaway PoCs to prove SABR actually plays end to end. they did their job, so i closed #68 rather than keep pushing onto a messy branch.

this is the real integration, rebuilt clean. the SABR extractor stands on its own here, and i reworked the buffering to be reader-driven (pacing + cache eviction follow what the player has actually read, not the play head, so it doesn't deadlock or blow up memory anymore), plus the Safari client-version fix and the live-metadata foundation. on the client side i split it too: the media3 migration is its own PR (InfinityLoop1308/PipePipeClient#45) and the SABR client sits on top.

still WIP, not for merge yet (FORCE_SABR_FOR_TESTING is on so every video goes through SABR while i hammer it), but this is where it continues from now :)

[Priveetee]

I've been working on this whenever I had some free time. I tested it a lot (u know me by now, im a little paranoid 🥲) on a S25, a Z Fold 5 and a Pixel 8, mostly the app/code itself, and found and fixed a shit load of bugs along the way 🫠: forward seeks, rewind after the stream buffered to the end, return from background, a crash on decoder init... u can see it all in the commit history.

Honestly I'm starting to reach my limits testing alone, so I think this is getting close to review-ready @InfinityLoop1308.

No rush at all, as we say in French: y a pas le feu au lac :)

If u want me to split this PR differently (or anything else really), don't hesitate to tell me.

One thing I'd really like your take on: how we wire SABR into the app. My take is it shouldn't be SABR-only, I'd rather marry both, keep the current delivery and have SABR live next to it (as an option / fallback) so we don't regress anyone.

But u might see it differently, so im genuinely curious what u think. Right now FORCE_SABR_FOR_TESTING is still true in YoutubeStreamExtractor so every video goes through SABR while I test, that's not the end state, I'll flip it back.

Last thing: it's pushed with private static final boolean DIAG = false; in YoutubeSabrSession. Flip it to true and it logs every SABR round (segments, buffered ranges, protection status), that's how I caught most of the bugs above.

Could be handy to hand a debug build to users who hit issues so their reports are actually useful.

[Priveetee]

Oh and I almost forgot the fun part 😄

I wrote actual docs for the extractor! the whole thing's in there: architecture, extraction flow, the info model, the downloader, paging, adding a service, the YouTube service... and then it dives deep into SABR (UMP framing, part-type decoding, building the VideoPlaybackAbrRequest, the session driver, the buffered-range model + seeking). plus a bunch of neat hand-drawn schemas to make it all click

Here

had a real blast writing it, curious what u think :) (If u have time of course)

[Priveetee]

the other day i was watching some LaSalle videos in fullscreen (1080p60 vp9) and kept getting this micro-freeze: the picture would hang for a moment every ~6-7s, and it got worse the longer the video played. only happened in fullscreen, and only on some videos.

threw some logging on the SABR pipeline to see what was going on, and the webm segment index was never getting built. the parser was bailing with "Invalid WebM element size" on the vp9 inits (itag 303): a SABR/DASH init's Segment element declares its full media size (basically the whole video), way bigger than the few-KB init we actually have, so readElement threw and the whole parse gave up 😅

with no index we were falling back to a uniform segment-duration guess (averageDurationMs), which drifts from the real variable-length segments (~4.8-5.4s here). so the chunk timing slowly desyncs from the actual samples and the picture stutters/freezes, worse as the drift adds up. mp4/avc was fine the whole time (its sidx parsed), this was webm-only, which is why it only showed on vp9.

fix is small: when an element overruns the init buffer, clamp it to the buffer instead of failing the parse. the cues are located by the explicit index range anyway, so we only ever read what we have :p

tested on my pixel 8: 1080p60 vp9 plays smooth now, no more freeze !

[Priveetee]

fixed the 4K OOM crashes (rewind + forward-seek deadlock): on a big seek the cache held two disconnected spans and busted the byte cap, so i collapse it to a window around the seek target now. also stream the SABR response instead of buffering the whole body + byte-sized the back-buffer so 4K keeps its read-ahead. tested pixel 8 (1080p/1440p/4K, seek + relaunch), 0 OOM.

[Priveetee]

honestly every time i think i'm done i find another damn bug 😂 but for real this time, i think it's ready.

@InfinityLoop1308 it's your call now. i'm kinda reaching my limits here, these days i'm spending way more time on niche bugs and performance than on the actual implementation, which is probably a good sign the core is solid.

the way i see it there are two paths: either i do the work to wire SABR into the app properly alongside HLS (imo it shouldn't be SABR-only), or you tell me the direction you'd prefer and i roll with that. happy either way.

one thing for sure, we'll want testing on more devices now, my poor pixel 8 has really been through it 😅 and one phone only tells you so much (decoders behave so wildly from one device to the next). but honestly i'm pretty excited about where this landed, it feels genuinely close now and the core's holding up really well :)

[InfinityLoop1308]

Awesome work :)

I'm going to test it on my devices and think about the next step.

[Priveetee]

Hello!

I know we had a busy week, but by any chance have you had a little time to think about the direction we can take with this?

Have a great day :p

[InfinityLoop1308]

I'm testing :) audio works pretty fine here but I noticed an issue that in the first few seconds it seems doing a jump. Not sure, still testing.

[InfinityLoop1308]

and, of course once I finish testing I'll directly merge the PR (with FORCE_SABR added as a advanced pref)

[InfinityLoop1308]

I must say this is impressively complete 👍👍

[Priveetee]

honestly thank you, that's really kind :) it was a long road on the SABR side, so hearing that from you means a lot !

quick thought before you wire the FORCE_SABR pref: FORCE_SABR is really just a testing flag, it forces SABR all the time with no fallback, so as a raw user toggle it could break playback for someone who flips it on a video/device where SABR isn't happy.

i'd rather go one of two ways:

• a proper setting (e.g. "prefer SABR") backed by a fallback, so if SABR fails it drops back to the normal DASH/HLS path instead of failing
• or fully seamless: keep DASH/HLS as the default and only fall back to SABR when the normal path fails (the 403 case), so the user never has to touch a toggle

not married to it, just feels cleaner than exposing the test flag. wdyt? i can wire the fallback if u want :p

[Priveetee]

I'm so dumb I didn't read correctly you clearly said

(with FORCE_SABR added as a advanced pref)

Wtf i need some sleep lmao 😭😂

[InfinityLoop1308]

ok I have found 2 issues:

1. when progress reach 0:04 for the first time, it will jump to 0:10. If reset the progress to 0:00 this won't happen.
2. manually changing resolution results in a exception or infinite loop

both can be 100% reproduced on any videos

Exception

• User Action: play stream
• Request: Loading failed for [ハチ - ドーナツホール 2024 , HACHI - DONUT HOLE 2024]: https://www.youtube.com/watch?v=y7fu_nNQAEQ
• Content Country: JP
• Content Language: en-US
• App Language: en_US
• Service: YouTube (Anonymous)
• Version: 5.2.0-beta3
• OS: Linux Android 15 - 35

Crash log

org.schabi.newpipe.player.mediasource.FailedMediaSource$StreamInfoLoadException: Unable to resolve source from stream info. URL: https://www.youtube.com/watch?v=y7fu_nNQAEQ, audio count: 4, video count: 18, 0
	at org.schabi.newpipe.player.playback.MediaSourceManager$$ExternalSyntheticLambda3.apply(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:222)
	at org.jsoup.parser.Parser.onSuccess(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:18)
	at org.jsoup.parser.Parser.onSuccess(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:110)
	at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.onSuccess(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:3)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeToSingle$ToSingleMaybeSubscriber.onSuccess(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:9)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onNext(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:67)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.drain(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:55)
	at io.reactivex.rxjava3.internal.operators.maybe.MaybeConcatArray$ConcatMaybeObserver.request(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:12)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableElementAtMaybe$ElementAtSubscriber.onSubscribe(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:44)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableDefer.subscribeActual(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:26)
	at io.reactivex.rxjava3.core.Flowable.subscribe(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:6)
	at io.reactivex.rxjava3.internal.operators.flowable.FlowableElementAtMaybe.subscribeActual(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:9)
	at io.reactivex.rxjava3.core.Maybe.subscribe(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:1)
	at io.reactivex.rxjava3.internal.operators.single.SingleJust.subscribeActual(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:87)
	at io.reactivex.rxjava3.core.Single.subscribe(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:6)
	at io.reactivex.rxjava3.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:5)
	at io.reactivex.rxjava3.core.Scheduler$DisposeTask.run(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:10)
	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.run(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:22)
	at io.reactivex.rxjava3.internal.schedulers.ScheduledRunnable.call(r8-map-id-c63030bcc34ed4ef3423d4cd59133edce20bab7f7d9937297562dbafcd6582c6:1)
	at java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:348)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1156)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:651)
	at java.lang.Thread.run(Thread.java:1119)

---

[Priveetee]

thanks for testing and the repro :) pushed 2 fixes to the client PR InfinityLoop1308/PipePipeClient#47:

1. the first-few-seconds jump (0:04 -> 0:10): the first audio chunk declared a stale ~5s end (computed before the init metadata lands with the real ~10s audio segment duration), so the end-clip dropped the real 5-10s of audio. fixed by not end-clipping the chunks, the container timestamps are the source of truth. validated on a cold start where it used to be 100%: zero discontinuity now.

2. the quality-change exception/loop: pickVideoFormat could return null when nothing was decodable at/under the chosen resolution, which threw StreamInfoLoadException and looped. it now falls back to the best decodable format. couldn't repro this one on my Pixel 8 (it has HW VP9+AV1 so the null path never triggers), could you confirm on your side since you hit it 100%?

[InfinityLoop1308]

I can confirm both are now fixed. However I notice a new issue (regression?), the progress resets when switching between audio and video.

[Priveetee]

I can confirm both are now fixed. However I notice a new issue (regression?), the progress resets when switching between audio and video.

aww shoot let me see I think I know whre to dig i kept u updated

[InfinityLoop1308]

btw,

the quality-change exception/loop: pickVideoFormat could return null when nothing was decodable at/under the chosen resolution, which threw StreamInfoLoadException and looped. it now falls back to the best decodable format.

actually my device supports all the formats but before the fix it doesn't work. do you think there are some detecting issues?

[Priveetee]

two things:

the quality-change one: yeah you're right, that's a detection issue, not your device. the HW-decoder check guesses from the decoder NAME (treats c2.android / c2.google / omx.google as software, the rest as hardware), which misflags a real HW decoder on some setups, so a format your device actually decodes gets marked undecodable -> the null -> the crash.

My fallback just stops the crash; the clean fix is MediaCodecInfo.isHardwareAccelerated() instead of guessing from the name (with a name fallback under API 29 since we still support 23).

I can wire that up so your exact resolution is honored, not the fallback.

The audio/video switch: dug into it but i can't repro the reset on my Pixel 8 😭 quick toggle and a 60s background both held position, no reset, no discontinuity in the logs.

The SABR switch just re-enables the video track on the live source (no reload, no seek), and my 2 fixes don't touch it. how do you trigger it (background / screen off / popup?), what's your "minimize on app switch" setting, and how long is it backgrounded?

A logcat around it would help a lot :p

[InfinityLoop1308]

OK, so I downgraded the apk to the version before the new fix and found that I can no longer reproduce the exception / loop there either. It seems to be an intermittent edge case rather than something fixed by the latest commit.

I took a closer look at the format-selection path, and the latest fallback commit does not appear to change the result. pickHardwareFriendlyVideo(info, preferredHeight) already falls back to info.findBestVideoFormat() when no matching “decodable” format is found. If that first call returns null, calling it again with maxHeight = 0 should still return null, since that would mean the SABR info contains no video formats at all. Therefore, the newly added fallback is probably unreachable and unlikely to be what resolved the issue.

More importantly, I don't think hardware-decoder detection should be part of this format-selection policy. As we discussed before, the codec capabilities advertised by Android devices are not reliable. SABR should follow the existing “Enable advanced formats” preference instead.

For the same reason, replacing the codec-name heuristic with MediaCodecInfo.isHardwareAccelerated() would not add much practical value. It may classify the decoder implementation more accurately, but it still cannot reliably tell us whether a specific codec, profile, resolution and frame-rate combination will work correctly on that device.

I think the cleaner solution is to remove the independent hardware-decoder filtering from the SABR path and make it follow the same format-selection policy as the normal playback path.

It would also be useful to preserve the original IOException instead of reducing every resolver failure to Unable to resolve source from stream info. If this issue happens again, we need to know whether it came from format selection, the SABR probe, session creation, or another part of the resolver path.

[Priveetee]

Ok thx for the feedback, I'm gonna work on it. Thx a lot 😊

[InfinityLoop1308]

I reproduced the reset issue again and checked the logs. The audio/video player switch does reload the media source on this path. Each reset releases both decoders, enters buffering at position 0, and creates new VP9/AAC decoders.

The switch goes through handleIntent(), where a player type change unconditionally calls reloadPlayQueueManager():

if (oldPlayerType != playerType && playQueue != null) {
    reloadPlayQueueManager();
}

This is separate from useVideoSource(), so the live-source track-toggle optimization does not cover explicit AUDIO/VIDEO player type switches. This reload neither saves the current recovery position nor sets seekOnNextSabrReload, so shouldSeek() rejects the SABR recovery seek and playback starts from 0.

Relevant log:

MediaSessionService: playbackState=PLAYING, position=182940
MediaSessionService: playbackState=NONE, position=182940
WifiService: releaseWifiLock uid=10520
MediaSessionService: playbackState=BUFFERING, position=0
MediaSessionService: playbackState=BUFFERING, position=0
DMCodecAdapterFactory: Creating an asynchronous MediaCodec adapter for track type video
CCodec: allocate(c2.qti.vp9.decoder)
DMCodecAdapterFactory: Creating an asynchronous MediaCodec adapter for track type audio
CCodec: allocate(c2.android.aac.decoder)
MediaSessionService: playbackState=PLAYING, position=10

MediaSessionService: playbackState=NONE, position=5684
WifiService: releaseWifiLock uid=10520
MediaSessionService: playbackState=BUFFERING, position=0
MediaSessionService: playbackState=BUFFERING, position=0
DMCodecAdapterFactory: Creating an asynchronous MediaCodec adapter for track type video
CCodec: allocate(c2.qti.vp9.decoder)
DMCodecAdapterFactory: Creating an asynchronous MediaCodec adapter for track type audio
CCodec: allocate(c2.android.aac.decoder)
MediaSessionService: playbackState=PLAYING, position=10

MediaSessionService: playbackState=NONE, position=105000
WifiService: releaseWifiLock uid=10520
MediaSessionService: playbackState=BUFFERING, position=0
MediaSessionService: playbackState=BUFFERING, position=0
DMCodecAdapterFactory: Creating an asynchronous MediaCodec adapter for track type video
CCodec: allocate(c2.qti.vp9.decoder)
DMCodecAdapterFactory: Creating an asynchronous MediaCodec adapter for track type audio
CCodec: allocate(c2.android.aac.decoder)
MediaSessionService: playbackState=PLAYING, position=10

[Priveetee]

thx a lot for the diagnosis, was right on the money. got all three done on the client side:

• player type switch (audio <-> video): dropped setRecovery() + seekOnNextSabrReload into the handleIntent type-change block, the two bits that were missing like u said. tested both ways on my phone, position holds now (video->audio and audio->video), no more snap back to 0.
• hw-decoder filter: killed the standalone filtering, it just follows the advanced-formats pref + the resolver's itag now, same as the normal path.
• resolver error: SABR surfaces the real IOException with its cause instead of the generic "unable to resolve source", flagged no-retry so the cause doesn't get eaten.

pushed to the client branch. happy to redo any of these differently if u'd rather.

[InfinityLoop1308]

can confirm the progress doesn't reset anymore.

I think the pr is ready for merge now. Can you add the force sabr pref in advanced settings?

[Priveetee]

Oki doki gimme a day or two and it will be done :p