HEVC and Dolby Vision support

This chapter says which HEVC and Dolby Vision sources, AV1 Profile 10 included, the custom pipeline plays, by which route, and on what evidence, and which AV1 sources fall back to a decoder package. It is the engine’s support matrix: what getCustomPlaybackEligibility accepts and the route it selects, whatever the host. A host may restrict it further; the Jellyfin plugin’s book has the matrix of what Jellyfin negotiates for each row. It comes from the source and the unit tests, not from playback. test/capability/CustomPlaybackEligibility.test.ts covers route selection, and the Jellyfin plugin’s support matrix tests assert every row, with its fallbacks, through getCustomPlaybackEligibility.

Read every “Yes” as conditional:

• the evidence listed for at least one of the row’s routes must pass on the running browser and GPU;
• the host must pass the route flags and authorized keys of CustomPlaybackEligibilityOptions for that evidence.

HEVC is eligible only in the containers that capability/CustomContainerCodecSupport.ts pairs with it (MP4, M4V, MOV, Matroska, MPEG-TS, M2TS, MTS). Resolution, level, frame rate, and bitrate are never gates. The route names are defined in Routes, and the probe and authorization names in Evidence.

Every HEVC row also falls back last to GPU, when the host offers a GPU decoder package whose header accepts the stream: the plain and range extension rows, and a Dolby Vision row’s declared base. A Dolby Vision row with an RPU route also falls back to GPU-DV, RPU reconstruction from the package’s frame pool, after its native and raw routes and its declared base’s native and raw routes, and before that base’s own GPU route; the authorization is the raw RPU route’s key of the BL format.

Plain HEVC

The SDR rows cover every transfer the engine reads as SDR: BT.709, SMPTE 170M, sRGB, and the BT.2020 10 and 12-bit transfers, which use the BT.709 curve. They take BT.709, BT.601 (SMPTE 170M, SMPTE 240M, or BT.470 BG), or BT.2020 primaries and matrix. The VF-SDR routes leave the color conversion to Chrome, while the raw SDR routes convert a gamut other than BT.709 themselves, through the raw SDR key of the stream’s matrix, primaries, and range, which authorizes on first use outside BT.709. Jellyfin’s HDR10+ flag does not change these rows: HDR10+ metadata applies only on a PQ transfer, so an SDR or HLG stream that Jellyfin flags as carrying it takes its SDR or HLG row, and its metadata is ignored.

The HLG row includes HLG-compatible streams, whose VUI signals the BT.2020 10-bit transfer and whose alternative transfer characteristics SEI names HLG (see Evidence).

Range extensions

The PixelFormat must be exact, and an explicit BitDepth must agree with it.

Dolby Vision

A variant is written as the Dolby Vision profile and compatibility ID (CCID), then the HEVC profile. Main 10 at 10 bits unless stated. The engine reads the Dolby Vision configuration and the color fields; it ignores VideoRange and VideoRangeType.

Dolby Vision over AV1

Profile 10 is single-layer: the RPU travels in an ITU-T T.35 metadata OBU of each temporal unit, and the base layer is AV1 in any profile and depth. VF-DV5 and VF-DV8 here decode Main at 10 bits in hardware through the engine’s own AV1 path, every sequence header neutralized to limited BT.709, and reconstruct from the external frames; RAW-DV decodes natively in software, into I420P10 for Main at 10 bits, or into the exact format of an AV1 raw format variant (see AV1 outside Main 10). Every AV1 track takes that path, whatever its route, so the declared base of a stream with HDR10+ metadata applies its HDR10+ metadata OBUs too (see Decisions). Only an RPU route loads the RPU parser; any other route removes the RPU OBUs without parsing them.

The external routes present 8-bit-quantized frames, as every external route does, so their reconstruction differs from RAW-DV by a few code values, more where the RPU’s mapping is steep.

The engine maps the MP4 dav1 sample entry of a 10.0 stream to AV1 itself (see Decisions).

Dolby Vision outside HEVC and AV1:

• Profile 9 (AVC, 8-bit) has no RPU route, because the engine owns no AVC decode path. Its declared SDR base plays through the H.264 routes.
• The retired Profiles 0 to 3 and 6 have no RPU route and present only a declared base.

AV1 outside Main 10

Native routes decode AV1 Main at 4:2:0: VF-SDR at 8 bits, and at 10 bits VF-PQ and VF-HLG where the AV1 external keys are authorized, RAW I420P10 through the raw SDR keys, and RAW I420P10 PQ or HLG. AV1 outside Main 10 takes raw planes in the exact format of its raw format variant, which its Jellyfin Profile and PixelFormat name, through av1RawFormats[variant], a 192x192 key and inter temporal unit decoded with prefer-software. Every AV1 stream falls back last to GPU, in its own format, when the host offers an AV1 decoder package whose header accepts it. The AV1 package takes pictures up to level 6.x, 16384x8704 and 35,651,584 samples, 8K included, on a device whose largest binding holds half its frame store and its packet ring: at 8192x4352 two buffers of 535 MB and a ring of 626 MB at 4:2:0 10-bit, two of 1.07 GB and a ring of 1 GiB at 4:4:4 12-bit (see WebAssembly build); eligibility skips it with the failed check otherwise.

Chrome’s software AV1 decoder decodes and copies the 8 and 10-bit variants, and refuses 12-bit AV1 at decode, so the 12-bit variants read unsupported there and take GPU. hasPotentialCustomPlaybackVideoRoute keeps AV1 outside Main 4:2:0 potential for these routes. On GPU the decoder receives the track’s av1C (see Codec private data).

H.264 beyond 8-bit 4:2:0

Native H.264 decodes Constrained Baseline, Baseline, Main, and High at 8-bit 4:2:0 to VF-SDR, each profile through its own h264Profiles probe. A profile beyond High takes raw planes in the exact format of its raw format variant through h264RawFormats[variant]: an I, a P, then a B access unit at 192x192, decoded with prefer-software, whose B frame must output before the P frame. The profile must code the variant’s chroma format: High 10 and High 10 Intra code 4:2:0, High 4:2:2 and High 4:2:2 Intra up to 4:2:2, and High 4:4:4 Predictive, High 4:4:4 Intra, the removed High 4:4:4, and CAVLC 4:4:4 up to 4:4:4. The PixelFormat names the variant, and FFmpeg’s yuvj names of a full-range 8-bit stream count as their yuv formats. No GPU decoder package decodes H.264, so a variant without its capability or raw key is refused.

Chrome’s software H.264 decoder is FFmpeg’s, which outputs the same planes as the reference decode the vectors pin, while prefer-hardware reports these profiles unsupported. The worker decodes them through Mediabunny’s sample path, as native H.264, which orders B frames, reads avcC or Annex B as the container gives it, and takes the presentation times the fork’s AVI demuxer gives each picture.

Interlaced video

The deinterlacing pass presents interlaced frames on every route but Dolby Vision’s (see Architecture and Eligibility and routes). This table covers every codec, beyond this chapter’s HEVC and Dolby Vision.

A stream whose metadata calls it interlaced but whose frames report progressive presents them untouched, and a frame without any fields takes the stream’s scan. The decoder’s and the parser’s reports override the container’s: an MPEG-2 or VC-1 stream whose Matroska header calls it progressive still deinterlaces each picture it reports interlaced.

Rejected

Routes

A route is the eligibility output: videoOutputMode, videoDecoderBackend, rawVideoFrameFormat, dolbyVisionProfile, and nativeHDRTransfer, plus gpuVideoDecoder on GPU. The host maps it to a presenter input mode.

Eligibility and routes gives the order in which they are tried.

HDR10+ dynamic metadata is applied on VF-PQ, DV base PQ, RAW PQ, and GPU PQ, and on the RPU routes (VF-DV5, VF-DV8, RAW-DV, and GPU-DV) to a frame whose RPU has no L1. HEVC carries it in SEI, and AV1 in ITU-T T.35 metadata OBUs, whose trailing bits the engine removes as dav1d does: it drops the trailing zero bytes, then the byte holding the trailing one bit, which must be 0x80, or the frame’s metadata is malformed. VP9 has no metadata of its own, so Matroska and WebM carry each frame’s ITU-T T.35 message beside it in a BlockAdditional, and VP9 in MP4 carries none. The engine recognizes the message by its T.35 header whatever its BlockAddID, so WebM, which has no BlockAdditionMapping, reads like Matroska, which maps FFmpeg’s BlockAddID 4 to the ITU-T T.35 type. A frame’s scene peak and average drive every tone map operator; only the ST 2094-40 operator follows a frame’s Bezier curve, adapted from its targeted display, and without a curve it builds libplacebo’s brightness-matching curve from the scene’s knee. A frame without a curve, such as every profile A frame, never has its targeted display read. A frame without HDR10+ metadata of its own, or whose payload fails to parse, takes the last metadata before it in decode order, as in FFmpeg (see Decisions). Conflicting metadata, unsupported metadata, and a curve with a targeted display of 0 tone-map their frame statically and end the carried metadata until the next valid payload. Each decode attempt and each seek starts without carried metadata.

An RPU route tone-maps each frame from its RPU, as libplacebo ranks the metadata: from its L1 scene peak and average, else from HDR10+, else statically from the peak of the mastering display the RPU names (source_max_pq), which an RPU without display management metadata leaves at FFmpeg’s default of 4000 nits. An L1 whose maximum or average is 0, as an RPU whose CM v2.9 section codes no L1 presents, counts as none.

Evidence

Probes, all in capability/CustomDecodeCapabilities.ts:

GPU has no probe: a package’s own vectors run only when the host verifies it (see Decoder packages).

GPU authorizations, all in validation/:

I420P10:dovi-rpu-v1 and the I420P10 Profile 7 keys are part of the default Dolby Vision prewarm. The Profile 4 keys in any format, and the Profile 7 and single-layer keys of other formats, authorize on first use, so a host must wait for them before it advertises or starts a stream that needs one. The external HDR keys authorize on first use too: an HDR negotiation waits for the HEVC pair (waitForExternalHDRAuthorizationPrewarm), which shares one decode, and an AV1 HDR stream waits for its own key.

Other conditions:

• Native VideoFrame HDR and the DV base routes also need explicit ColorTransfer, ColorPrimaries, and ColorSpace values.
• The SPS parser never rejects a VUI color description. It maps each code to a WebCodecs name or to unspecified (null), and an SPS without VUI has unspecified color. Only the native HDR route check is strict: limited range, BT.2020 primaries, the BT.2020 non-constant-luminance matrix, and the route’s transfer.
• The transfer that check reads is the key access unit’s alternative transfer characteristics SEI value (payload type 147) when one is present, and the VUI transfer otherwise. Without the SEI, VF-HLG and DV base HLG also accept the BT.2020 10 and 12-bit VUI transfers (14 and 15) of HLG-compatible streams. An SEI naming another transfer rejects the HLG route, and a malformed SEI counts as absent, as in FFmpeg.
• P20 is MV-HEVC. NAL units with a nuh_layer_id above 0, the second view, are dropped before decode, and SEI of other layers is ignored.
• A separate-track P7 (a base track and an EL track) selects the same routes. test/capability/CustomPlaybackEligibility.test.ts covers it.
• An RPU the parser rejects ends the custom session, which then falls back. The parser follows FFmpeg’s dovi_rpudec.c: a display-metadata extension block that runs past the data or falls short of its level’s fields, an L2 ms_weight below -1, or more than 7 static or 25 dynamic blocks reject the RPU, an L1 in the CM v4.0 section counts as an L1 of zeros, and unknown, other misplaced, or padded blocks are skipped. Leftover bits before the CRC32, or an AV1 EMDF wrapper whose size or protection bits FFmpeg refuses, also reject it, while a CRC32 mismatch and bytes after the 0x80 terminator do not, as in FFmpeg. It also reads what FFmpeg does not: polynomial and MMR pieces mixed within one component, polynomial linear interpolation (see Decisions), and any mapping color space or chroma format.

Not supported

• HDR10, HDR10Plus, or HLG at 8 bits, under any profile.
• Profile and bit depth contradictions: Main at 10 bits, Main 10 at 8 bits, Main 10 at 12 bits.
• 14-bit and 16-bit range extensions.
• Monochrome range extensions outside GPU.
• High Throughput 4:4:4 profiles, Screen Content Coding (Screen-Extended) profiles, Main Still Picture, and named Intra aliases such as Main 4:4:4 10 Intra. Generic Rext with an intra constraint is the ordinary variant.
• Interlaced HEVC of any profile or range: HEVC codes interlaced video as field pictures, which no decoder weaves into the frames the deinterlacing pass takes (see Interlaced video).
• Dolby Vision:
  • the configurations in Rejected;
  • RPU reconstruction for AVC Profile 9, and for AV1 Profile 10 in a raw format variant the browser does not qualify, such as 12 bits in Chrome;
  • RPUs the parser rejects: a linear interpolation piece next to an MMR piece, an RPU format extension, missing sequence information, display metadata compression above method 1, and a header from which no profile is inferred while the container declares none.

Known issues

• Firefox on Windows has no WebCodecs HEVC (see Decisions), so HEVC plays there only through GPU. Dolby Vision then plays through GPU-DV, Profile 5 included, when the host offers the HEVC package and authorized the RPU route’s key. Firefox decodes 10-bit AV1 to 8-bit BGRX, so no raw AV1 route, P10 included, qualifies there.