Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Codec vectors

A codec vector is a known input with a known answer. Qualification vectors ship to the browser, where the capability probes check native decoders against them before a route is advertised. A decoder package’s self-test vectors ship inside the package, and run only when a host verifies it (see Decoder packages). Test vectors only feed tests and generators.

The generators are Python 3.10 or later, in scripts/codec_vector_assets/, with their tests in scripts/codec_vector_assets/test/. They write the committed bin/codec_vector_assets/, except the local playback media generators, which write the ignored bin/playback_smoke_media/. A generate_ script makes its output from nothing but the repository and FFmpeg, while a create_ script derives it from media you supply. Hand-made vectors live in src/capability/vectors/.

Generate or check everything

From the engine root:

python scripts/generate_all_codec_vector_assets.py            # Writes bin/codec_vector_assets/
python scripts/generate_all_codec_vector_assets.py --check    # Fails unless every committed asset matches
python -m unittest discover -s scripts/codec_vector_assets/test -p "*_test.py"

generate_all_codec_vector_assets.py runs every generator of committed assets in order, and passes --regenerate-sources to the TrueHD generator so its source streams are re-encoded and compared too. It continues past a failure and exits with status 1 if any generator failed. It does not run the local playback media generators.

bin/codec_vector_assets/ is the one committed folder in bin/, so every change to an asset shows in review. .gitattributes stores it without line-ending conversion, so the committed bytes are the generated bytes.

Every committed asset reproduces with FFmpeg and FFprobe 2026-03-01-git-862338fe31-full_build-www.gyan.dev on PATH. The HEVC range-extension, AV1 raw-format, H.264 raw-format, AV1 Dolby Vision, HDR10+ AV1, HDR10+ VP9, and WMA decoder generators refuse any other build.

Committed assets

ScriptWrites, in bin/codec_vector_assets/
generate_truehd_capability_vectors.pytruehd/TrueHDExactCapabilityVectors.ts, from the synthetic streams in truehd/. --regenerate-sources re-encodes those streams first
generate_seven_point_one_downmix_reference.pydownmix-reference/seven-point-one.json. --verify-external also runs the referenced mpv and FFmpeg builds against it
generate_HEVC_range_extension_vectors.pyhevc-range-extension/*.hevc. --inspect prints the evidence that capability/HEVCRangeExtensionCapabilities.ts records
generate_AV1_raw_format_vectors.pyav1-raw-format/*.obu. --inspect prints the evidence that capability/AV1RawFormatCapabilities.ts records
generate_H264_raw_format_vectors.pyh264-raw-format/*.h264. --inspect prints the evidence that capability/H264RawFormatCapabilities.ts records
generate_mpeg2_decoder_vectors.pyOutside bin/: wgpud/mpeg2/vectors/, one Matroska MPEG-2 stream per chroma format from a deterministic syntax exerciser, since no available encoder writes field pictures, dual prime, 16x8 motion, concealment vectors, and 4:4:4 together; FFmpeg checks each stream
generate_jpeg2000_decoder_vectors.pyOutside bin/: wgpud/jpeg2000/vectors/, one stream per output format from FFmpeg’s native and OpenJPEG encoders, across both wavelets, tiles, quality layers, every progression order, and SOP and EPH markers, in QuickTime files and in Motion JPEG 2000 files it muxes itself, and for the route tests OpenJPEG’s Digital Cinema 2K profile in X’Y’Z’ at 12 bits, which FFmpeg’s Matroska muxer writes as a V_MS/VFW/FOURCC track of mjp2 beside 5.1 FLAC and a JPEG cover attachment
generate_av1_decoder_vectors.pyOutside bin/: wgpud/av1/vectors/, one Matroska stream per output format from FFmpeg’s libaom-av1 encoder, as AV1 decoder vectors describes
generate_mpeg4_decoder_vectors.pyOutside bin/: wgpud/mpeg4/vectors/, the mpeg4 project’s self-test stream from libxvid and a stream per codec and container its routes take from FFmpeg’s encoders, as MPEG-4 decoder media describes
generate_wma_decoder_vectors.pyOutside bin/: wgpud/wma/vectors/, the wma project’s media, FFmpeg’s WMA 1 and WMA 2 encodes in AVI and Matroska, remuxes of FFmpeg’s FATE samples of the other three codecs, and the WMA 2 packets beside H.264 video for the decode worker tests, and known-answers.json, FFmpeg’s decode of each audio-only medium, as WMA decoder vectors describes
generate_eac3_joc_vectors.pyOutside bin/: wgpud/eac3/vectors/, the eac3 project’s Dolby Atmos media, remuxes of freely licensed E-AC-3 JOC streams, and known-answers.json, FFmpeg’s decode of each, as E-AC-3 Atmos vectors describes
generate_decoder_package_vectors.pyOutside bin/: each decoder project’s wgpud/<project>/<project>.vectors, from the streams its decoder.json lists, demuxed with FFprobe and hashed from FFmpeg’s decode, so a project’s vectors are FFmpeg’s known answers; a vector’s codec private data is the stream’s extradata, or with codecPrivate bitmap-info-header a BITMAPINFOHEADER of its size and FourCC before it
generate_dolby_vision_AV1_vectors.pydolby-vision-av1/profile10.{0,1,2,4}.{mp4,mkv} and expectations.json, from the RPU payloads in src/capability/vectors/test/dolby-vision-rpu/
generate_HDR10_plus_AV1_vectors.pyhdr10plus-av1/hdr10plus.{mp4,mkv} and expectations.json
generate_HDR10_plus_VP9_vectors.pyhdr10plus-vp9/hdr10plus.{webm,mkv} and expectations.json
generate_AVI_demux_vectors.pyavi/*.avi, FFmpeg encodes in FFmpeg’s AVI muxer: H.264 High with B-frames in a pyramid and VBR MP3, H.264 with CBR MP3, VP8 with PCM, VP9 with AAC, AV1 with AC-3, HEVC with B-frames in a pyramid and an open group under HEVC and under the zero tag FFmpeg writes without one, MPEG-4 Part 2 under XVID (no route), and VP9 with 5.1 24-bit and float PCM and 5.1 FLAC as WAVE_FORMAT_EXTENSIBLE, stereo FLAC, stereo Vorbis, and three-channel 16-bit PCM as 2.1 and 3.0(back); and, rewritten by avi_rewriting.py and required to decode in FFmpeg to the encode’s frames, the VP9 encode in three OpenDML RIFF lists as FFmpeg’s muxer writes a file past 1 GiB, and an MPEG-4 Part 2 encode with B-frames under DX50 packed as DivX 5 packs it
generate_transport_stream_demux_vectors.pympeg-ts/*.{ts,m2ts}: MPEG-1 with MP3 in MPEG-TS alone, since BDAV M2TS carries neither; MPEG-2 with B-frames and AC-3, and as mpeg2video-ac3-start600 with stereo AC-3 and every timestamp 600 s on, as a Blu-ray title’s program clock starts, FFprobe checking each encode’s start time; and the VC-1 qualification vector, which FFmpeg cannot encode, rebuilt as an elementary stream with its headers and frame start codes in band and remuxed, and in MPEG-TS also as vc1-advanced-unregistered.ts, whose PMT the generator strips of the VC-1 registration descriptor FFmpeg’s muxer writes, recomputing its lengths and CRC
generate_program_stream_demux_vectors.pympeg-ps/*.{mpg,vob}: the MPEG-TS vectors’ MPEG-1 with MP3 in an MPEG-1 system stream from FFmpeg’s mpeg muxer, and MPEG-2 with B-frames and AC-3 in a DVD-Video program stream from its vob muxer, which carries the AC-3 in private stream 1
generate_chroma_siting_vectors.pychroma-siting/*: H.264 and HEVC whose SPS declares a 4:2:0 chroma location only in band, in MPEG-TS, M2TS, AVI (H.264), and avc3 or hev1 MP4 and Matroska whose record the generator empties of parameter sets; and HEVC outside the bounded SPS parser’s bounds with its SPS in its hvcC: an interlaced field sequence, Main 12, and 4:2:2 10-bit. FFprobe checks each file’s codec, tag, pixel format, chroma location, frame count, and emptied record size

Two helpers serve them: generated_output.py (shared write, check, and install helpers) and engine_layout.py (the engine’s folders, read from tools/constants.json through tools/constants.py). avi_rewriting.py rewrites an FFmpeg AVI into the OpenDML layout FFmpeg’s muxer writes past 1 GiB, with a smaller RIFF list limit, and into DivX’s packed bitstream. The TrueHD generator computes the TypeScript import paths it embeds from the same constants, so moving a folder only needs a rerun.

The AV1 Dolby Vision and both HDR10+ generators take the error they raise, the pinned FFmpeg check, JSON comparison, and bit packing from media_tools.py. Both HDR10+ generators take the HDR10+ frame model, its ITU-T T.35 serialization, and its known answers for expectations.json and FFprobe from HDR10_plus_metadata.py.

The truehd decoder project’s tests import the TrueHD module as #codec_vector_assets/*, and scripts/library-assets.mjs serves the range extension, AV1 raw-format, and H.264 raw-format streams.

The generators work in one of three modes:

  • Deterministic generators rewrite their output, and with --check fail when a committed file differs from a fresh run: the TrueHD module, the 7.1 downmix reference, the MPEG-2 decoder streams, and the decoder projects’ vectors files.
  • Encoder-dependent generators compare a fresh encode with the committed bytes: the JPEG 2000, AV1, and MPEG-4 decoder media and the TrueHD source streams under --regenerate-sources. Another encoder build writes different bytes, so a differing encode never replaces a committed asset, while a missing one is installed. With --check, a missing asset fails as well.
  • The HEVC range-extension, AV1 raw-format, H.264 raw-format, AV1 Dolby Vision, HDR10+ AV1, HDR10+ VP9, and WMA decoder generators refuse any FFmpeg build but the pinned one. Before writing each encode the HEVC generator checks the access-unit lengths, the picture types, the profile and constraint flags of the VPS and SPS, and the decoded fingerprints. The AV1 raw-format generator runs the checks listed in AV1 raw-format vectors, and the H.264 raw-format generator those in H.264 raw-format vectors. The AV1 Dolby Vision generator runs the checks listed in AV1 Dolby Vision vectors, the HDR10+ AV1 generator those in HDR10+ AV1 vectors, and the HDR10+ VP9 generator those in HDR10+ VP9 vectors. The WMA decoder generator installs its media as the encoder-dependent generators do and rewrites its known answers from the committed media.

HEVC range-extension vectors

These Annex B files are deterministic WebCodecs capability probes. They are 192x192, because common NVIDIA HEVC decoders reject smaller coded sizes. They prove only that the exact codec configuration decodes, copies to the expected WebCodecs raw format, and produces the expected fingerprints. They prove nothing about performance, resolution or level limits, seeking, or transport.

Toolchain: FFmpeg 2026-03-01-git-862338fe31-full_build-www.gyan.dev, libavcodec 62.24.100, libx265 4.1+225-1b48507eb.

The source is FFmpeg testsrc2 at 192x192 and one frame per second. Each of the nine vectors is one IDR access unit followed by one P access unit, which qualifies general inter decode for the exact tuple.

Every vector uses these x265 settings, serial because there is no thread pool and one frame thread:

-preset fast -crf 32
-x265-params info=0:pools=none:frame-threads=1:wpp=0:log-level=error:keyint=30:min-keyint=30:scenecut=0:bframes=0:repeat-headers=1

info=0 prevents the x265 user-data SEI, so nothing is stripped after encoding. For example:

ffmpeg -f lavfi -i "testsrc2=size=192x192:rate=1:duration=2" \
  -frames:v 2 -pix_fmt yuv422p10le -c:v libx265 \
  -profile:v main422-10 -preset fast -crf 32 \
  -x265-params "info=0:pools=none:frame-threads=1:wpp=0:log-level=error:keyint=30:min-keyint=30:scenecut=0:bframes=0:repeat-headers=1" \
  -f hevc -y main422-10.hevc

After each encode the generator rewrites the profile, tier, and level fields of the VPS and SPS, removing and reinserting the emulation-prevention bytes around the rewritten RBSP:

  • Level: every vector signals Level 3.1 (general_level_idc 93). x265 gets no level-idc, because with CRF it would enforce the level through VBV, which it reports as non-deterministic. Without it x265 signals Level 1 and the slices are byte-identical, so the generator writes 93 instead.
  • Profile: x265 does not emit general profile IDC 4 for 4:2:0 at 8 or 10 bits, so for rext420-8 and rext420-10 the generator starts from conforming Main or Main 10 syntax and rewrites the fields to profile IDC 4, compatibility flag 4, and the general constraints 9F.88 or 9D.88.
FilePixel formatx265 profile
rext420-8.hevcyuv420pmain, rewritten
main422-8.hevcyuv422pmain422-10
main444-8.hevcyuv444pmain444-8
rext420-10.hevcyuv420p10lemain10, rewritten
main422-10.hevcyuv422p10lemain422-10
main444-10.hevcyuv444p10lemain444-10
main12-420.hevcyuv420p12lemain12
main422-12.hevcyuv422p12lemain422-12
main444-12.hevcyuv444p12lemain444-12

For every vector the generator verifies the FFprobe access-unit lengths and I/P types, the VPS and SPS constraint flags, and raw plane fingerprints computed as in production. Without --check, it replaces each committed file once its checks pass. --inspect prints the measured evidence without comparing or writing.

test/capability/HEVCRangeExtensionCapabilities.test.ts verifies the VPS constraint bytes, the intra and one-picture flags, the access-unit lengths, and the I/P slice sequence. Fingerprints use the production 32-bit FNV-1a sampler in capability/CustomDecodeCapabilities.ts: the plane dimensions, then a 64 by 36 uniform grid of samples over each output plane.

Re-encoding with another toolchain may not be byte-identical. Do not update an access-unit length, a constraint string, or a fingerprint until the static vector tests and the decoded raw output have been inspected together.

AV1 raw-format vectors

These low-overhead OBU streams (AV1 Section 5) qualify the raw-plane route for the AV1 formats outside Main 10, which the raw AV1 HDR probe qualifies. Like the HEVC range-extension vectors they are 192x192, fetched as library assets, and prove only that the exact codec configuration decodes, copies to the expected WebCodecs raw format, and produces the expected fingerprints. scripts/library-assets.mjs serves each av1-raw-format/<variant>.obu as webgpu-player/av1-raw/<variant>.bin.

Toolchain: FFmpeg 2026-03-01-git-862338fe31-full_build-www.gyan.dev, libavcodec 62.24.100, libaom 3.13.1-271-g85d21cdb24, libdav1d 1.5.3-18-gdaef396.

The source is FFmpeg testsrc2 at 192x192 and one frame per second. Each of the eight vectors is one key temporal unit followed by one inter temporal unit, back to back, and only the key unit carries the sequence header.

Every vector uses these libaom settings, serial because there is one thread, no row multithreading, and one tile:

-usage good -cpu-used 6 -crf 40 -b:v 0 -g 30 -lag-in-frames 0 -threads 1 -row-mt 0 -tiles 1x1

Good-quality usage clamps its speed to 6, so a higher -cpu-used writes the same bytes. No lookahead keeps each temporal unit to one shown frame, with no hidden alternate reference. For example:

ffmpeg -f lavfi -i "testsrc2=size=192x192:rate=1:duration=2" \
  -frames:v 2 -map_metadata -1 -fflags +bitexact -flags:v +bitexact \
  -pix_fmt yuv422p10le -c:v libaom-av1 \
  -usage good -cpu-used 6 -crf 40 -b:v 0 -g 30 -lag-in-frames 0 \
  -threads 1 -row-mt 0 -tiles 1x1 \
  -f obu -y professional422-10.obu
FilePixel formatJellyfin profileseq_profileCodec string
main420-8.obuyuv420pMain0av01.0.00M.08.0.110
high444-8.obuyuv444pHigh1av01.1.00M.08.0.000
high444-10.obuyuv444p10leHigh1av01.1.00M.10.0.000
professional422-8.obuyuv422pProfessional2av01.2.00M.08.0.100
professional422-10.obuyuv422p10leProfessional2av01.2.00M.10.0.100
professional420-12.obuyuv420p12leProfessional2av01.2.00M.12.0.110
professional422-12.obuyuv422p12leProfessional2av01.2.00M.12.0.100
professional444-12.obuyuv444p12leProfessional2av01.2.00M.12.0.000

Before writing each vector the generator checks:

  • the FFprobe temporal-unit lengths, one OBU demuxer packet each;
  • the I/P picture types of libdav1d’s decode;
  • the sequence header’s seq_profile, BitDepth, mono_chrome 0, and subsampling;
  • raw plane fingerprints of libdav1d’s decode, computed as in production.

Without --check, it replaces each committed file once its checks pass. --inspect prints the measured evidence without comparing or writing.

test/capability/AV1RawFormatCapabilities.test.ts reads the committed bytes and verifies the temporal-unit lengths, the key then inter frame types, and the sequence header’s profile, depth, and subsampling. The fingerprints use the same production sampler as the HEVC range-extension vectors. Re-encoding with another toolchain may not be byte-identical, so update a length or a fingerprint only after inspecting the static vector tests and the decoded raw output together.

H.264 raw-format vectors

These Annex B streams qualify the raw-plane route for H.264 beyond 8-bit 4:2:0, the profiles beyond High. Like the AV1 raw-format vectors they are 192x192, fetched as library assets, and prove only that the exact codec configuration decodes, copies to the expected WebCodecs raw format, and produces the expected fingerprints. scripts/library-assets.mjs serves each h264-raw-format/<variant>.h264 as webgpu-player/h264-raw/<variant>.bin.

Toolchain: FFmpeg 2026-03-01-git-862338fe31-full_build-www.gyan.dev, libavcodec 62.24.100, libx264 core 165 r3223 0480cb0, whose version the generator reads from the user data SEI of an unfiltered encode.

The source is FFmpeg testsrc2 at 192x192 and one frame per second. Each of the five vectors is an IDR access unit, a P access unit, and a B access unit, back to back in decode order, and only the IDR access unit carries the SPS and PPS. The B picture presents between the other two, so its frame outputs before the P frame’s, and a probe that expected decode order would fail. The capability definitions stamp each access unit with its presentation time: 0 s, 2 s, and 1 s.

Every vector uses these libx264 settings, serial because there is one thread and no sliced threads, with one B-frame, no adaptive B-frame placement, and no pyramid:

-preset medium -crf 30 -threads 1
-x264-params keyint=30:min-keyint=30:scenecut=0:bframes=1:b-adapt=0:b-pyramid=none:threads=1:sliced-threads=0
-bsf:v filter_units=remove_types=6

The bitstream filter drops every SEI NAL unit, x264’s version and option string among them. For example:

ffmpeg -f lavfi -i "testsrc2=size=192x192:rate=1:duration=3" \
  -frames:v 3 -map_metadata -1 -fflags +bitexact -flags:v +bitexact \
  -pix_fmt yuv422p10le -c:v libx264 -profile:v high422 \
  -preset medium -crf 30 -threads 1 \
  -x264-params "keyint=30:min-keyint=30:scenecut=0:bframes=1:b-adapt=0:b-pyramid=none:threads=1:sliced-threads=0" \
  -bsf:v filter_units=remove_types=6 -f h264 -y high422-10.h264
FilePixel formatx264 profileprofile_idcCodec string
high420-10.h264yuv420p10lehigh10110avc1.6E000B
high422-8.h264yuv422phigh422122avc1.7A000B
high422-10.h264yuv422p10lehigh422122avc1.7A000B
high444-8.h264yuv444phigh444244avc1.F4000B
high444-10.h264yuv444p10lehigh444244avc1.F4000B

Before writing each vector the generator checks:

  • the FFprobe access-unit lengths, one packet of FFmpeg’s H.264 demuxer each;
  • the slice types I, P, B in decode order, from each access unit’s first slice header, and the picture types I, B, P of FFmpeg’s decode in presentation order;
  • the SPS’s profile_idc, chroma_format_idc, and both bit depths, and the codec string, which is the SPS’s profile, constraint flags, and level;
  • raw plane fingerprints of FFmpeg’s decode in presentation order, computed as in production.

FFmpeg’s H.264 decoder is bit-exact, and Chrome’s software H.264 decoder is FFmpeg’s, so Chrome decodes each vector to the pinned fingerprints. Without --check, it replaces each committed file once its checks pass. --inspect prints the measured evidence without comparing or writing.

test/capability/H264RawFormatCapabilities.test.ts reads the committed bytes and verifies the access-unit lengths, the NAL unit types, the slice types, the SPS’s profile, chroma format, and bit depths, and the codec string. Re-encoding with another toolchain may not be byte-identical, so update a length or a fingerprint only after inspecting the static vector tests and the decoded raw output together.

AV1 decoder vectors

These are the av1 decoder project’s self-test media, one Matroska stream per output format. wgpud/av1/decoder.json lists them with the decoder options -c:v libdav1d, so the package’s known answers are libdav1d’s decode, film grain included. libaom’s decoder decodes every frame of every stream to the same bytes as libdav1d. generate_av1_decoder_vectors.py --print-declarations prints that list.

FFmpeg’s libaom-av1 encoder encodes each stream in one thread at speed 0 and CRF 50, and the generator runs the nine encodes at once. Each stream is 13 frames at 24 frames per second with a key frame every 12, so it holds two key frames. A 19-frame lookahead makes four hidden alternate references, three or four of which show_existing_frame shows.

The source is a lavfi canvas of 256x192 that rotates, zooms, and pans as a whole, from the identity at the first frame, which libaom codes with global and local warped motion:

  • a frozen 64x64 Game of Life pattern of two colours at (0, 0), copied to (128, 64) and (128, 128), where IntraBC may copy it;
  • a second pattern at (64, 0), for palette;
  • two testsrc2 pictures, whose moving parts give OBMC, compound, and inter-intra prediction;
  • smooth planes that geq draws at (192, 64), drifting each frame, for 64-point transforms. lavfi’s gradients source draws random points whatever its seed, so it is not used.

libaom lets an IntraBC block copy only from more than four 64x64 superblocks back in its tile, in raster order. With 64x64 superblocks the tiles are 3+1 columns and 2+1 rows of superblocks, so the copy at (128, 64) and the pattern at (0, 0) share a tile three superblocks wide. With 128x128 superblocks there is one tile, and the copy at (128, 128) is one superblock row below the pattern.

Every encode also asks for screen content tools (tune-content=screen), segmentation (-aq-mode 1), delta quantizer and loop filter levels, and quantization matrices. It turns off the temporal filtering of key frames, which blurs the copies IntraBC needs.

FileFormatSuperblocks and tilesFilm grain
av1-420-250x186.mkvI42064x64, four non-uniform tilesSchedule A
av1-420p10-253x187.mkvI420P1064x64, four non-uniform tileslibaom test table 3
av1-420p12-256x192-sb128.mkvI420P12128x128, one tileSchedule B
av1-422-254x190.mkvI42264x64, four non-uniform tileslibaom test table 1
av1-422p10-249x183-sb128.mkvI422P10128x128, one tileSchedule A
av1-422p12-256x192.mkvI422P1264x64, four non-uniform tileslibaom test table 6
av1-444-251x189-sb128.mkvI444128x128, one tileSchedule B
av1-444p10-256x188.mkvI444P1064x64, four non-uniform tileslibaom test table 14
av1-444p12-247x191.mkvI444P1264x64, four non-uniform tilesSchedule A

A schedule is a libaom film grain table the generator writes beside the encode, one entry per range of frames:

  • schedule A: autoregression lag 0 with chroma scaled from luma, then no grain, then lag 1 without overlap, then lag 3;
  • schedule B: lag 2 with chroma scaled from luma and without overlap, then lag 3, then no grain, then lag 1.

The table has bare line feeds, because libaom’s reader stops at a carriage return without an error and the encode then has no grain. libaom’s test tables carry no lag 0 or 1, but they clip to the restricted range, which a table cannot ask for.

The streams hold no super-resolution, reference scaling, dual filters, lossless frames, intra-only frames, or switch frames, so a self-test does not cover them. The pinned FFmpeg cannot ask its libaom for super-resolution or a frame size change, because that libaom takes no superres-mode or resize-mode key-value option, and libaom’s speed features disable dual filters at every speed.

MPEG-4 decoder media

These are the mpeg4 decoder project’s media: its self-test stream, and a stream of each codec in each container the engine routes it from, which the route tests play. wgpud/mpeg4/decoder.json lists the self-test stream with the decoder options -threads 1 -flags +bitexact, FFmpeg’s C code in one thread, and with codecPrivate bitmap-info-header, the codec private data the engine passes the package.

The self-test stream is 200x120 from libxvid, with global motion compensation, quarter-sample motion, and two B-VOPs between anchors. Its source is testsrc2 at twice the size, turned and zoomed into its center a little more each frame, so libxvid codes S-VOPs among its P-VOPs. libxvid writes a raw stream, from which the generator drops the N-VOPs libxvid writes for its delayed frames, as a native Matroska muxer drops them. FFmpeg then remuxes it into Matroska with the presentation timestamps it generates, since FFmpeg’s libxvid output in Matroska stamps each P-VOP before the B-VOPs shown ahead of it.

FileStream
mpeg4-gmc-qpel-b.mkvThe self-test stream, V_MPEG4/ISO/ASP
mpeg4-b.mp4FFmpeg’s MPEG-4 Part 2 with B-VOPs, mp4v
mpeg4-b-xvid.aviFFmpeg’s MPEG-4 Part 2 with B-VOPs under XVID, its VOL in the first packet
mpeg4-b-vfw.mkvFFmpeg’s MPEG-4 Part 2 with B-VOPs in a V_MS/VFW/FOURCC track of XVID, at decode-order timestamps
msmpeg4v3.aviMS-MPEG4 V3 under DIV3
msmpeg4v3.mkvMS-MPEG4 V3 in V_MPEG4/MS/V3
msmpeg4v2-vfw.mkvMS-MPEG4 V2 in a V_MS/VFW/FOURCC track of MP42
msmpeg4v2.aviMS-MPEG4 V2 under MP42
h263.3gpH.263 at sub-QCIF, s263
h263.aviH.263 at sub-QCIF under H263
h263p-vfw.mkvH.263+ at 200x120 with advanced intra coding, the deblocking filter, and unrestricted motion vectors, in a V_MS/VFW/FOURCC track of H263

The other streams are 16 frames with a key frame every 8, the MPEG-4 Part 2 and MS-MPEG4 ones at 176x144. Every encoder runs in one thread, since the MPEG-4 Part 2 encoder’s slice threads change its bitstream with the thread count. FFmpeg’s Matroska muxer writes MPEG-4 Part 2 only as a native track, so the generator rewrites its file into mpeg4-b-vfw.mkv. The rewrite sets the codec ID, a CodecPrivate of the BITMAPINFOHEADER FFmpeg’s muxers write followed by the VOL, and block timestamps one frame apart in decode order, as an AVI stamps its frames. It drops the seek head and the cues, whose offsets it moves.

Each medium must probe as its codec, FourCC, and size, show its picture types, and decode one frame per packet in the reference decode without an error. The self-test stream must also present its frames in the order of their timestamps.

WMA decoder vectors

These are the wma decoder project’s media and FFmpeg’s known answers for them, which wgpud/wma/test/WMAPackage.test.ts decodes through the package and the engine’s WAVEFORMATEX reader, and two media with video for the engine’s decode worker tests, which play only sources with video. The generator writes them with the pinned FFmpeg, libavcodec 62.24.100.

FileStreamSource
wmav1.aviWMA 1, 48 kHz stereo, 96 kb/s, 12 packetsFFmpeg’s wmav1 encoder
wmav1.mkvThe same packets in an A_MS/ACM trackA remux of wmav1.avi
wmav2.aviWMA 2, 44.1 kHz stereo, 128 kb/s, 11 packetsFFmpeg’s wmav2 encoder
wmav2.mkvThe same packets in an A_MS/ACM trackA remux of wmav2.avi
wmapro-5.1.mkvWMA Pro, 48 kHz 5.1 with a WAVE_FORMAT_EXTENSIBLE header, 2 packetswmapro/latin_192_mulitchannel_cut.wma
wmalossless-16.mkvWMA Lossless, 44.1 kHz stereo 16-bit, 2 packetslossless-audio/luckynight-partial.wma
wmalossless-24.mkvWMA Lossless, 48 kHz stereo 24-bit, 2 packets, the whole samplelossless-audio/Mega_Weird_Audio_Test_24bit.wma
wmavoice.mkvWMA Voice, 8 kHz mono, 4 packetswmavoice/streaming_CBR-7K.wma
wmav2-h264.aviThe packets of wmav2.avi beside 5 frames of 64x64 H.264 at 10 frames per second; no known answerlibx264 and the packets of wmav2.avi
wmav2-h264.mkvThe same, the audio in an A_MS/ACM tracklibx264 and the packets of wmav2.avi

The two encodes are half a second of a 440 Hz tone on the left channel and a 660 Hz tone on the right, from lavfi’s aevalsrc, so a decode in the wrong channel order shows:

ffmpeg -f lavfi -i "aevalsrc=exprs=0.5*sin(2*PI*440*t)|0.25*sin(2*PI*660*t):sample_rate=44100:duration=0.5:channel_layout=stereo" \
  -c:a wmav2 -b:a 128k -map_metadata -1 -fflags +bitexact -flags:a +bitexact -f avi wmav2.avi
ffmpeg -i wmav2.avi -map 0:a -c copy -map_metadata -1 -fflags +bitexact -flags:a +bitexact -f matroska wmav2.mkv

The Matroska files are remuxes of the AVI files, so both carry the same packets, which the generator checks, and no Matroska CodecDelay. With a CodecDelay, FFmpeg’s decode would skip the encoder’s 2048 priming frames instead of the 4096 frames of its decoder’s own delay, which the package, without the container’s skip, skips.

The two media with video take the packets of wmav2.avi too, beside libx264’s encode of testsrc2 in one thread with the ultrafast preset and a key frame every 5 frames:

ffmpeg -f lavfi -i testsrc2=size=64x64:rate=10 -i wmav2.avi -map 0:v -map 1:a -frames:v 5 -pix_fmt yuv420p -c:v libx264 -preset ultrafast -g 5 \
  -threads 1 -flags:v +bitexact -c:a copy -map_metadata -1 -fflags +bitexact -flags:a +bitexact -f matroska wmav2-h264.mkv

FFmpeg has no WMA Pro, WMA Lossless, or WMA Voice encoder, so the other media come from FFmpeg’s FATE suite, https://fate-suite.ffmpeg.org/. The generator downloads each sample over HTTPS, refuses one whose SHA-256 differs, and remuxes its first packets into Matroska:

SampleSHA-256
wmapro/latin_192_mulitchannel_cut.wma3f3a18ce22ac061d4abc38b553a1d00cfe6795d76e2366c1aa5aa7a4c8c230b8
lossless-audio/luckynight-partial.wma33d7dc28f717abcc8ef7f35c1c836840503e2d888c3dec833c53314eba81aa1b
lossless-audio/Mega_Weird_Audio_Test_24bit.wma1ce71dcd98db91d23896e84c602c774fc93eeb6afc94fd762382744d680d2bbd
wmavoice/streaming_CBR-7K.wma049a5a4c758b07b874af9a8a2473ff2a3cc8234f79e387dfd6daf6780a58c6d5
ffmpeg -i latin_192_mulitchannel_cut.wma -map 0:a -c copy -frames:a 2 -map_metadata -1 -fflags +bitexact -flags:a +bitexact -f matroska wmapro-5.1.mkv

The 5.1 WMA Pro sample is the suite’s one multichannel WMA Pro sample; its fourth packet is cut short, and the remux keeps the first two, about three seconds. FFmpeg’s Matroska muxer writes its six channels as WAVE_FORMAT_EXTENSIBLE, with the channel mask of 5.1 with back speakers.

known-answers.json names the FFmpeg build and the decoder options -cpuflags 0, FFmpeg’s C code without assembly, and records per vector without video its codec, sample rate, channel count, WAVE channel mask, frame count, and the SHA-256 of FFmpeg’s interleaved 32-bit float PCM:

ffmpeg -cpuflags 0 -c:a wmapro -i wmapro-5.1.mkv -f f32le wmapro-5.1.f32

An answer is FFmpeg’s whole decode, the frames FFmpeg’s decoder holds back until it drains at the end of the stream included, which WMA Lossless and WMA 1 and 2 have; the test drains the package after the last packet. A layout without a channel mask, as WMA 1 and 2 have, records the mask of FFmpeg’s default layout of its channel count, which the bridge reports.

The package decodes every vector to the same bytes as FFmpeg’s C code, WMA Lossless’s integers and every lossy codec alike, so the test compares the SHA-256. FFmpeg’s default decode, with its x86 assembly, differs from it by up to 7e-7 in WMA Voice and 2e-7 in WMA 1, WMA 2, and WMA Pro, and matches it in WMA Lossless.

E-AC-3 Atmos vectors

These are the eac3 decoder project’s Dolby Atmos media and FFmpeg’s known answers for them, which wgpud/eac3/test/EAC3Package.test.ts decodes through the package at every render target. generate_eac3_joc_vectors.py downloads each source at a pinned commit, requires its pinned SHA-256, and remuxes its first packets into Matroska with the pinned FFmpeg, checking that the packets are the source’s.

FileStreamSource
joc-5.1.mkvE-AC-3 with JOC, 48 kHz 5.1 core, 15 JOC objects, OAMD of a 5.1 bed and 10 dynamic objects, EMDF in audio-block skip fields, 16 packetsThe first packets of sample_eac3joc.ec3 from androidx/media commit 8c6678b, libraries/test_data/src/test/assets/media/ts/, Apache-2.0, a Dolby encode
joc-5.1.2-dependent.mkvE-AC-3 with JOC, a 5.1 independent substream and a dependent one with the top front pair, JOC downmix configuration 2, 15 objects, EMDF in the dependent substream’s auxiliary data, 1 packetidx2.ec3 from OpenJOC commit e3e330a, crates/openjoc-api/tests/fixtures/admission/, Apache-2.0, a synthetic encode

known-answers.json holds FFmpeg’s decode of each vector with its C code (-cpuflags 0): the stock decode the package outputs whenever its target needs no objects, which it matches byte for byte.

AV1 Dolby Vision vectors

These test vectors are Dolby Vision Profile 10 in MP4 and in Matroska, one pair per sub-profile: 10.0 (CCID 0), 10.1 (CCID 1, BT.2020 PQ base), 10.2 (CCID 2, BT.709 base), and 10.4 (CCID 4, BT.2020 HLG base). Each is a libaom AV1 Main 10 encode of testsrc2, 192x192 and four frames with a key frame every second frame. Every temporal unit carries one Dolby Vision ITU-T T.35 metadata OBU, and the container signals the configuration record: a dvvC box in a dav1 (10.0) or av01 sample entry, or a Matroska BlockAdditionMapping.

The RPUs are the dovi_tool payloads from src/capability/vectors/test/dolby-vision-rpu/, carried byte for byte as the EMDF payload of the T.35 message:

  • 10.0 alternates profile5.bin and profile5-02.bin;
  • 10.1 and 10.2 use profile8.bin;
  • 10.4 uses profile84.bin.

Each frame’s RPU therefore parses to the same output as the HEVC parse of its source file, which is the known answer. expectations.json records, per vector, the container, sample entry, Dolby Vision configuration, and color, and per frame the source RPU, the key flag, and the SHA-256 of the T.35 message.

FFmpeg’s own AV1 Dolby Vision encoding is not used. Its encoders rebuild each RPU from the decoded metadata, which reorders extension blocks, and its command line cannot signal 10.0, which needs the RPU header when the encoder is set up. Instead the generator inserts the T.35 OBUs after the sequence header, where libaom places metadata, adds a dvvC box to an intermediate MP4, and lets FFmpeg remux that file into both vectors, so FFmpeg writes the container records. It then renames the 10.0 MP4 sample entry to dav1, which FFmpeg cannot write. Before writing, it checks:

  • the RPU and key flag of every temporal unit;
  • the sequence header color;
  • the MP4 sample entry and boxes;
  • what FFprobe reads back from both containers;
  • that FFmpeg parses the same Dolby Vision metadata from each T.35 OBU as from the same RPU carried as HEVC NAL 62.

test/demux/dolby-vision/DolbyVisionAV1DemuxIntegration.test.ts opens every vector with Mediabunny, splits each temporal unit with the engine’s splitter, and requires the parser’s packed output to equal the HEVC parse of the source RPU. The vectors prove the demux, the split, and the parse; they prove nothing about decode or presentation.

HDR10+ AV1 vectors

These test vectors are one AV1 Main 10 encode in BT.2020 PQ, in MP4 (hdr10plus.mp4) and in Matroska (hdr10plus.mkv). It is a libaom encode of testsrc2 with the settings of the AV1 Dolby Vision vectors, 192x192 and six frames with a key frame every third frame. The generator writes every metadata payload itself, field by field, and inserts the metadata OBUs before each temporal unit’s frame, where libaom places metadata:

  • each key frame’s unit carries a mastering display color volume (MDCV) OBU and a content light level (CLL) OBU: BT.2020 primaries, a D65 white point, 1000 and 0.005 nits, MaxCLL 940, and MaxFALL 410;
  • the first, second, and sixth units carry a profile B HDR10+ ITU-T T.35 OBU, each with its own curve, targeted display, and distribution;
  • the fourth unit carries a profile A HDR10+ OBU, without a curve and with a targeted display of 0, whose message ends in a zero byte that must stay when the trailing bits are removed;
  • the third and fifth units carry no HDR10+.

FFmpeg muxes the stream into the MP4 and remuxes the MP4 into Matroska. Both keep the metadata OBUs in each sample, and the av1C record FFmpeg writes repeats the first unit’s MDCV and CLL OBUs. expectations.json records the color, the coded MDCV and CLL values, and per frame the key flag, whether its unit carries MDCV and CLL, the coded HDR10+ values, and the T.35 message without its trailing bits.

Before writing, the generator checks:

  • the metadata OBUs of every temporal unit, read back from the stream;
  • the codec, color, packet sizes, and key flags FFprobe reads from the stream and from both containers, and that neither container carries static HDR metadata of its own;
  • the side data FFmpeg decodes for every frame of both containers, the independent oracle: libdav1d reports the MDCV and CLL on every frame from the first key frame on, and HDR10+ only on the frame whose unit carries it.

test/demux/hdr/AV1HDR10PlusDemuxIntegration.test.ts reads both vectors’ packets with Mediabunny and requires the engine’s extraction and static scan to read every frame as recorded. test/pipeline/CustomDecodeWorkerAV1Route.test.ts plays them through the worker with a stand-in WebCodecs decoder, and requires every posted frame to carry its recorded HDR10+ result. The vectors prove the demux, the extraction, and the routing; they prove nothing about decode or presentation.

HDR10+ VP9 vectors

These test vectors are one VP9 Profile 2 encode, 10-bit in BT.2020 PQ, in WebM (hdr10plus.webm) and in Matroska (hdr10plus.mkv). It is 192x192 at 24 frames per second, eight frames with a key frame every fourth frame, and each group of four frames is one cluster. VP9 has no metadata of its own, so each frame’s HDR10+ is an ITU-T T.35 message in a BlockAdditional of its block, under FFmpeg’s BlockAddID 4:

  • the first, second, sixth, seventh, and eighth frames carry profile B metadata with different curves, targeted displays, and distributions, up to 15 percentiles and 15 anchors, and the last two are identical;
  • the fourth frame carries profile A metadata, without a curve and with a targeted display of 0;
  • the third frame and the second key frame carry none.

Toolchain: the pinned FFmpeg, with libx265 4.1+225-1b48507eb and libvpx v1.16.0-56-g4fcebeabe. FFmpeg writes it all:

  1. libx265 encodes testsrc2 with one dhdr10-info entry per frame, so FFmpeg decodes HDR10+ frame side data.
  2. The sidedata filter deletes that side data from the frames without HDR10+, and setparams tags every frame BT.2020 PQ, because libvpx-vp9 keeps HDR10+ only when the encoder opens on a PQ transfer, which it reads from the first frame.
  3. libvpx-vp9 encodes Profile 2 in one thread, without lookahead or alternate reference frames, so every packet is one shown frame, and passes each frame’s HDR10+ to its packet.
  4. The WebM muxer writes each packet’s HDR10+ as a BlockAdditional, without a BlockAdditionMapping, which WebM does not have.
  5. FFmpeg remuxes the WebM into Matroska, which adds the BlockAdditionMapping of BlockAddID 4 to the ITU-T T.35 type.

expectations.json records the color, and per frame the key flag, the coded HDR10+ values, and the BlockAdditional bytes.

Before writing, the generator checks:

  • every block’s BlockAdditionals, read back with its own EBML walk, against the frame’s ST 2094-40 serialization in HDR10_plus_metadata.py, and the DocType, Colour, BlockAdditionMapping, key frames, and clusters of each container;
  • the codec, profile, color, key flags, and HDR10+ side data FFprobe reads from every packet of both containers, the independent oracle.

test/demux/MatroskaBlockAdditions.test.ts reads both vectors’ block additions from the fork’s packet side data, through Mediabunny’s formats and the worker’s, and fails when any block’s additions no longer arrive as recorded. test/demux/hdr/HDR10PlusVP9DemuxIntegration.test.ts plays both through the owned VP9 path with a stand-in decoder, from the start and from the second key frame, and requires every frame’s HDR10+ result to be the recorded one. test/pipeline/CustomDecodeWorkerOwnedVP9.test.ts plays them through the worker with a stand-in WebCodecs decoder, and requires every posted frame to carry its recorded HDR10+ result. The vectors prove the demux, the extraction, and the routing; they prove nothing about decode or presentation.

Local playback media

These write media for smoke tests through a host in a real browser. The generate_ scripts write to the ignored bin/playback_smoke_media/ unless given --output-directory, and the create_ scripts write to the output path they are given. Real playback evidence comes from the host, not from the checks above.

ScriptWrites
generate_playback_smoke_media.pyHDR10 and HLG HEVC Main 10 playback media, and audio-switch variants
generate_static_HDR_validation_vectors.pyStatic HDR validation media
generate_native_HEVC_High_Tier_validation_vector.pyA native HEVC High Tier stream
generate_dolby_vision_AV1_playback_media.py1080p Dolby Vision Profile 10 AV1 with stereo AAC, per sub-profile, in MP4 and Matroska. The RPUs do not describe the test pattern, so the files prove only that the pipeline runs
create_separate_track_dolby_vision_vector.pyFrom an interleaved Profile 7 Matroska: one BL track, and one Profile 7 EL and RPU track
create_dual_track_dolby_vision_MP4_vector.pyFrom the separate-track output: an MP4 with a base HEVC track and a dependent dvh1 or dvhe enhancement track
create_profile7_playback_vector.pyFrom the separate-track output: a dual-track MP4 for full playback tests. Its BL is scaled to 1080p and re-encoded, so it is no color reference
create_dual_PID_dolby_vision_TS_vector.pyFrom a two-track Profile 7 file: a dual-PID MPEG-TS whose EL entry carries a Profile 7 dependency descriptor
create_container_only_HVCE_vector.pyA Matroska copy whose wrapped EL parameter sets are replaced by filler, so the EL decodes only from the container’s hvcE

The create_ scripts transform a Profile 7 source you supply. They and both AV1 Dolby Vision generators take an option for each tool they run: FFmpeg and FFprobe from --ffmpeg and --ffprobe or PATH, and MKVToolNix from --mkvtoolnix-directory, then %ProgramFiles%\MKVToolNix on Windows, then PATH. create_container_only_HVCE_vector.py rewrites the Matroska file itself and runs no tool. media_tools.py holds that lookup, the bounded tool runner they share, and a check that refuses an output path naming the input file, by case variant or link included.

Hand-made vectors

FolderHolds
src/capability/vectors/qualification/vc1/The VC-1 Advanced stream that the vc1 decoder project’s vector holds
src/capability/vectors/test/dts/Public-domain dcadec samples, packet tables, and reference WAVs, and FFmpeg’s stereo and 5.0 DTS core encodes with one tone per channel (PROVENANCE.txt)
src/capability/vectors/test/ac3/FFmpeg’s AC-3 encode of a 3/2 bed that gains its LFE, one tone per channel, for the eac3 package’s 5.0 and 5.1 layouts (PROVENANCE.txt)
src/capability/vectors/test/truehd/A Matroska remux of one generated TrueHD stream (PROVENANCE.txt)
src/capability/vectors/test/dolby-vision-rpu/RPU payloads, copied unmodified from dovi_tool assets/tests (MIT), for the RPU parser, color transform, and Dolby Vision integration tests and the AV1 Dolby Vision generators
src/capability/vectors/test/hdr10plus-tool/x265’s --dhdr10-opt HDR10+ profile A sample and the metadata hdr10plus_tool extracts from it, copied unmodified from hdr10plus_tool assets/hevc_tests (MIT, PROVENANCE.txt pins the commit and SHA-256s), for the HDR10+ parser and carry tests

Tests

Most scripts have a test module in scripts/codec_vector_assets/test/; the TrueHD, 7.1 downmix reference, and decoder package vector generators have none. A test adds the generator folder to the import path, so it also runs on its own:

python scripts/codec_vector_assets/test/generated_output_test.py