Implement Fragmented WebVTT vtte/vttc #1639
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
peaBerberian
added
work-in-progress
peaBerberian
force-pushed
the
feat/vttc
branch
from
0abb771 to
0392a72
Compare
peaBerberian
force-pushed
the
feat/vttc
branch
2 times, most recently
from
54c9f43 to
2343307
Compare
peaBerberian
force-pushed
the
feat/vttc
branch
7 times, most recently
from
ec871f8 to
c4dadbd
Compare
|
@peaBerberian tested with the steam in the PR description, and also some internal static DASH streams. Only tested browser, but working well. I can test again when this moves out of work in progress state. |
peaBerberian
removed
the
work-in-progress
OK thanks. I just removed the work-in-progress label because I resolved all the remaining points I was unhappy with. It is now in review phase. To facilitate future tests, I bundled this branch through github's CI which makes it possible to just put in a package.json: "rx-player": "https://github.com/peaBerberian/rx-player/releases/download/autobuild.6ff637df7d0d2ee8e4851960d62206b73a12d1b7/rx-player.tgz" |
peaBerberian
added
the
Priority: 2 (Medium)
peaBerberian
changed the title
peaBerberian
force-pushed
the
dev
branch
2 times, most recently
from
00fc806 to
b7216b4
Compare
src/core/segment_sinks/implementations/text/text_segment_sink.ts
Outdated
Show resolved
Hide resolved
As #1638 reported, we do not support some (or all?) ways of communicating webvtt subtitles in mp4 files. I initially assumed that webvtt embedded in an mp4 file worked like for TTML (a format we're more used to): the plain subtitle format directly inserted in an `mdat` box. Turns out, some contents (like https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel-wvtt.ism/.mpd), actually rely on the metadata from other ISOBMFF boxes (mainly the tfdt and trun boxes) to provide timing information with the `mdat` reserved for text, identifiers and style information in a new binary format following the ISOBMFF way of doing things (new webvtt-specific boxes). Weirdly enough, that format does not look at all like the WEBVTT we're used to beside the fact that it uses the same identifier and "settings" concept. --- As now our subtitle parser has to have the context of the whole mp4 file (and not of the mp4 segment), and as that parser also has to rely on state (a `timescale` value) provided by an initialization segment, I had to update the architecture of how subtitles are communicated: they can now be communicated as string or `BufferSource` types (the latter leading the text encoding detection), and a supplementary `timescale` argument (defaulting to `1`) is now always provided to parsers. The vast majority of parsers now do not make use of that `timescale` value which is kind of ugly though, we may want to find a better solution.
|
✅ Automated performance checks have passed on commit DetailsPerformance tests 1st run outputNo significative change in performance for tests:
|
|
✅ Automated performance checks have passed on commit DetailsPerformance tests 1st run outputNo significative change in performance for tests:
|
Florent-Bouisset
approved these changes
Jun 16, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
NOTE: I also deployed a demo just to have feedbacks here: https://developers.canal-plus.com/rx-player/vttc/
For example with a content reproducing the issue: https://developers.canal-plus.com/rx-player/vttc/#!tech_4=DASH!manifest_2w=https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel-wvtt.ism/.mpd
As #1638 reported, we do not support some (or all?) ways of communicating webvtt subtitles in mp4 files.
I initially assumed that webvtt embedded in an mp4 file worked like for TTML (a format we're more used to): the plain subtitle format directly inserted in an
mdatbox.Turns out, some contents (like https://demo.unified-streaming.com/k8s/features/stable/video/tears-of-steel/tears-of-steel-wvtt.ism/.mpd), actually rely on the metadata from other ISOBMFF boxes (mainly the tfdt and trun boxes) to provide timing information with the
mdatreserved for text, identifiers and style information in a new binary format following the ISOBMFF way of doing things (new webvtt-specific boxes).Weirdly enough, that format does not look at all like the WEBVTT we're used to beside the fact that it uses the same identifier and "settings" concept.
To implement this, I didn't find the spec so I mainly looked around what other players were doing, I hope I did not bring other player bugs or some copyright trap while doing this :D
As now our subtitle parser has to have the context of the whole mp4 file (and not only the
mdat's content), and as that parser also has to rely on state (atimescalevalue) provided by an initialization segment, I had to update the architecture of how subtitles are communicated: they can now be communicated as string orBufferSourcetypes (the latter leading to text encoding detection), and a supplementarytimescaleargument (defaulting to1) is now always provided to parsers.