DASH: Be more optimistic on the buffer depth of DASH contents

Overview
--------

We were debugging some ad insertion usage, where an ad would be
dynamically inserted at a live content's start on the initial Manifest
request.

An application reported to us that while playing such content with the
RxPlayer, we were starting a few seconds into the ad instead of right
at the beginning of it. When looking at segment requests we saw that the
first media segment was never requested: we always started with the second
one instead.

Issue
-----

After investigation, I found out that one of the issue, was that the
`MPD@timeShiftBufferDepth` attribute of the produced stream was on the
first Manifest request exactly equal to the duration of the video segments
generated until now.

Yet the RxPlayer always interpreted that it could only play segments
whose ending timestamp is inferior or equal to the "live edge" (the live
position basically, which monotically increases as time increases) and,
more importantly here, whose starting timestamp is superior or equal to
that same live edge minus the `timeShiftBufferDepth`.

All this is not shocking, and other players have a similar interpretation
here, but I noticed that this was why we were not requesting the first
segment on the problematic content: as some milliseconds passed between
the time the Manifest is parsed and the time at which we began choosing a
segment to request, our "live edge" as further in the future (by the same
amount of milliseconds) and thus the minimum timestamp we can request will
also be further in the future - and ahead of the first segment position
by the same amount of milliseconds.

How to fix this
---------------

So the fix seemed to be to either raise the `MPD@timeShiftBufferDepth`
server-side or to raise the tolerance of what can still be requested
RxPlayer-side.

I dived into the DASH specification (5th edition) - which is a little
hard to interpret here - but I found the following passage:

> Assume the parameter availabilityTimeOffset is determined as the sum of all values of
@availabilityTimeOffset on all levels that are processed in determining the URL for the
corresponding segment. If the attribute @availabilityTimeOffset is not present, the value is of
availabilityTimeOffset is 0. Then for services with MPD@type='dynamic', the Segment
availability start time Tavail[i] for a Segment i in a specific Period is determined as
MPD@availabilityStartTime + PeriodStart + MediaSegment[i].startTime +
MediaSegment[i].duration - availabilityTimeOffset and the Segment availability end time is
determined as MPD@availabilityStartTime + PeriodStart + MediaSegment[i].startTime +
@timeshiftBufferDepth + 2*MediaSegment[i].duration

At the very end, it seems that we have to re-apply a segment's duration,
on top of the `timeShiftBufferDepth` to know when it is not available
anymore (they indicate "2 times the duration").

So I guess we may have been too pessimistic with how we previously
considered whether a segment could be requested or not.

How other players treat this
----------------------------

Because the people behind the ad-insertion tool told me they succeeded
to rely on the shaka-player, I looked at what they did on this issue.

It turns out that they have roughly the same interpretation that we had
for the minimum requestable position (`live edge -
timeshiftBufferDepth`) but their "live edge" was basically offset by `-
MPD@maxSegmentDuration`, so in the end they were removing the MPD's
maximum segment duration from the minimum requestable position
relatively to us.

If I had to guess why they do that, it may be that they clonflate the
"live edge" concept (the timestamp pointing to the current last media
data basically) and what in the RxPlayer we call the "maximum safe
position" (the last timestamp for which we're sure that a segment is
currently available) where the RxPlayer make those two very different
concepts and properties.

Author: peaBerberian

src/parsers/manifest/dash/common/__tests__/manifest_bounds_calculator.test.ts

@@ -9,9 +9,9 @@ describe("DASH parsers - ManifestBoundsCalculator", () => {
       availabilityStartTime: 0,
       serverTimestampOffset: undefined,
     });
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(undefined);
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(undefined);
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(undefined);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(undefined);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(3)).toEqual(undefined);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(5)).toEqual(undefined);
   });
 
   it("should return 0 through `getEstimatedMinimumSegmentTime` for a static content", () => {
@@ -21,10 +21,10 @@ describe("DASH parsers - ManifestBoundsCalculator", () => {
       availabilityStartTime: 0,
       serverTimestampOffset: 555555,
     });
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(0);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(0);
     manifestBoundsCalculator.setLastPosition(5555, 2135);
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(0);
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(0);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(4)).toEqual(0);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(5)).toEqual(0);
   });
 
   it("should return 0 through `getEstimatedMinimumSegmentTime` if the `serverTimestampOffset` was never set nor the last position for a dynamic content with no timeShiftBufferDepth", () => {
@@ -34,9 +34,9 @@ describe("DASH parsers - ManifestBoundsCalculator", () => {
       availabilityStartTime: 0,
       serverTimestampOffset: undefined,
     });
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(0);
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(0);
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(0);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(0);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(5)).toEqual(0);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(28)).toEqual(0);
   });
 
   it("should return `false` through `lastPositionIsKnown` if `setLastPositionOffset` was never called", () => {
@@ -47,7 +47,7 @@ describe("DASH parsers - ManifestBoundsCalculator", () => {
       serverTimestampOffset: undefined,
     });
     expect(manifestBoundsCalculator.lastPositionIsKnown()).toEqual(false);
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(undefined);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(undefined);
     expect(manifestBoundsCalculator.lastPositionIsKnown()).toEqual(false);
   });
 
@@ -86,9 +86,10 @@ describe("DASH parsers - ManifestBoundsCalculator", () => {
     });
     manifestBoundsCalculator.setLastPosition(1000, 10);
     performanceNow = 25000;
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(1010);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(1010);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(50)).toEqual(960);
     performanceNow = 35000;
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(1020);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(1020);
     mockPerformanceNow.mockRestore();
   });
 
@@ -104,18 +105,27 @@ describe("DASH parsers - ManifestBoundsCalculator", () => {
       serverTimestampOffset: 7000,
     });
     manifestBoundsCalculator.setLastPosition(3000, 10);
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(
       7 + 5 - 4 - 3,
     );
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(4)).toEqual(
+      7 + 5 - 4 - 3 - 4,
+    );
     performanceNow = 25000;
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(
       7 + 25 - 4 - 3,
     );
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(10)).toEqual(
+      7 + 25 - 4 - 3 - 10,
+    );
     performanceNow = 35000;
     manifestBoundsCalculator.setLastPosition(84546464, 5642);
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(
       7 + 35 - 4 - 3,
     );
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(5)).toEqual(
+      7 + 35 - 4 - 3 - 5,
+    );
     mockPerformanceNow.mockRestore();
   });
 
@@ -132,10 +142,12 @@ describe("DASH parsers - ManifestBoundsCalculator", () => {
     });
     manifestBoundsCalculator.setLastPosition(1000, 0);
     performanceNow = 50000;
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(1045);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(1045);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(40)).toEqual(1005);
     manifestBoundsCalculator.setLastPosition(0, 0);
     performanceNow = 55000;
-    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime()).toEqual(50);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(0)).toEqual(50);
+    expect(manifestBoundsCalculator.getEstimatedMinimumSegmentTime(40)).toEqual(10);
     mockPerformanceNow.mockRestore();
   });
 

src/parsers/manifest/dash/common/indexes/template.ts

@@ -557,7 +557,9 @@ export default class TemplateRepresentationIndex implements IRepresentationIndex
     }
 
     const { duration, timescale } = this._index;
-    const firstPosition = this._manifestBoundsCalculator.getEstimatedMinimumSegmentTime();
+    const firstPosition = this._manifestBoundsCalculator.getEstimatedMinimumSegmentTime(
+      duration / timescale,
+    );
     if (firstPosition === undefined) {
       return undefined;
     }

src/parsers/manifest/dash/common/indexes/timeline/timeline_representation_index.ts

@@ -796,7 +796,9 @@ export default class TimelineRepresentationIndex implements IRepresentationIndex
     if (!this._isDynamic) {
       return;
     }
-    const firstPosition = this._manifestBoundsCalculator.getEstimatedMinimumSegmentTime();
+    const firstPosition = this._manifestBoundsCalculator.getEstimatedMinimumSegmentTime(
+      (this._index.timeline[0]?.duration ?? 0) / this._index.timescale,
+    );
     if (isNullOrUndefined(firstPosition)) {
       return; // we don't know yet
     }

src/parsers/manifest/dash/common/manifest_bounds_calculator.ts

@@ -109,10 +109,14 @@ export default class ManifestBoundsCalculator {
   /**
    * Estimate a minimum bound for the content from the last set segment time
    * and buffer depth.
-   * Consider that it is only an estimation, not the real value.
+   * Consider that it is only an estimate, not the real value.
+   * @param {number} segmentDuration - In DASH, the buffer depth actually also
+   * depend on a corresponding's segment duration (e.g. a segment become
+   * unavailable once the `timeShiftBufferDepth` + its duration has elapsed).
+   * This argument can thus be set the approximate duration of a segment.
    * @return {number|undefined}
    */
-  getEstimatedMinimumSegmentTime(): number | undefined {
+  getEstimatedMinimumSegmentTime(segmentDuration: number): number | undefined {
     if (!this._isDynamic || this._timeShiftBufferDepth === null) {
       return 0;
     }
@@ -121,7 +125,7 @@ export default class ManifestBoundsCalculator {
     if (maximumBound === undefined) {
       return undefined;
     }
-    const minimumBound = maximumBound - this._timeShiftBufferDepth;
+    const minimumBound = maximumBound - (this._timeShiftBufferDepth + segmentDuration);
     return minimumBound;
   }
 

src/parsers/manifest/dash/common/parse_mpd.ts

@@ -258,6 +258,7 @@ function parseCompleteIntermediateRepresentation(
     args.referenceDateTime,
   );
   const timeShiftBufferDepth = rootAttributes.timeShiftBufferDepth;
+  const maxSegmentDuration = rootAttributes.maxSegmentDuration;
   const { externalClockOffset: clockOffset, unsafelyBaseOnPreviousManifest } = args;
 
   const { externalClockOffset } = args;
@@ -279,7 +280,6 @@ function parseCompleteIntermediateRepresentation(
     manifestBoundsCalculator,
     manifestProfiles: mpdIR.attributes.profiles,
     receivedTime: args.manifestReceivedTime,
-    timeShiftBufferDepth,
     unsafelyBaseOnPreviousManifest,
     xlinkInfos,
     xmlNamespaces: mpdIR.attributes.namespaces,
@@ -381,6 +381,20 @@ function parseCompleteIntermediateRepresentation(
     // can go even lower in terms of depth
     minimumTime = minimumSafePosition;
     timeshiftDepth = timeShiftBufferDepth ?? null;
+    if (timeshiftDepth !== null) {
+      // The DASH spec implies that a segment is still available after a given
+      // `timeShiftBufferDepth` for a time equal to its duration
+      // (What I interpret from "ISO/IEC 23009-1 fifth edition 2022-08
+      // A.3.4 Media Segment list restrictions).
+      //
+      // This `timeshiftDepth` property is global for the whole Manifest (and
+      // not per segment), thus we cannot do exactly that, but we can take the
+      // anounced `maxSegmentDuration` by default instead. This may be a little
+      // too optimistic, but would in reality not lead to a lot of issues as
+      // this `timeshiftDepth` property is not the one that should be relied on
+      // to know which segment can or cannot be requested anymore.
+      timeshiftDepth += maxSegmentDuration ?? 0;
+    }
     if (
       timeshiftDepth !== null &&
       minimumTime !== undefined &&

tests/integration/scenarios/dash_live.test.js

@@ -250,8 +250,8 @@ describe("DASH live content (SegmentTimeline)", function () {
       });
 
       expect(manifestLoaderCalledTimes).to.equal(1);
-      await checkAfterSleepWithBackoff(null, () => {
-        expect(player.getMinimumPosition()).to.be.closeTo(1527507768, 1);
+      await checkAfterSleepWithBackoff({ maxTimeMs: 2000 }, () => {
+        expect(player.getMinimumPosition()).to.be.closeTo(1527507763, 1);
       });
     });
   });

tests/integration/scenarios/dash_live_multi_periods.test.js

@@ -57,7 +57,7 @@ describe("DASH live content multi-periods (SegmentTemplate)", function () {
     const maxPos = player.getMaximumPosition();
     expect(maxPos).to.be.closeTo(now, 2);
     const minPos = player.getMinimumPosition();
-    expect(minPos).to.be.closeTo(now - manifestInfos.tsbd, 2);
+    expect(minPos).to.be.closeTo(now - manifestInfos.tsbd - 2, 2);
     expect(manifestLoaderCalledTimes).to.equal(1);
   });
 });

tests/integration/scenarios/dash_live_utc_timings.test.js

@@ -31,7 +31,7 @@ describe("DASH live - UTCTimings", () => {
         { maxTimeMs: 1000 },
         {
           resolveWhen() {
-            expect(player.getMinimumPosition()).to.be.closeTo(1553521448, 3);
+            expect(player.getMinimumPosition()).to.be.closeTo(1553521446, 3);
             expect(player.getMaximumPosition()).to.be.closeTo(1553521748, 3);
           },
           untilSuccess() {
@@ -56,7 +56,7 @@ describe("DASH live - UTCTimings", () => {
         { maxTimeMs: 1000 },
         {
           resolveWhen() {
-            expect(player.getMinimumPosition()).to.be.closeTo(1553521748, 1);
+            expect(player.getMinimumPosition()).to.be.closeTo(1553521746, 1);
             expect(player.getMaximumPosition()).to.be.closeTo(1553522048, 1);
           },
           untilSuccess() {
@@ -95,7 +95,7 @@ describe("DASH live - UTCTimings", () => {
         { maxTimeMs: 1000 },
         {
           resolveWhen() {
-            expect(player.getMinimumPosition()).to.be.closeTo(1558791848, 3);
+            expect(player.getMinimumPosition()).to.be.closeTo(1558791846, 3);
           },
           untilSuccess() {
             expect(player.getMinimumPosition()).to.equal(null);
@@ -119,7 +119,7 @@ describe("DASH live - UTCTimings", () => {
         { maxTimeMs: 1000 },
         {
           resolveWhen() {
-            expect(player.getMinimumPosition()).to.be.closeTo(1553521748, 1);
+            expect(player.getMinimumPosition()).to.be.closeTo(1553521746, 1);
             expect(player.getMaximumPosition()).to.be.closeTo(1553522048, 1);
           },
           untilSuccess() {
@@ -158,7 +158,7 @@ describe("DASH live - UTCTimings", () => {
               Date.now() / 1000 - manifestInfos.availabilityStartTime;
             const minimumPosition = maximumPosition - timeShiftBufferDepth;
 
-            expect(player.getMinimumPosition()).to.be.closeTo(minimumPosition, 3);
+            expect(player.getMinimumPosition()).to.be.closeTo(minimumPosition - 2, 3);
             expect(player.getMaximumPosition()).to.be.closeTo(maximumPosition, 3);
           },
           untilSuccess() {
@@ -184,7 +184,7 @@ describe("DASH live - UTCTimings", () => {
         { maxTimeMs: 1000 },
         {
           resolveWhen() {
-            expect(player.getMinimumPosition()).to.be.closeTo(1553521748, 1);
+            expect(player.getMinimumPosition()).to.be.closeTo(1553521746, 1);
             expect(player.getMaximumPosition()).to.be.closeTo(1553522048, 1);
           },
           untilSuccess() {
@@ -218,7 +218,7 @@ describe("DASH live - UTCTimings", () => {
         { maxTimeMs: 1000 },
         {
           resolveWhen() {
-            expect(player.getMinimumPosition()).to.be.closeTo(1553521448, 3);
+            expect(player.getMinimumPosition()).to.be.closeTo(1553521446, 3);
           },
           untilSuccess() {
             expect(player.getMinimumPosition()).to.equal(null);
@@ -242,7 +242,7 @@ describe("DASH live - UTCTimings", () => {
         { maxTimeMs: 1000 },
         {
           resolveWhen() {
-            expect(player.getMinimumPosition()).to.be.closeTo(1553521748, 1);
+            expect(player.getMinimumPosition()).to.be.closeTo(1553521746, 1);
             expect(player.getMaximumPosition()).to.be.closeTo(1553522048, 1);
           },
           untilSuccess() {