Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 66 additions & 63 deletions doc/api/Miscellaneous/MultiThreading.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,13 +109,10 @@ import RxPlayer from "rx-player/minimal";
// Import the MULTI_THREAD experimental feature
import { MULTI_THREAD } from "rx-player/experimental/features";

// To simplify this example, we'll directly import "embedded" versions of the
// supplementary code used by the `MULTI_THREAD` feature.
// We could also load them on demand through URLs
import {
EMBEDDED_WORKER, // Code which will run in the "Worker"
EMBEDDED_DASH_WASM, // To be able to play DASH contents
} from "rx-player/experimental/features/embeds";
// To simplify this example, we'll directly import an "embedded" version of the
// supplementary code loaded by the `MULTI_THREAD` feature.
// We could also load it on demand through an URL
import { EMBEDDED_WORKER } from "rx-player/experimental/features/embeds";

// Add the MULTI_THREAD feature, like any other feature
RxPlayer.addFeatures([MULTI_THREAD]);
Expand All @@ -125,19 +122,14 @@ const player = new RxPlayer(/* your usual options */);

// After instantiation, you can at any time "attach" a WebWorker so any
// following `loadVideo` call can rely on it when possible.
player
.attachWorker({
workerUrl: EMBEDDED_WORKER,
dashWasmUrl: EMBEDDED_DASH_WASM,
})
.catch((err) => {
console.error("An error arised while initializing the worker", err);
// Note the if `attachWorker` rejects, the next `loadVideo` / `reload` calls
// will not rely on the "multithread" mode anymore.
//
// However the last-loaded content may fail on error if it was already
// loading in "multithread" mode.
});
player.attachWorker({ workerUrl: EMBEDDED_WORKER }).catch((err) => {
console.error("An error arised while initializing the worker", err);
// Note the if `attachWorker` rejects, the next `loadVideo` / `reload` calls
// will not rely on the "multithread" mode anymore.
//
// However the last-loaded content may fail on error if it was already
// loading in "multithread" mode.
});

// Any further call to `loadVideo` may rely on WebWorker if possible (see
// limitations below)
Expand All @@ -163,8 +155,8 @@ if (currentModeInfo === null) {
Note that the `"multithread"` mode will only run on a `loadVideo` call if all the
following conditions are respected:

- Your supported platforms both are compatible to the `WebWorker` browser feature (the
great majority are) and WebAssembly (very old platforms might not).
- Your supported platforms are compatible to the `WebWorker` browser feature (the great
majority are).

- You've added the `MULTI_THREAD` feature to the `RxPlayer` class (as shown in examples)
before that `loadVideo` call.
Expand Down Expand Up @@ -228,52 +220,59 @@ have already been played on the main thread.

## How to rely on "multithread" mode

### Step 1: Obtaining the Worker file and the WebAssembly file
### Step 1: Obtaining the Worker file

The RxPlayer will need to be able to obtain two files to be able to run in multithread
mode:
The RxPlayer will need to be able to obtain an external file to be able to run in
multithread mode, `worker.js`, which is the code running in a WebWorker concurrently to
your application.

- `worker.js`: The "worker" file, which will run in a WebWorker concurrently to your
application.
You can find it at any of the following places:

- `mpd-parser.wasm`: The DASH WebAssembly parser, today the only supported transport's
parser in a "multithread" scenario (Microsoft Smooth streaming, local contents and
Metaplaylist are not yet supported).

You can find them at any of the following places:

- The easiest way is to just import in your application the "embedded" versions of each of
them, exported through the `"rx-player/experimental/features/embeds"` path:
- The easiest way is to just import in your application its "embedded" version, exported
through the `"rx-player/experimental/features/embeds"` path:

```js
import {
EMBEDDED_WORKER, // "embedded" version of the `worker.js` file
EMBEDDED_DASH_WASM, // "embedded" version of the `mpd-parser.wasm` file
} from "rx-player/experimental/features/embeds";
import { EMBEDDED_WORKER } from "rx-player/experimental/features/embeds";
```

This allows to bypass the need to store and serve separately those two files. Note
however that including those "embeds" in your application may sensibly increase its
size.
This allows to bypass the need to store and serve separately that file.

If you would prefer more control and a smaller bundle size, you may instead consider the
other following ways to load those as separate files. This will lead to smaller file
sizes and they will only be loaded on demand, but at a maintenance cost: you'll have to
store and serve those files yourself as well as not forget to update them each time you
update the RxPlayer.
other following ways to it as a separate file.

- With every release note published on GitHub as `worker.js` (you should only use the file
linked to the RxPlayer's version you're using),

- It is also available as `dist/worker.js` from the root directory of the project
published on npm. As such, it might already be found in your project's directory, for
example in the `node_modules` directory (most probably in `node_modules/rx-player/dist/`
depending on your project).

#### Optional: Obtaining DASH WASM parser

- With every release note published on GitHub (you should only use the files linked to the
RxPlayer's version you're using), as `worker.js` and `mpd-parser.wasm` respectively.
Optionally, for very specific use-cases where you've seen improvements with it, you can
also rely on our [DASH WebAssembly parser](./DASH_WASM_Parser.md) in those "multithread"
scenarios. Note that this is unneeded for most usages.

- They are also available as respectively as `dist/worker.js` and `dist/mpd-parser.wasm`
from the root directory of the project published on npm. As such, they might already be
found in your project's directory, for example in the `node_modules` directory (most
probably in `node_modules/rx-player/dist/` depending on your project).
To do this, you have to explicitely provide the corresponding WebAssembly file. Like for
the worker file, it can either be found:

Once you've retrieved those files and unless you relied on the embedded versions, you will
need to store them and give its URL to the RxPlayer so it will be able to load them
(embedded versions may be used directly instead, like an URL, in the `attachWorker` method
shown below).
- As an "embedded" version, exported through the
`"rx-player/experimental/features/embeds"` path:

```js
import { EMBEDDED_DASH_WASM } from "rx-player/experimental/features/embeds";
```

Note however that the embedded version of this WebAssembly file is much bigger than the
original file. If it is an issue for you, you might want to store and serve the
WebAssembly file itself instead.

- With every release note published on GitHub as `mpd-parser.js` (like for `worker.js`,
only use the one linked to the RxPlayer's version you're using),

- It is also available as `dist/mpd-parser.js` from the root directory of the project
published on npm.

### Step 2: importing the `MULTI_THREAD` feature and adding it

Expand All @@ -297,23 +296,27 @@ builds to reduce bundle size when that feature isn't needed.
### Step 3: Attaching a WebWorker to your RxPlayer instance

After instantiating an `RxPlayer`, you can link it to its own `WebWorker` by calling the
`attachWorker` method on this instance and by providing both the `worker.js` and
`mpd-parser.wasm` files by providing their URL (obtained in step 1) - or the embedded
version - to it:
`attachWorker` method on this instance and by providing the `worker.js` file (and
optionally the `mpd-parser.wasm` file if you wanted WebAssembly capabilities) by providing
either its URL (obtained in step 1) or its embedded version:

```js
const player = new RxPlayer({
/* ... */
});
player
.attachWorker({
workerUrl: URL_TO_WORKER_FILE,
dashWasmUrl: URL_TO_DASH_WASM_FILE,
})
.attachWorker({ workerUrl: URL_TO_WORKER_FILE })
.then(() => console.log("Worker succesfully attached!"))
.catch((err) => {
console.error("An error arised while initializing the worker", err);
});

// NOTE: With the WebAssembly file, it would have been:
// player.attachWorker({
// workerUrl: URL_TO_WORKER_FILE,
// dashWasmUrl: URL_TO_DASH_WASM_FILE,
// })
// // ...
```

As you can see, `attachWorker` returns a Promise resolving once the Worker attachment
Expand Down
19 changes: 8 additions & 11 deletions src/core/main/worker/worker_main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import type {
IReferenceUpdateMessage,
} from "../../../multithread_types";
import { MainThreadMessageType, WorkerMessageType } from "../../../multithread_types";
import DashFastJsParser from "../../../parsers/manifest/dash/fast-js-parser";
import DashWasmParser from "../../../parsers/manifest/dash/wasm-parser";
import { ObservationPosition } from "../../../playback_observer";
import type { IWorkerPlaybackObservation } from "../../../playback_observer/worker_playback_observer";
Expand Down Expand Up @@ -73,6 +74,7 @@ export default function initializeWorkerMain() {
// TODO allow worker-side feature-switching? Not sure how
const dashWasmParser = new DashWasmParser();
features.dashParsers.wasm = dashWasmParser;
features.dashParsers.fastJs = DashFastJsParser;
features.transports.dash = createDashPipelines;

Comment on lines 75 to 79
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to prefer assigning those property over calling the exposed methods in src/features/list like addDashFeature, dashWasmFeature .. ?

Copy link
Collaborator Author

@peaBerberian peaBerberian Feb 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question and I've thought about it.

However it seems that features exported in src/features/list, because they are the exact same files imported by applications, assume a main thread environment. This is to reduce the number of features to import for an application (the end goal being a simpler API).

As such, importing src/features/lib/dash.ts also adds a MediaSourceContentInitializer to play content in main thread and a MainCodecSupportProber to perform codec checks on main thread.

Because we don't want those in a worker, in the end I kept just manually selecting the exact feature we want in worker_main.ts

/**
Expand All @@ -94,19 +96,12 @@ export default function initializeWorkerMain() {
const diffWorker = Date.now() - performance.now();
mainThreadTimestampDiff.setValueIfChanged(diffWorker - diffMain);
updateLoggerLevel(msg.value.logLevel, msg.value.sendBackLogs);
dashWasmParser.initialize({ wasmUrl: msg.value.dashWasmUrl }).then(
() => {
sendMessage({ type: WorkerMessageType.InitSuccess, value: null });
},
(err) => {
if (msg.value.dashWasmUrl !== undefined && dashWasmParser.isCompatible()) {
dashWasmParser.initialize({ wasmUrl: msg.value.dashWasmUrl }).catch((err) => {
const error = err instanceof Error ? err.toString() : "Unknown Error";
log.error("Worker: Could not initialize DASH_WASM parser", error);
sendMessage({
type: WorkerMessageType.InitError,
value: { errorMessage: error, kind: "dashWasmInitialization" },
});
},
);
});
}

if (!msg.value.hasVideo || msg.value.hasMseInWorker) {
contentPreparer.disposeCurrentContent();
Expand All @@ -119,6 +114,8 @@ export default function initializeWorkerMain() {
features.codecSupportProber = msg.value.hasMseInWorker
? MainCodecSupportProber
: WorkerCodecSupportProber;

sendMessage({ type: WorkerMessageType.InitSuccess, value: null });
break;

case MainThreadMessageType.LogLevelUpdate:
Expand Down
2 changes: 1 addition & 1 deletion src/features/features_object.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import type { IFeaturesObject } from "./types";
* @type {Object}
*/
const features: IFeaturesObject = {
dashParsers: { wasm: null, js: null },
dashParsers: { wasm: null, native: null, fastJs: null },
codecSupportProber: null,
createDebugElement: null,
directfile: null,
Expand Down
6 changes: 3 additions & 3 deletions src/features/list/__tests__/dash.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

import MediaSourceContentInitializer from "../../../main_thread/init/media_source_content_initializer";
import mainCodecSupportProber from "../../../mse/main_codec_support_prober";
import dashJsParser from "../../../parsers/manifest/dash/js-parser";
import nativeDashParser from "../../../parsers/manifest/dash/native-parser";
import DASHFeature from "../../../transports/dash";
import type { IFeaturesObject } from "../../types";
import addDASHFeature from "../dash";
Expand All @@ -25,13 +25,13 @@ describe("Features list - DASH", () => {
it("should add DASH in the current features", () => {
const featureObject = {
transports: {},
dashParsers: { js: null, wasm: null },
dashParsers: { fastJs: null, native: null, wasm: null },
mainThreadMediaSourceInit: null,
} as unknown as IFeaturesObject;
addDASHFeature(featureObject);
expect(featureObject).toEqual({
transports: { dash: DASHFeature },
dashParsers: { js: dashJsParser, wasm: null },
dashParsers: { native: nativeDashParser, fastJs: null, wasm: null },
mainThreadMediaSourceInit: MediaSourceContentInitializer,
codecSupportProber: mainCodecSupportProber,
});
Expand Down
5 changes: 3 additions & 2 deletions src/features/list/__tests__/dash_wasm.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,12 @@ describe("Features list - DASH WASM Parser", () => {

const featureObject = {
transports: {},
dashParsers: { js: null, wasm: null },
dashParsers: { native: null, fastJs: null, wasm: null },
} as unknown as IFeaturesObject;
DASH_WASM._addFeature(featureObject);
expect(featureObject.transports).toEqual({ dash: DASHFeature });
expect(featureObject.dashParsers.js).toEqual(null);
expect(featureObject.dashParsers.native).toEqual(null);
expect(featureObject.dashParsers.fastJs).toEqual(null);
expect(featureObject.dashParsers.wasm).toBeInstanceOf(DashWasmParser);
});
});
4 changes: 2 additions & 2 deletions src/features/list/dash.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@

import MediaSourceContentInitializer from "../../main_thread/init/media_source_content_initializer";
import mainCodecSupportProber from "../../mse/main_codec_support_prober";
import dashJsParser from "../../parsers/manifest/dash/js-parser";
import dashJsParser from "../../parsers/manifest/dash/native-parser";
import dash from "../../transports/dash";
import type { IFeaturesObject } from "../types";

Expand All @@ -28,7 +28,7 @@ function addDASHFeature(features: IFeaturesObject): void {
if (features.transports.dash === undefined) {
features.transports.dash = dash;
}
features.dashParsers.js = dashJsParser;
features.dashParsers.native = dashJsParser;
features.mainThreadMediaSourceInit = MediaSourceContentInitializer;
features.codecSupportProber = mainCodecSupportProber;
}
Expand Down
17 changes: 13 additions & 4 deletions src/features/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -61,8 +61,13 @@ export type IHTMLTextTracksBuffer = new (
*/
export type INativeTextTracksBuffer = new (mediaElement: HTMLMediaElement) => SegmentSink;

export type IDashJsParser = (
document: Document,
export type IDashNativeParser = (
dom: Document,
args: IMPDParserArguments,
) => IDashParserResponse<string>;

export type IDashFastJsParser = (
xml: string,
args: IMPDParserArguments,
) => IDashParserResponse<string>;

Expand Down Expand Up @@ -143,9 +148,13 @@ export interface IFeaturesObject {
*/
wasm: DashWasmParser | null;
/**
* JavaScript-based Manifest DASH parser.
* Entirely JavaScript-based Manifest DASH parser.
*/
fastJs: IDashFastJsParser | null;
/**
* JavaScript+Browser's DOMParser-based Manifest DASH parser.
*/
js: IDashJsParser | null;
native: IDashNativeParser | null;
};
/** Implement text track rendering through `<track>` HTML elements. */
nativeTextDisplayer: typeof NativeTextDisplayer | null;
Expand Down
7 changes: 0 additions & 7 deletions src/main_thread/api/public_api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ import {
} from "../../compat/event_listeners";
import getStartDate from "../../compat/get_start_date";
import hasMseInWorker from "../../compat/has_mse_in_worker";
import hasWebassembly from "../../compat/has_webassembly";
import isDebugModeEnabled from "../../compat/is_debug_mode_enabled";
import type {
IAdaptationChoice,
Expand Down Expand Up @@ -427,12 +426,6 @@ class Player extends EventEmitter<IPublicAPIEvent> {
new WorkerInitializationError("INCOMPATIBLE_ERROR", "Worker unavailable"),
);
}
if (!hasWebassembly) {
log.warn("API: Cannot rely on a WebWorker: WebAssembly unavailable");
return rej(
new WorkerInitializationError("INCOMPATIBLE_ERROR", "WebAssembly unavailable"),
);
}
if (typeof workerSettings.workerUrl === "string") {
this._priv_worker = new Worker(workerSettings.workerUrl);
} else {
Expand Down
2 changes: 1 addition & 1 deletion src/multithread_types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ export interface IInitMessage {
type: MainThreadMessageType.Init;
value: {
/** Link to the DASH_WASM's feature WebAssembly file to parse DASH MPDs. */
dashWasmUrl: string;
dashWasmUrl: string | undefined;
/**
* If `true` the final element on the current page displaying the content
* can display video content.
Expand Down
Loading