mediafoundation: Introduce Media Foundation-based encoding support on WoA #11993

thirumalai-qcom · 2025-03-24T05:36:08Z

Description

This PR introduces the media foundation plugin to OBS Studio, enabling Media Foundation-based encoding for H264, HEVC and AV1. This plugin supports hardware-accelerated encoding on Windows on ARM (WoA) devices, specifically those utilizing Qualcomm hardware encoders. The Key changes include:

mediafoundation: Add base files for MFT encoders support on WoA
mediafoundation: Add MFT H264 encoder for WoA
mediafoundation: Add MFT HEVC encoder for WoA
mediafoundation: Add MFT AV1 encoder for WoA
cmake: Add mediafoundation plugin to OBS windows build

Motivation and Context

The primary motivation for these changes is to enable hardware-accelerated encoding on Windows on ARM (WoA) devices by reintroducing the win-mf plugin in OBS Studio. With the growing adoption of WoA devices, ensuring OBS Studio can leverage hardware encoders on this platform improves performance and efficiency, reducing CPU load while maintaining high-quality encoding.

By integrating the mediafoundation plugin into the build system, we enable native support for Media Foundation-based encoding on WoA, ensuring users on this platform can leverage hardware acceleration for streaming and recording. This change ensures better utilization of Qualcomm hardware encoders, providing a more efficient and optimized encoding experience.

How Has This Been Tested?

The changes have been tested by building OBS Studio on a Windows on ARM (WoA) device with Qualcomm hardware. The build completed successfully, and basic functionality tests confirmed that OBS Studio runs as expected with the media foundation plugin. Media Foundation-based encoding for H264, HEVC and AV1 was verified to function correctly, ensuring proper hardware acceleration on WoA devices utilizing Qualcomm hardware.

Types of changes

New feature (non-breaking change which adds functionality)

Checklist:

My code has been run through clang-format.
I have read the contributing document.
My code is not on the master branch.
The code has been tested.
All commit messages are properly formatted and commits squashed where appropriate.
I have included updates to all appropriate documentation.

PatTheMav

I'm wary about just reintroducing the old code as-is, particularly as we have progressed to C++17 and expect modern C++ coding practices, including but not limited to:

Use of anonymous namespaces instead of static functions is deprecated in C++ (static functions are only used for class methods in modern C++).
Use of constexpr as it's preferable over macros in almost all situations as they enable the compiler to optimise code better and sometimes resolve entire code paths at compile time
Abuse of macros instead of functions marked as constexpr (see above) which makes the code harder to debug

Just merging this would mean re-importing the code smell as well and I just don't see someone else volunteering to "refactor" it after it's been merged again.

plugins/win-mf/CMakeLists.txt

plugins/CMakeLists.txt

plugins/win-mf/data/locale/en-US.ini

derrod · 2025-03-25T15:28:14Z

I tested this on my Snapdragon X laptop and it does work as expected.

But Ideally this plugin should be either rewritten from the ground up, or replaced wholesale with just using the FFmpeg implemenation. At the very least, the AAC encoder could probably be removed. Of course since this is ancient it also doesn't support AV1 (which the X Elite chips do support).

Aside from what Pat mentioned, the naming also conflicts with #10471 right now.

plugins/win-mf/mf-aac.cpp

dd-han · 2025-04-13T16:22:39Z

Will this work with my older Surface Pro X (SQ2)?

If it works I can help test it.

thirumalai-qcom · 2025-05-20T12:48:47Z

Will this work with my older Surface Pro X (SQ2)?
If it works I can help test it.

@dd-han Thanks for offering to help.
I believe it should work with your Surface Pro X [SQ2] which is based on the Snapdragon 8cx, If you get a change to try it out, please let me know how it goes.

SA-Root · 2025-05-31T13:27:03Z

Hi, I tried using version 31.1.0 beta 1 on my pc with 8cx Gen 3, and I got 2 problems with MF based encoding:

When using h264_mf encoder, it seems to be using CPU to encode the video.
When I tried to use hevc_mf encoder, OBS immediately crashed when I click the Start Recording button. The corresponding log files are attached below.

2025-05-31 21-12-18.txt
Crash 2025-05-31 21-14-41.txt

RytoEX · 2025-05-31T16:02:51Z

Hi, I tried using version 31.1.0 beta 1 on my pc with 8cx Gen 3, and I got 2 problems with MF based encoding:

When using h264_mf encoder, it seems to be using CPU to encode the video.

You are using the Media Foundation encoder via FFmpeg. That is not what is in this PR, and this PR has not been merged, so it is not in OBS Studio 31.1.0 Beta 1. Please use our forums or Discord for assistance.

driver1998 · 2025-06-13T10:00:07Z

@SA-Root When using h264_mf encoder, it seems to be using CPU to encode the video.

You need to pass additional flags to make hardware encoding work with the ffmpeg backend.
See https://driver1998.github.io/posts/ffmpeg-hardware-video-encoding-on-qualcomm-woa/ (Chinese only)

PatTheMav

Left a first round of reviews, will still have to look at the HEVC and H264 encoder implementations.

Except for the missing newline issues I haven't commented on every repeat issue, so things like functions marked inline in compilation units, opaque variable names, "magic" global variables, should be fixed throughout and not just were I first encountered them in review.

plugins/win-mf/CMakeLists.txt

plugins/win-mf/mf-aac-encoder.cpp

plugins/win-mf/mf-encoder-descriptor.cpp

plugins/win-mf/mf-encoder-descriptor.hpp

plugins/win-mf/mf-hevc-encoder.cpp

plugins/win-mf/mf-hevc-encoder.hpp

derrod · 2025-07-10T19:25:27Z

Couple more notes:

You should remove the AAC encoder, it was removed due to glitches and we don't really have a reason to reintroduce it
- If you insist on keeping it, add the .codecs property so it is correctly recognised by the encoder selectors, currently it's not selectable due to missing codec information
MediaFoundation does support DirectX textures as encoder input since Windows 8.1 and that should be preferred over copying data to system RAM

plugins/win-mf/mf-encoder-descriptor.cpp

plugins/win-mf/mf-h264.cpp

PatTheMav · 2025-08-19T15:14:32Z

Couple more notes:

You should remove the AAC encoder, it was removed due to glitches and we don't really have a reason to reintroduce it

If you insist on keeping it, add the .codecs property so it is correctly recognised by the encoder selectors, currently it's not selectable due to missing codec information

MediaFoundation does support DirectX textures as encoder input since Windows 8.1 and that should be preferred over copying data to system RAM

@thirumalai-qcom did you have a chance to review these notes as well?

thirumalai-qcom · 2025-08-22T11:15:29Z

I tested this on my Snapdragon X laptop and it does work as expected.

@derrod Thanks for Confirming.

At the very least, the AAC encoder could probably be removed.

Apologies for the delayed response, there were some internal discussions on this. The AAC related changes have been dropped from this PR.

Of course since this is ancient it also doesn't support AV1 (which the X Elite chips do support).

There are also plans to add AV1 support. I'll get back on this shortly.

MediaFoundation does support DirectX textures as encoder input since Windows 8.1 and that should be preferred over copying data to system RAM

@PatTheMav @derrod
Got this preference confirmed from @Fenrirthviti , will keep you posted on this. 👍

PatTheMav · 2025-09-04T17:33:20Z

I tested this on my Snapdragon X laptop and it does work as expected.

@derrod Thanks for Confirming.

At the very least, the AAC encoder could probably be removed.

Apologies for the delayed response, there were some internal discussions on this. The AAC related changes have been dropped from this PR.

Of course since this is ancient it also doesn't support AV1 (which the X Elite chips do support).

There are also plans to add AV1 support. I'll get back on this shortly.

MediaFoundation does support DirectX textures as encoder input since Windows 8.1 and that should be preferred over copying data to system RAM

@PatTheMav @derrod Got this preference confirmed from @Fenrirthviti , will keep you posted on this. 👍

Just as a heads-up, because there's another PR that introduces support for Media Foundation-based capture devices, we have a potential conflict between both using the win-mf subdirectory.

I suggested a solution to this on the other PR that I'd recommend this PR leans into as well: #10471 (comment)

thirumalai-qcom · 2025-09-08T11:54:48Z

@derrod
Of course since this is ancient it also doesn't support AV1 (which the X Elite chips do support).
There are also plans to add AV1 support. I'll get back on this shortly.

Happy to announce, AV1 support has now been successfully integrated into this PR. 🥳 The CI-generated binaries of this PR include AV1 functionality.

Just as a heads-up, because there's another PR that introduces support for Media Foundation-based capture devices, we have a potential conflict between both using the win-mf subdirectory.

I suggested a solution to this on the other PR that I'd recommend this PR leans into as well: #10471 (comment)

@PatTheMav, Thanks for the heads-up regarding the potential conflict with the win-mf subdirectory. I've reviewed your suggestion and applied the recommended changes. Please take a look and let me know if any further improvements are needed.

PatTheMav · 2025-10-07T16:07:19Z

@PatTheMav, Thanks for the heads-up regarding the potential conflict with the win-mf subdirectory. I've reviewed your suggestion and applied the recommended changes. Please take a look and let me know if any further improvements are needed.

Sorry for the delayed response - I saw that, much appreciated.

I'll have to look into both PRs a bit more closely as both initially wanted to "be" win-mf, but that's not possible as they would clobber each other. So I need to figure out what the correct way is for this scenario and come up with a naming scheme you could follow.

This commit re-introduces the base files required for Media Foundation Transform (MFT) support in the mediafoundation plugin, specifically targeting Windows on ARM devices with Qualcomm hardware. It includes core implementation files, language configuration file for localization, and the initial CMakeLists.txt for integration.

This commit introduces H264 encoder support using the Media Foundation Transform (MFT) interface, specifically targeting Windows on ARM (WoA) devices with Qualcomm hardware. It adds mf-h264.cpp and mf-h264-encoder.cpp/.hpp to implement the encoding logic.

This commit introduces HEVC encoder support using the Media Foundation Transform (MFT) interface, specifically targeting Windows on ARM (WoA) devices with Qualcomm hardware. It adds mf-hevc.cpp and mf-hevc-encoder.cpp/.hpp to implement the encoding logic.

This commit introduces AV1 encoder support using the Media Foundation Transform (MFT) interface, specifically targeting Windows on ARM (WoA) devices with Qualcomm hardware. It adds mf-av1.cpp and mf-av1-encoder.cpp/.hpp to implement the encoding logic.

This commit updates the CMakeLists.txt file in the plugins directory to incorporate the mediafoundation plugin into OBS build system. This ensures that Media Foundation-based encoding is available for Windows on ARM (WoA) devices, utilizing Qualcomm hardware for efficient video Encoding.

thirumalai-qcom · 2025-11-06T07:21:54Z

Hi @RytoEX & @PatTheMav
All comments from the previous review rounds have been addressed and resolved. Could you please take a moment to review and let us know if there are any additional comments or if we’re good to proceed with approval? Thanks in advance.

PatTheMav

Looks fine apart from the few things I found. This will probably require a small refactor to change the plugin ID and also final output directory once we have guidance, but those should be less impactful final changes.

PatTheMav · 2025-11-06T17:08:10Z

plugins/mediafoundation/encoders/CMakeLists.txt

+add_library(win-mf MODULE)
+add_library(OBS::win-mf ALIAS win-mf)


Just a small heads-up, this name will need to change (here and throughout the plugin code) as it should distinguish itself from the Media Foundation-based capture plugin. I asked for guidance what that name should be, so that'd hopefully just be a final "refactor->rename" operation.

PatTheMav · 2025-11-06T17:08:35Z

plugins/mediafoundation/encoders/CMakeLists.txt

+    mf-common.cpp
+    mf-encoder-descriptor.cpp
+    mf-h264-encoder.cpp
+    mf-h264.cpp
+    mf-hevc-encoder.cpp
+    mf-av1.cpp
+    mf-av1-encoder.cpp
+    mf-hevc.cpp
+    mf-plugin.cpp
+    mf-common.hpp
+    mf-encoder-descriptor.hpp
+    mf-h264-encoder.hpp
+    mf-hevc-encoder.hpp
+    mf-av1-encoder.hpp


These should be sorted alphabetically.

PatTheMav · 2025-11-06T17:09:08Z

plugins/mediafoundation/encoders/mf-av1-encoder.cpp

+	CHECK_HR_ERROR(o->SetUINT32(MF_MT_INTERLACE_MODE, MFVideoInterlaceMode::MFVideoInterlace_Progressive));
+	CHECK_HR_ERROR(MFSetAttributeRatio(o, MF_MT_PIXEL_ASPECT_RATIO, 1, 1));
+	CHECK_HR_ERROR(o->SetUINT32(MF_MT_VIDEO_LEVEL, (UINT32)-1));
+	//CHECK_HR_ERROR(o->SetUINT32(MF_MT_MPEG2_PROFILE, MapProfile(profile)));


Is there a good reason to retain this as a comment, or is this unused and thus can be removed?

PatTheMav · 2025-11-06T17:10:18Z

plugins/mediafoundation/encoders/mf-av1.cpp

+//#undef MFTEXT
+//#undef MFP


If those lines are not needed anymore, they should be removed.

PatTheMav · 2025-11-06T17:12:23Z

plugins/mediafoundation/encoders/mf-h264.cpp

+//#undef MFTEXT
+//#undef MFP


If those lines are not needed anymore they should be removed.

PatTheMav · 2025-11-06T17:13:43Z

plugins/mediafoundation/encoders/mf-plugin.cpp

+}
+
+OBS_DECLARE_MODULE()
+OBS_MODULE_USE_DEFAULT_LOCALE("win-mf", "en-US")


As noted above, the id will need to change. Will reply back once we have guidance for the preferred name.

WizardCM added Seeking Testers Build artifacts on CI New Feature New feature or plugin labels Mar 24, 2025

thirumalai-qcom force-pushed the mft-support branch 2 times, most recently from 55aa652 to ea69f55 Compare March 25, 2025 11:32

thirumalai-qcom marked this pull request as ready for review March 25, 2025 11:55

PatTheMav reviewed Mar 25, 2025

View reviewed changes

plugins/win-mf/CMakeLists.txt Outdated Show resolved Hide resolved

plugins/CMakeLists.txt Outdated Show resolved Hide resolved

plugins/win-mf/data/locale/en-US.ini Show resolved Hide resolved

derrod reviewed Mar 30, 2025

View reviewed changes

plugins/win-mf/mf-aac.cpp Outdated Show resolved Hide resolved

thirumalai-qcom force-pushed the mft-support branch from ea69f55 to 109de95 Compare April 3, 2025 13:35

thirumalai-qcom force-pushed the mft-support branch 4 times, most recently from 13c8aac to 2aa95e6 Compare May 20, 2025 11:10

thirumalai-qcom requested review from PatTheMav and RytoEX May 20, 2025 12:49

rbqvq mentioned this pull request Jun 2, 2025

build(windows): add arm64 initial support LizardByte/Sunshine#3905

Draft

15 tasks

thirumalai-qcom force-pushed the mft-support branch from 2aa95e6 to e066af7 Compare June 13, 2025 10:52

thirumalai-qcom force-pushed the mft-support branch from e066af7 to f7fd3aa Compare July 10, 2025 08:23

PatTheMav requested changes Jul 10, 2025

View reviewed changes

thirumalai-qcom force-pushed the mft-support branch from f7fd3aa to b521f72 Compare August 4, 2025 06:20

thirumalai-qcom changed the title ~~win-mf: Reintroduce win-mf plugin for Media Foundation-based encoding on WoA~~ WIP:: win-mf: Reintroduce win-mf plugin for Media Foundation-based encoding on WoA Aug 4, 2025

thirumalai-qcom force-pushed the mft-support branch 2 times, most recently from 88a1a72 to 8ba0168 Compare August 12, 2025 12:31

PatTheMav reviewed Aug 19, 2025

View reviewed changes

plugins/win-mf/mf-encoder-descriptor.cpp Show resolved Hide resolved

plugins/win-mf/mf-h264.cpp Outdated Show resolved Hide resolved

thirumalai-qcom force-pushed the mft-support branch 2 times, most recently from 06727f5 to 2df9b97 Compare August 22, 2025 10:48

thirumalai-qcom changed the title ~~WIP:: win-mf: Reintroduce win-mf plugin for Media Foundation-based encoding on WoA~~ win-mf: Reintroduce win-mf plugin for Media Foundation-based encoding on WoA Aug 23, 2025

thirumalai-qcom force-pushed the mft-support branch from 2df9b97 to 353400c Compare August 23, 2025 08:07

thirumalai-qcom requested review from PatTheMav and derrod August 23, 2025 08:09

thirumalai-qcom force-pushed the mft-support branch from 353400c to 4554231 Compare September 3, 2025 19:21

wang-bin mentioned this pull request Sep 4, 2025

X Elite GPU encoding is not working gyroflow/gyroflow#974

Open

4 tasks

thirumalai-qcom force-pushed the mft-support branch 2 times, most recently from c299d3b to 69c00db Compare September 4, 2025 13:15

thirumalai-qcom force-pushed the mft-support branch from 69c00db to 88a6b4c Compare September 8, 2025 11:04

thirumalai-qcom changed the title ~~win-mf: Reintroduce win-mf plugin for Media Foundation-based encoding on WoA~~ mediafoundation: Introduce Media Foundation-based encoding support on WoA Sep 8, 2025

thirumalai-qcom force-pushed the mft-support branch from 88a6b4c to d35e742 Compare September 30, 2025 14:22

thirumalai-qcom added 5 commits November 6, 2025 12:35

thirumalai-qcom force-pushed the mft-support branch from d35e742 to 19a4f9f Compare November 6, 2025 07:05

PatTheMav reviewed Nov 6, 2025

View reviewed changes

		add_library(win-mf MODULE)
		add_library(OBS::win-mf ALIAS win-mf)

		//#undef MFTEXT
		//#undef MFP

		//#undef MFTEXT
		//#undef MFP

Uh oh!

mediafoundation: Introduce Media Foundation-based encoding support on WoA #11993

Are you sure you want to change the base?

mediafoundation: Introduce Media Foundation-based encoding support on WoA #11993

Uh oh!

Conversation

thirumalai-qcom commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Motivation and Context

How Has This Been Tested?

Types of changes

Checklist:

Uh oh!

PatTheMav left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

derrod commented Mar 25, 2025

Uh oh!

Uh oh!

dd-han commented Apr 13, 2025

Uh oh!

thirumalai-qcom commented May 20, 2025

Uh oh!

SA-Root commented May 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RytoEX commented May 31, 2025

Uh oh!

driver1998 commented Jun 13, 2025

Uh oh!

PatTheMav left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

derrod commented Jul 10, 2025

Uh oh!

Uh oh!

Uh oh!

PatTheMav commented Aug 19, 2025

Uh oh!

thirumalai-qcom commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PatTheMav commented Sep 4, 2025

Uh oh!

thirumalai-qcom commented Sep 8, 2025

Uh oh!

PatTheMav commented Oct 7, 2025

Uh oh!

thirumalai-qcom commented Nov 6, 2025

Uh oh!

PatTheMav left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

PatTheMav Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

PatTheMav Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

PatTheMav Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

PatTheMav Nov 6, 2025

Choose a reason for hiding this comment

Uh oh!

thirumalai-qcom commented Mar 24, 2025 •

edited

Loading

SA-Root commented May 31, 2025 •

edited

Loading

thirumalai-qcom commented Aug 22, 2025 •

edited

Loading

PatTheMav left a comment •

edited

Loading