{lib}[SYSTEM] ffnvcodec v11.1.5.2 + add it as build dependency in FFmpeg 5.0.1 #17271

Micket · 2023-02-07T13:39:23Z

(created using eb --new-pr)

I also need to test this as an optional dependency to ffmpeg.

Micket · 2023-02-07T14:01:22Z

I wasn't sure if to call this ffnvcodec or nv-codec-headers.

Micket · 2023-02-07T14:11:44Z

Test report by @Micket
SUCCESS
Build succeeded for 1 out of 1 (1 easyconfigs in total)
vera-c2 - Linux Rocky Linux 8.6, x86_64, Intel Xeon Processor (Skylake), Python 3.6.8
See https://gist.github.com/37cac73034b94eceda85511bece60c3d for a full test report.

Micket · 2023-02-08T15:54:19Z

So, ffnvcodec are just headers, and ffmpeg should do runtime checks. I think this will just build FFmpeg with support for nvenc, similar to how we do with OpenMPI's cuda support.
There is basically minimal impact (except for a few seconds of build time?) for those who don't have nvidia GPUs.

It doesn't seem to care about CUDA runtime either, the actual codec libraries are part of the drivers. So, that's great. No need to make CUDA suffix variants of FFmpeg for this feature i think.

But it needs to be tested to ensure it has no negative impact on systems without nvidia gpus (or drivers)

Micket · 2023-02-08T15:57:03Z

Test report by @Micket
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
vera-c2 - Linux Rocky Linux 8.6, x86_64, Intel Xeon Processor (Skylake), Python 3.6.8
See https://gist.github.com/bdfd05ed7ec3c622c01a9862bb78166c for a full test report.

zao · 2023-02-08T17:58:20Z

Test report by @zao
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
eb-rocky8 - Linux Rocky Linux 8.6, x86_64, AMD Ryzen 9 3900X 12-Core Processor, Python 3.9.7
See https://gist.github.com/e29910a21cd90a279645bc9d7a9aae97 for a full test report.

Micket · 2023-02-08T22:02:36Z

Test report by @Micket
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
vera-c2 - Linux Rocky Linux 8.6, x86_64, Intel Xeon Processor (Skylake), Python 3.6.8
See https://gist.github.com/dc59ef0f865da9df3c6f6b8341351199 for a full test report.

Micket · 2023-02-08T23:43:25Z

Tested to confirm that it builds fine even if you lack cuda and gpu (and such a build will work at runtime from a machine with nvidia gpu). We should be able to apply this to all FFmpegs going forward.

I'm not sure how to interpret the different versions of ffnvcodec though, or what the compatibility with nvidia drivers and/or FFmpeg is.

Micket · 2023-02-09T20:42:23Z

@boegelbot please test @ generoso

boegelbot · 2023-02-09T20:45:09Z

@Micket: Request for testing this PR well received on login1

PR test command 'EB_PR=17271 EB_ARGS= EB_CONTAINER= /opt/software/slurm/bin/sbatch --job-name test_PR_17271 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 10136

Test results coming soon (I hope)...

- notification for comment with ID 1424801064 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegel

lgtm

boegel · 2023-02-13T09:46:18Z

@boegelbot please test @ generoso

boegelbot · 2023-02-13T09:50:10Z

@boegel: Request for testing this PR well received on login1

PR test command 'EB_PR=17271 EB_ARGS= EB_CONTAINER= /opt/software/slurm/bin/sbatch --job-name test_PR_17271 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 10214

Test results coming soon (I hope)...

- notification for comment with ID 1427634359 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2023-02-13T09:55:33Z

Test report by @boegelbot
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
cns1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/4f93a0b32e037dd7fc40824da61f1d35 for a full test report.

boegel · 2023-02-13T10:07:01Z

@boegelbot please test @ jsc-zen2

boegelbot · 2023-02-13T10:07:12Z

@boegel: Request for testing this PR well received on jsczen2l1.int.jsc-zen2.easybuild-test.cluster

PR test command 'EB_PR=17271 EB_ARGS= /opt/software/slurm/bin/sbatch --mem-per-cpu=4000M --job-name test_PR_17271 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen2.sh' executed!

exit code: 0
output:

Submitted batch job 2180

Test results coming soon (I hope)...

- notification for comment with ID 1427663971 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2023-02-13T10:10:08Z

Test report by @boegelbot
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
jsczen2c1.int.jsc-zen2.easybuild-test.cluster - Linux Rocky Linux 8.5, x86_64, AMD EPYC 7742 64-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/73def7c62ee7ee7d82afa9da1284af8b for a full test report.

boegel · 2023-02-13T10:11:20Z

Test report by @boegel
SUCCESS
Build succeeded for 2 out of 2 (2 easyconfigs in total)
node3158.skitty.os - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/924b7608faf1ad3eb3627db3479c8239 for a full test report.

boegel · 2023-02-13T10:11:33Z

Going in, thanks @Micket!

adding easyconfigs: nv-codec-headers-11.1.5.2-GCCcore-11.3.0.eb

cebbba6

Micket added the new label Feb 7, 2023

Micket marked this pull request as draft February 7, 2023 14:01

Rename to ffnvcodec

aa95f18

Micket changed the title ~~{lib}[GCCcore/11.3.0] nv-codec-headers v11.1.5.2~~ {lib}[GCCcore/11.3.0] ffnvcodec v11.1.5.2 Feb 7, 2023

Micket marked this pull request as ready for review February 8, 2023 09:51

Micket changed the title ~~{lib}[GCCcore/11.3.0] ffnvcodec v11.1.5.2~~ {lib}[GCCcore/11.3.0] ffnvcodec v11.1.5.2 + used in FFmpeg 5.0.1 Feb 8, 2023

Add dependency to FFmpeg

bffb5c6

Drop ffnvcodec to SYSTEM

82261a1

Micket changed the title ~~{lib}[GCCcore/11.3.0] ffnvcodec v11.1.5.2 + used in FFmpeg 5.0.1~~ {lib}[SYSTEM] ffnvcodec v11.1.5.2 + used in FFmpeg 5.0.1 Feb 8, 2023

Remove leftover cruft from ffnvcodec

cc569cc

boegel changed the title ~~{lib}[SYSTEM] ffnvcodec v11.1.5.2 + used in FFmpeg 5.0.1~~ {lib}[SYSTEM] ffnvcodec v11.1.5.2 + add it as dependency in FFmpeg 5.0.1 Feb 13, 2023

boegel approved these changes Feb 13, 2023

View reviewed changes

boegel added the enhancement label Feb 13, 2023

boegel added this to the next release (4.7.1?) milestone Feb 13, 2023

boegel merged commit b6c3c6d into easybuilders:develop Feb 13, 2023

boegel changed the title ~~{lib}[SYSTEM] ffnvcodec v11.1.5.2 + add it as dependency in FFmpeg 5.0.1~~ {lib}[SYSTEM] ffnvcodec v11.1.5.2 + add it as build dependency in FFmpeg 5.0.1 Feb 15, 2023

Micket deleted the 20230207133918_new_pr_nv-codec-headers11152 branch April 21, 2023 09:18

{lib}[SYSTEM] ffnvcodec v11.1.5.2 + add it as build dependency in FFmpeg 5.0.1 #17271

{lib}[SYSTEM] ffnvcodec v11.1.5.2 + add it as build dependency in FFmpeg 5.0.1 #17271

Uh oh!

Conversation

Micket commented Feb 7, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Micket commented Feb 7, 2023

Uh oh!

Micket commented Feb 7, 2023

Uh oh!

Micket commented Feb 8, 2023

Uh oh!

Micket commented Feb 8, 2023

Uh oh!

zao commented Feb 8, 2023

Uh oh!

Micket commented Feb 8, 2023

Uh oh!

Micket commented Feb 8, 2023

Uh oh!

Micket commented Feb 9, 2023

Uh oh!

boegelbot commented Feb 9, 2023

Uh oh!

boegel left a comment

Choose a reason for hiding this comment

Uh oh!

boegel commented Feb 13, 2023

Uh oh!

boegelbot commented Feb 13, 2023

Uh oh!

boegelbot commented Feb 13, 2023

Uh oh!

boegel commented Feb 13, 2023

Uh oh!

boegelbot commented Feb 13, 2023

Uh oh!

boegelbot commented Feb 13, 2023

Uh oh!

boegel commented Feb 13, 2023

Uh oh!

boegel commented Feb 13, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Micket commented Feb 7, 2023 •

edited

Loading