NvFBC patching fails on 555 driver series (linux)

[Sid127]

Installed the AUR package nvlax-git on Arch Linux and had this assert pop.

(2/2) Applying nvlax
[+] libnvidia-encode.so
[+] patched successfully
Assertion '(!((ZydisDecoderDecodeFull(&decoder, v_backtrack_bytes.data(), v_backtrack_bytes.size(), &instr, operands)) & 0x80000000u))' failed (ERROR)
  in file lax_fbc.cc, line 85
  function: int main(int, char**)
terminate called after throwing an instance of 'ppk::assert::AssertionException'
  what():  
error: command terminated by signal 6: Aborted

[UpYoursMicrosoft]

Same issue here. I use the nvlax-git AUR package to enable NvFBC capture in Sunshine. It falls back to capturing through X11 but that was causing screen tearing in the video stream when the system was under load. The screen tearing wasn't happening on the physical display.

For anyone else who finds this issue and has a similar setup, I fixed it by temporarily downgrading lib32-nvidia-utils, libxnvctrl, nvidia, nvidia-settings, nvidia-utils, linux (since I'm not using nvidia-dkms), and linux-headers (since I need a different DKMS module). The most recent package versions in my cache that worked for me are below. You should be able to skip downgrading the kernel if you use nvidia-dkms but YMMV.

lib32-nvidia-utils 550.90.07-1
libxnvctrl 550.78-1
nvidia 550.90.07-4
nvidia-settings 550.78-1
nvidia-utils 550.90.07-3
linux 6.9.6.arch1-1
linux-headers 6.9.6.arch1-1

[UpYoursMicrosoft]

For the record, I switched to nvidia-patch which does the job. Nvlax was great because I forgot it was even there. That's the best kind of software.