Fix MPS

[goanpeca]

Fix MPS by detaching before moving to cpu and moving to numpy array

[Copilot]

Pull Request Overview

This PR fixes an issue with MPS (Metal Performance Shaders) compatibility by adding detach() calls before moving tensors to CPU in the FID score calculation.

Key Changes

• Added .detach() calls before .cpu() for mu1, mu2, sigma1, and sigma2 tensors to prevent gradient tracking issues when moving from GPU to CPU

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

[vfdev-5]

@goanpeca please post here the error you see without this PR to understand better why we need now to call detach

[goanpeca]

@vfdev-5 it was a suggestion from copilot on the first PR to add the detach method before the CPU.

Added .detach() calls before .cpu() for mu1, mu2, sigma1, and sigma2 tensors to prevent gradient tracking issues when
moving from GPU to CPU

Detach the tensor from the computational graph (if it requires gradients):
If the tensor was created with requires_grad=True or is part of a computation where gradients are being tracked, you need to detach it from the computational graph using the .detach() method. This prevents errors related to attempting to convert a tensor that requires gradients to a NumPy array, as NumPy arrays do not have a concept of gradients.

Is this something that could apply to mu1, mu2, sigma1, sigma2?

[goanpeca]

@vfdev-5 the error was something on

tests/ignite/metrics/gan/test_fid.py:51: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
ignite/metrics/gan/fid.py:40: in fid_score
    covmean, _ = scipy.linalg.sqrtm(sigma1.mm(sigma2).cpu(), disp=False)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

...
          if disp is False:
              try:
  >               arg2 = norm(res @ res - A, 'fro')**2 / norm(A, 'fro')
                              ^^^^^^^^^^^^^
  E               TypeError: unsupported operand type(s) for -: 'numpy.ndarray' and 'Tensor'
  
  .venv/lib/python3.11/site-packages/scipy/linalg/_matfuncs.py:547: TypeError

so that is why we had to add a .numpy() call. For the detach see previous message, it was a suggestion from copilot, but if it does not apply, I can revert it!

[vfdev-5]

Thanks Gonzalo!