Skip to content

Conversation

@Essoz
Copy link
Collaborator

@Essoz Essoz commented Feb 20, 2025

This PR resolves dispatch-related compatibility issues caused by wrapping torch objects using Proxy.

The PR resolves it by adding a __torch_function__ implementation to the Proxy definition which automatically invokes any API to be dispatched with the original objects instead of the proxied ones.

After this PR, we are able to instrument the word_language_model input in the pytorch example repo. Previously, instrumentation it leads to crashes as torch cannot find kernel implementations for the proxied tensors.

@Essoz
Copy link
Collaborator Author

Essoz commented Feb 20, 2025

Tested the whether this PR allow us to get rid of all the unproxy wrapped applied to builtin APIs to avoid passing Proxy into backend which leads to segmentation fault.

This PR allowed us to pass through the forward phase without any problems. But we crashed at loss computation at F.nll_loss in the MNIST pipeline. Problem seems to be inside the handle_torch_function inside the nll_loss implementation.

@Essoz
Copy link
Collaborator Author

Essoz commented Feb 20, 2025

did a little bit deeper search. The code crashes at l1627 in torch.overrides.py.
image

@Essoz
Copy link
Collaborator Author

Essoz commented Feb 20, 2025

Ok the problem is caused because torch_func_method is the regular pytorch Tensor __torch_function__ instead of our own unproxying torch function.

image

Note how type(args[0]) is a proxy and is being passed to C.

@Essoz
Copy link
Collaborator Author

Essoz commented Feb 20, 2025

renaming Proxy._unwrapping__torch_function__ to Proxy.__torch_function__ solved the problem:

torch sometimes rely on hasattr(cls, '__torch_function__') to determine whether torch function needs to be handled for a type and this hasattr isn't controlled by us.

@Essoz Essoz merged commit ea1a8dd into main Feb 20, 2025
2 checks passed
@Essoz Essoz mentioned this pull request Feb 20, 2025
3 tasks
@Essoz Essoz added the enhancement New feature or request label Mar 17, 2025
@Essoz Essoz deleted the proxy-torch-function branch April 30, 2025 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants