About jit of multi-host pmap not implemented #32709
Unanswered
matrix97317
asked this question in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Q:NotImplementedError: jit of multi-host pmap not implemented (and jit-of-pmap can cause extra data movement anyway, so maybe you don't want it after all).
When I don't use jit, I run into out-of-memory issues, but when I use jit and a multi-machine training environment with pmap for all-reduce, I encounter compatibility problems between jit and pmap. Are there any good solutions
Beta Was this translation helpful? Give feedback.
All reactions