Skip to content

update doc for perf_train_cpu_many, add intel mpi introduction #18576

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 12, 2022
Merged

update doc for perf_train_cpu_many, add intel mpi introduction #18576

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3eadfcc
update doc for perf_train_cpu_many, add mpi introduction
sywangyi Aug 11, 2022
20cc8e3
Update docs/source/en/perf_train_cpu_many.mdx
sywangyi Aug 12, 2022
6933951
Update docs/source/en/perf_train_cpu_many.mdx
sywangyi Aug 12, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion docs/source/en/perf_train_cpu_many.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,14 @@ pip install oneccl_bind_pt=={pytorch_version} -f https://software.intel.com/ipex
where `{pytorch_version}` should be your PyTorch version, for instance 1.12.0.
Check more approaches for [oneccl_bind_pt installation](https://github.com/intel/torch-ccl).

### Usage in Trainer
## Intel® MPI library
Use this standards-based MPI implementation to deliver flexible, efficient, scalable cluster messaging on Intel® architecture. This component is part of the Intel® oneAPI HPC Toolkit.
it could be installed from [mpi installation](https://www.intel.com/content/www/us/en/developer/articles/tool/oneapi-standalone-components.html#mpi).

The following "Usage in Trainer" takes mpirun in Intel® MPI library as an example.


## Usage in Trainer
To enable multi CPU distributed training in the Trainer with the ccl backend, users should add **`--xpu_backend ccl`** in the command arguments.

Let's see an example with the [question-answering example](https://github.com/huggingface/transformers/tree/main/examples/pytorch/question-answering)
Expand Down