NCCL_LAUNCH_MODE=PARALLEL 环境变量打开，多卡下容易hang住

[ccmeteorljh]

paddle version:1.3.0

+-------------------------------+----------------------+----------------------+
|   4  Tesla P40           Off  | 00000000:49:00.0 Off |                    0 |
| N/A   41C    P0    57W / 250W |  21729MiB / 22912MiB |    100%      Default |
+-------------------------------+----------------------+----------------------+
|   5  Tesla P40           Off  | 00000000:4A:00.0 Off |                    0 |
| N/A   38C    P0    57W / 250W |  21729MiB / 22912MiB |    100%      Default |
+-------------------------------+----------------------+----------------------+
|   6  Tesla P40           Off  | 00000000:4B:00.0 Off |                    0 |
| N/A   38C    P0    58W / 250W |  21729MiB / 22912MiB |    100%      Default |
+-------------------------------+----------------------+----------------------+
|   7  Tesla P40           Off  | 00000000:4C:00.0 Off |                    0 |
| N/A   38C    P0    51W / 250W |  21729MiB / 22912MiB |      0%      Default

在transformer，image_classification, deeplabv3+ 都会出现；

[Xreki]

您好，请问使用几张卡？大概执行多久之后hang住？可以看出hang在什么地方吗？

[gavin1332]

https://docs.nvidia.com/deeplearning/sdk/nccl-developer-guide/docs/env.html#nccl-launch-mode

[paddle-bot-old]

Since you haven't replied for more than a year, we have closed this issue/pr.
If the problem is not solved or there is a follow-up one, please reopen it at any time and we will continue to follow up.
由于您超过一年未回复，我们将关闭这个issue/pr。
若问题未解决或有后续问题，请随时重新打开，我们会继续跟进。