[hybrid] optimize pipeline performance with recompute and amp

[wangxicoding]

PR types

Performance optimization

PR changes

Others

Describe

optimize pipeline performance with recompute and amp.
Main optimization points

1. Recompute doesn't set op_device attr to backward op, will led Pipeline send/recv sub grad.
   
   After we add op_device attr to recompute backward op, Pipeline send/recv is ok.
   
2. Add concat and split into amp gray_list. Before we add, concat only run fp32.
   
   After we add concat into amp gray_list, concat can run will fp16 if inputs is fp16
   
3. When use Pipeline, if op's device is all, this op will compute in all PipelineStage. While when we use AMP, AMP will insert cast op after this device:all op, and cast op's device is not all, the Pipeline will send cast var.
   
   For optimize, we set this cast op as device:all if prev op is device:all, and Pipeline will not send this var.
   

Performance test

Test with Ernie3.0, "hidden_size": 4096, "num_attention_heads": 128, "num_hidden_layers": 76, "num_sharing_layers": 64,
16cards 32G V100，mp=8, pp=2, amp, recompute, gbs=32, micro_bs=2

develop(tokens/s)	PR(tokens/s)	improve
2164.5	2445.2	12.96%

[paddle-bot-old]

Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

[gongweibao]

介绍一下做了哪些改动？为何能提升性能？提升多少性能？

[sandyhouse]

LGTM

[JZ-LIANG]

LGTM

[zhiqiu]

LGTM for backward.py