diff --git a/README.md b/README.md index d9ef44fa2b5697..6b3f3ef86fe1bc 100644 --- a/README.md +++ b/README.md @@ -21,7 +21,7 @@ PaddlePaddle is originated from industrial practices with dedication and commitm ## Installation -### Latest PaddlePaddle Release: [v2.0](https://github.com/PaddlePaddle/Paddle/tree/release/2.0) +### Latest PaddlePaddle Release: [v2.1](https://github.com/PaddlePaddle/Paddle/tree/release/2.1) Our vision is to enable deep learning for everyone via PaddlePaddle. Please refer to our [release announcement](https://github.com/PaddlePaddle/Paddle/releases) to track the latest features of PaddlePaddle. @@ -33,9 +33,9 @@ pip install paddlepaddle pip install paddlepaddle-gpu ``` -More infomation about installation, please view [Quick Install](https://www.paddlepaddle.org.cn/install/quick) +For more information about installation, please view [Quick Install](https://www.paddlepaddle.org.cn/install/quick) -Now our developers can acquire Tesla V100 online computing resources for free. If you create a program by AI Studio, you will obtain 10 hours to train models online per day. [Click here to start](https://aistudio.baidu.com/aistudio/index). +Now our developers can acquire Tesla V100 online computing resources for free. If you create a program by AI Studio, you will obtain 8 hours to train models online per day. [Click here to start](https://aistudio.baidu.com/aistudio/index). ## FOUR LEADING TECHNOLOGIES @@ -46,13 +46,13 @@ Now our developers can acquire Tesla V100 online computing resources for free. I - **Support Ultra-Large-Scale Training of Deep Neural Networks** - PaddlePaddle has made breakthroughs in ultra-large-scale deep neural networks training. It launched the world's first large-scale open-source training platform that supports the training of deep networks with 100 billions of features and trillions of parameters using data sources distributed over hundreds of nodes. PaddlePaddle overcomes the online deep learning challenges for ultra-large-scale deep learning models, and further achieved the real-time model updating with more than 1 trillion parameters. + PaddlePaddle has made breakthroughs in ultra-large-scale deep neural networks training. It launched the world's first large-scale open-source training platform that supports the training of deep networks with 100 billion features and trillions of parameters using data sources distributed over hundreds of nodes. PaddlePaddle overcomes the online deep learning challenges for ultra-large-scale deep learning models, and further achieved real-time model updating with more than 1 trillion parameters. [Click here to learn more](https://github.com/PaddlePaddle/Fleet) - **High-Performance Inference Engines for Comprehensive Deployment Enviroments** - PaddlePaddle is not only compatible with models trained in 3rd party open-source frameworks , but also offers complete inference products for various production scenarios. Our inference product line includes [Paddle Inference](https://paddle-inference.readthedocs.io/en/latest/product_introduction/summary.html): Native inference library for high performance server and cloud inference; [Paddle Serving](https://github.com/PaddlePaddle/Serving): A service-oriented framework suitable for distributed and pipeline productions; [Paddle Lite](https://github.com/PaddlePaddle/Paddle-Lite): Ultra-Lightweight inference engine for mobile and IoT enviroments; [Paddle.js](https://www.paddlepaddle.org.cn/paddle/paddlejs): A frontend inference engine for browser and mini apps. Futhermore, by great amounts of optimization with leading hardwares in each scenarios, Paddle inference engines outperform most of the other mainstream frameworks. + PaddlePaddle is not only compatible with models trained in 3rd party open-source frameworks , but also offers complete inference products for various production scenarios. Our inference product line includes [Paddle Inference](https://paddle-inference.readthedocs.io/en/latest/product_introduction/summary.html): Native inference library for high-performance server and cloud inference; [Paddle Serving](https://github.com/PaddlePaddle/Serving): A service-oriented framework suitable for distributed and pipeline productions; [Paddle Lite](https://github.com/PaddlePaddle/Paddle-Lite): Ultra-Lightweight inference engine for mobile and IoT environments; [Paddle.js](https://www.paddlepaddle.org.cn/paddle/paddlejs): A frontend inference engine for browser and mini-apps. Furthermore, by great amounts of optimization with leading hardware in each scenario, Paddle inference engines outperform most of the other mainstream frameworks. - **Industry-Oriented Models and Libraries with Open Source Repositories** @@ -87,6 +87,11 @@ We provide [English](https://www.paddlepaddle.org.cn/documentation/docs/en/guide - [Github Issues](https://github.com/PaddlePaddle/Paddle/issues): bug reports, feature requests, install issues, usage issues, etc. - QQ discussion group: 793866180 (PaddlePaddle). - [Forums](https://ai.baidu.com/forum/topic/list/168?pageNo=1): discuss implementations, research, etc. + +## Courses + +- [Server Deployments](https://aistudio.baidu.com/aistudio/course/introduce/19084): Courses intorducing high performance server deployments via local and remote services. +- [Edge Deployments](https://aistudio.baidu.com/aistudio/course/introduce/22690): Courses intorducing edge deployments from mobile, IoT to web and applets. ## Copyright and License PaddlePaddle is provided under the [Apache-2.0 license](LICENSE). diff --git a/README_cn.md b/README_cn.md index f80e703d107ef1..cc8afde7dd2662 100644 --- a/README_cn.md +++ b/README_cn.md @@ -19,7 +19,7 @@ ## 安装 -### PaddlePaddle最新版本: [v2.0](https://github.com/PaddlePaddle/Paddle/tree/release/2.0) +### PaddlePaddle最新版本: [v2.1](https://github.com/PaddlePaddle/Paddle/tree/release/2.1) 跟进PaddlePaddle最新特性请参考我们的[版本说明](https://github.com/PaddlePaddle/Paddle/releases) @@ -32,7 +32,7 @@ pip install paddlepaddle-gpu ``` 更多安装信息详见官网 [安装说明](https://www.paddlepaddle.org.cn/install/quick) -PaddlePaddle用户可领取**免费Tesla V100在线算力资源**，训练模型更高效。**每日登陆即送10小时**，[前往使用免费算力](https://aistudio.baidu.com/aistudio/index)。 +PaddlePaddle用户可领取**免费Tesla V100在线算力资源**，训练模型更高效。**每日登陆即送8小时**，[前往使用免费算力](https://aistudio.baidu.com/aistudio/index)。 ## 四大领先技术 @@ -84,6 +84,11 @@ PaddlePaddle用户可领取**免费Tesla V100在线算力资源**，训练模型 - 欢迎您通过[Github Issues](https://github.com/PaddlePaddle/Paddle/issues)来提交问题、报告与建议 - QQ群: 793866180 (PaddlePaddle) - [论坛](https://ai.baidu.com/forum/topic/list/168): 欢迎大家在PaddlePaddle论坛分享在使用PaddlePaddle中遇到的问题和经验, 营造良好的论坛氛围 + +## 课程 + +- [服务器部署](https://aistudio.baidu.com/aistudio/course/introduce/19084): 详细介绍高性能服务器端部署实操，包含本地端及服务化Serving部署等 +- [端侧部署](https://aistudio.baidu.com/aistudio/course/introduce/22690): 详细介绍端侧多场景部署实操，从移端端设备、IoT、网页到小程序部署 ## 版权和许可证 PaddlePaddle由[Apache-2.0 license](LICENSE)提供 diff --git a/cmake/external/box_ps.cmake b/cmake/external/box_ps.cmake index adfc6dba1f083e..85e1f94fd2c67f 100644 --- a/cmake/external/box_ps.cmake +++ b/cmake/external/box_ps.cmake @@ -49,7 +49,10 @@ ExternalProject_Add( DOWNLOAD_NO_PROGRESS 1 UPDATE_COMMAND "" CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${BOX_PS_INSTALL_ROOT} + -DCMAKE_BUILD_TYPE=${THIRD_PARTY_BUILD_TYPE} CMAKE_CACHE_ARGS -DCMAKE_INSTALL_PREFIX:PATH=${BOX_PS_INSTALL_ROOT} + -DCMAKE_BUILD_TYPE=${THIRD_PARTY_BUILD_TYPE} + BUILD_BYPRODUCTS ${BOX_PS_LIB} ) ADD_LIBRARY(box_ps SHARED IMPORTED GLOBAL) SET_PROPERTY(TARGET box_ps PROPERTY IMPORTED_LOCATION ${BOX_PS_LIB}) diff --git a/cmake/external/brpc.cmake b/cmake/external/brpc.cmake index 2d72b6eb56deaa..1a45cfa0a1e514 100644 --- a/cmake/external/brpc.cmake +++ b/cmake/external/brpc.cmake @@ -45,23 +45,24 @@ ExternalProject_Add( PREFIX ${BRPC_SOURCES_DIR} UPDATE_COMMAND "" CMAKE_ARGS -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER} - -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER} - -DCMAKE_CXX_FLAGS=${CMAKE_CXX_FLAGS} - -DCMAKE_C_FLAGS=${CMAKE_C_FLAGS} - -DCMAKE_INSTALL_PREFIX=${BRPC_INSTALL_DIR} - -DCMAKE_INSTALL_LIBDIR=${BRPC_INSTALL_DIR}/lib - -DCMAKE_POSITION_INDEPENDENT_CODE=ON - -DCMAKE_BUILD_TYPE=${THIRD_PARTY_BUILD_TYPE} - -DCMAKE_PREFIX_PATH=${prefix_path} - -DWITH_GLOG=ON - -DIOBUF_WITH_HUGE_BLOCK=ON - -DBRPC_WITH_RDMA=${WITH_BRPC_RDMA} - ${EXTERNAL_OPTIONAL_ARGS} - LIST_SEPARATOR | + -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER} + -DCMAKE_CXX_FLAGS=${CMAKE_CXX_FLAGS} + -DCMAKE_C_FLAGS=${CMAKE_C_FLAGS} + -DCMAKE_INSTALL_PREFIX=${BRPC_INSTALL_DIR} + -DCMAKE_INSTALL_LIBDIR=${BRPC_INSTALL_DIR}/lib + -DCMAKE_POSITION_INDEPENDENT_CODE=ON + -DCMAKE_BUILD_TYPE=${THIRD_PARTY_BUILD_TYPE} + -DCMAKE_PREFIX_PATH=${prefix_path} + -DWITH_GLOG=ON + -DIOBUF_WITH_HUGE_BLOCK=ON + -DBRPC_WITH_RDMA=${WITH_BRPC_RDMA} + ${EXTERNAL_OPTIONAL_ARGS} + LIST_SEPARATOR | CMAKE_CACHE_ARGS -DCMAKE_INSTALL_PREFIX:PATH=${BRPC_INSTALL_DIR} - -DCMAKE_INSTALL_LIBDIR:PATH=${BRPC_INSTALL_DIR}/lib - -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=ON - -DCMAKE_BUILD_TYPE:STRING=${THIRD_PARTY_BUILD_TYPE} + -DCMAKE_INSTALL_LIBDIR:PATH=${BRPC_INSTALL_DIR}/lib + -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=ON + -DCMAKE_BUILD_TYPE:STRING=${THIRD_PARTY_BUILD_TYPE} + BUILD_BYPRODUCTS ${BRPC_LIBRARIES} ) # ADD_DEPENDENCIES(extern_brpc protobuf ssl crypto leveldb gflags glog gtest snappy) ADD_DEPENDENCIES(extern_brpc protobuf ssl crypto leveldb gflags glog snappy) diff --git a/cmake/external/cryptopp.cmake b/cmake/external/cryptopp.cmake index a30164ada2791b..f7f7a9b52e895d 100644 --- a/cmake/external/cryptopp.cmake +++ b/cmake/external/cryptopp.cmake @@ -72,6 +72,7 @@ ExternalProject_Add( CMAKE_CACHE_ARGS -DCMAKE_INSTALL_PREFIX:PATH=${CRYPTOPP_INSTALL_DIR} -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=${THIRD_PARTY_BUILD_TYPE} + BUILD_BYPRODUCTS ${CRYPTOPP_LIBRARIES} ) ADD_LIBRARY(cryptopp STATIC IMPORTED GLOBAL) diff --git a/cmake/external/dgc.cmake b/cmake/external/dgc.cmake index bc8611f3862cd1..3c64e1ea11ecd6 100644 --- a/cmake/external/dgc.cmake +++ b/cmake/external/dgc.cmake @@ -39,6 +39,7 @@ ExternalProject_Add( && cp ${DGC_SOURCES_DIR}/build/lib/libdgc.a ${DGC_LIBRARIES} && cp ${DGC_SOURCES_DIR}/build/include/dgc.h ${DGC_INCLUDE_DIR}/dgc/ BUILD_IN_SOURCE 1 + BUILD_BYPRODUCTS ${DGC_LIBRARIES} ) ADD_LIBRARY(dgc STATIC IMPORTED GLOBAL) diff --git a/cmake/external/gflags.cmake b/cmake/external/gflags.cmake index 576598b4ac6e3b..8360761de6fb98 100644 --- a/cmake/external/gflags.cmake +++ b/cmake/external/gflags.cmake @@ -61,6 +61,7 @@ ExternalProject_Add( CMAKE_CACHE_ARGS -DCMAKE_INSTALL_PREFIX:PATH=${GFLAGS_INSTALL_DIR} -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=${THIRD_PARTY_BUILD_TYPE} + BUILD_BYPRODUCTS ${GFLAGS_LIBRARIES} ) ADD_LIBRARY(gflags STATIC IMPORTED GLOBAL) diff --git a/cmake/external/glog.cmake b/cmake/external/glog.cmake index 05b98e2b56a33a..d2bb1e62e83de3 100644 --- a/cmake/external/glog.cmake +++ b/cmake/external/glog.cmake @@ -64,6 +64,7 @@ ExternalProject_Add( -DCMAKE_INSTALL_LIBDIR:PATH=${GLOG_INSTALL_DIR}/lib -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=${THIRD_PARTY_BUILD_TYPE} + BUILD_BYPRODUCTS ${GLOG_LIBRARIES} ) ADD_LIBRARY(glog STATIC IMPORTED GLOBAL) diff --git a/cmake/external/gloo.cmake b/cmake/external/gloo.cmake index e8db13a694f557..03e45e3e5c67b0 100644 --- a/cmake/external/gloo.cmake +++ b/cmake/external/gloo.cmake @@ -32,7 +32,7 @@ cache_third_party(extern_gloo TAG ${GLOO_TAG} DIR GLOO_SOURCE_DIR) - if(WITH_ASCEND OR WITH_ASCEND_CL) +if(WITH_ASCEND OR WITH_ASCEND_CL) ExternalProject_Add( extern_gloo ${EXTERNAL_PROJECT_LOG_ARGS} @@ -47,6 +47,7 @@ cache_third_party(extern_gloo && mkdir -p ${GLOO_LIBRARY_DIR} ${GLOO_INCLUDE_DIR}/gloo INSTALL_COMMAND ${CMAKE_COMMAND} -E copy ${GLOO_SOURCE_DIR}/build/gloo/libgloo.a ${GLOO_LIBRARY_DIR} COMMAND ${CMAKE_COMMAND} -E copy_directory "${GLOO_SOURCE_DIR}/gloo/" "${GLOO_INCLUDE_DIR}/gloo" + BUILD_BYPRODUCTS ${GLOO_LIBRARIES} ) else() ExternalProject_Add( @@ -63,6 +64,7 @@ else() && mkdir -p ${GLOO_LIBRARY_DIR} ${GLOO_INCLUDE_DIR}/gloo INSTALL_COMMAND ${CMAKE_COMMAND} -E copy ${GLOO_SOURCE_DIR}/build/gloo/libgloo.a ${GLOO_LIBRARY_DIR} COMMAND ${CMAKE_COMMAND} -E copy_directory "${GLOO_SOURCE_DIR}/gloo/" "${GLOO_INCLUDE_DIR}/gloo" + BUILD_BYPRODUCTS ${GLOO_LIBRARIES} ) endif() diff --git a/cmake/external/gtest.cmake b/cmake/external/gtest.cmake index 3db12f084eb5a3..e7d4783a9593a7 100644 --- a/cmake/external/gtest.cmake +++ b/cmake/external/gtest.cmake @@ -79,6 +79,8 @@ ExternalProject_Add( CMAKE_CACHE_ARGS -DCMAKE_INSTALL_PREFIX:PATH=${GTEST_INSTALL_DIR} -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=${THIRD_PARTY_BUILD_TYPE} + BUILD_BYPRODUCTS ${GTEST_LIBRARIES} + BUILD_BYPRODUCTS ${GTEST_MAIN_LIBRARIES} ) ADD_LIBRARY(gtest STATIC IMPORTED GLOBAL) diff --git a/cmake/external/leveldb.cmake b/cmake/external/leveldb.cmake index 79dc403e67d526..c36f49d3bd354a 100644 --- a/cmake/external/leveldb.cmake +++ b/cmake/external/leveldb.cmake @@ -33,6 +33,7 @@ ExternalProject_Add( && cp ${LEVELDB_SOURCES_DIR}/src/extern_leveldb/libleveldb.a ${LEVELDB_LIBRARIES} && cp -r ${LEVELDB_SOURCES_DIR}/src/extern_leveldb/include ${LEVELDB_INSTALL_DIR}/ BUILD_IN_SOURCE 1 + BUILD_BYPRODUCTS ${LEVELDB_LIBRARIES} ) ADD_DEPENDENCIES(extern_leveldb snappy) diff --git a/cmake/external/libmct.cmake b/cmake/external/libmct.cmake index c10a662485c2d1..d318bc7d0f3c3f 100644 --- a/cmake/external/libmct.cmake +++ b/cmake/external/libmct.cmake @@ -49,7 +49,9 @@ ExternalProject_Add( DOWNLOAD_NO_PROGRESS 1 UPDATE_COMMAND "" CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${LIBMCT_INSTALL_ROOT} + -DCMAKE_BUILD_TYPE=${THIRD_PARTY_BUILD_TYPE} CMAKE_CACHE_ARGS -DCMAKE_INSTALL_PREFIX:PATH=${LIBMCT_INSTALL_ROOT} + -DCMAKE_BUILD_TYPE=${THIRD_PARTY_BUILD_TYPE} ) add_library(libmct INTERFACE) diff --git a/cmake/external/libxsmm.cmake b/cmake/external/libxsmm.cmake index 0d09576286d907..fae8154eb1cb03 100644 --- a/cmake/external/libxsmm.cmake +++ b/cmake/external/libxsmm.cmake @@ -18,8 +18,8 @@ SET(LIBXSMM_SOURCES_DIR ${THIRD_PARTY_PATH}/libxsmm) SET(LIBXSMM_INSTALL_DIR ${THIRD_PARTY_PATH}/install/libxsmm) SET(LIBXSMM_INCLUDE_DIR "${LIBXSMM_INSTALL_DIR}/include" CACHE PATH "LIBXSMM include directory." FORCE) SET(LIBXSMM_LIBRARY_DIR "${LIBXSMM_INSTALL_DIR}/lib" CACHE PATH "LIBXSMM library directory." FORCE) -SET(LIBXSMM_LIBS "${LIBXSMM_LIBRARY_DIR}/libxsmm.a" - "${LIBXSMM_LIBRARY_DIR}/libxsmmnoblas.a") +SET(LIBXSMM_LIB "${LIBXSMM_LIBRARY_DIR}/libxsmm.a") +SET(LIBXSMMNOBLAS_LIB "${LIBXSMM_LIBRARY_DIR}/libxsmmnoblas.a") ExternalProject_Add( extern_libxsmm @@ -32,10 +32,12 @@ ExternalProject_Add( BUILD_IN_SOURCE 1 BUILD_COMMAND $(MAKE) --silent PREFIX=${LIBXSMM_INSTALL_DIR} CXX=g++ CC=gcc WARP=0 install INSTALL_COMMAND "" + BUILD_BYPRODUCTS ${LIBXSMM_LIB} + BUILD_BYPRODUCTS ${LIBXSMMNOBLAS_LIB} ) ADD_LIBRARY(libxsmm STATIC IMPORTED GLOBAL) -SET_PROPERTY(TARGET libxsmm PROPERTY IMPORTED_LOCATION "${LIBXSMM_LIBRARY_DIR}/libxsmm.a") -SET_PROPERTY(TARGET libxsmm PROPERTY IMPORTED_LOCATION "${LIBXSMM_LIBRARY_DIR}/libxsmmnoblas.a") +SET_PROPERTY(TARGET libxsmm PROPERTY IMPORTED_LOCATION "${LIBXSMM_LIB}") +SET_PROPERTY(TARGET libxsmm PROPERTY IMPORTED_LOCATION "${LIBXSMMNOBLAS_LIB}") MESSAGE(STATUS "Libxsmm library: ${LIBXSMM_LIBS}") include_directories(${LIBXSMM_INCLUDE_DIR}) diff --git a/cmake/external/mkldnn.cmake b/cmake/external/mkldnn.cmake index e99d59bbed6fd4..9963237ff188cf 100644 --- a/cmake/external/mkldnn.cmake +++ b/cmake/external/mkldnn.cmake @@ -20,7 +20,7 @@ SET(MKLDNN_SOURCE_DIR ${THIRD_PARTY_PATH}/mkldnn/src/extern_mkldnn) SET(MKLDNN_INSTALL_DIR ${THIRD_PARTY_PATH}/install/mkldnn) SET(MKLDNN_INC_DIR "${MKLDNN_INSTALL_DIR}/include" CACHE PATH "mkldnn include directory." FORCE) SET(MKLDNN_REPOSITORY ${GIT_URL}/oneapi-src/oneDNN.git) -SET(MKLDNN_TAG 748528a2d3204b5f401c14a9aacdec16accd5ead) +SET(MKLDNN_TAG 593e0de6267d2575f3e4c9e9818f0f11253d093a) # Introduce variables: @@ -43,8 +43,10 @@ IF(NOT WIN32) SET(MKLDNN_FLAG "${MKLDNN_FLAG} -Wno-unused-result -Wno-unused-value") SET(MKLDNN_CFLAG "${CMAKE_C_FLAGS} ${MKLDNN_FLAG}") SET(MKLDNN_CXXFLAG "${CMAKE_CXX_FLAGS} ${MKLDNN_FLAG}") + SET(MKLDNN_LIB "${MKLDNN_INSTALL_DIR}/${LIBDIR}/libdnnl.so" CACHE FILEPATH "mkldnn library." FORCE) ELSE() SET(MKLDNN_CXXFLAG "${CMAKE_CXX_FLAGS} /EHsc") + SET(MKLDNN_LIB "${MKLDNN_INSTALL_DIR}/bin/mkldnn.lib" CACHE FILEPATH "mkldnn library." FORCE) ENDIF(NOT WIN32) cache_third_party(${MKLDNN_PROJECT} @@ -77,12 +79,8 @@ ExternalProject_Add( -DCMAKE_CXX_FLAGS=${MKLDNN_CXXFLAG} -DDNNL_BUILD_TESTS=OFF -DDNNL_BUILD_EXAMPLES=OFF CMAKE_CACHE_ARGS -DCMAKE_INSTALL_PREFIX:PATH=${MKLDNN_INSTALL_DIR} + BUILD_BYPRODUCTS ${MKLDNN_LIB} ) -if(WIN32) - SET(MKLDNN_LIB "${MKLDNN_INSTALL_DIR}/bin/mkldnn.lib" CACHE FILEPATH "mkldnn library." FORCE) -else(WIN32) - SET(MKLDNN_LIB "${MKLDNN_INSTALL_DIR}/${LIBDIR}/libdnnl.so" CACHE FILEPATH "mkldnn library." FORCE) -endif(WIN32) ADD_LIBRARY(shared_mkldnn SHARED IMPORTED GLOBAL) SET_PROPERTY(TARGET shared_mkldnn PROPERTY IMPORTED_LOCATION ${MKLDNN_LIB}) diff --git a/cmake/external/mklml.cmake b/cmake/external/mklml.cmake index d99cb1952951c4..a4df5756ce015d 100644 --- a/cmake/external/mklml.cmake +++ b/cmake/external/mklml.cmake @@ -50,6 +50,10 @@ cache_third_party(${MKLML_PROJECT} URL ${MKLML_URL} DIR MKLML_SOURCE_DIR) +# Ninja Generator can not establish the correct dependency relationship between the imported library with target, +# the product file in the ExternalProject need to be specified manually, please refer to +# https://stackoverflow.com/questions/54866067/cmake-and-ninja-missing-and-no-known-rule-to-make-it +# It is the same to all other ExternalProject. ExternalProject_Add( ${MKLML_PROJECT} ${EXTERNAL_PROJECT_LOG_ARGS} @@ -63,7 +67,9 @@ ExternalProject_Add( BUILD_COMMAND "" UPDATE_COMMAND "" INSTALL_COMMAND ${CMAKE_COMMAND} -E copy_directory ${MKLML_SOURCE_DIR}/include ${MKLML_INC_DIR} && - ${CMAKE_COMMAND} -E copy_directory ${MKLML_SOURCE_DIR}/lib ${MKLML_LIB_DIR} + ${CMAKE_COMMAND} -E copy_directory ${MKLML_SOURCE_DIR}/lib ${MKLML_LIB_DIR} + BUILD_BYPRODUCTS ${MKLML_LIB} + BUILD_BYPRODUCTS ${MKLML_IOMP_LIB} ) INCLUDE_DIRECTORIES(${MKLML_INC_DIR}) diff --git a/cmake/external/protobuf.cmake b/cmake/external/protobuf.cmake index c108c05368c915..a2b6ddadb625f6 100644 --- a/cmake/external/protobuf.cmake +++ b/cmake/external/protobuf.cmake @@ -239,6 +239,10 @@ endif() -DCMAKE_VERBOSE_MAKEFILE:BOOL=OFF -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=ON ${OPTIONAL_CACHE_ARGS} + BUILD_BYPRODUCTS ${PROTOBUF_INSTALL_DIR}/lib/libprotobuf${CMAKE_STATIC_LIBRARY_SUFFIX} + BUILD_BYPRODUCTS ${PROTOBUF_INSTALL_DIR}/lib/libprotobuf-lite${CMAKE_STATIC_LIBRARY_SUFFIX} + BUILD_BYPRODUCTS ${PROTOBUF_INSTALL_DIR}/lib/libprotoc${CMAKE_STATIC_LIBRARY_SUFFIX} + BUILD_BYPRODUCTS ${PROTOBUF_INSTALL_DIR}/bin/protoc${CMAKE_EXECUTABLE_SUFFIX} ) ENDFUNCTION() diff --git a/cmake/external/pslib.cmake b/cmake/external/pslib.cmake index bdfd335172d877..40d198b2958339 100644 --- a/cmake/external/pslib.cmake +++ b/cmake/external/pslib.cmake @@ -53,7 +53,10 @@ ExternalProject_Add( DOWNLOAD_NO_PROGRESS 1 UPDATE_COMMAND "" CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${PSLIB_INSTALL_ROOT} + -DCMAKE_BUILD_TYPE=${THIRD_PARTY_BUILD_TYPE} CMAKE_CACHE_ARGS -DCMAKE_INSTALL_PREFIX:PATH=${PSLIB_INSTALL_ROOT} + -DCMAKE_BUILD_TYPE=${THIRD_PARTY_BUILD_TYPE} + BUILD_BYPRODUCTS ${PSLIB_LIB} ) ADD_LIBRARY(pslib SHARED IMPORTED GLOBAL) diff --git a/cmake/external/pslib_brpc.cmake b/cmake/external/pslib_brpc.cmake index 7b00474a650706..d69c27a197b25a 100644 --- a/cmake/external/pslib_brpc.cmake +++ b/cmake/external/pslib_brpc.cmake @@ -52,7 +52,10 @@ ExternalProject_Add( DOWNLOAD_NO_PROGRESS 1 UPDATE_COMMAND "" CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${PSLIB_BRPC_INSTALL_ROOT} + -DCMAKE_BUILD_TYPE=${THIRD_PARTY_BUILD_TYPE} CMAKE_CACHE_ARGS -DCMAKE_INSTALL_PREFIX:PATH=${PSLIB_BRPC_INSTALL_ROOT} + -DCMAKE_BUILD_TYPE=${THIRD_PARTY_BUILD_TYPE} + BUILD_BYPRODUCTS ${PSLIB_BRPC_LIB} ) ADD_LIBRARY(pslib_brpc SHARED IMPORTED GLOBAL) diff --git a/cmake/external/snappy.cmake b/cmake/external/snappy.cmake index ab9cb02307c1f0..fb4c1c7cc8a3d5 100644 --- a/cmake/external/snappy.cmake +++ b/cmake/external/snappy.cmake @@ -22,8 +22,15 @@ set(SNAPPY_INCLUDE_DIR "${SNAPPY_INSTALL_DIR}/include" CACHE PATH "snappy includ if(WIN32) SET(SNAPPY_CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /wd4244 /wd4267") + IF(NOT EXISTS "${SNAPPY_INSTALL_DIR}/lib/libsnappy.lib") + add_custom_command(TARGET extern_snappy POST_BUILD + COMMAND cmake -E copy ${SNAPPY_INSTALL_DIR}/lib/snappy.lib ${SNAPPY_INSTALL_DIR}/lib/libsnappy.lib + ) + ENDIF() + set(SNAPPY_LIBRARIES "${SNAPPY_INSTALL_DIR}/lib/libsnappy.lib") else() SET(SNAPPY_CMAKE_CXX_FLAGS ${CMAKE_CXX_FLAGS}) + set(SNAPPY_LIBRARIES "${SNAPPY_INSTALL_DIR}/lib/libsnappy.a") endif() ExternalProject_Add( @@ -33,35 +40,26 @@ ExternalProject_Add( PREFIX ${SNAPPY_SOURCES_DIR} UPDATE_COMMAND "" CMAKE_ARGS -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER} - -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER} - -DCMAKE_C_FLAGS=${CMAKE_C_FLAGS} - -DCMAKE_C_FLAGS_DEBUG=${CMAKE_C_FLAGS_DEBUG} - -DCMAKE_C_FLAGS_RELEASE=${CMAKE_C_FLAGS_RELEASE} - -DCMAKE_CXX_FLAGS=${SNAPPY_CMAKE_CXX_FLAGS} - -DCMAKE_CXX_FLAGS_RELEASE=${CMAKE_CXX_FLAGS_RELEASE} - -DCMAKE_CXX_FLAGS_DEBUG=${CMAKE_CXX_FLAGS_DEBUG} - -DCMAKE_INSTALL_PREFIX=${SNAPPY_INSTALL_DIR} - -DCMAKE_INSTALL_LIBDIR=${SNAPPY_INSTALL_DIR}/lib - -DCMAKE_POSITION_INDEPENDENT_CODE=ON - -DBUILD_TESTING=OFF - -DSNAPPY_BUILD_TESTS:BOOL=OFF - -DCMAKE_BUILD_TYPE=${THIRD_PARTY_BUILD_TYPE} - ${EXTERNAL_OPTIONAL_ARGS} + -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER} + -DCMAKE_C_FLAGS=${CMAKE_C_FLAGS} + -DCMAKE_C_FLAGS_DEBUG=${CMAKE_C_FLAGS_DEBUG} + -DCMAKE_C_FLAGS_RELEASE=${CMAKE_C_FLAGS_RELEASE} + -DCMAKE_CXX_FLAGS=${SNAPPY_CMAKE_CXX_FLAGS} + -DCMAKE_CXX_FLAGS_RELEASE=${CMAKE_CXX_FLAGS_RELEASE} + -DCMAKE_CXX_FLAGS_DEBUG=${CMAKE_CXX_FLAGS_DEBUG} + -DCMAKE_INSTALL_PREFIX=${SNAPPY_INSTALL_DIR} + -DCMAKE_INSTALL_LIBDIR=${SNAPPY_INSTALL_DIR}/lib + -DCMAKE_POSITION_INDEPENDENT_CODE=ON + -DBUILD_TESTING=OFF + -DSNAPPY_BUILD_TESTS:BOOL=OFF + -DCMAKE_BUILD_TYPE=${THIRD_PARTY_BUILD_TYPE} + ${EXTERNAL_OPTIONAL_ARGS} CMAKE_CACHE_ARGS -DCMAKE_INSTALL_PREFIX:PATH=${SNAPPY_INSTALL_DIR} - -DCMAKE_INSTALL_LIBDIR:PATH=${SNAPPY_INSTALL_DIR}/lib - -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=ON - -DCMAKE_BUILD_TYPE:STRING=${THIRD_PARTY_BUILD_TYPE} + -DCMAKE_INSTALL_LIBDIR:PATH=${SNAPPY_INSTALL_DIR}/lib + -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=ON + -DCMAKE_BUILD_TYPE:STRING=${THIRD_PARTY_BUILD_TYPE} + BUILD_BYPRODUCTS ${SNAPPY_LIBRARIES} ) -IF(WIN32) - IF(NOT EXISTS "${SNAPPY_INSTALL_DIR}/lib/libsnappy.lib") - add_custom_command(TARGET extern_snappy POST_BUILD - COMMAND cmake -E copy ${SNAPPY_INSTALL_DIR}/lib/snappy.lib ${SNAPPY_INSTALL_DIR}/lib/libsnappy.lib - ) - ENDIF() - set(SNAPPY_LIBRARIES "${SNAPPY_INSTALL_DIR}/lib/libsnappy.lib") -else(WIN32) - set(SNAPPY_LIBRARIES "${SNAPPY_INSTALL_DIR}/lib/libsnappy.a") -endif (WIN32) add_library(snappy STATIC IMPORTED GLOBAL) set_property(TARGET snappy PROPERTY IMPORTED_LOCATION ${SNAPPY_LIBRARIES}) diff --git a/cmake/external/warpctc.cmake b/cmake/external/warpctc.cmake index 6597e259aa890f..532ebaaf5c0643 100644 --- a/cmake/external/warpctc.cmake +++ b/cmake/external/warpctc.cmake @@ -32,6 +32,14 @@ SET(WARPCTC_INCLUDE_DIR "${WARPCTC_INSTALL_DIR}/include" SET(WARPCTC_LIB_DIR "${WARPCTC_INSTALL_DIR}/lib" CACHE PATH "Warp-ctc Library Directory" FORCE) +IF(WIN32) + SET(WARPCTC_LIBRARIES "${WARPCTC_INSTALL_DIR}/bin/warpctc${CMAKE_SHARED_LIBRARY_SUFFIX}" + CACHE FILEPATH "Warp-ctc Library" FORCE) +else(WIN32) + SET(WARPCTC_LIBRARIES "${WARPCTC_INSTALL_DIR}/lib/libwarpctc${CMAKE_SHARED_LIBRARY_SUFFIX}" + CACHE FILEPATH "Warp-ctc Library" FORCE) +ENDIF(WIN32) + IF(CMAKE_CXX_COMPILER_ID STREQUAL "Clang" OR CMAKE_CXX_COMPILER_ID STREQUAL "AppleClang" OR WIN32) SET(USE_OMP OFF) ELSE() @@ -59,7 +67,7 @@ if(WITH_ASCEND OR WITH_ASCEND_CL) -DCMAKE_C_FLAGS=${CMAKE_C_FLAGS} -DCMAKE_C_FLAGS_DEBUG=${CMAKE_C_FLAGS_DEBUG} -DCMAKE_C_FLAGS_RELEASE=${CMAKE_C_FLAGS_RELEASE} - "-DCMAKE_CXX_FLAGS=${CMAKE_CXX_FLAGS}" + -DCMAKE_CXX_FLAGS=${CMAKE_CXX_FLAGS} -DCMAKE_CXX_FLAGS_RELEASE=${CMAKE_CXX_FLAGS_RELEASE} -DCMAKE_CXX_FLAGS_DEBUG=${CMAKE_CXX_FLAGS_DEBUG} -DCMAKE_INSTALL_PREFIX=${WARPCTC_INSTALL_DIR} @@ -76,6 +84,7 @@ if(WITH_ASCEND OR WITH_ASCEND_CL) CMAKE_CACHE_ARGS -DCMAKE_BUILD_TYPE:STRING=${THIRD_PARTY_BUILD_TYPE} -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=ON -DCMAKE_INSTALL_PREFIX:PATH=${WARPCTC_INSTALL_DIR} + BUILD_BYPRODUCTS ${WARPCTC_LIBRARIES} ) else() if(WIN32) @@ -125,18 +134,10 @@ else() CMAKE_CACHE_ARGS -DCMAKE_BUILD_TYPE:STRING=${THIRD_PARTY_BUILD_TYPE} -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=ON -DCMAKE_INSTALL_PREFIX:PATH=${WARPCTC_INSTALL_DIR} + BUILD_BYPRODUCTS ${WARPCTC_LIBRARIES} ) endif() - -IF(WIN32) - SET(WARPCTC_LIBRARIES "${WARPCTC_INSTALL_DIR}/bin/warpctc${CMAKE_SHARED_LIBRARY_SUFFIX}" - CACHE FILEPATH "Warp-ctc Library" FORCE) -else(WIN32) - SET(WARPCTC_LIBRARIES "${WARPCTC_INSTALL_DIR}/lib/libwarpctc${CMAKE_SHARED_LIBRARY_SUFFIX}" - CACHE FILEPATH "Warp-ctc Library" FORCE) -ENDIF(WIN32) - MESSAGE(STATUS "warp-ctc library: ${WARPCTC_LIBRARIES}") get_filename_component(WARPCTC_LIBRARY_PATH ${WARPCTC_LIBRARIES} DIRECTORY) INCLUDE_DIRECTORIES(${WARPCTC_INCLUDE_DIR}) # For warpctc code to include its headers. diff --git a/cmake/external/xbyak.cmake b/cmake/external/xbyak.cmake index 610a692ef12c6a..eabcabf7430633 100644 --- a/cmake/external/xbyak.cmake +++ b/cmake/external/xbyak.cmake @@ -46,7 +46,9 @@ ExternalProject_Add( SOURCE_DIR ${XBYAK_SOURCE_DIR} # UPDATE_COMMAND "" CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${XBYAK_INSTALL_ROOT} + -DCMAKE_BUILD_TYPE=${THIRD_PARTY_BUILD_TYPE} CMAKE_CACHE_ARGS -DCMAKE_INSTALL_PREFIX:PATH=${XBYAK_INSTALL_ROOT} + -DCMAKE_BUILD_TYPE=${THIRD_PARTY_BUILD_TYPE} ) add_library(xbyak INTERFACE) diff --git a/cmake/external/xpu.cmake b/cmake/external/xpu.cmake index a8c33618a61359..a2d824877ea528 100644 --- a/cmake/external/xpu.cmake +++ b/cmake/external/xpu.cmake @@ -27,19 +27,18 @@ ELSEIF(WITH_CENTOS) SET(XPU_XRE_DIR_NAME "xre-centos7_x86_64") SET(XPU_XDNN_DIR_NAME "xdnn-centos7_x86_64") SET(XPU_XCCL_DIR_NAME "xccl-bdcentos_x86_64") + ELSE () SET(XPU_XRE_DIR_NAME "xre-ubuntu_x86_64") SET(XPU_XDNN_DIR_NAME "xdnn-ubuntu_x86_64") SET(XPU_XCCL_DIR_NAME "xccl-bdcentos_x86_64") ENDIF() -IF(NOT XPU_BASE_URL) - SET(XPU_BASE_URL "https://baidu-kunlun-product.cdn.bcebos.com/KL-SDK/klsdk-dev/20210527") -ENDIF() - +SET(XPU_BASE_URL_WITHOUT_DATE "https://baidu-kunlun-product.cdn.bcebos.com/KL-SDK/klsdk-dev") +SET(XPU_BASE_URL "${XPU_BASE_URL_WITHOUT_DATE}/20210625") SET(XPU_XRE_URL "${XPU_BASE_URL}/${XPU_XRE_DIR_NAME}.tar.gz" CACHE STRING "" FORCE) SET(XPU_XDNN_URL "${XPU_BASE_URL}/${XPU_XDNN_DIR_NAME}.tar.gz" CACHE STRING "" FORCE) -SET(XPU_XCCL_URL "${XPU_BASE_URL}/${XPU_XCCL_DIR_NAME}.tar.gz" CACHE STRING "" FORCE) +SET(XPU_XCCL_URL "${XPU_BASE_URL_WITHOUT_DATE}/20210623/${XPU_XCCL_DIR_NAME}.tar.gz" CACHE STRING "" FORCE) SET(XPU_PACK_DEPENCE_URL "https://baidu-kunlun-public.su.bcebos.com/paddle_depence/pack_paddle_depence.sh" CACHE STRING "" FORCE) SET(XPU_SOURCE_DIR "${THIRD_PARTY_PATH}/xpu") @@ -71,6 +70,8 @@ ExternalProject_Add( UPDATE_COMMAND "" CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${XPU_INSTALL_ROOT} CMAKE_CACHE_ARGS -DCMAKE_INSTALL_PREFIX:PATH=${XPU_INSTALL_ROOT} + BUILD_BYPRODUCTS ${XPU_API_LIB} + BUILD_BYPRODUCTS ${XPU_RT_LIB} ) INCLUDE_DIRECTORIES(${XPU_INC_DIR}) diff --git a/cmake/external/xxhash.cmake b/cmake/external/xxhash.cmake index bdd7df190ff106..0279d4e2a835c2 100644 --- a/cmake/external/xxhash.cmake +++ b/cmake/external/xxhash.cmake @@ -21,10 +21,7 @@ set(XXHASH_INCLUDE_DIR "${XXHASH_INSTALL_DIR}/include") set(XXHASH_REPOSITORY ${GIT_URL}/Cyan4973/xxHash.git) set(XXHASH_TAG v0.6.5) -cache_third_party(extern_xxhash - REPOSITORY ${XXHASH_REPOSITORY} - TAG ${XXHASH_TAG} - DIR XXHASH_SOURCE_DIR) +INCLUDE_DIRECTORIES(${XXHASH_INCLUDE_DIR}) IF(APPLE) SET(BUILD_CMD sed -i \"\" "s/-Wstrict-prototypes -Wundef/-Wstrict-prototypes -Wundef -fPIC/g" ${XXHASH_SOURCE_DIR}/Makefile && make lib) @@ -32,6 +29,17 @@ ELSEIF(UNIX) SET(BUILD_CMD sed -i "s/-Wstrict-prototypes -Wundef/-Wstrict-prototypes -Wundef -fPIC/g" ${XXHASH_SOURCE_DIR}/Makefile && make lib) ENDIF() +if (WIN32) + set(XXHASH_LIBRARIES "${XXHASH_INSTALL_DIR}/lib/xxhash.lib") +else() + set(XXHASH_LIBRARIES "${XXHASH_INSTALL_DIR}/lib/libxxhash.a") +endif () + +cache_third_party(extern_xxhash + REPOSITORY ${XXHASH_REPOSITORY} + TAG ${XXHASH_TAG} + DIR XXHASH_SOURCE_DIR) + if(WIN32) ExternalProject_Add( extern_xxhash @@ -54,6 +62,7 @@ if(WIN32) -DBUILD_SHARED_LIBS=OFF ${OPTIONAL_CACHE_ARGS} TEST_COMMAND "" + BUILD_BYPRODUCTS ${XXHASH_LIBRARIES} ) else() ExternalProject_Add( @@ -68,16 +77,10 @@ else() BUILD_COMMAND ${BUILD_CMD} INSTALL_COMMAND make PREFIX=${XXHASH_INSTALL_DIR} install TEST_COMMAND "" + BUILD_BYPRODUCTS ${XXHASH_LIBRARIES} ) endif() -if (WIN32) - set(XXHASH_LIBRARIES "${XXHASH_INSTALL_DIR}/lib/xxhash.lib") -else() - set(XXHASH_LIBRARIES "${XXHASH_INSTALL_DIR}/lib/libxxhash.a") -endif () -INCLUDE_DIRECTORIES(${XXHASH_INCLUDE_DIR}) - add_library(xxhash STATIC IMPORTED GLOBAL) set_property(TARGET xxhash PROPERTY IMPORTED_LOCATION ${XXHASH_LIBRARIES}) include_directories(${XXHASH_INCLUDE_DIR}) diff --git a/cmake/external/zlib.cmake b/cmake/external/zlib.cmake index 4464787a0c2a64..f1a015f6304a38 100644 --- a/cmake/external/zlib.cmake +++ b/cmake/external/zlib.cmake @@ -25,6 +25,12 @@ set(ZLIB_TAG v1.2.8) INCLUDE_DIRECTORIES(${ZLIB_INCLUDE_DIR}) # For zlib code to include its own headers. INCLUDE_DIRECTORIES(${THIRD_PARTY_PATH}/install) # For Paddle code to include zlib.h. +IF(WIN32) + SET(ZLIB_LIBRARIES "${ZLIB_INSTALL_DIR}/lib/zlibstatic.lib" CACHE FILEPATH "zlib library." FORCE) +ELSE(WIN32) + SET(ZLIB_LIBRARIES "${ZLIB_INSTALL_DIR}/lib/libz.a" CACHE FILEPATH "zlib library." FORCE) +ENDIF(WIN32) + cache_third_party(extern_zlib REPOSITORY ${ZLIB_REPOSITORY} TAG ${ZLIB_TAG} @@ -51,12 +57,8 @@ ExternalProject_Add( CMAKE_CACHE_ARGS -DCMAKE_INSTALL_PREFIX:PATH=${ZLIB_INSTALL_DIR} -DCMAKE_POSITION_INDEPENDENT_CODE:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=${THIRD_PARTY_BUILD_TYPE} + BUILD_BYPRODUCTS ${ZLIB_LIBRARIES} ) -IF(WIN32) - SET(ZLIB_LIBRARIES "${ZLIB_INSTALL_DIR}/lib/zlibstatic.lib" CACHE FILEPATH "zlib library." FORCE) -ELSE(WIN32) - SET(ZLIB_LIBRARIES "${ZLIB_INSTALL_DIR}/lib/libz.a" CACHE FILEPATH "zlib library." FORCE) -ENDIF(WIN32) ADD_LIBRARY(zlib STATIC IMPORTED GLOBAL) SET_PROPERTY(TARGET zlib PROPERTY IMPORTED_LOCATION ${ZLIB_LIBRARIES}) diff --git a/cmake/third_party.cmake b/cmake/third_party.cmake index e3a78d3cf3bfe0..aa31745c21340c 100644 --- a/cmake/third_party.cmake +++ b/cmake/third_party.cmake @@ -253,15 +253,17 @@ if(WITH_GPU) set(URL "https://paddlepaddledeps.bj.bcebos.com/externalErrorMsg.tar.gz" CACHE STRING "" FORCE) file_download_and_uncompress(${URL} "externalError" MD5 c0749523ebb536eb7382487d645d9cd4) # download file externalErrorMsg.tar.gz if(WITH_TESTING) - # copy externalErrorMsg.pb for unittest 'enforce_test' + # copy externalErrorMsg.pb, just for unittest can get error message correctly. set(SRC_DIR ${THIRD_PARTY_PATH}/externalError/data) if(WIN32 AND (NOT "${CMAKE_GENERATOR}" STREQUAL "Ninja")) - set(DST_DIR ${CMAKE_BINARY_DIR}/paddle/fluid/third_party/externalError/data) + set(DST_DIR1 ${CMAKE_BINARY_DIR}/paddle/fluid/third_party/externalError/data) else() - set(DST_DIR ${CMAKE_BINARY_DIR}/paddle/third_party/externalError/data) + set(DST_DIR1 ${CMAKE_BINARY_DIR}/paddle/third_party/externalError/data) endif() + set(DST_DIR2 ${CMAKE_BINARY_DIR}/python/paddle/include/third_party/externalError/data) add_custom_command(TARGET download_externalError POST_BUILD - COMMAND ${CMAKE_COMMAND} -E copy_directory ${SRC_DIR} ${DST_DIR} + COMMAND ${CMAKE_COMMAND} -E copy_directory ${SRC_DIR} ${DST_DIR1} + COMMAND ${CMAKE_COMMAND} -E copy_directory ${SRC_DIR} ${DST_DIR2} COMMENT "copy_directory from ${SRC_DIR} to ${DST_DIR}") endif() endif(WITH_GPU) diff --git a/paddle/fluid/framework/CMakeLists.txt b/paddle/fluid/framework/CMakeLists.txt index c06260b72e6ee7..652ef95c8d9456 100644 --- a/paddle/fluid/framework/CMakeLists.txt +++ b/paddle/fluid/framework/CMakeLists.txt @@ -29,7 +29,7 @@ add_subdirectory(io) proto_library(framework_proto SRCS framework.proto) proto_library(op_def_proto SRCS op_def.proto) -cc_library(op_def_api SRCS op_def_api.cc DEPS op_def_proto) +cc_library(op_def_api SRCS op_def_api.cc DEPS op_def_proto boost) FILE(GLOB OP_DEF_FILES ${PADDLE_SOURCE_DIR}/paddle/fluid/operators/compat/*.pbtxt) FILE(WRITE ${CMAKE_CURRENT_BINARY_DIR}/op_def.pbtxt @@ -110,7 +110,7 @@ cc_test(reader_test SRCS reader_test.cc DEPS reader) cc_library(threadpool SRCS threadpool.cc DEPS enforce) cc_test(threadpool_test SRCS threadpool_test.cc DEPS threadpool) -cc_library(var_type_traits SRCS var_type_traits DEPS lod_tensor selected_rows framework_proto) +cc_library(var_type_traits SRCS var_type_traits.cc DEPS lod_tensor selected_rows framework_proto) if (WITH_GPU) target_link_libraries(var_type_traits dynload_cuda) endif() @@ -261,7 +261,7 @@ if(WITH_DISTRIBUTE) dist_multi_trainer.cc trainer_factory.cc trainer.cc data_feed_factory.cc heterxpu_trainer.cc data_feed.cc device_worker.cc hogwild_worker.cc hetercpu_worker.cc ps_gpu_worker.cc - heterbox_worker.cc heterbox_trainer.cc ps_gpu_trainer.cc downpour_worker.cc downpour_worker_opt.cc + ps_gpu_trainer.cc downpour_worker.cc downpour_worker_opt.cc pull_dense_worker.cc section_worker.cc device_worker_factory.cc data_set.cc DEPS op_registry device_context scope framework_proto trainer_desc_proto glog fs shell fleet_wrapper heter_wrapper ps_gpu_wrapper box_wrapper lodtensor_printer @@ -282,7 +282,7 @@ if(WITH_DISTRIBUTE) dist_multi_trainer.cc trainer_factory.cc trainer.cc data_feed_factory.cc heterxpu_trainer.cc data_feed.cc device_worker.cc hogwild_worker.cc hetercpu_worker.cc - heterbox_worker.cc heterbox_trainer.cc downpour_worker.cc downpour_worker_opt.cc + downpour_worker.cc downpour_worker_opt.cc pull_dense_worker.cc section_worker.cc device_worker_factory.cc data_set.cc DEPS op_registry device_context scope framework_proto data_feed_proto heter_service_proto trainer_desc_proto glog lod_rank_table fs shell fleet_wrapper heter_wrapper box_wrapper lodtensor_printer feed_fetch_method @@ -296,7 +296,7 @@ if(WITH_DISTRIBUTE) dist_multi_trainer.cc trainer_factory.cc trainer.cc data_feed_factory.cc heterxpu_trainer.cc data_feed.cc device_worker.cc hogwild_worker.cc hetercpu_worker.cc ps_gpu_worker.cc - heterbox_worker.cc heterbox_trainer.cc ps_gpu_trainer.cc downpour_worker.cc downpour_worker_opt.cc + ps_gpu_trainer.cc downpour_worker.cc downpour_worker_opt.cc pull_dense_worker.cc section_worker.cc device_worker_factory.cc data_set.cc DEPS op_registry device_context scope framework_proto data_feed_proto heter_service_proto trainer_desc_proto glog lod_rank_table fs shell fleet_wrapper heter_wrapper ps_gpu_wrapper box_wrapper lodtensor_printer feed_fetch_method @@ -316,7 +316,7 @@ elseif(WITH_PSLIB) dist_multi_trainer.cc trainer_factory.cc trainer.cc data_feed_factory.cc heterxpu_trainer.cc data_feed.cc device_worker.cc hogwild_worker.cc hetercpu_worker.cc ps_gpu_worker.cc - heterbox_worker.cc heterbox_trainer.cc ps_gpu_trainer.cc downpour_worker.cc downpour_worker_opt.cc + ps_gpu_trainer.cc downpour_worker.cc downpour_worker_opt.cc pull_dense_worker.cc section_worker.cc device_worker_factory.cc data_set.cc DEPS op_registry device_context scope framework_proto data_feed_proto heter_service_proto trainer_desc_proto glog lod_rank_table fs shell fleet_wrapper heter_wrapper ps_gpu_wrapper box_wrapper lodtensor_printer feed_fetch_method @@ -326,7 +326,7 @@ else() dist_multi_trainer.cc trainer_factory.cc trainer.cc data_feed_factory.cc heterxpu_trainer.cc data_feed.cc device_worker.cc hogwild_worker.cc hetercpu_worker.cc ps_gpu_worker.cc - heterbox_worker.cc heterbox_trainer.cc ps_gpu_trainer.cc downpour_worker.cc downpour_worker_opt.cc + ps_gpu_trainer.cc downpour_worker.cc downpour_worker_opt.cc pull_dense_worker.cc section_worker.cc device_worker_factory.cc data_set.cc DEPS op_registry device_context scope framework_proto data_feed_proto heter_service_proto trainer_desc_proto glog lod_rank_table fs shell fleet_wrapper heter_wrapper ps_gpu_wrapper box_wrapper lodtensor_printer feed_fetch_method @@ -370,10 +370,10 @@ endif (NOT WIN32) cc_library(dlpack_tensor SRCS dlpack_tensor.cc DEPS tensor dlpack) cc_test(dlpack_tensor_test SRCS dlpack_tensor_test.cc DEPS dlpack_tensor glog) -cc_library(op_compatible_info SRCS op_compatible_info DEPS string_helper proto_desc) +cc_library(op_compatible_info SRCS op_compatible_info.cc DEPS string_helper proto_desc) cc_test(op_compatible_info_test SRCS op_compatible_info_test.cc DEPS op_compatible_info proto_desc string_helper glog) -cc_library(save_load_util SRCS save_load_util DEPS tensor scope layer) +cc_library(save_load_util SRCS save_load_util.cc DEPS tensor scope layer) cc_test(save_load_util_test SRCS save_load_util_test.cc DEPS save_load_util tensor scope layer) cc_library(generator SRCS generator.cc DEPS enforce place) diff --git a/paddle/fluid/framework/attribute.h b/paddle/fluid/framework/attribute.h index 66b988ee1f1fb6..e9e18757656339 100644 --- a/paddle/fluid/framework/attribute.h +++ b/paddle/fluid/framework/attribute.h @@ -208,15 +208,27 @@ Attribute GetAttrValue(const proto::OpDesc::Attr& attr_desc); class AttrReader { public: - explicit AttrReader(const AttributeMap& attrs) : attrs_(attrs) {} + explicit AttrReader(const AttributeMap& attrs) + : attrs_(attrs), default_attrs_(nullptr) {} + + AttrReader(const AttributeMap& attrs, const AttributeMap& default_attrs) + : attrs_(attrs), default_attrs_(&default_attrs) {} template inline const T& Get(const std::string& name) const { - PADDLE_ENFORCE_NE(attrs_.count(name), 0, + auto it = attrs_.find(name); + bool found = it != attrs_.end(); + if (!found) { + if (default_attrs_ != nullptr) { + it = default_attrs_->find(name); + found = it != default_attrs_->end(); + } + } + PADDLE_ENFORCE_EQ(found, true, platform::errors::NotFound( "Attribute (%s) should be in AttributeMap.", name)); - Attribute& attr = const_cast(attrs_.at(name)); + Attribute& attr = const_cast(it->second); ExtractAttribute extract_attr(name); T* attr_value = extract_attr(attr); return *attr_value; @@ -224,6 +236,7 @@ class AttrReader { private: const AttributeMap& attrs_; + const AttributeMap* default_attrs_; }; // check whether a value(attribute) fit a certain limit @@ -234,8 +247,8 @@ class GreaterThanChecker { void operator()(const T& value) const { PADDLE_ENFORCE_GT( value, lower_bound_, - platform::errors::OutOfRange( - "Check for attribute value greater than a certain value failed.")); + platform::errors::OutOfRange("Check for attribute value greater than " + "a certain value failed.")); } private: @@ -332,9 +345,9 @@ class TypedAttrChecker { TypedAttrChecker& SetDefault(const T& default_value) { PADDLE_ENFORCE_EQ( default_value_setter_.empty(), true, - platform::errors::AlreadyExists( - "Attribute (%s) has a default value and cannot be set repeatedly.", - attr_name_)); + platform::errors::AlreadyExists("Attribute (%s) has a default value " + "and cannot be set repeatedly.", + attr_name_)); default_value_setter_.push_back(DefaultValueSetter(default_value)); return *this; } @@ -345,8 +358,8 @@ class TypedAttrChecker { return *this; } - void operator()(AttributeMap* attr_map, - bool get_default_value_only = false) const { + void operator()(AttributeMap* attr_map, bool get_default_value_only = false, + bool only_check_exist_value = false) const { if (get_default_value_only) { if (!default_value_setter_.empty()) { attr_map->emplace(attr_name_, default_value_setter_[0]()); @@ -354,21 +367,32 @@ class TypedAttrChecker { return; } - auto it = attr_map->find(attr_name_); - if (it == attr_map->end()) { - // user do not set this attr - PADDLE_ENFORCE_EQ( - default_value_setter_.empty(), false, - platform::errors::InvalidArgument( - "Attribute (%s) is not set correctly.", attr_name_)); - // default_value_setter_ has no more than one element - attr_map->emplace(attr_name_, default_value_setter_[0]()); - } - it = attr_map->find(attr_name_); - ExtractAttribute extract_attr(attr_name_); - T* attr_value = extract_attr(it->second); - for (const auto& checker : value_checkers_) { - checker(*attr_value); + if (only_check_exist_value) { + auto it = attr_map->find(attr_name_); + if (it != attr_map->end()) { + ExtractAttribute extract_attr(attr_name_); + T* attr_value = extract_attr(it->second); + for (const auto& checker : value_checkers_) { + checker(*attr_value); + } + } + } else { + auto it = attr_map->find(attr_name_); + if (it == attr_map->end()) { + // user do not set this attr + PADDLE_ENFORCE_EQ( + default_value_setter_.empty(), false, + platform::errors::InvalidArgument( + "Attribute (%s) is not set correctly.", attr_name_)); + // default_value_setter_ has no more than one element + auto tmp = attr_map->emplace(attr_name_, default_value_setter_[0]()); + it = tmp.first; + } + ExtractAttribute extract_attr(attr_name_); + T* attr_value = extract_attr(it->second); + for (const auto& checker : value_checkers_) { + checker(*attr_value); + } } } @@ -380,7 +404,7 @@ class TypedAttrChecker { // check whether op's all attributes fit their own limits class OpAttrChecker { - typedef std::function AttrChecker; + typedef std::function AttrChecker; public: template @@ -390,18 +414,19 @@ class OpAttrChecker { return *(checker.target>()); } - void Check(AttributeMap* attr_map, bool explicit_only = false) const { + void Check(AttributeMap* attr_map, bool explicit_only = false, + bool only_check_exist_value = false) const { auto checker_num = attr_checkers_.size(); if (explicit_only) checker_num = explicit_checker_num_; for (size_t i = 0; i < checker_num; ++i) { - attr_checkers_[i](attr_map, false); + attr_checkers_[i](attr_map, false, only_check_exist_value); } } - AttributeMap GetAttrsDefaultValuesMap() const { + AttributeMap GetDefaultAttrsMap() const { AttributeMap default_values_map; for (const auto& checker : attr_checkers_) { - checker(&default_values_map, true); + checker(&default_values_map, true, false); } return default_values_map; } @@ -410,15 +435,26 @@ class OpAttrChecker { explicit_checker_num_ = attr_checkers_.size(); } + void InitDefaultAttributeMap() { + for (const auto& checker : attr_checkers_) { + checker(&default_attrs_, true, false); + } + } + + const AttributeMap& GetDefaultAttrMap() const { return default_attrs_; } + private: std::vector attr_checkers_; + AttributeMap default_attrs_; + // in order to improve the efficiency of dynamic graph mode, // we divede the attribute into explicit type and implicit type. // for explicit attribute, we mean the attribute added in the customized // op makers, usually it's defined in the overloaded Make method. // for implicit attribute, we mean the attribute added outside of the Make - // method like "op_role", "op_role_var", and they are useless in dynamic graph + // method like "op_role", "op_role_var", and they are useless in dynamic + // graph // mode size_t explicit_checker_num_; }; diff --git a/paddle/fluid/framework/custom_operator.cc b/paddle/fluid/framework/custom_operator.cc index c4b833ec94c294..b1c5ff86d19790 100644 --- a/paddle/fluid/framework/custom_operator.cc +++ b/paddle/fluid/framework/custom_operator.cc @@ -781,10 +781,12 @@ void RegisterOperatorWithMetaInfo( const imperative::NameVarBaseMap& var_base_map_in, const imperative::NameVarBaseMap& var_base_map_out, const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs, const std::map& inplace_map) { CustomGradOpMaker maker( type, var_base_map_in, var_base_map_out, attrs, inplace_map, grad_op_name, grad_op_inputs, grad_op_outputs); + maker.SetDygraphDefaultAttrsMap(default_attrs); return maker(); }; diff --git a/paddle/fluid/framework/details/op_registry.h b/paddle/fluid/framework/details/op_registry.h index df5370e42ee9f3..27f55e237f5168 100644 --- a/paddle/fluid/framework/details/op_registry.h +++ b/paddle/fluid/framework/details/op_registry.h @@ -249,8 +249,10 @@ struct OpInfoFiller { const imperative::NameVarBaseMap& var_base_map_in, const imperative::NameVarBaseMap& var_base_map_out, const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs, const std::map& inplace_map) { T maker(type, var_base_map_in, var_base_map_out, attrs, inplace_map); + maker.SetDygraphDefaultAttrsMap(default_attrs); return maker(); }; } diff --git a/paddle/fluid/framework/device_worker.h b/paddle/fluid/framework/device_worker.h index db83cd55889c43..b40099542cfd5d 100644 --- a/paddle/fluid/framework/device_worker.h +++ b/paddle/fluid/framework/device_worker.h @@ -444,107 +444,6 @@ class HeterCpuWorker : public HogwildWorker { }; #endif -#if (defined PADDLE_WITH_CUDA || defined PADDLE_WITH_HIP || \ - defined PADDLE_WITH_XPU) && \ - (defined PADDLE_WITH_PSLIB) -class HeterBoxWorker : public HogwildWorker { - public: - HeterBoxWorker() {} - virtual ~HeterBoxWorker() {} - virtual void Initialize(const TrainerDesc& desc); - virtual void TrainFiles(); - virtual void SetNeedDump(bool need_dump_field); - virtual void SetChannelWriter(ChannelObject* queue); - virtual void SetWorkerNum(int num) { worker_num_ = num; } - virtual void CacheProgram(const ProgramDesc& main_program) { - new (&program_) ProgramDesc(main_program); - } - void ProduceTasks() override; - virtual void SetStream(const gpuStream_t stream) { copy_stream_ = stream; } - virtual void SetEvent(const gpuEvent_t event) { event_ = event; } - virtual void TrainFilesWithProfiler() {} - void ResetStat(); - - protected: - std::shared_ptr fleet_ptr_; - void FillSparseValue(std::shared_ptr task, size_t table_id); - void PushGradients(); - void CollectLabelInfo(std::shared_ptr task, size_t table_id); - void AdjustInsWeight(std::shared_ptr task); - void DumpParam(); - void CopySparseTable(); - void CopyDenseTable(); - void CopyDenseVars(); - - private: - int mpi_rank_; - std::mutex mutex_; - std::vector send_var_list_; - int worker_num_; - ProgramDesc program_; - HeterObjectPool object_pool_; - bool need_dump_param_; - std::vector dump_param_; - bool need_to_push_dense_; - bool need_dump_field_; - bool dump_slot_; - bool need_to_push_sparse_; - std::vector dump_fields_; - ChannelWriter writer_; - DownpourWorkerParameter param_; - float scale_datanorm_; - // just save the value in param_ for easy access - std::map label_var_name_; - std::map> sparse_key_names_; - std::map> sparse_value_names_; - std::map> sparse_grad_names_; - std::map> dense_value_names_; - std::map> dense_grad_names_; - platform::Place root_place_; - // actually pushed feasign of each table - std::map> sparse_push_keys_; - - // skipped ops - std::vector skip_ops_; - - std::vector<::std::future> push_sparse_status_; - std::vector<::std::future> push_dense_status_; - - // adjust ins weight - AdjustInsWeightConfig adjust_ins_weight_config_; - std::vector nid_show_; - // check nan and inf during training - std::vector check_nan_var_names_; - // copy table - CopyTableConfig copy_table_config_; - std::map table_dependency_; - std::vector> copy_sparse_tables_; - std::vector> copy_dense_tables_; - std::unordered_map> feasign_set_; - paddle::framework::Channel> pull_queue_; - paddle::framework::Channel> push_queue_; - gpuEvent_t event_; - gpuStream_t copy_stream_; - int batch_cnt_{0}; - std::atomic done_cnt_{0}; - - double total_time_; - double read_time_; - double pack_time_; - double pull_sparse_local_time_; - double op_all_time_; - double xpu_op_time_; - double xpu_wait_time_; - double cpu_op_time_; - double collect_label_time_; - double fill_sparse_time_; - double push_sparse_time_; - double gpu_2_cpu_time_; - double cpu_2_gpu_time_; - uint64_t total_inst_; -}; -#endif - #if (defined PADDLE_WITH_NCCL || defined PADDLE_WITH_RCCL) && \ (defined PADDLE_WITH_PSLIB) class PSGPUWorker : public HogwildWorker { diff --git a/paddle/fluid/framework/distributed_strategy.proto b/paddle/fluid/framework/distributed_strategy.proto index a63dfd7b091f7e..29eef3eabc6e00 100644 --- a/paddle/fluid/framework/distributed_strategy.proto +++ b/paddle/fluid/framework/distributed_strategy.proto @@ -47,6 +47,7 @@ message HybridConfig { optional int32 dp_degree = 1 [ default = -1 ]; optional int32 mp_degree = 2 [ default = 1 ]; optional int32 pp_degree = 3 [ default = 1 ]; + optional int32 sharding_degree = 4 [ default = 1 ]; } message AMPConfig { @@ -118,6 +119,16 @@ message ExecutionStrategy { optional bool use_thread_barrier = 4 [ default = false ]; } +message GradientScaleConfig { + // Optional value ['avg', 'sum', 'customized'] + // If avg, loss@grad will be divided by the number of devices, + // that is, the gradient will be accumulated and averaged among + // multiple devices. + // Else if sum, the gradient will accumulated among multiple + // devices. + optional string scale_strategy = 1 [ default = 'avg' ]; +} + message AsyncConfig { optional int32 k_steps = 1 [ default = -1 ]; optional int32 max_merge_var_num = 2 [ default = 1 ]; @@ -194,6 +205,7 @@ message DistributedStrategy { optional TensorParallelConfig tensor_parallel_configs = 113; optional BuildStrategy build_strategy = 201; optional ExecutionStrategy execution_strategy = 202; + optional GradientScaleConfig gradient_scale_configs = 203; } message DistributedJobInfo { diff --git a/paddle/fluid/framework/fleet/fleet_wrapper.cc b/paddle/fluid/framework/fleet/fleet_wrapper.cc index 3cd8b55026e518..dfe94cf1eb39ae 100644 --- a/paddle/fluid/framework/fleet/fleet_wrapper.cc +++ b/paddle/fluid/framework/fleet/fleet_wrapper.cc @@ -551,16 +551,36 @@ void FleetWrapper::PullSparseVarsSync( for (auto& t : *fea_values) { pull_result_ptr.push_back(t.data()); } - auto status = pslib_ptr_->_worker_ptr->pull_sparse( - pull_result_ptr.data(), table_id, fea_keys->data(), fea_keys->size()); - pull_sparse_status.push_back(std::move(status)); - for (auto& t : pull_sparse_status) { - t.wait(); - auto status = t.get(); - if (status != 0) { - LOG(ERROR) << "fleet pull sparse failed, status[" << status << "]"; - sleep(sleep_seconds_before_fail_exit_); - exit(-1); + + int32_t cnt = 0; + while (true) { + pull_sparse_status.clear(); + auto status = pslib_ptr_->_worker_ptr->pull_sparse( + pull_result_ptr.data(), table_id, fea_keys->data(), fea_keys->size()); + pull_sparse_status.push_back(std::move(status)); + bool flag = true; + for (auto& t : pull_sparse_status) { + t.wait(); + int32_t status = -1; + try { + status = t.get(); + } catch (const std::future_error& e) { + VLOG(0) << "Caught a future_error with code" << e.code() + << ", Message:" << e.what(); + } + if (status != 0) { + VLOG(0) << "fleet pull sparse failed, status[" << status << "]"; + sleep(sleep_seconds_before_fail_exit_); + flag = false; + cnt++; + } + if (cnt > 3) { + VLOG(0) << "fleet pull sparse failed, retry 3 times"; + exit(-1); + } + } + if (flag) { + break; } } #endif diff --git a/paddle/fluid/framework/fleet/ps_gpu_wrapper.h b/paddle/fluid/framework/fleet/ps_gpu_wrapper.h index cfb23d1be2acfe..81b2b0a12b2c37 100644 --- a/paddle/fluid/framework/fleet/ps_gpu_wrapper.h +++ b/paddle/fluid/framework/fleet/ps_gpu_wrapper.h @@ -209,6 +209,15 @@ class PSGPUWrapper { void EndPass() { HeterPs_->end_pass(); } void ShowOneTable(int index) { HeterPs_->show_one_table(index); } + void Finalize() { + VLOG(3) << "PSGPUWrapper Begin Finalize."; + if (s_instance_ == nullptr) { + return; + } + s_instance_ = nullptr; + VLOG(3) << "PSGPUWrapper Finalize Finished."; + } + private: static std::shared_ptr s_instance_; Dataset* dataset_; diff --git a/paddle/fluid/framework/grad_op_desc_maker.h b/paddle/fluid/framework/grad_op_desc_maker.h index b0247fe795b3ea..ebbfd446a03de2 100644 --- a/paddle/fluid/framework/grad_op_desc_maker.h +++ b/paddle/fluid/framework/grad_op_desc_maker.h @@ -219,6 +219,19 @@ class SingleGradOpMaker public: using GradOpBaseMakerBase::GradOpBaseMakerBase; + virtual const framework::Attribute& GetAttr(const std::string& name) const { + auto it = Attrs().find(name); + if (it == Attrs().end()) { + it = this->DefaultAttrsMap().find(name); + PADDLE_ENFORCE_EQ(it != this->DefaultAttrsMap().end(), true, + platform::errors::NotFound( + "Cannot find attribute [%s] in operator [%s]", name, + this->ForwardOpType())); + } + + return it->second; + } + std::shared_ptr operator()() const final { auto node = this->NewGradNode(); auto& inplace_map = this->GetInplaceMap(); @@ -228,6 +241,7 @@ class SingleGradOpMaker { imperative::TracedGradOp traced_grad_op(node); try { + traced_grad_op.SetDefaultAttrsMap(this->DefaultAttrsMap()); this->Apply(&traced_grad_op); } catch (platform::EnforceNotMet& exception) { framework::AppendErrorOpHint(traced_grad_op.Type(), &exception); diff --git a/paddle/fluid/framework/heterbox_trainer.cc b/paddle/fluid/framework/heterbox_trainer.cc deleted file mode 100644 index 1f6dc39ae851df..00000000000000 --- a/paddle/fluid/framework/heterbox_trainer.cc +++ /dev/null @@ -1,275 +0,0 @@ -/* Copyright (c) 2016 PaddlePaddle Authors. All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - -http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. */ - -#include -#include -#include -#include "io/fs.h" -#include "paddle/fluid/framework/data_feed_factory.h" -#include "paddle/fluid/framework/data_set.h" -#include "paddle/fluid/framework/device_worker_factory.h" -#include "paddle/fluid/framework/fleet/fleet_wrapper.h" -#include "paddle/fluid/framework/trainer.h" -#if (defined PADDLE_WITH_CUDA || defined PADDLE_WITH_HIP || \ - defined PADDLE_WITH_XPU) && \ - (defined PADDLE_WITH_PSLIB) -#if defined(PADDLE_WITH_CUDA) || defined(PADDLE_WITH_HIP) -#include "paddle/fluid/platform/cuda_device_guard.h" -#endif -namespace paddle { -namespace framework { - -void HeterBoxTrainer::Initialize(const TrainerDesc& trainer_desc, - Dataset* dataset) { - thread_num_ = trainer_desc.thread_num(); - param_ = trainer_desc.downpour_param(); - for (int i = 0; i < param_.dense_table_size(); ++i) { - uint64_t table_id = static_cast(param_.dense_table(i).table_id()); - auto table = param_.dense_table(i); - dense_grad_names_[table_id].resize(table.dense_grad_name_size()); - for (int j = 0; j < table.dense_grad_name_size(); ++j) { - dense_grad_names_[table_id][j] = table.dense_grad_name(j); - } - } - RegisterHeterCallback(); - scale_datanorm_ = trainer_desc.scale_datanorm(); - int place_num = trainer_desc.worker_places_size(); - const std::vector readers = - dataset->GetReaders(); - for (int i = 0; i < place_num; ++i) { - int num = trainer_desc.worker_places(i); -#if defined(PADDLE_WITH_CUDA) || defined(PADDLE_WITH_HIP) - platform::CUDAPlace place = platform::CUDAPlace(num); - platform::CUDADeviceGuard guard(place.device); - gpuStream_t stream; -#ifdef PADDLE_WITH_HIP - PADDLE_ENFORCE_CUDA_SUCCESS(hipStreamCreate(&stream)); -#else - PADDLE_ENFORCE_CUDA_SUCCESS(cudaStreamCreate(&stream)); -#endif - copy_streams_.push_back(stream); - places_.push_back(place); - gpuEvent_t event; -#ifdef PADDLE_WITH_HIP - PADDLE_ENFORCE_CUDA_SUCCESS( - hipEventCreateWithFlags(&event, hipEventDisableTiming)); -#else - PADDLE_ENFORCE_CUDA_SUCCESS( - cudaEventCreateWithFlags(&event, cudaEventDisableTiming)); -#endif - events_.push_back(event); -#endif -#ifdef PADDLE_WITH_XPU - platform::XPUPlace place = platform::XPUPlace(num); - places_.push_back(place); -#endif - } - for (int i = 0; i < trainer_desc.downpour_param().stat_var_names_size(); - i++) { - need_merge_var_names_.push_back( - trainer_desc.downpour_param().stat_var_names(i)); - } - VLOG(3) << "going to initialize pull dense worker"; - pull_dense_worker_ = PullDenseWorker::GetInstance(); - pull_dense_worker_->Initialize(trainer_desc); - VLOG(3) << "initialize pull dense worker"; - SetDebug(trainer_desc.debug()); - fleet_ptr_ = FleetWrapper::GetInstance(); - trainer_desc_ = trainer_desc; - workers_.resize(place_num); - for (int i = 0; i < place_num; ++i) { - workers_[i] = DeviceWorkerFactory::CreateDeviceWorker( - trainer_desc.device_worker_name()); - workers_[i]->SetDeviceIndex(i); - workers_[i]->SetDataFeed(readers[i]); - workers_[i]->Initialize(trainer_desc); - workers_[i]->SetWorkerNum(place_num); - } -} - -void HeterBoxTrainer::DumpWork(int tid) {} - -void HeterBoxTrainer::RegisterHeterCallback() { - auto fleet_ptr = FleetWrapper::GetInstance(); - fleet_ptr->RegisterHeterCallback([this](int worker, int taskid) { - // workers_[worker]->Schedule(taskid); - }); -} - -void HeterBoxTrainer::InitTrainerEnv(const ProgramDesc& main_program, - const platform::Place& place) { - for (size_t i = 0; i < places_.size(); ++i) { - workers_[i]->SetPlace(places_[i]); - workers_[i]->SetStream(copy_streams_[i]); - workers_[i]->SetEvent(events_[i]); - workers_[i]->SetReaderPlace(platform::CPUPlace()); - workers_[i]->SetRootScope(root_scope_); - workers_[i]->CreateDeviceResource(main_program); // Program - workers_[i]->BindingDataFeedMemory(); -#ifdef PADDLE_WITH_PSLIB - workers_[i]->CacheProgram(main_program); -#endif - } - for (size_t num = 0; num < places_.size(); ++num) { - auto place = places_[num]; - Scope* scope = workers_[num]->GetThreadScope(); - auto stream = copy_streams_[num]; - auto event = events_[num]; - auto dev_id = BOOST_GET_CONST(platform::CUDAPlace, place).device; - platform::CUDADeviceGuard guard(dev_id); - auto& block = main_program.Block(0); - for (auto& var : block.AllVars()) { - if (var->Persistable()) { - auto name = var->Name(); - Variable* root_var = root_scope_->FindVar(name); - if (!root_var) { - continue; - } - LoDTensor* root_tensor = root_var->GetMutable(); - auto* ptr = scope->Var(name); - InitializeVariable(ptr, proto::VarType::LOD_TENSOR); - LoDTensor* thread_tensor = ptr->GetMutable(); - -#define HeterMemcpyFunc(cpp_type, proto_type) \ - do { \ - if (root_tensor->type() == proto_type) { \ - HeterMemCpy(thread_tensor, root_tensor, place, stream); \ - } \ - } while (0) - _ForEachDataType_(HeterMemcpyFunc); - } - } -#ifdef PADDLE_WITH_HIP - PADDLE_ENFORCE_CUDA_SUCCESS(hipEventRecord(event, stream)); - hipEventSynchronize(event); -#else - PADDLE_ENFORCE_CUDA_SUCCESS(cudaEventRecord(event, stream)); - cudaEventSynchronize(event); -#endif - } - place_ = place; -} - -template -void HeterBoxTrainer::HeterMemCpy(LoDTensor* thread_tensor, - LoDTensor* root_tensor, - const paddle::platform::Place& thread_place, - gpuStream_t stream) { - T* thread_ptr = - thread_tensor->mutable_data(root_tensor->dims(), thread_place); - T* root_ptr = root_tensor->data(); - if (platform::is_cpu_place(root_tensor->place())) { - memory::Copy(BOOST_GET_CONST(platform::CUDAPlace, thread_place), thread_ptr, - platform::CPUPlace(), root_ptr, - sizeof(T) * root_tensor->numel(), stream); - } else { - memory::Copy(BOOST_GET_CONST(platform::CUDAPlace, thread_place), thread_ptr, - BOOST_GET_CONST(platform::CUDAPlace, root_tensor->place()), - root_ptr, sizeof(T) * root_tensor->numel(), stream); - } -} - -void HeterBoxTrainer::InitOtherEnv(const ProgramDesc& main_program) { - pull_dense_worker_->SetRootScope(root_scope_); - pull_dense_worker_->CreatePinVar(); - for (size_t i = 0; i < places_.size(); ++i) { - pull_dense_worker_->AddThreadScope(workers_[i]->GetThreadScope()); - pull_dense_worker_->AddPlace(places_[i]); -#if defined(PADDLE_WITH_CUDA) || defined(PADDLE_WITH_HIP) - pull_dense_worker_->AddStream(copy_streams_[i]); -#endif - } - VLOG(3) << "init other env done."; -} - -void HeterBoxTrainer::Run() { - int pull_thread_num = 3 * places_.size(); - for (size_t thidx = 0; thidx < places_.size(); ++thidx) { - workers_[thidx]->device_reader_->Start(); - std::dynamic_pointer_cast( - workers_[thidx]) - ->ResetStat(); - } - for (int i = 0; i < pull_thread_num; ++i) { - int worker_id = i % places_.size(); - pull_threads_.push_back( - std::thread(&DeviceWorker::ProduceTasks, workers_[worker_id].get())); - } - for (size_t thidx = 0; thidx < places_.size(); ++thidx) { - threads_.push_back( - std::thread(&DeviceWorker::TrainFiles, workers_[thidx].get())); - } -} - -template -void HeterBoxTrainer::MergeToRootScope(LoDTensor* root_tensor, - LoDTensor* tensor) { - LoDTensor tmp_root; - TensorCopy(*root_tensor, platform::CPUPlace(), &tmp_root); - T* tmp_root_data = tmp_root.data(); - LoDTensor tmp_tensor; - TensorCopy(*tensor, platform::CPUPlace(), &tmp_tensor); - T* data = tmp_tensor.data(); - for (int i = 0; i < tmp_tensor.numel(); i++) { - tmp_root_data[i] += data[i]; - } - TensorCopy(tmp_root, platform::CPUPlace(), root_tensor); -} - -Scope* HeterBoxTrainer::GetWorkerScope(int thread_id) { return nullptr; } - -void HeterBoxTrainer::Finalize() { - for (auto& th : pull_threads_) { - th.join(); - } - for (auto& th : threads_) { - th.join(); - } - for (size_t i = 0; i < need_merge_var_names_.size(); i++) { - Variable* root_var = root_scope_->FindVar(need_merge_var_names_[i]); - if (root_var == nullptr) { - continue; - } - LoDTensor* root_tensor = root_var->GetMutable(); - - for (size_t j = 0; j < places_.size(); j++) { - Scope* cur_thread_scope = workers_[j]->GetThreadScope(); - Variable* thread_var = - cur_thread_scope->FindVar(need_merge_var_names_[i]); - if (thread_var == nullptr) { - continue; - } - LoDTensor* thread_tensor = thread_var->GetMutable(); -#define MergeCallback(cpp_type, proto_type) \ - do { \ - if (root_tensor->type() == proto_type) { \ - if (thread_tensor->type() != proto_type) { \ - VLOG(0) << "Error: thread id=" << j << ", need_merge_var_names_[" << i \ - << "] " << need_merge_var_names_[i] \ - << ", root tensor type=" << root_tensor->type() \ - << ", thread tensor type=" << thread_tensor->type(); \ - exit(-1); \ - } \ - MergeToRootScope(root_tensor, thread_tensor); \ - } \ - } while (0) - _ForEachDataType_(MergeCallback); - } - } - pull_dense_worker_->MergeDenseParam(); - root_scope_->DropKids(); -} -} // namespace framework -} // namespace paddle -#endif diff --git a/paddle/fluid/framework/heterbox_worker.cc b/paddle/fluid/framework/heterbox_worker.cc deleted file mode 100644 index b7df88218cbd4d..00000000000000 --- a/paddle/fluid/framework/heterbox_worker.cc +++ /dev/null @@ -1,753 +0,0 @@ -/* Copyright (c) 2019 PaddlePaddle Authors. All Rights Reserved. - -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. */ - -#include "paddle/fluid/framework/device_worker.h" -#include "paddle/fluid/framework/device_worker_factory.h" -#include "paddle/fluid/framework/fleet/fleet_wrapper.h" -#include "paddle/fluid/framework/heter_util.h" -#include "paddle/fluid/platform/cpu_helper.h" -#include "paddle/fluid/string/string_helper.h" - -#if (defined PADDLE_WITH_CUDA || defined PADDLE_WITH_XPU) && \ - (defined PADDLE_WITH_PSLIB) -#include "paddle/fluid/platform/cuda_device_guard.h" - -#if defined _WIN32 || defined __APPLE__ -#else -#define _LINUX -#endif - -namespace paddle { -namespace framework { - -void HeterBoxWorker::Initialize(const TrainerDesc& desc) { - param_ = desc.downpour_param(); - mpi_rank_ = desc.mpi_rank(); - trainer_desc_ = desc; - for (int i = 0; i < trainer_desc_.xpu_recv_list_size(); ++i) { - send_var_list_.push_back(trainer_desc_.xpu_recv_list(i)); - } - for (int i = 0; i < param_.sparse_table_size(); ++i) { - uint64_t table_id = - static_cast(param_.sparse_table(i).table_id()); - TableParameter table = param_.sparse_table(i); - sparse_key_names_[table_id].resize(table.sparse_key_name_size()); - for (int j = 0; j < table.sparse_key_name_size(); ++j) { - sparse_key_names_[table_id][j] = table.sparse_key_name(j); - } - sparse_value_names_[table_id].resize(table.sparse_value_name_size()); - for (int j = 0; j < table.sparse_value_name_size(); ++j) { - sparse_value_names_[table_id][j] = table.sparse_value_name(j); - } - sparse_grad_names_[table_id].resize(table.sparse_grad_name_size()); - for (int j = 0; j < table.sparse_grad_name_size(); ++j) { - sparse_grad_names_[table_id][j] = table.sparse_grad_name(j); - } - label_var_name_[table_id] = table.label_var_name(); - sparse_push_keys_[table_id] = std::vector(); - } - - for (int i = 0; i < param_.dense_table_size(); ++i) { - uint64_t table_id = static_cast(param_.dense_table(i).table_id()); - auto table = param_.dense_table(i); - dense_value_names_[table_id].resize(table.dense_value_name_size()); - for (int j = 0; j < table.dense_value_name_size(); ++j) { - dense_value_names_[table_id][j] = table.dense_value_name(j); - } - dense_grad_names_[table_id].resize(table.dense_grad_name_size()); - for (int j = 0; j < table.dense_grad_name_size(); ++j) { - dense_grad_names_[table_id][j] = table.dense_grad_name(j); - } - } - - skip_ops_.resize(param_.skip_ops_size()); - for (int i = 0; i < param_.skip_ops_size(); ++i) { - skip_ops_[i] = param_.skip_ops(i); - } - for (int i = 0; i < param_.stat_var_names_size(); ++i) { - stat_var_name_map_[param_.stat_var_names(i)] = 1; - } - - need_to_push_sparse_ = param_.push_sparse(); - need_to_push_dense_ = param_.push_dense(); - - fleet_ptr_ = FleetWrapper::GetInstance(); - fetch_config_ = desc.fetch_config(); - use_cvm_ = desc.use_cvm(); - // for sparse value accessor, embedding only - no_cvm_ = desc.no_cvm(); - scale_datanorm_ = desc.scale_datanorm(); - dump_slot_ = desc.dump_slot(); - dump_fields_.resize(desc.dump_fields_size()); - for (int i = 0; i < desc.dump_fields_size(); ++i) { - dump_fields_[i] = desc.dump_fields(i); - } - adjust_ins_weight_config_ = desc.adjust_ins_weight_config(); - need_dump_param_ = false; - dump_param_.resize(desc.dump_param_size()); - for (int i = 0; i < desc.dump_param_size(); ++i) { - dump_param_[i] = desc.dump_param(i); - } - if (desc.dump_param_size() != 0) { - need_dump_param_ = true; - } - for (int i = 0; i < desc.check_nan_var_names_size(); ++i) { - check_nan_var_names_.push_back(desc.check_nan_var_names(i)); - } - copy_table_config_ = desc.copy_table_config(); - for (int i = 0; i < copy_table_config_.src_sparse_tables_size(); ++i) { - uint64_t src_table = copy_table_config_.src_sparse_tables(i); - uint64_t dest_table = copy_table_config_.dest_sparse_tables(i); - VLOG(3) << "copy_sparse_tables_ push back " << src_table << "->" - << dest_table; - copy_sparse_tables_.push_back(std::make_pair(src_table, dest_table)); - } - for (int i = 0; i < copy_table_config_.src_dense_tables_size(); ++i) { - uint64_t src_table = copy_table_config_.src_dense_tables(i); - uint64_t dest_table = copy_table_config_.dest_dense_tables(i); - VLOG(3) << "copy_dense_tables_ push back " << src_table << "->" - << dest_table; - copy_dense_tables_.push_back(std::make_pair(src_table, dest_table)); - } - for (auto& m : copy_table_config_.table_denpendency_map()) { - if (sparse_key_names_.find(m.key()) != sparse_key_names_.end()) { - // currently only support one dependency - for (auto& value : m.values()) { - table_dependency_[m.key()] = value; - } - } - } - pull_queue_ = paddle::framework::MakeChannel>(); - push_queue_ = paddle::framework::MakeChannel>(); -} - -void HeterBoxWorker::SetChannelWriter(ChannelObject* queue) { - writer_.Reset(queue); -} - -void HeterBoxWorker::SetNeedDump(bool need_dump_field) { - need_dump_field_ = need_dump_field; -} - -void HeterBoxWorker::DumpParam() {} - -void HeterBoxWorker::CollectLabelInfo(std::shared_ptr task, - size_t table_idx) { - if (no_cvm_) { - return; - } - uint64_t table_id = static_cast( - param_.program_config(0).pull_sparse_table_id(table_idx)); - - TableParameter table; - for (auto i : param_.sparse_table()) { - if (i.table_id() == table_id) { - table = i; - break; - } - } - auto& feature = (task->features_)[table_id]; - auto& feature_label = (task->feature_labels_)[table_id]; - Scope* scope = task->scope_; - feature_label.resize(feature.size()); - Variable* var = scope->FindVar(label_var_name_[table_id]); - LoDTensor* tensor = var->GetMutable(); - int64_t* label_ptr = tensor->data(); - - size_t global_index = 0; - for (size_t i = 0; i < sparse_key_names_[table_id].size(); ++i) { - VLOG(3) << "sparse_key_names_[" << i - << "]: " << sparse_key_names_[table_id][i]; - Variable* fea_var = scope->FindVar(sparse_key_names_[table_id][i]); - if (fea_var == nullptr) { - continue; - } - LoDTensor* tensor = fea_var->GetMutable(); - CHECK(tensor != nullptr) << "tensor of var " - << sparse_key_names_[table_id][i] << " is null"; - - // skip slots which do not have embedding - Variable* emb_var = scope->FindVar(sparse_value_names_[table_id][i]); - if (emb_var == nullptr) { - continue; - } - int64_t* ids = tensor->data(); - size_t fea_idx = 0; - // tensor->lod()[0].size() == batch_size + 1 - for (auto lod_idx = 1u; lod_idx < tensor->lod()[0].size(); ++lod_idx) { - for (; fea_idx < tensor->lod()[0][lod_idx]; ++fea_idx) { - // should be skipped feasign defined in protobuf - if (ids[fea_idx] == 0u) { - continue; - } - feature_label[global_index++] = - static_cast(label_ptr[lod_idx - 1]); - } - } - } - CHECK(global_index == feature.size()) - << "expect fea info size:" << feature.size() << " real:" << global_index; -} - -void HeterBoxWorker::FillSparseValue(std::shared_ptr task, - size_t table_idx) { - uint64_t table_id = static_cast( - param_.program_config(0).pull_sparse_table_id(table_idx)); - - TableParameter table; - for (auto i : param_.sparse_table()) { - if (i.table_id() == table_id) { - table = i; - break; - } - } - - auto& fea_value = (task->feature_values_)[table_id]; - Scope* scope = task->scope_; - auto fea_idx = 0u; - - std::vector init_value(table.fea_dim()); - for (size_t i = 0; i < sparse_key_names_[table_id].size(); ++i) { - std::string slot_name = sparse_key_names_[table_id][i]; - std::string emb_slot_name = sparse_value_names_[table_id][i]; - Variable* var = scope->FindVar(slot_name); - if (var == nullptr) { - continue; - } - LoDTensor* tensor = var->GetMutable(); - CHECK(tensor != nullptr) << "tensor of var " << slot_name << " is null"; - int64_t* ids = tensor->data(); - int len = tensor->numel(); - Variable* var_emb = scope->FindVar(emb_slot_name); - if (var_emb == nullptr) { - continue; - } - LoDTensor* tensor_emb = var_emb->GetMutable(); - float* ptr = tensor_emb->mutable_data({len, table.emb_dim()}, - platform::CPUPlace()); - // memset(ptr, 0, sizeof(float) * len * table.emb_dim()); - auto& tensor_lod = tensor->lod()[0]; - LoD data_lod{tensor_lod}; - tensor_emb->set_lod(data_lod); - - bool is_nid = (adjust_ins_weight_config_.need_adjust() && - adjust_ins_weight_config_.nid_slot() == emb_slot_name); - if (is_nid) { - nid_show_.clear(); - } - int nid_ins_index = 0; - - for (int index = 0; index < len; ++index) { - if (use_cvm_ || no_cvm_) { - if (ids[index] == 0u) { - memcpy(ptr + table.emb_dim() * index, init_value.data(), - sizeof(float) * table.emb_dim()); - if (is_nid) { - nid_show_.push_back(-1); - ++nid_ins_index; - } - continue; - } - memcpy(ptr + table.emb_dim() * index, fea_value[fea_idx].data(), - sizeof(float) * table.emb_dim()); - if (is_nid && - static_cast(index) == tensor->lod()[0][nid_ins_index]) { - nid_show_.push_back(fea_value[fea_idx][0]); - ++nid_ins_index; - } - fea_idx++; - } else { - if (ids[index] == 0u) { - memcpy(ptr + table.emb_dim() * index, init_value.data() + 2, - sizeof(float) * table.emb_dim()); - if (is_nid) { - nid_show_.push_back(-1); - ++nid_ins_index; - } - continue; - } - memcpy(ptr + table.emb_dim() * index, fea_value[fea_idx].data() + 2, - sizeof(float) * table.emb_dim()); - if (is_nid && - static_cast(index) == tensor->lod()[0][nid_ins_index]) { - nid_show_.push_back(fea_value[fea_idx][0]); - ++nid_ins_index; - } - fea_idx++; - } - } - } -} - -void HeterBoxWorker::AdjustInsWeight(std::shared_ptr task) { -#ifdef _LINUX - // check var and tensor not null - Scope* scope = task->scope_; - if (!adjust_ins_weight_config_.need_adjust()) { - VLOG(0) << "need_adjust=false, skip adjust ins weight"; - return; - } - Variable* nid_var = scope->FindVar(adjust_ins_weight_config_.nid_slot()); - if (nid_var == nullptr) { - VLOG(0) << "nid slot var " << adjust_ins_weight_config_.nid_slot() - << " is nullptr, skip adjust ins weight"; - return; - } - LoDTensor* nid_tensor = nid_var->GetMutable(); - if (nid_tensor == nullptr) { - VLOG(0) << "tensor of nid slot var " << adjust_ins_weight_config_.nid_slot() - << " is nullptr, skip adjust ins weight"; - return; - } - Variable* ins_weight_var = - scope->FindVar(adjust_ins_weight_config_.ins_weight_slot()); - if (ins_weight_var == nullptr) { - VLOG(0) << "ins weight var " << adjust_ins_weight_config_.ins_weight_slot() - << " is nullptr, skip adjust ins weight"; - return; - } - LoDTensor* ins_weight_tensor = ins_weight_var->GetMutable(); - if (ins_weight_tensor == nullptr) { - VLOG(0) << "tensor of ins weight tensor " - << adjust_ins_weight_config_.ins_weight_slot() - << " is nullptr, skip adjust ins weight"; - return; - } - - float* ins_weights = ins_weight_tensor->data(); - size_t len = ins_weight_tensor->numel(); // len = batch size - // here we assume nid_show slot only has one feasign in each instance - CHECK(len == nid_show_.size()) << "ins_weight size should be equal to " - << "nid_show size, " << len << " vs " - << nid_show_.size(); - float nid_adjw_threshold = adjust_ins_weight_config_.nid_adjw_threshold(); - float nid_adjw_ratio = adjust_ins_weight_config_.nid_adjw_ratio(); - int64_t nid_adjw_num = 0; - double nid_adjw_weight = 0.0; - size_t ins_index = 0; - for (size_t i = 0; i < len; ++i) { - float nid_show = nid_show_[i]; - VLOG(3) << "nid_show " << nid_show; - if (nid_show < 0) { - VLOG(3) << "nid_show < 0, continue"; - continue; - } - float ins_weight = 1.0; - if (nid_show >= 0 && nid_show < nid_adjw_threshold) { - ins_weight = log(M_E + - (nid_adjw_threshold - nid_show) / nid_adjw_threshold * - nid_adjw_ratio); - // count nid adjw insnum and weight - ++nid_adjw_num; - nid_adjw_weight += ins_weight; - // choose large ins weight - VLOG(3) << "ins weight new " << ins_weight << ", ins weight origin " - << ins_weights[ins_index]; - if (ins_weight > ins_weights[ins_index]) { - VLOG(3) << "ins " << ins_index << " weight changes to " << ins_weight; - ins_weights[ins_index] = ins_weight; - } - ++ins_index; - } - } - VLOG(3) << "nid adjw info: total_adjw_num: " << nid_adjw_num - << ", avg_adjw_weight: " << nid_adjw_weight; -#endif -} - -void HeterBoxWorker::TrainFiles() { - VLOG(3) << "Begin to train files"; - platform::SetNumThreads(1); - need_to_push_dense_ = false; - while (1) { - VLOG(3) << "before heter task"; - std::shared_ptr task; - - if (!pull_queue_->Get(task)) { - VLOG(3) << "get task"; - break; - } - VLOG(3) << "get task done"; - Scope* scope = task->scope_->kids().front(); - VLOG(3) << "get kid done"; - // do computation here - task->timeline.Start(); - for (auto& op : ops_) { - if (op->HasAttr("op_device")) { - auto device = op->Attr("op_device"); - if (device != "gpu") { - continue; - } - } - bool need_skip = false; - for (auto t = 0u; t < skip_ops_.size(); ++t) { - if (op->Type().find(skip_ops_[t]) != std::string::npos) { - need_skip = true; - break; - } - } - if (!need_skip) { - op->Run(*(scope), place_); - } - } - platform::DeviceContextPool::Instance().Get(place_)->Wait(); - task->timeline.Pause(); - task->xpu_op_time += task->timeline.ElapsedSec(); - task->total_time += task->timeline.ElapsedSec(); - push_queue_->Put(task); - } -} - -void HeterTask::PackGpuTask(Scope* thread_scope, DataFeed* reader, - const ProgramDesc& program) { - auto& block = program.Block(0); - if (!scope_) { - scope_ = &(thread_scope->NewScope()); - for (auto& var : block.AllVars()) { - if (!var->Persistable()) { - auto* ptr = scope_->Var(var->Name()); - InitializeVariable(ptr, var->GetType()); - } - } - } - reader->AssignFeedVar(*scope_); - cur_batch_ = reader->Next(); -} - -void HeterBoxWorker::ResetStat() { - total_time_ = 0; - read_time_ = 0; - pack_time_ = 0; - pull_sparse_local_time_ = 0; - op_all_time_ = 0; - xpu_op_time_ = 0; - xpu_wait_time_ = 0; - cpu_op_time_ = 0; - collect_label_time_ = 0; - fill_sparse_time_ = 0; - push_sparse_time_ = 0; - gpu_2_cpu_time_ = 0; - cpu_2_gpu_time_ = 0; - total_inst_ = 0; -} - -void HeterBoxWorker::ProduceTasks() { - need_to_push_dense_ = false; - while (1) { - std::shared_ptr task; - task = object_pool_.Get(); - task->Reset(); - { - std::lock_guard lock(mutex_); - task->timeline.Start(); - task->PackGpuTask(thread_scope_, device_reader_, program_); - task->timeline.Pause(); - task->pack_time = task->timeline.ElapsedSec(); - task->total_time += task->pack_time; - if (task->cur_batch_ <= 0) { - if (!pull_queue_->Closed() && batch_cnt_ == done_cnt_) { - pull_queue_->Close(); - } - break; - } - batch_cnt_ += 1; - } - for (int i = 0; i < param_.program_config(0).pull_sparse_table_id_size(); - ++i) { - uint64_t tid = static_cast( - param_.program_config(0).pull_sparse_table_id(i)); - TableParameter table; - for (auto j : param_.sparse_table()) { - if (j.table_id() == tid) { - table = j; - break; - } - } - task->timeline.Start(); - fleet_ptr_->HeterPullSparseVars(thread_id_, task, tid, - sparse_key_names_[tid], table.fea_dim(), - sparse_value_names_[tid]); - task->timeline.Pause(); - task->pull_sparse_local_time += task->timeline.ElapsedSec(); - task->total_time += task->timeline.ElapsedSec(); - - task->timeline.Start(); - CollectLabelInfo(task, i); - task->timeline.Pause(); - task->collect_label_time += task->timeline.ElapsedSec(); - task->total_time += task->timeline.ElapsedSec(); - - task->timeline.Start(); - FillSparseValue(task, i); - task->timeline.Pause(); - task->fill_sparse_time += task->timeline.ElapsedSec(); - task->total_time += task->timeline.ElapsedSec(); - - auto nid_iter = std::find(sparse_value_names_[tid].begin(), - sparse_value_names_[tid].end(), - adjust_ins_weight_config_.nid_slot()); - if (nid_iter != sparse_value_names_[tid].end()) { - AdjustInsWeight(task); - } - } - - task->timeline.Start(); - size_t op_index = 0; - for (; op_index < ops_.size(); ++op_index) { - auto& op = ops_[op_index]; - if (op->HasAttr("op_device")) { - auto device = op->Attr("op_device"); - if (device == "gpu") { - break; - } - } - bool need_skip = false; - for (auto t = 0u; t < skip_ops_.size(); ++t) { - if (op->Type().find(skip_ops_[t]) != std::string::npos) { - need_skip = true; - break; - } - } - if (!need_skip) { - op->Run(*(task->scope_), platform::CPUPlace()); - } - } - - task->timeline.Pause(); - task->cpu_op_time += task->timeline.ElapsedSec(); - task->total_time += task->timeline.ElapsedSec(); - - task->timeline.Start(); - // prepare for gpu - Scope* cpu_scope = task->scope_; - Scope* gpu_scope = nullptr; - if (cpu_scope->kids().empty()) { - gpu_scope = &cpu_scope->NewScope(); - } else { - gpu_scope = cpu_scope->kids().front(); - } - for (const std::string& name : send_var_list_) { - const LoDTensor& cpu_tensor = cpu_scope->FindVar(name)->Get(); - LoDTensor* gpu_tensor = gpu_scope->Var(name)->GetMutable(); - gpu_tensor->set_lod(cpu_tensor.lod()); - gpu_tensor->Resize(cpu_tensor.dims()); - gpu_tensor->set_layout(cpu_tensor.layout()); - void* gpu_ptr = gpu_tensor->mutable_data(place_, cpu_tensor.type()); - const void* cpu_ptr = cpu_tensor.data(); - memory::Copy(BOOST_GET_CONST(platform::CUDAPlace, place_), gpu_ptr, - platform::CPUPlace(), cpu_ptr, - cpu_tensor.numel() * SizeOfType(cpu_tensor.type()), - copy_stream_); - } - task->timeline.Pause(); - task->cpu_2_gpu_time += task->timeline.ElapsedSec(); - task->total_time += task->timeline.ElapsedSec(); - pull_queue_->Put(task); - push_queue_->Get(task); - - int need_copy_grad = 1; - task->timeline.Start(); - for (; op_index < ops_.size(); ++op_index) { - auto& op = ops_[op_index]; - if (op->HasAttr("op_device")) { - auto device = op->Attr("op_device"); - if (device == "gpu") { - continue; - } - } - bool need_skip = false; - for (auto t = 0u; t < skip_ops_.size(); ++t) { - if (op->Type().find(skip_ops_[t]) != std::string::npos) { - need_skip = true; - break; - } - } - if (!need_skip) { - need_copy_grad = 0; - op->Run(*(task->scope_), platform::CPUPlace()); - } - } - task->timeline.Pause(); - task->cpu_op_time += task->timeline.ElapsedSec(); - task->total_time += task->timeline.ElapsedSec(); - - VLOG(3) << "fill sparse value for all sparse table done."; - for (std::string& var_name : check_nan_var_names_) { - Variable* var = (task->scope_)->FindVar(var_name); - if (var == nullptr) { - continue; - } - LoDTensor* tensor = var->GetMutable(); - if (tensor == nullptr) { - continue; - } - PADDLE_ENFORCE_EQ(framework::TensorContainsInf(*tensor), false, - platform::errors::InvalidArgument( - "Tensor %s contains Inf.", var_name)); - PADDLE_ENFORCE_EQ(framework::TensorContainsNAN(*tensor), false, - platform::errors::InvalidArgument( - "Tensor %s contains NAN.", var_name)); - } - - if (need_to_push_sparse_) { - // push gradients here - for (int i = 0; i < param_.program_config(0).push_sparse_table_id_size(); - ++i) { - uint64_t tid = static_cast( - param_.program_config(0).push_sparse_table_id(i)); - TableParameter table; - for (auto i : param_.sparse_table()) { - if (i.table_id() == tid) { - table = i; - break; - } - } - Scope* src_scope = task->scope_; - Scope* dest_scope = nullptr; - task->timeline.Start(); - if (need_copy_grad) { - if (cpu_scope->kids().empty()) { - dest_scope = &src_scope->NewScope(); - } else { - dest_scope = src_scope->kids().front(); - } - auto dev_id = BOOST_GET_CONST(platform::CUDAPlace, place_).device; - platform::CUDADeviceGuard guard(dev_id); - - for (const std::string& name : sparse_grad_names_[tid]) { - const LoDTensor& src_tensor = - src_scope->FindVar(name)->Get(); - LoDTensor* dest_tensor = - dest_scope->Var(name)->GetMutable(); - dest_tensor->set_lod(src_tensor.lod()); - dest_tensor->Resize(src_tensor.dims()); - dest_tensor->set_layout(src_tensor.layout()); - void* dest_ptr = dest_tensor->mutable_data(platform::CPUPlace(), - src_tensor.type()); - const void* src_ptr = src_tensor.data(); - memory::Copy(platform::CPUPlace(), dest_ptr, - BOOST_GET_CONST(platform::CUDAPlace, place_), src_ptr, - src_tensor.numel() * SizeOfType(src_tensor.type()), - copy_stream_); - } - } else { - dest_scope = task->scope_; - } - task->timeline.Pause(); - task->gpu_2_cpu_time += task->timeline.ElapsedSec(); - task->total_time += task->timeline.ElapsedSec(); - - task->timeline.Start(); - fleet_ptr_->HeterPushSparseVars( - task, *(dest_scope), tid, sparse_key_names_[tid], - sparse_grad_names_[tid], table.emb_dim(), &push_sparse_status_, - use_cvm_, dump_slot_, no_cvm_); - task->timeline.Pause(); - task->push_sparse_time += task->timeline.ElapsedSec(); - task->total_time += task->timeline.ElapsedSec(); - } - } - - if (need_to_push_sparse_) { - VLOG(3) << "push sparse gradient done."; - int32_t tmp_push_sparse_wait_times = -1; - static uint32_t push_sparse_wait_times = - static_cast(tmp_push_sparse_wait_times); - if (push_sparse_status_.size() >= push_sparse_wait_times) { - for (auto& t : push_sparse_status_) { - t.wait(); - } - push_sparse_status_.resize(0); - } - - if (tmp_push_sparse_wait_times == -1) { - push_sparse_status_.resize(0); - } - } - { - std::lock_guard lock(mutex_); - total_time_ += task->total_time; - read_time_ += task->read_time; - pack_time_ += task->pack_time; - pull_sparse_local_time_ += task->pull_sparse_local_time; - op_all_time_ += task->op_all_time; - xpu_op_time_ += task->xpu_op_time; - xpu_wait_time_ += task->xpu_wait_time; - cpu_op_time_ += task->cpu_op_time; - collect_label_time_ += task->collect_label_time; - fill_sparse_time_ += task->fill_sparse_time; - push_sparse_time_ += task->push_sparse_time; - gpu_2_cpu_time_ += task->gpu_2_cpu_time; - cpu_2_gpu_time_ += task->cpu_2_gpu_time; - total_inst_ += task->cur_batch_; - } - done_cnt_.fetch_add(1, std::memory_order_relaxed); - if (thread_id_ == 0) { - // should be configured here - if (done_cnt_ > 0 && done_cnt_ % 100 == 0) { - fprintf(stderr, "cpu_2_gpu total time: %fs\n", - cpu_2_gpu_time_ / done_cnt_); - fprintf(stderr, "gpu_2_cpu run total time: %fs\n", - gpu_2_cpu_time_ / done_cnt_); - fprintf(stderr, "cpu op run total time: %fs\n", - cpu_op_time_ / done_cnt_); - fprintf(stderr, "xpu op run total time: %fs\n", - xpu_op_time_ / done_cnt_); - fprintf(stderr, "xpu wait total time: %fs\n", - xpu_wait_time_ / done_cnt_); - fprintf(stderr, "pack task time: %fs\n", pack_time_ / done_cnt_); - fprintf(stderr, "train total time: %fs\n", total_time_ / done_cnt_); - fprintf(stderr, "pull sparse local time: %fs\n", - pull_sparse_local_time_ / done_cnt_); - fprintf(stderr, "fill sparse time: %fs\n", - fill_sparse_time_ / done_cnt_); - fprintf(stderr, "push sparse time: %fs\n", - push_sparse_time_ / done_cnt_); - fprintf(stderr, "collect label time: %fs\n", - collect_label_time_ / done_cnt_); - fprintf(stderr, "mean read time: %fs\n", read_time_ / done_cnt_); - fprintf(stderr, "IO percent: %f\n", read_time_ / total_time_ * 100); - fprintf(stderr, "cpu_2_gpu run percent: %f\n", - cpu_2_gpu_time_ / total_time_ * 100); - fprintf(stderr, "gpu_2_cpu run percent: %f\n", - gpu_2_cpu_time_ / total_time_ * 100); - fprintf(stderr, "cpu op run percent: %f\n", - cpu_op_time_ / total_time_ * 100); - fprintf(stderr, "xpu op run percent: %f\n", - xpu_op_time_ / total_time_ * 100); - fprintf(stderr, "xpu wait percent: %f\n", - xpu_wait_time_ / total_time_ * 100); - fprintf(stderr, "pack task percent: %f\n", - pack_time_ / total_time_ * 100); - fprintf(stderr, "pull sparse local time percent: %f\n", - pull_sparse_local_time_ / total_time_ * 100); - fprintf(stderr, "collect label time percent: %f\n", - collect_label_time_ / total_time_ * 100); - fprintf(stderr, "fill sparse time percent: %f\n", - fill_sparse_time_ / total_time_ * 100); - fprintf(stderr, "push sparse time percent: %f\n", - push_sparse_time_ / total_time_ * 100); - fprintf(stderr, "%6.2f instances/s\n", total_inst_ / total_time_); - } - } - - VLOG(3) << "done taskid = " << task->taskid_; - task->scope_->DropKids(); - object_pool_.Push(task); - } -} - -} // end namespace framework -} // end namespace paddle -#endif diff --git a/paddle/fluid/framework/ir/adaptive_pool2d_convert_global_pass.cc b/paddle/fluid/framework/ir/adaptive_pool2d_convert_global_pass.cc index 62d79f987a6702..0e2bb3eaad536f 100644 --- a/paddle/fluid/framework/ir/adaptive_pool2d_convert_global_pass.cc +++ b/paddle/fluid/framework/ir/adaptive_pool2d_convert_global_pass.cc @@ -24,6 +24,46 @@ namespace paddle { namespace framework { namespace ir { +AdaptivePool2dConvertGlobalPass::AdaptivePool2dConvertGlobalPass() { + AddOpCompat(OpCompat("pool2d")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("pooling_type") + .IsStringIn({"max", "avg"}) + .End() + .AddAttr("ksize") + .IsType>() + .End() + .AddAttr("global_pooling") + .IsBoolEQ(true) + .End() + .AddAttr("strides") + .IsType>() + .End() + .AddAttr("paddings") + .IsType>() + .End() + .AddAttr("exclusive") + .IsType() + .End() + .AddAttr("adaptive") + .IsBoolEQ(false) + .End() + .AddAttr("ceil_mode") + .IsType() + .End() + .AddAttr("data_format") + .IsStringIn({"NHWC", "NCHW"}) + .End() + .AddAttr("padding_algorithm") + .IsStringIn({"EXPLICIT", "SAME", "VALID"}) + .End(); +} + void AdaptivePool2dConvertGlobalPass::ApplyImpl(ir::Graph* graph) const { std::string name_scope = "adaptive_pool2d_convert_global_pass"; FusePassBase::Init(name_scope, graph); diff --git a/paddle/fluid/framework/ir/adaptive_pool2d_convert_global_pass.h b/paddle/fluid/framework/ir/adaptive_pool2d_convert_global_pass.h index f16f030d518d02..4a1405004e247d 100644 --- a/paddle/fluid/framework/ir/adaptive_pool2d_convert_global_pass.h +++ b/paddle/fluid/framework/ir/adaptive_pool2d_convert_global_pass.h @@ -31,6 +31,7 @@ class Graph; */ class AdaptivePool2dConvertGlobalPass : public FusePassBase { public: + AdaptivePool2dConvertGlobalPass(); virtual ~AdaptivePool2dConvertGlobalPass() {} protected: diff --git a/paddle/fluid/framework/ir/attention_lstm_fuse_pass.cc b/paddle/fluid/framework/ir/attention_lstm_fuse_pass.cc index 34c6777195f843..8f6c6968f60dd8 100644 --- a/paddle/fluid/framework/ir/attention_lstm_fuse_pass.cc +++ b/paddle/fluid/framework/ir/attention_lstm_fuse_pass.cc @@ -23,6 +23,61 @@ namespace paddle { namespace framework { namespace ir { +AttentionLSTMFusePass::AttentionLSTMFusePass() { + AddOpCompat(OpCompat("while")) + .AddInput("X") // A set of variables, unconstrained + .End() + .AddInput("Condition") // An scalar + .IsTensor() + .End() + .AddOutput("Out") // A set of variables, unconstrained + .End() + .AddOutput("StepScopes") // A vector of local scope, unconstrained + .End() + .AddAttr("sub_block") + .IsType() + .End(); + + AddOpCompat(OpCompat("fill_constant")) + .AddInput("ValueTensor") + .IsTensor() + .IsOptional() + .End() + .AddInput("ShapeTensor") + .IsTensor() + .IsOptional() + .End() + .AddInput("ShapeTensorList") // vector> + .IsOptional() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("dtype") + .IsNumGE(0) + .IsNumLE(25) + .End() + .AddAttr("shape") + .IsType>() + .End() + .AddAttr("value") + .IsType() + .End(); + + AddOpCompat(OpCompat("sequence_expand")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("ref_level") + .IsNumGE(-1) + .End(); +} struct Param { std::string X = "concat_0.tmp_0"; std::string C0 = "cell_init"; @@ -43,7 +98,7 @@ struct Param { void PrepareParameters(Graph* graph, const Param& param, ir::Node* lstm_op); -void FindWhileOp(Graph* graph) { +void AttentionLSTMFusePass::FindWhileOp(Graph* graph) const { GraphPatternDetector gpd; std::unordered_set fused_external_ops( {35, 36, 37, 38, 43, 44, 49, 45, 46, 47, 41, 42, 53, 54, 48, @@ -60,6 +115,10 @@ void FindWhileOp(Graph* graph) { auto handle = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass in op compat failed."; + return; + } auto* while_pat_node = gpd.pattern().RetrieveNode("while"); auto* while_node = subgraph.at(while_pat_node); marked_nodes.insert(while_node); diff --git a/paddle/fluid/framework/ir/attention_lstm_fuse_pass.h b/paddle/fluid/framework/ir/attention_lstm_fuse_pass.h index 48e3989a5314c6..5d4896a6db103c 100644 --- a/paddle/fluid/framework/ir/attention_lstm_fuse_pass.h +++ b/paddle/fluid/framework/ir/attention_lstm_fuse_pass.h @@ -23,8 +23,14 @@ namespace ir { class Graph; class AttentionLSTMFusePass : public FusePassBase { + public: + AttentionLSTMFusePass(); + protected: void ApplyImpl(ir::Graph* graph) const override; + + private: + void FindWhileOp(Graph* graph) const; }; } // namespace ir diff --git a/paddle/fluid/framework/ir/conv_affine_channel_fuse_pass.cc b/paddle/fluid/framework/ir/conv_affine_channel_fuse_pass.cc index 56d5831f3329b9..e4ac89f04ff679 100644 --- a/paddle/fluid/framework/ir/conv_affine_channel_fuse_pass.cc +++ b/paddle/fluid/framework/ir/conv_affine_channel_fuse_pass.cc @@ -94,6 +94,77 @@ void recompute_bias_and_weights(const Scope* scope, ir::Node* conv_weight, } } +ConvAffineChannelFusePass::ConvAffineChannelFusePass() { + AddOpCompat(OpCompat("conv2d")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("Filter") + .IsTensor() + .End() + .AddInput("Bias") + .IsTensor() + .IsOptional() + .End() + .AddInput("ResidualData") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Output") + .IsTensor() + .End() + .AddAttr("strides") + .IsType>() + .End() + .AddAttr("paddings") + .IsType>() + .End() + .AddAttr("padding_algorithm") + .IsStringIn({"EXPLICIT", "SAME", "VALID"}) + .End() + .AddAttr("groups") + .IsNumGE(1) + .End() + .AddAttr("dilations") + .IsType>() + .End() + .AddAttr("data_format") + .IsStringIn({"NCHW", "NHWC"}) + .End(); + + AddOpCompat(OpCompat("affine_channel")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Scale") + .IsTensor() + .End() + .AddInput("Bias") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("data_layout") + .IsStringIn({"NCHW", "NHWC", "AnyLayout"}) + .End(); + + AddOpCompat(OpCompat("elementwise_add")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("axis") + .IsNumEQ(1) + .End(); +} + void ConvAffineChannelFusePass::ApplyImpl(ir::Graph* graph) const { PADDLE_ENFORCE_NOT_NULL( graph, platform::errors::InvalidArgument("Graph cannot be nullptr.")); @@ -116,6 +187,11 @@ void ConvAffineChannelFusePass::ApplyImpl(ir::Graph* graph) const { int found_conv_ac_count = 0; auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "ConvAffineChannelFusePass in op compat failed."; + return; + } + VLOG(4) << "handle ConvAffineChannel fuse"; GET_CONV_BN_NODES(conv_ac_pattern); @@ -149,6 +225,7 @@ void ConvAffineChannelFusePass::ApplyImpl(ir::Graph* graph) const { desc.SetType("elementwise_add"); desc.SetAttr("axis", 1); desc.SetAttr("use_mkldnn", conv->Op()->GetAttrIfExists("use_mkldnn")); + auto eltwise_op = g->CreateOpNode(&desc); // OpDesc will be copied. GraphSafeRemoveNodes(graph, {ac_scale, ac_bias, affine_channel}); @@ -164,6 +241,75 @@ void ConvAffineChannelFusePass::ApplyImpl(ir::Graph* graph) const { AddStatis(found_conv_ac_count); } +ConvEltwiseAddAffineChannelFusePass::ConvEltwiseAddAffineChannelFusePass() { + AddOpCompat(OpCompat("conv2d")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("Filter") + .IsTensor() + .End() + .AddInput("Bias") + .IsTensor() + .IsOptional() + .End() + .AddInput("ResidualData") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Output") + .IsTensor() + .End() + .AddAttr("strides") + .IsType>() + .End() + .AddAttr("paddings") + .IsType>() + .End() + .AddAttr("padding_algorithm") + .IsStringIn({"EXPLICIT", "SAME", "VALID"}) + .End() + .AddAttr("groups") + .IsNumGE(1) + .End() + .AddAttr("dilations") + .IsType>() + .End() + .AddAttr("data_format") + .IsStringIn({"NCHW", "NHWC"}) + .End(); + AddOpCompat(OpCompat("affine_channel")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Scale") + .IsTensor() + .End() + .AddInput("Bias") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("data_layout") + .IsStringIn({"NCHW", "NHWC", "AnyLayout"}) + .End(); + AddOpCompat(OpCompat("elementwise_add")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("axis") + .IsNumEQ(1) + .End(); +} + void ConvEltwiseAddAffineChannelFusePass::ApplyImpl(ir::Graph* graph) const { PADDLE_ENFORCE_NOT_NULL( graph, platform::errors::InvalidArgument("Graph cannot be nullptr.")); @@ -186,6 +332,12 @@ void ConvEltwiseAddAffineChannelFusePass::ApplyImpl(ir::Graph* graph) const { int found_conv_ac_count = 0; auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) + << "ConvEltwiseAddAffineChannelFusePass in op compat failed."; + return; + } + VLOG(4) << "handle ConvBN fuse"; GET_CONV_BN_NODES(conv_ac_pattern); diff --git a/paddle/fluid/framework/ir/conv_affine_channel_fuse_pass.h b/paddle/fluid/framework/ir/conv_affine_channel_fuse_pass.h index 916384ec447045..8cfaf5c6a89f06 100644 --- a/paddle/fluid/framework/ir/conv_affine_channel_fuse_pass.h +++ b/paddle/fluid/framework/ir/conv_affine_channel_fuse_pass.h @@ -31,6 +31,7 @@ class Graph; class ConvAffineChannelFusePass : public FusePassBase { public: + ConvAffineChannelFusePass(); virtual ~ConvAffineChannelFusePass() {} protected: @@ -40,6 +41,7 @@ class ConvAffineChannelFusePass : public FusePassBase { class ConvEltwiseAddAffineChannelFusePass : public FusePassBase { public: + ConvEltwiseAddAffineChannelFusePass(); virtual ~ConvEltwiseAddAffineChannelFusePass() {} protected: diff --git a/paddle/fluid/framework/ir/conv_bn_fuse_pass.cc b/paddle/fluid/framework/ir/conv_bn_fuse_pass.cc index 03a78ec3a21375..c362eec34b0683 100644 --- a/paddle/fluid/framework/ir/conv_bn_fuse_pass.cc +++ b/paddle/fluid/framework/ir/conv_bn_fuse_pass.cc @@ -149,17 +149,21 @@ ConvBNFusePass::ConvBNFusePass() { .IsTensor() .End() .AddInput("Bias") + .IsTensor() .IsOptional() .End() .AddInput("ResidualData") + .IsTensor() .IsOptional() .End() .AddOutput("Output") .IsTensor() .End() .AddAttr("strides") + .IsType>() .End() .AddAttr("paddings") + .IsType>() .End() .AddAttr("padding_algorithm") .IsOptional() @@ -169,6 +173,7 @@ ConvBNFusePass::ConvBNFusePass() { .IsNumGE(1) .End() .AddAttr("dilations") + .IsType>() .End() .AddAttr("data_format") .IsStringIn({"NCHW", "NHWC", "AnyLayout"}) @@ -205,6 +210,10 @@ ConvBNFusePass::ConvBNFusePass() { .AddOutput("Y") .IsTensor() .End() + .AddOutput("ReserveSpace") + .IsTensor() + .IsOptional() + .End() .AddAttr("epsilon") .IsNumLE(0.001f) .IsNumGE(0.0f) @@ -375,17 +384,21 @@ ConvEltwiseAddBNFusePass::ConvEltwiseAddBNFusePass() { .IsTensor() .End() .AddInput("Bias") + .IsTensor() .IsOptional() .End() .AddInput("ResidualData") + .IsTensor() .IsOptional() .End() .AddOutput("Output") .IsTensor() .End() .AddAttr("strides") + .IsType>() .End() .AddAttr("paddings") + .IsType>() .End() .AddAttr("padding_algorithm") .IsStringIn({"EXPLICIT", "SAME", "VALID"}) @@ -395,6 +408,7 @@ ConvEltwiseAddBNFusePass::ConvEltwiseAddBNFusePass() { .IsNumGE(1) .End() .AddAttr("dilations") + .IsType>() .End() .AddAttr("data_format") .IsStringIn({"NCHW", "NHWC", "AnyLayout"}) @@ -431,6 +445,10 @@ ConvEltwiseAddBNFusePass::ConvEltwiseAddBNFusePass() { .AddOutput("Y") .IsTensor() .End() + .AddOutput("ReserveSpace") + .IsTensor() + .IsOptional() + .End() .AddAttr("epsilon") .IsNumLE(0.001f) .IsNumGE(0.0f) @@ -575,31 +593,85 @@ ConvTransposeBNFusePass::ConvTransposeBNFusePass() { .IsTensor() .End() .AddInput("Bias") + .IsTensor() .IsOptional() .End() .AddOutput("Output") .IsTensor() .End() + .AddAttr("output_padding") + .IsType>() + .IsOptional() + .End() + .AddAttr("output_size") + .IsType>() + .IsOptional() + .End() + .AddAttr("groups") + .IsNumGE(1) + .End() + .AddAttr("dilations") + .IsType>() + .End() .AddAttr("strides") + .IsType>() .End() .AddAttr("paddings") + .IsType>() .End() .AddAttr("padding_algorithm") .IsStringIn({"EXPLICIT", "SAME", "VALID"}) + .End() + .AddAttr("data_format") + .IsStringIn({"NCHW", "NHWC", "AnyLayout"}) + .End(); +} + +ConvTransposeEltwiseAddBNFusePass::ConvTransposeEltwiseAddBNFusePass() { + AddOpCompat(OpCompat("conv2d_transpose")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("Filter") + .IsTensor() + .End() + .AddInput("Bias") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Output") + .IsTensor() + .End() + .AddAttr("output_padding") + .IsType>() + .IsOptional() + .End() + .AddAttr("output_size") + .IsType>() .IsOptional() .End() .AddAttr("groups") .IsNumGE(1) .End() .AddAttr("dilations") + .IsType>() + .End() + .AddAttr("strides") + .IsType>() + .End() + .AddAttr("paddings") + .IsType>() + .End() + .AddAttr("padding_algorithm") + .IsStringIn({"EXPLICIT", "SAME", "VALID"}) .End() .AddAttr("data_format") .IsStringIn({"NCHW", "NHWC", "AnyLayout"}) .End(); } -ConvTransposeEltwiseAddBNFusePass::ConvTransposeEltwiseAddBNFusePass() { - AddOpCompat(OpCompat("conv2d_transpose")) +DepthwiseConvBNFusePass::DepthwiseConvBNFusePass() { + AddOpCompat(OpCompat("depthwise_conv2d")) .AddInput("Input") .IsTensor() .End() @@ -607,23 +679,31 @@ ConvTransposeEltwiseAddBNFusePass::ConvTransposeEltwiseAddBNFusePass() { .IsTensor() .End() .AddInput("Bias") + .IsTensor() + .IsOptional() + .End() + .AddInput("ResidualData") + .IsTensor() .IsOptional() .End() .AddOutput("Output") .IsTensor() .End() .AddAttr("strides") + .IsType>() .End() .AddAttr("paddings") + .IsType>() .End() .AddAttr("padding_algorithm") - .IsStringIn({"EXPLICIT", "SAME", "VALID"}) .IsOptional() + .IsStringIn({"EXPLICIT", "SAME", "VALID"}) .End() .AddAttr("groups") .IsNumGE(1) .End() .AddAttr("dilations") + .IsType>() .End() .AddAttr("data_format") .IsStringIn({"NCHW", "NHWC", "AnyLayout"}) diff --git a/paddle/fluid/framework/ir/conv_bn_fuse_pass.h b/paddle/fluid/framework/ir/conv_bn_fuse_pass.h index c78dfc2a487cae..b976aab0eeae20 100644 --- a/paddle/fluid/framework/ir/conv_bn_fuse_pass.h +++ b/paddle/fluid/framework/ir/conv_bn_fuse_pass.h @@ -17,8 +17,6 @@ #include #include "paddle/fluid/framework/ir/fuse_pass_base.h" -#include "paddle/fluid/framework/ir/graph.h" -#include "paddle/fluid/framework/ir/graph_pattern_detector.h" namespace paddle { namespace framework { @@ -27,12 +25,10 @@ namespace ir { /* * Fuse the Conv and BatchNorm to a ConvBNMKLDNNOp. */ -class Graph; class ConvBNFusePass : public FusePassBase { public: ConvBNFusePass(); - virtual ~ConvBNFusePass() {} virtual std::string conv_type() const { return "conv2d"; } protected: @@ -43,7 +39,6 @@ class ConvBNFusePass : public FusePassBase { class ConvEltwiseAddBNFusePass : public FusePassBase { public: ConvEltwiseAddBNFusePass(); - virtual ~ConvEltwiseAddBNFusePass() {} virtual std::string conv_type() const { return "conv2d"; } protected: @@ -54,19 +49,18 @@ class ConvEltwiseAddBNFusePass : public FusePassBase { class ConvTransposeBNFusePass : public ConvBNFusePass { public: ConvTransposeBNFusePass(); - virtual ~ConvTransposeBNFusePass() {} std::string conv_type() const { return "conv2d_transpose"; } }; class ConvTransposeEltwiseAddBNFusePass : public ConvEltwiseAddBNFusePass { public: ConvTransposeEltwiseAddBNFusePass(); - virtual ~ConvTransposeEltwiseAddBNFusePass() {} std::string conv_type() const { return "conv2d_transpose"; } }; class DepthwiseConvBNFusePass : public ConvBNFusePass { public: + DepthwiseConvBNFusePass(); std::string conv_type() const { return "depthwise_conv2d"; } }; diff --git a/paddle/fluid/framework/ir/conv_elementwise_add2_act_fuse_pass.cc b/paddle/fluid/framework/ir/conv_elementwise_add2_act_fuse_pass.cc index e7656171700b4f..f2a295694dcb96 100644 --- a/paddle/fluid/framework/ir/conv_elementwise_add2_act_fuse_pass.cc +++ b/paddle/fluid/framework/ir/conv_elementwise_add2_act_fuse_pass.cc @@ -52,6 +52,56 @@ framework::proto::OpDesc PrepareOpDesc( desc.Flush(); return *desc.Proto(); } +ConvElementwiseAdd2ActFusePass::ConvElementwiseAdd2ActFusePass() { + AddOpCompat(OpCompat("conv2d")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("Filter") + .IsTensor() + .End() + .AddOutput("Output") + .IsTensor() + .End() + .AddAttr("strides") + .End() + .AddAttr("paddings") + .End() + .AddAttr("padding_algorithm") + .IsStringIn({"EXPLICIT", "SAME", "VALID"}) + .End() + .AddAttr("groups") + .IsNumGE(1) + .End() + .AddAttr("dilations") + .End() + .AddAttr("data_format") + .IsStringIn({"NHWC", "NCHW"}) + .End(); + + AddOpCompat(OpCompat("elementwise_add")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("axis") + // the first elementwise_add-axis needs to be 1, the second has to be -1 + .IsIntIn({1, -1}) + .End(); + + AddOpCompat(OpCompat("relu")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End(); +} void ConvElementwiseAdd2ActFusePass::ApplyImpl(ir::Graph* graph) const { const std::string pattern_name = "conv_elementwise_add2_act_fuse"; @@ -66,6 +116,10 @@ void ConvElementwiseAdd2ActFusePass::ApplyImpl(ir::Graph* graph) const { auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass op compat failed."; + return; + } GET_NODES; auto base_op_desc = *conv_op->Op()->Proto(); diff --git a/paddle/fluid/framework/ir/conv_elementwise_add2_act_fuse_pass.h b/paddle/fluid/framework/ir/conv_elementwise_add2_act_fuse_pass.h index e68f57d4ae9982..3d5e5788fed2d0 100644 --- a/paddle/fluid/framework/ir/conv_elementwise_add2_act_fuse_pass.h +++ b/paddle/fluid/framework/ir/conv_elementwise_add2_act_fuse_pass.h @@ -24,6 +24,7 @@ class Graph; class ConvElementwiseAdd2ActFusePass : public FusePassBase { public: + ConvElementwiseAdd2ActFusePass(); virtual ~ConvElementwiseAdd2ActFusePass() {} protected: diff --git a/paddle/fluid/framework/ir/conv_elementwise_add_act_fuse_pass.cc b/paddle/fluid/framework/ir/conv_elementwise_add_act_fuse_pass.cc index ac6e22862d6299..c89984f3846917 100644 --- a/paddle/fluid/framework/ir/conv_elementwise_add_act_fuse_pass.cc +++ b/paddle/fluid/framework/ir/conv_elementwise_add_act_fuse_pass.cc @@ -48,6 +48,60 @@ framework::proto::OpDesc PrepareOpDesc( return *desc.Proto(); } +ConvElementwiseAddActFusePass::ConvElementwiseAddActFusePass() { + AddOpCompat(OpCompat("conv2d")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("Filter") + .IsTensor() + .End() + .AddInput("ResidualData") + .IsOptional() + .End() + .AddOutput("Output") + .IsTensor() + .End() + .AddAttr("strides") + .End() + .AddAttr("paddings") + .End() + .AddAttr("padding_algorithm") + .IsOptional() + .IsStringIn({"EXPLICIT", "SAME", "VALID"}) + .End() + .AddAttr("groups") + .IsNumGE(1) + .End() + .AddAttr("dilations") + .End() + .AddAttr("data_format") + .IsStringIn({"NCHW", "NHWC", "AnyLayout"}) + .End(); + + AddOpCompat(OpCompat("elementwise_add")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("axis") + .IsNumEQ(1) + .End(); + + AddOpCompat(OpCompat("relu")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End(); +} + void ConvElementwiseAddActFusePass::ApplyImpl(ir::Graph* graph) const { const std::string pattern_name = "conv_elementwise_add_act_fuse"; FusePassBase::Init(pattern_name, graph); @@ -63,6 +117,10 @@ void ConvElementwiseAddActFusePass::ApplyImpl(ir::Graph* graph) const { auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass in op compat failed."; + return; + } GET_NODES; auto base_op_desc = *conv_op->Op()->Proto(); diff --git a/paddle/fluid/framework/ir/conv_elementwise_add_act_fuse_pass.h b/paddle/fluid/framework/ir/conv_elementwise_add_act_fuse_pass.h index 933092c7db7d38..d28f212f49e71b 100644 --- a/paddle/fluid/framework/ir/conv_elementwise_add_act_fuse_pass.h +++ b/paddle/fluid/framework/ir/conv_elementwise_add_act_fuse_pass.h @@ -24,6 +24,7 @@ class Graph; class ConvElementwiseAddActFusePass : public FusePassBase { public: + ConvElementwiseAddActFusePass(); virtual ~ConvElementwiseAddActFusePass() {} protected: diff --git a/paddle/fluid/framework/ir/conv_elementwise_add_fuse_pass.cc b/paddle/fluid/framework/ir/conv_elementwise_add_fuse_pass.cc index 170b8fb8c80fa7..248a71ede14beb 100644 --- a/paddle/fluid/framework/ir/conv_elementwise_add_fuse_pass.cc +++ b/paddle/fluid/framework/ir/conv_elementwise_add_fuse_pass.cc @@ -29,6 +29,52 @@ namespace ir { GET_IR_NODE(elementwise_add_in_y); \ GET_IR_NODE(elementwise_add_out); +ConvElementwiseAddFusePass::ConvElementwiseAddFusePass() { + AddOpCompat(OpCompat("conv2d")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("Filter") + .IsTensor() + .End() + .AddInput("ResidualData") + .IsOptional() + .End() + .AddOutput("Output") + .IsTensor() + .End() + .AddAttr("strides") + .End() + .AddAttr("paddings") + .End() + .AddAttr("padding_algorithm") + .IsOptional() + .IsStringIn({"EXPLICIT", "SAME", "VALID"}) + .End() + .AddAttr("groups") + .IsNumGE(1) + .End() + .AddAttr("dilations") + .End() + .AddAttr("data_format") + .IsStringIn({"NCHW", "NHWC", "AnyLayout"}) + .End(); + + AddOpCompat(OpCompat("elementwise_add")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("axis") + .IsNumEQ(1) + .End(); +} + void ConvElementwiseAddFusePass::ApplyImpl(ir::Graph* graph) const { const std::string pattern_name = "conv_elementwise_add_fuse"; FusePassBase::Init(pattern_name, graph); @@ -44,6 +90,10 @@ void ConvElementwiseAddFusePass::ApplyImpl(ir::Graph* graph) const { auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass in op compat failed."; + return; + } GET_NODES; auto base_op_desc = *conv_op->Op()->Proto(); diff --git a/paddle/fluid/framework/ir/conv_elementwise_add_fuse_pass.h b/paddle/fluid/framework/ir/conv_elementwise_add_fuse_pass.h index 7198a7488e052b..0913dc5c002271 100644 --- a/paddle/fluid/framework/ir/conv_elementwise_add_fuse_pass.h +++ b/paddle/fluid/framework/ir/conv_elementwise_add_fuse_pass.h @@ -24,6 +24,7 @@ class Graph; class ConvElementwiseAddFusePass : public FusePassBase { public: + ConvElementwiseAddFusePass(); virtual ~ConvElementwiseAddFusePass() {} protected: diff --git a/paddle/fluid/framework/ir/delete_quant_dequant_filter_op_pass.cc b/paddle/fluid/framework/ir/delete_quant_dequant_filter_op_pass.cc index 4379bba6380c59..4ce91999207a2b 100644 --- a/paddle/fluid/framework/ir/delete_quant_dequant_filter_op_pass.cc +++ b/paddle/fluid/framework/ir/delete_quant_dequant_filter_op_pass.cc @@ -32,6 +32,37 @@ namespace ir { GET_IR_NODE(quant_dequant_op_outscale); \ GET_IR_NODE(any_op2); +DeleteQuantDequantFilterOpPass::DeleteQuantDequantFilterOpPass() { + AddOpCompat(OpCompat("fake_quantize_dequantize_abs_max")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("OutScale") + .IsTensor() + .End() + .AddAttr("bit_length") + .IsIntIn({8, 16}) + .End(); + AddOpCompat(OpCompat("fake_channel_wise_quantize_dequantize_abs_max")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("OutScale") + .IsTensor() + .End() + .AddAttr("bit_length") + .IsIntIn({8, 16}) + .End() + .AddAttr("quant_axis") + .IsIntIn({0, 1}) + .End(); +} // Delete quant_dequant_op, then quantize and dequantize weight void DeleteQuantDequantFilterOpPass::ApplyImpl(ir::Graph* graph) const { const std::string pattern_name = "delete_quantdequant_filter_op_pattern"; @@ -50,6 +81,11 @@ void DeleteQuantDequantFilterOpPass::ApplyImpl(ir::Graph* graph) const { Graph* g) { GET_NODES; + if (!IsCompat(*quant_dequant_op->Op())) { + LOG(WARNING) << "quant_dequant_op in delete_quant_dequant_filter_op_pass " + "compat check failed."; + return; + } std::unordered_set nodes2rm = {}; int bit_length = BOOST_GET_CONST(int, quant_dequant_op->Op()->GetAttr("bit_length")); diff --git a/paddle/fluid/framework/ir/delete_quant_dequant_filter_op_pass.h b/paddle/fluid/framework/ir/delete_quant_dequant_filter_op_pass.h index 0409032d93816a..23049aac9622ee 100644 --- a/paddle/fluid/framework/ir/delete_quant_dequant_filter_op_pass.h +++ b/paddle/fluid/framework/ir/delete_quant_dequant_filter_op_pass.h @@ -16,16 +16,14 @@ #include #include "paddle/fluid/framework/ir/fuse_pass_base.h" -#include "paddle/fluid/framework/ir/graph_pattern_detector.h" namespace paddle { namespace framework { namespace ir { -class Graph; - class DeleteQuantDequantFilterOpPass : public FusePassBase { public: + DeleteQuantDequantFilterOpPass(); virtual ~DeleteQuantDequantFilterOpPass() {} protected: diff --git a/paddle/fluid/framework/ir/fc_elementwise_layernorm_fuse_pass.cc b/paddle/fluid/framework/ir/fc_elementwise_layernorm_fuse_pass.cc index 6f7a52fce59330..d3cf3319adfc5e 100644 --- a/paddle/fluid/framework/ir/fc_elementwise_layernorm_fuse_pass.cc +++ b/paddle/fluid/framework/ir/fc_elementwise_layernorm_fuse_pass.cc @@ -196,7 +196,7 @@ FCElementwiseLayerNormFusePass::FCElementwiseLayerNormFusePass() { .IsTensor() .End() .AddAttr("axis") - .IsNumEQ(-1) + .IsIntIn({-1, 0}) .End(); } diff --git a/paddle/fluid/framework/ir/fc_gru_fuse_pass.cc b/paddle/fluid/framework/ir/fc_gru_fuse_pass.cc index 921e1ea513961d..e1260f62ddb649 100644 --- a/paddle/fluid/framework/ir/fc_gru_fuse_pass.cc +++ b/paddle/fluid/framework/ir/fc_gru_fuse_pass.cc @@ -30,8 +30,137 @@ namespace ir { class Node; -static int BuildFusion(Graph* graph, const std::string& name_scope, - Scope* scope, bool with_fc_bias) { +MulGRUFusePass::MulGRUFusePass() { + AddOpCompat(OpCompat("gru")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("H0") + .IsTensor() + .IsOptional() + .End() + .AddInput("Weight") + .IsTensor() + .End() + .AddInput("Bias") + .IsTensor() + .End() + .AddOutput("BatchGate") + .IsTensor() + .End() + .AddOutput("BatchResetHiddenPrev") + .IsTensor() + .End() + .AddOutput("BatchHidden") + .IsTensor() + .End() + .AddOutput("Hidden") + .IsTensor() + .End() + .AddAttr("activation") + .IsStringIn({"sigmoid", "tanh", "relu", "identity"}) + .End() + .AddAttr("gate_activation") + .IsStringIn({"sigmoid", "tanh", "relu", "identity"}) + .End() + .AddAttr("is_reverse") + .IsType() + .End() + .AddAttr("origin_mode") + .IsType() + .IsOptional() + .End(); + AddOpCompat(OpCompat("mul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("x_num_col_dims") + .IsNumEQ(1) + .End() + .AddAttr("y_num_col_dims") + .IsNumEQ(1) + .End(); +} + +FCGRUFusePass::FCGRUFusePass() { + AddOpCompat(OpCompat("gru")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("H0") + .IsTensor() + .IsOptional() + .End() + .AddInput("Weight") + .IsTensor() + .End() + .AddInput("Bias") + .IsTensor() + .End() + .AddOutput("BatchGate") + .IsTensor() + .End() + .AddOutput("BatchResetHiddenPrev") + .IsTensor() + .End() + .AddOutput("BatchHidden") + .IsTensor() + .End() + .AddOutput("Hidden") + .IsTensor() + .End() + .AddAttr("activation") + .IsStringIn({"sigmoid", "tanh", "relu", "identity"}) + .End() + .AddAttr("gate_activation") + .IsStringIn({"sigmoid", "tanh", "relu", "identity"}) + .End() + .AddAttr("is_reverse") + .IsType() + .End() + .AddAttr("origin_mode") + .IsType() + .IsOptional() + .End(); + AddOpCompat(OpCompat("mul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("x_num_col_dims") + .IsNumEQ(1) + .End() + .AddAttr("y_num_col_dims") + .IsNumEQ(1) + .End(); + AddOpCompat(OpCompat("elementwise_add")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("axis") + .IsNumGE(-1) + .End(); +} + +int FCGRUFusePass::BuildFusion(Graph* graph, const std::string& name_scope, + Scope* scope, bool with_fc_bias) const { GraphPatternDetector gpd; auto* pattern = gpd.mutable_pattern(); @@ -133,6 +262,10 @@ static int BuildFusion(Graph* graph, const std::string& name_scope, int fusion_count{0}; auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass in op compat failed."; + return; + } auto* x_n = subgraph.at(x); GET_IR_NODE_FROM_SUBGRAPH(w, w, fc_pattern); GET_IR_NODE_FROM_SUBGRAPH(mul, mul, fc_pattern); @@ -189,8 +322,8 @@ static int BuildFusion(Graph* graph, const std::string& name_scope, void MulGRUFusePass::ApplyImpl(ir::Graph* graph) const { FusePassBase::Init(name_scope_, graph); - int fusion_count = - BuildFusion(graph, name_scope_, param_scope(), false /*with_fc_bias*/); + int fusion_count = MulGRUFusePass::BuildFusion( + graph, name_scope_, param_scope(), false /*with_fc_bias*/); AddStatis(fusion_count); } @@ -198,8 +331,8 @@ void MulGRUFusePass::ApplyImpl(ir::Graph* graph) const { void FCGRUFusePass::ApplyImpl(ir::Graph* graph) const { FusePassBase::Init(name_scope_, graph); - int fusion_count = - BuildFusion(graph, name_scope_, param_scope(), true /*with_fc_bias*/); + int fusion_count = FCGRUFusePass::BuildFusion( + graph, name_scope_, param_scope(), true /*with_fc_bias*/); AddStatis(fusion_count); } diff --git a/paddle/fluid/framework/ir/fc_gru_fuse_pass.h b/paddle/fluid/framework/ir/fc_gru_fuse_pass.h index 73f00504d34d5f..421f3ef46d7f5c 100644 --- a/paddle/fluid/framework/ir/fc_gru_fuse_pass.h +++ b/paddle/fluid/framework/ir/fc_gru_fuse_pass.h @@ -18,7 +18,6 @@ #include "paddle/fluid/framework/ir/fuse_pass_base.h" #include "paddle/fluid/framework/ir/graph.h" -#include "paddle/fluid/framework/ir/graph_pattern_detector.h" namespace paddle { namespace framework { @@ -26,21 +25,22 @@ namespace ir { // The MulGRUFusePass and MulGRUFusePass will fuse to the same FusionGRU op. -class Graph; - class FCGRUFusePass : public FusePassBase { public: + FCGRUFusePass(); virtual ~FCGRUFusePass() {} protected: void ApplyImpl(ir::Graph* graph) const override; - const std::string name_scope_{"fc_gru_fuse"}; + int BuildFusion(Graph* graph, const std::string& name_scope, Scope* scope, + bool with_fc_bias) const; }; // Just FC without bias -class MulGRUFusePass : public FusePassBase { +class MulGRUFusePass : public FCGRUFusePass { public: + MulGRUFusePass(); virtual ~MulGRUFusePass() {} protected: diff --git a/paddle/fluid/framework/ir/fc_lstm_fuse_pass.cc b/paddle/fluid/framework/ir/fc_lstm_fuse_pass.cc index 6bd956ef0d53c9..35704f1f3309e1 100644 --- a/paddle/fluid/framework/ir/fc_lstm_fuse_pass.cc +++ b/paddle/fluid/framework/ir/fc_lstm_fuse_pass.cc @@ -29,8 +29,149 @@ namespace ir { class Node; -int BuildFusion(Graph* graph, const std::string& name_scope, Scope* scope, - bool with_fc_bias) { +MulLstmFusePass::MulLstmFusePass() { + AddOpCompat(OpCompat("lstm")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("H0") + .IsTensor() + .IsOptional() + .End() + .AddInput("C0") + .IsTensor() + .IsOptional() + .End() + .AddInput("Weight") + .IsTensor() + .End() + .AddInput("Bias") + .IsTensor() + .End() + .AddOutput("Hidden") + .IsTensor() + .End() + .AddOutput("Cell") + .IsTensor() + .End() + .AddOutput("BatchGate") + .IsTensor() + .End() + .AddOutput("BatchCellPreAct") + .IsTensor() + .End() + .AddAttr("use_peepholes") + .IsType() + .End() + .AddAttr("is_reverse") + .IsType() + .End() + .AddAttr("gate_activation") + .IsStringIn({"sigmoid", "tanh", "relu", "identity"}) + .End() + .AddAttr("cell_activation") + .IsStringIn({"sigmoid", "tanh", "relu", "identity"}) + .End() + .AddAttr("candidate_activation") + .IsStringIn({"sigmoid", "tanh", "relu", "identity"}) + .End(); + AddOpCompat(OpCompat("mul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("x_num_col_dims") + .IsNumEQ(1) + .End() + .AddAttr("y_num_col_dims") + .IsNumEQ(1) + .End(); +} + +FCLstmFusePass::FCLstmFusePass() { + AddOpCompat(OpCompat("lstm")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("H0") + .IsTensor() + .IsOptional() + .End() + .AddInput("C0") + .IsTensor() + .IsOptional() + .End() + .AddInput("Weight") + .IsTensor() + .End() + .AddInput("Bias") + .IsTensor() + .End() + .AddOutput("Hidden") + .IsTensor() + .End() + .AddOutput("Cell") + .IsTensor() + .End() + .AddOutput("BatchGate") + .IsTensor() + .End() + .AddOutput("BatchCellPreAct") + .IsTensor() + .End() + .AddAttr("use_peepholes") + .IsType() + .End() + .AddAttr("is_reverse") + .IsType() + .End() + .AddAttr("gate_activation") + .IsStringIn({"sigmoid", "tanh", "relu", "identity"}) + .End() + .AddAttr("cell_activation") + .IsStringIn({"sigmoid", "tanh", "relu", "identity"}) + .End() + .AddAttr("candidate_activation") + .IsStringIn({"sigmoid", "tanh", "relu", "identity"}) + .End(); + AddOpCompat(OpCompat("mul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("x_num_col_dims") + .IsNumEQ(1) + .End() + .AddAttr("y_num_col_dims") + .IsNumEQ(1) + .End(); + AddOpCompat(OpCompat("elementwise_add")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("axis") + .IsNumGE(-1) + .End(); +} + +int FCLstmFusePass::BuildFusion(Graph* graph, const std::string& name_scope, + Scope* scope, bool with_fc_bias) const { GraphPatternDetector gpd; auto* pattern = gpd.mutable_pattern(); @@ -140,6 +281,10 @@ int BuildFusion(Graph* graph, const std::string& name_scope, Scope* scope, auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass in op compat failed."; + return; + } GET_IR_NODE_FROM_SUBGRAPH(lstm, lstm, lstm_pattern); GET_IR_NODE_FROM_SUBGRAPH(Weight, Weight, lstm_pattern); GET_IR_NODE_FROM_SUBGRAPH(Bias, Bias, lstm_pattern); diff --git a/paddle/fluid/framework/ir/fc_lstm_fuse_pass.h b/paddle/fluid/framework/ir/fc_lstm_fuse_pass.h index d37f53b15f06b7..60b4953c2ec0a8 100644 --- a/paddle/fluid/framework/ir/fc_lstm_fuse_pass.h +++ b/paddle/fluid/framework/ir/fc_lstm_fuse_pass.h @@ -31,16 +31,19 @@ class Graph; class FCLstmFusePass : public FusePassBase { public: + FCLstmFusePass(); virtual ~FCLstmFusePass() {} protected: void ApplyImpl(ir::Graph* graph) const override; - + int BuildFusion(Graph* graph, const std::string& name_scope, Scope* scope, + bool with_fc_bias) const; const std::string name_scope_{"fc_lstm_fuse"}; }; -class MulLstmFusePass : public FusePassBase { +class MulLstmFusePass : public FCLstmFusePass { public: + MulLstmFusePass(); virtual ~MulLstmFusePass() {} protected: diff --git a/paddle/fluid/framework/ir/graph_pattern_detector.cc b/paddle/fluid/framework/ir/graph_pattern_detector.cc index 3476ce8610ee34..37a8ec12680aba 100644 --- a/paddle/fluid/framework/ir/graph_pattern_detector.cc +++ b/paddle/fluid/framework/ir/graph_pattern_detector.cc @@ -2265,8 +2265,8 @@ PDNode *patterns::Bfloat16Placement::operator()( std::unordered_set( {"concat", "conv2d", "conv2d_transpose", "elementwise_add", "elementwise_mul", "fc", "fusion_gru", "fusion_lstm", "gelu", - "layer_norm", "matmul", "pool2d", "relu", "reshape2", "softmax", - "sum", "transpose2"}); + "layer_norm", "matmul", "matmul_v2", "pool2d", "relu", "reshape2", + "softmax", "split", "sum", "transpose2"}); if (!bfloat16_enabled_op_types.empty()) { supported_op_types = bfloat16_enabled_op_types; } @@ -2340,16 +2340,7 @@ PDNode *patterns::DuplicatedInputs::operator()() { PDNode *patterns::MKLDNNInPlace::operator()() { const std::unordered_set &supported_op_types = { - "abs", - "elementwise_mul", - "elementwise_add", - "gelu", - "leaky_relu", - "relu", - "softmax", - "sqrt", - "swish", - "tanh"}; + "abs", "gelu", "leaky_relu", "relu", "softmax", "sqrt", "swish", "tanh"}; auto possible_inplace_op = pattern->NewNode(inplace_to_be_op_repr()) ->assert_is_ops(supported_op_types); diff --git a/paddle/fluid/framework/ir/map_matmul_to_mul_pass.cc b/paddle/fluid/framework/ir/map_matmul_to_mul_pass.cc index c36123f65f6644..9542d3d3d43f31 100644 --- a/paddle/fluid/framework/ir/map_matmul_to_mul_pass.cc +++ b/paddle/fluid/framework/ir/map_matmul_to_mul_pass.cc @@ -16,6 +16,7 @@ #include #include +#include "paddle/fluid/framework/op_proto_maker.h" #include "paddle/fluid/framework/op_version_registry.h" #include "paddle/fluid/platform/enforce.h" @@ -26,6 +27,157 @@ namespace ir { class Node; +MapMatmul2MulPass::MapMatmul2MulPass() { + AddOpCompat(OpCompat("matmul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("alpha") + .IsNumGE(0.99f) + .IsNumLE(1.01f) + .End() + .AddAttr("transpose_X") + .IsBoolEQ(false) + .End() + .AddAttr("transpose_Y") + .IsBoolEQ(false) + .End(); + + AddOpCompat(OpCompat("mul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("x_num_col_dims") + .IsNumGE(1) + .End() + .AddAttr("y_num_col_dims") + .IsNumEQ(1) + .End(); +} + +Flatten2MatmulFusePass::Flatten2MatmulFusePass() { + AddOpCompat(OpCompat("matmul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("alpha") + .IsNumGE(0.99f) + .IsNumLE(1.01f) + .End() + .AddAttr("transpose_X") + .IsBoolEQ(false) + .End() + .AddAttr("transpose_Y") + .IsBoolEQ(false) + .End(); + + AddOpCompat(OpCompat("flatten2")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("XShape") + .IsTensor() + .End() + .AddAttr("axis") + .IsNumGE(0) + .End(); + + AddOpCompat(OpCompat("mul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("x_num_col_dims") + .IsNumGE(1) + .End() + .AddAttr("y_num_col_dims") + .IsNumEQ(1) + .End(); +} + +Squeeze2MatmulFusePass::Squeeze2MatmulFusePass() { + AddOpCompat(OpCompat("matmul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("alpha") + .IsNumGE(0.99f) + .IsNumLE(1.01f) + .End() + .AddAttr("transpose_X") + .IsBoolEQ(false) + .End() + .AddAttr("transpose_Y") + .IsBoolEQ(false) + .End(); + + AddOpCompat(OpCompat("Squeeze2")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("XShape") + .IsTensor() + .End() + .AddAttr("axes") + .IsType>() + .End(); + + AddOpCompat(OpCompat("mul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("x_num_col_dims") + .IsNumEQ(1) + .End() + .AddAttr("y_num_col_dims") + .IsNumEQ(1) + .End(); +} + void MapMatmul2MulPass::ApplyImpl(ir::Graph* graph) const { PADDLE_ENFORCE_NOT_NULL( graph, platform::errors::InvalidArgument("Graph cannot be nullptr.")); @@ -39,6 +191,11 @@ void MapMatmul2MulPass::ApplyImpl(ir::Graph* graph) const { int found_count = 0; auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass in op compat failed."; + return; + } + VLOG(4) << "map matmul to mul"; GET_IR_NODE_FROM_SUBGRAPH(matmul_in_x, matmul_in_x, matmul_pattern); GET_IR_NODE_FROM_SUBGRAPH(matmul_in_y, matmul_in_y, matmul_pattern); @@ -82,6 +239,11 @@ void MapMatmul2MulPass::ApplyImpl(ir::Graph* graph) const { IR_NODE_LINK_TO(mul_node, matmul_out); GraphSafeRemoveNodes(graph, {matmul_op}); ++found_count; + + if (!IsCompat(desc)) { + LOG(WARNING) << "MapMatmul2MulPass in out mul op compat failed."; + return; + } } }; @@ -103,6 +265,10 @@ void Squeeze2MatmulFusePass::ApplyImpl(ir::Graph* graph) const { auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { VLOG(4) << "fuse squeeze2+matmul to mul"; + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass in op compat failed."; + return; + } GET_IR_NODE_FROM_SUBGRAPH(squeeze2_in_x, squeeze2_in_x, fuse_pattern); GET_IR_NODE_FROM_SUBGRAPH(squeeze2_op, squeeze2_op, fuse_pattern); GET_IR_NODE_FROM_SUBGRAPH(matmul_in_x, matmul_in_x, fuse_pattern); @@ -152,6 +318,10 @@ void Squeeze2MatmulFusePass::ApplyImpl(ir::Graph* graph) const { IR_NODE_LINK_TO(mul_node, matmul_out); GraphSafeRemoveNodes(graph, {squeeze2_op, matmul_in_x, matmul_op}); ++found_count; + if (!IsCompat(desc)) { + LOG(WARNING) << "Squeeze2MatmulFusePass in out mul op compat failed."; + return; + } } }; @@ -159,6 +329,68 @@ void Squeeze2MatmulFusePass::ApplyImpl(ir::Graph* graph) const { AddStatis(found_count); } +Reshape2MatmulFusePass::Reshape2MatmulFusePass() { + AddOpCompat(OpCompat("reshape2")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Shape") + .IsTensor() + .IsOptional() + .End() + .AddInput("ShapeTensor") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("XShape") + .IsTensor() + .End() + .AddAttr("shape") // ints + .IsType>() + .End(); + + AddOpCompat(OpCompat("matmul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("alpha") + .IsNumGT(0.99999f) + .IsNumLT(1.00001f) + .End() + .AddAttr("transpose_X") + .IsBoolEQ("False") + .End() + .AddAttr("transpose_Y") + .IsBoolEQ("False") + .End(); + + AddOpCompat(OpCompat("mul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("x_num_col_dims") + .IsNumEQ(1) + .End() + .AddAttr("y_num_col_dims") + .IsNumEQ(1) + .End(); +} + void Reshape2MatmulFusePass::ApplyImpl(ir::Graph* graph) const { PADDLE_ENFORCE_NOT_NULL( graph, platform::errors::InvalidArgument("Graph cannot be nullptr.")); @@ -172,6 +404,10 @@ void Reshape2MatmulFusePass::ApplyImpl(ir::Graph* graph) const { int found_count = 0; auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass in op compat failed."; + return; + } VLOG(4) << "fuse reshape2+matmul to mul"; GET_IR_NODE_FROM_SUBGRAPH(reshape2_in_x, reshape2_in_x, fuse_pattern); GET_IR_NODE_FROM_SUBGRAPH(reshape2_op, reshape2_op, fuse_pattern); @@ -218,6 +454,10 @@ void Reshape2MatmulFusePass::ApplyImpl(ir::Graph* graph) const { desc.SetAttr("X_scale", matmul_op->Op()->GetAttr("X_scale")); desc.SetAttr("weight_scale", matmul_op->Op()->GetAttr("weight_scale")); } + if (!IsCompat(desc)) { + LOG(WARNING) << "reshape2 matmul pass in out mul op compat failed."; + return; + } auto mul_node = g->CreateOpNode(&desc); IR_NODE_LINK_TO(reshape2_in_x, mul_node); IR_NODE_LINK_TO(matmul_in_y, mul_node); @@ -244,6 +484,11 @@ void Flatten2MatmulFusePass::ApplyImpl(ir::Graph* graph) const { int found_count = 0; auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass in op compat failed."; + return; + } + VLOG(4) << "fuse flatten2+matmul to mul"; GET_IR_NODE_FROM_SUBGRAPH(flatten2_in_x, flatten2_in_x, fuse_pattern); GET_IR_NODE_FROM_SUBGRAPH(flatten2_op, flatten2_op, fuse_pattern); @@ -301,6 +546,11 @@ void Flatten2MatmulFusePass::ApplyImpl(ir::Graph* graph) const { IR_NODE_LINK_TO(mul_node, matmul_out); GraphSafeRemoveNodes(graph, {flatten2_op, matmul_in_x, matmul_op}); ++found_count; + + if (!IsCompat(desc)) { + LOG(WARNING) << "Flatten2MatmulFusePass in out mul op compat failed."; + return; + } } }; diff --git a/paddle/fluid/framework/ir/map_matmul_to_mul_pass.h b/paddle/fluid/framework/ir/map_matmul_to_mul_pass.h index 85067a6f642fe4..192dcfc00f9d34 100644 --- a/paddle/fluid/framework/ir/map_matmul_to_mul_pass.h +++ b/paddle/fluid/framework/ir/map_matmul_to_mul_pass.h @@ -39,6 +39,7 @@ class Graph; class MapMatmul2MulPass : public FusePassBase { public: + MapMatmul2MulPass(); virtual ~MapMatmul2MulPass() {} protected: @@ -66,6 +67,7 @@ class MapMatmul2MulPass : public FusePassBase { class Squeeze2MatmulFusePass : public FusePassBase { public: + Squeeze2MatmulFusePass(); virtual ~Squeeze2MatmulFusePass() {} protected: @@ -95,6 +97,7 @@ class Squeeze2MatmulFusePass : public FusePassBase { class Reshape2MatmulFusePass : public FusePassBase { public: + Reshape2MatmulFusePass(); virtual ~Reshape2MatmulFusePass() {} protected: @@ -103,6 +106,7 @@ class Reshape2MatmulFusePass : public FusePassBase { class Flatten2MatmulFusePass : public FusePassBase { public: + Flatten2MatmulFusePass(); virtual ~Flatten2MatmulFusePass() {} protected: diff --git a/paddle/fluid/framework/ir/mkldnn/conv_activation_mkldnn_fuse_pass.cc b/paddle/fluid/framework/ir/mkldnn/conv_activation_mkldnn_fuse_pass.cc index 7c749d9274299a..79a31e5cdc7b33 100644 --- a/paddle/fluid/framework/ir/mkldnn/conv_activation_mkldnn_fuse_pass.cc +++ b/paddle/fluid/framework/ir/mkldnn/conv_activation_mkldnn_fuse_pass.cc @@ -49,6 +49,11 @@ void ConvActivationFusePass::ApplyImpl(ir::Graph* graph) const { auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { VLOG(4) << "handle " + conv_type() + "+" + activation_type() + " fuse"; + + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass op compat failed."; + return; + } GET_IR_NODE_FROM_SUBGRAPH(conv_weight, conv_weight, conv_activation_pattern); // Filter GET_IR_NODE_FROM_SUBGRAPH(conv_out, conv_out, @@ -97,6 +102,113 @@ void ConvActivationFusePass::ApplyImpl(ir::Graph* graph) const { AddStatis(found_conv_activation_count); } +ConvActivationFusePass::ConvActivationFusePass() { + AddOpCompat(OpCompat("conv2d")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("Filter") + .IsTensor() + .End() + .AddInput("Bias") + .IsOptional() + .IsTensor() + .End() + .AddOutput("Output") + .IsTensor() + .End() + .AddAttr("strides") + .IsType>() + .End() + .AddAttr("paddings") + .IsType>() + .End() + // IsStringIn({"EXPLICIT", "SAME", "VALID"}), MobileNetV2 has no this + // attribute + .AddAttr("padding_algorithm") + .IsOptional() + .IsStringIn({"EXPLICIT", "SAME", "VALID"}) + .End() + .AddAttr("groups") + .IsNumGE(1) + .End() + .AddAttr("dilations") + .IsType>() + .End() + // IsStringIn({"NHWC", "NCHW"}) MobileNetV2 has no this attribute + .AddAttr("data_format") + .IsOptional() + .IsStringIn({"NHWC", "NCHW", "AnyLayout"}) + .End(); + + AddOpCompat(OpCompat("relu")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End(); +} +Conv2DLeakyReLUFusePass::Conv2DLeakyReLUFusePass() { + AddOpCompat(OpCompat("leaky_relu")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + // float, default=0.02 + .AddAttr("alpha") + .IsType() + .End(); +} +Conv2DReLU6FusePass::Conv2DReLU6FusePass() { + AddOpCompat(OpCompat("relu6")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + // default = 6.0f + .AddAttr("threshold") + .IsType() + .End(); +} +Conv2DSwishFusePass::Conv2DSwishFusePass() { + AddOpCompat(OpCompat("swish")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End(); +} +Conv2DHardSwishFusePass::Conv2DHardSwishFusePass() { + AddOpCompat(OpCompat("hard_swish")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + // float, optional, default=6.0 + .AddAttr("threshold") + .IsOptional() + .IsType() + .End() + // float, optional, default=6.0 + .AddAttr("scale") + .IsOptional() + .IsType() + .End() + // float, optional, default=3.0 + .AddAttr("offset") + .IsOptional() + .IsType() + .End(); +} + } // namespace ir } // namespace framework } // namespace paddle diff --git a/paddle/fluid/framework/ir/mkldnn/conv_activation_mkldnn_fuse_pass.h b/paddle/fluid/framework/ir/mkldnn/conv_activation_mkldnn_fuse_pass.h index 2df27c420f6eca..d22773fb41904a 100644 --- a/paddle/fluid/framework/ir/mkldnn/conv_activation_mkldnn_fuse_pass.h +++ b/paddle/fluid/framework/ir/mkldnn/conv_activation_mkldnn_fuse_pass.h @@ -31,6 +31,7 @@ class Graph; class ConvActivationFusePass : public FusePassBase { public: + ConvActivationFusePass(); virtual ~ConvActivationFusePass() {} virtual std::string conv_type() const { return "conv2d"; } virtual std::string activation_type() const { return "relu"; } @@ -44,6 +45,7 @@ class ConvActivationFusePass : public FusePassBase { */ class Conv2DLeakyReLUFusePass : public ConvActivationFusePass { public: + Conv2DLeakyReLUFusePass(); std::string activation_type() const { return "leaky_relu"; } }; /* @@ -51,6 +53,7 @@ class Conv2DLeakyReLUFusePass : public ConvActivationFusePass { */ class Conv2DReLU6FusePass : public ConvActivationFusePass { public: + Conv2DReLU6FusePass(); std::string activation_type() const { return "relu6"; } }; /* @@ -58,6 +61,7 @@ class Conv2DReLU6FusePass : public ConvActivationFusePass { */ class Conv2DSwishFusePass : public ConvActivationFusePass { public: + Conv2DSwishFusePass(); std::string activation_type() const { return "swish"; } }; /* @@ -65,6 +69,7 @@ class Conv2DSwishFusePass : public ConvActivationFusePass { */ class Conv2DHardSwishFusePass : public ConvActivationFusePass { public: + Conv2DHardSwishFusePass(); std::string activation_type() const { return "hard_swish"; } }; } // namespace ir diff --git a/paddle/fluid/framework/ir/mkldnn/conv_activation_mkldnn_fuse_pass_tester.cc b/paddle/fluid/framework/ir/mkldnn/conv_activation_mkldnn_fuse_pass_tester.cc index 55bbad7a8875af..453197cda39154 100644 --- a/paddle/fluid/framework/ir/mkldnn/conv_activation_mkldnn_fuse_pass_tester.cc +++ b/paddle/fluid/framework/ir/mkldnn/conv_activation_mkldnn_fuse_pass_tester.cc @@ -15,6 +15,7 @@ #include "paddle/fluid/framework/ir/mkldnn/conv_activation_mkldnn_fuse_pass.h" #include +#include #include "paddle/fluid/framework/op_proto_maker.h" namespace paddle { @@ -30,9 +31,16 @@ void SetOp(ProgramDesc* prog, const std::string& type, const std::string& name, op->SetAttr("name", name); if (type == "conv2d") { op->SetAttr("use_mkldnn", use_mkldnn); + op->SetAttr("groups", 1); + op->SetAttr("padding_algorithm", std::string("EXPLICIT")); + op->SetAttr("data_format", std::string("NCHW")); + op->SetAttr("strides", std::vector({1, 1})); + op->SetAttr("dilations", std::vector({1, 1})); + op->SetAttr("paddings", std::vector({0, 0})); op->SetInput("Input", {inputs[0]}); op->SetInput("Filter", {inputs[1]}); op->SetInput("Bias", {inputs[2]}); + op->SetOutput("Output", outputs); } else if (is_activation) { op->SetAttr("use_mkldnn", use_mkldnn); op->SetInput("X", inputs); @@ -43,8 +51,9 @@ void SetOp(ProgramDesc* prog, const std::string& type, const std::string& name, } else if (type == "swish") { op->SetAttr("beta", 1.0f); } + op->SetOutput("Out", outputs); } - op->SetOutput("Out", outputs); + op->SetAttr(OpProtoAndCheckerMaker::OpRoleAttrName(), static_cast(OpRole::kForward)); } diff --git a/paddle/fluid/framework/ir/mkldnn/conv_bias_mkldnn_fuse_pass.cc b/paddle/fluid/framework/ir/mkldnn/conv_bias_mkldnn_fuse_pass.cc index 8d73a35bf09be8..efad207e172723 100644 --- a/paddle/fluid/framework/ir/mkldnn/conv_bias_mkldnn_fuse_pass.cc +++ b/paddle/fluid/framework/ir/mkldnn/conv_bias_mkldnn_fuse_pass.cc @@ -41,8 +41,10 @@ ConvBiasFusePass::ConvBiasFusePass() { .IsTensor() .End() .AddAttr("strides") + .IsType>() .End() .AddAttr("paddings") + .IsType>() .End() .AddAttr("padding_algorithm") .IsStringIn({"EXPLICIT", "SAME", "VALID"}) @@ -51,6 +53,7 @@ ConvBiasFusePass::ConvBiasFusePass() { .IsNumGE(1) .End() .AddAttr("dilations") + .IsType>() .End() .AddAttr("data_format") .IsStringIn({"NCHW", "NHWC"}) @@ -81,43 +84,67 @@ Conv2DTransposeBiasFusePass::Conv2DTransposeBiasFusePass() { .End() .AddInput("Bias") .IsTensor() + .IsOptional() .End() .AddOutput("Output") .IsTensor() .End() .AddAttr("output_padding") + .IsType>() + .IsOptional() .End() .AddAttr("output_size") - .IsNumGE(1) + .IsType>() + .IsOptional() .End() .AddAttr("groups") .IsNumGE(1) .End() .AddAttr("dilations") + .IsType>() .End() .AddAttr("strides") + .IsType>() .End() .AddAttr("paddings") + .IsType>() .End() .AddAttr("padding_algorithm") .IsStringIn({"EXPLICIT", "SAME", "VALID"}) .End() .AddAttr("data_format") - .IsStringIn({"NCHW", "NHWC"}) + .IsStringIn({"NCHW", "NHWC", "AnyLayout"}) .End(); +} - AddOpCompat(OpCompat("elementwise_add")) - .AddInput("X") +Conv3DBiasFusePass::Conv3DBiasFusePass() { + AddOpCompat(OpCompat("conv3d")) + .AddInput("Input") .IsTensor() .End() - .AddInput("Y") + .AddInput("Filter") .IsTensor() .End() - .AddOutput("Out") + .AddOutput("Output") .IsTensor() .End() - .AddAttr("axis") - .IsNumEQ(-1) + .AddAttr("strides") + .IsType>() + .End() + .AddAttr("paddings") + .IsType>() + .End() + .AddAttr("padding_algorithm") + .IsStringIn({"EXPLICIT", "SAME", "VALID"}) + .End() + .AddAttr("groups") + .IsNumGE(1) + .End() + .AddAttr("dilations") + .IsType>() + .End() + .AddAttr("data_format") + .IsStringIn({"NCHW", "NHWC"}) .End(); } diff --git a/paddle/fluid/framework/ir/mkldnn/conv_bias_mkldnn_fuse_pass.h b/paddle/fluid/framework/ir/mkldnn/conv_bias_mkldnn_fuse_pass.h index 20c683c094edfe..a74d7443ee1fe1 100644 --- a/paddle/fluid/framework/ir/mkldnn/conv_bias_mkldnn_fuse_pass.h +++ b/paddle/fluid/framework/ir/mkldnn/conv_bias_mkldnn_fuse_pass.h @@ -48,6 +48,7 @@ class Conv2DTransposeBiasFusePass : public ConvBiasFusePass { class Conv3DBiasFusePass : public ConvBiasFusePass { public: + Conv3DBiasFusePass(); std::string type() const override { return "conv3d"; } }; } // namespace ir diff --git a/paddle/fluid/framework/ir/mkldnn/conv_concat_relu_mkldnn_fuse_pass.cc b/paddle/fluid/framework/ir/mkldnn/conv_concat_relu_mkldnn_fuse_pass.cc index c4d7a12037293e..5fbfef08b7209b 100644 --- a/paddle/fluid/framework/ir/mkldnn/conv_concat_relu_mkldnn_fuse_pass.cc +++ b/paddle/fluid/framework/ir/mkldnn/conv_concat_relu_mkldnn_fuse_pass.cc @@ -23,7 +23,67 @@ namespace paddle { namespace framework { namespace ir { -class Graph; +ConvConcatReLUFusePass::ConvConcatReLUFusePass() { + AddOpCompat(OpCompat("conv2d")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("Filter") + .IsTensor() + .End() + .AddInput("Bias") + .IsTensor() + .IsOptional() + .End() + .AddInput("ResidualData") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Output") + .IsTensor() + .End() + .AddAttr("strides") + .IsType>() + .End() + .AddAttr("paddings") + .IsType>() + .End() + .AddAttr("padding_algorithm") + .IsOptional() + .IsStringIn({"EXPLICIT", "SAME", "VALID"}) + .End() + .AddAttr("groups") + .IsNumGE(1) + .End() + .AddAttr("dilations") + .IsType>() + .End() + .AddAttr("data_format") + .IsStringIn({"NCHW", "NHWC", "AnyLayout"}) + .End(); + + AddOpCompat(OpCompat("concat")) + .AddInput("X") // Input("X"): vector + .End() + .AddInput("AxisTensor") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("axis") + .IsNumGE(0) + .End(); + + AddOpCompat(OpCompat("relu")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End(); +} void ConvConcatReLUFusePass::FindConcatWithConvs( ir::Graph* graph, diff --git a/paddle/fluid/framework/ir/mkldnn/conv_concat_relu_mkldnn_fuse_pass.h b/paddle/fluid/framework/ir/mkldnn/conv_concat_relu_mkldnn_fuse_pass.h index f1faa84f3d59b7..af372dbf97c672 100644 --- a/paddle/fluid/framework/ir/mkldnn/conv_concat_relu_mkldnn_fuse_pass.h +++ b/paddle/fluid/framework/ir/mkldnn/conv_concat_relu_mkldnn_fuse_pass.h @@ -18,9 +18,6 @@ #include #include "paddle/fluid/framework/ir/fuse_pass_base.h" -#include "paddle/fluid/framework/ir/graph.h" -#include "paddle/fluid/framework/ir/graph_pattern_detector.h" -#include "paddle/fluid/framework/ir/pass.h" namespace paddle { namespace framework { @@ -31,10 +28,10 @@ namespace ir { * to a: * (multi ConvReLU) -> Concat -> next_op. */ -class Graph; class ConvConcatReLUFusePass : public FusePassBase { public: + ConvConcatReLUFusePass(); virtual ~ConvConcatReLUFusePass() {} protected: diff --git a/paddle/fluid/framework/ir/mkldnn/matmul_transpose_reshape_fuse_pass.cc b/paddle/fluid/framework/ir/mkldnn/matmul_transpose_reshape_fuse_pass.cc index 1f17a741f19094..e5bdb08fe4ab48 100644 --- a/paddle/fluid/framework/ir/mkldnn/matmul_transpose_reshape_fuse_pass.cc +++ b/paddle/fluid/framework/ir/mkldnn/matmul_transpose_reshape_fuse_pass.cc @@ -34,10 +34,13 @@ MatmulTransposeReshapeMKLDNNPass::MatmulTransposeReshapeMKLDNNPass() { .IsTensor() .End() .AddAttr("alpha") // unconstrained. can be any float value. + .IsType() .End() .AddAttr("transpose_X") // unconstrained. can be any bool value. + .IsType() .End() .AddAttr("transpose_Y") // unconstrained. can be any bool value. + .IsType() .End(); AddOpCompat(OpCompat("transpose2")) @@ -51,9 +54,7 @@ MatmulTransposeReshapeMKLDNNPass::MatmulTransposeReshapeMKLDNNPass() { .IsTensor() .End() .AddAttr("axis") // ints - .End() - .AddAttr("data_format") - .IsStringIn({"NHWC", "NCHW", "AnyLayout"}) + .IsType>() .End(); AddOpCompat(OpCompat("reshape2")) @@ -75,6 +76,7 @@ MatmulTransposeReshapeMKLDNNPass::MatmulTransposeReshapeMKLDNNPass() { .IsTensor() .End() .AddAttr("shape") // ints + .IsType>() .End(); } void MatmulTransposeReshapeMKLDNNPass::ApplyImpl(ir::Graph *graph) const { diff --git a/paddle/fluid/framework/ir/mkldnn/matmul_transpose_reshape_fuse_pass_tester.cc b/paddle/fluid/framework/ir/mkldnn/matmul_transpose_reshape_fuse_pass_tester.cc index ac4e6c383dad9d..d98d640e1002b1 100644 --- a/paddle/fluid/framework/ir/mkldnn/matmul_transpose_reshape_fuse_pass_tester.cc +++ b/paddle/fluid/framework/ir/mkldnn/matmul_transpose_reshape_fuse_pass_tester.cc @@ -28,7 +28,6 @@ void SetOp(ProgramDesc *prog, const std::string &type, op->SetOutput("Out", {outputs[0]}); if (type == "transpose2") { op->SetAttr("axis", std::vector({0, 2, 1, 3})); - op->SetAttr("data_format", std::string("NCHW")); op->SetOutput("XShape", {outputs[1]}); } if (type == "reshape2") { diff --git a/paddle/fluid/framework/ir/mkldnn/mkldnn_inplace_pass_tester.cc b/paddle/fluid/framework/ir/mkldnn/mkldnn_inplace_pass_tester.cc index 01abe5a8d281b6..90dc7801131074 100644 --- a/paddle/fluid/framework/ir/mkldnn/mkldnn_inplace_pass_tester.cc +++ b/paddle/fluid/framework/ir/mkldnn/mkldnn_inplace_pass_tester.cc @@ -167,7 +167,7 @@ TEST(MKLDNNInplacePass, inplace_softmax_branched) { TEST(MKLDNNInplacePass, inplace_elementwise_add) { // Two elementwise_add mkl-dnn enabled op instances to be made inplace - MKLDNNInplacePassTest().MainTest("elementwise_add", false, 1); + MKLDNNInplacePassTest().MainTest("elementwise_add", false, 0); } TEST(MKLDNNInplacePass, inplace_tanh) { MKLDNNInplacePassTest().MainTest("tanh", false, 1); diff --git a/paddle/fluid/framework/ir/mkldnn/reshape_transpose_matmul_mkldnn_fuse_pass.cc b/paddle/fluid/framework/ir/mkldnn/reshape_transpose_matmul_mkldnn_fuse_pass.cc index b4c53ec5f91ccb..26692849d977b5 100644 --- a/paddle/fluid/framework/ir/mkldnn/reshape_transpose_matmul_mkldnn_fuse_pass.cc +++ b/paddle/fluid/framework/ir/mkldnn/reshape_transpose_matmul_mkldnn_fuse_pass.cc @@ -23,6 +23,59 @@ namespace paddle { namespace framework { namespace ir { +ReshapeTransposeMatmulMkldnnFusePass::ReshapeTransposeMatmulMkldnnFusePass() { + AddOpCompat(OpCompat("reshape2")) + .AddInput("X") + .IsTensor() + .End() + // The reshape2 op for this pass should not have "Shape" and "ShapeTensor" + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("XShape") + .IsOptional() + .IsTensor() + .End() + .AddAttr("shape") + .IsType>() + .End(); + + AddOpCompat(OpCompat("transpose2")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("XShape") + .IsOptional() + .IsTensor() + .End() + .AddAttr("axis") + .IsType>() + .End(); + + AddOpCompat(OpCompat("matmul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("alpha") + .IsType() + .End() + .AddAttr("transpose_X") + .IsType() + .End() + .AddAttr("transpose_Y") + .IsType() + .End(); +} + void ReshapeTransposeMatmulMkldnnFusePass::Fuse( Graph *graph, bool with_reshape_xshape, bool with_transpose_xshape) const { GraphPatternDetector gpd; @@ -34,6 +87,11 @@ void ReshapeTransposeMatmulMkldnnFusePass::Fuse( int found_reshape_transpose_matmul_count = 0; auto handler = [&](const GraphPatternDetector::subgraph_t &subgraph, Graph *g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Op compatible check in " + "reshape_transpose_matmul_mkldnn_fuse_pass failed."; + return; + } VLOG(4) << "handle ReshapeTransposeMatmulMkldnn fuse"; GET_IR_NODE_FROM_SUBGRAPH(reshape_in, reshape_in, rtm_pattern); GET_IR_NODE_FROM_SUBGRAPH(reshape_op, reshape_op, rtm_pattern); diff --git a/paddle/fluid/framework/ir/mkldnn/reshape_transpose_matmul_mkldnn_fuse_pass.h b/paddle/fluid/framework/ir/mkldnn/reshape_transpose_matmul_mkldnn_fuse_pass.h index 7a53b3c498413e..4637d0659af8c5 100644 --- a/paddle/fluid/framework/ir/mkldnn/reshape_transpose_matmul_mkldnn_fuse_pass.h +++ b/paddle/fluid/framework/ir/mkldnn/reshape_transpose_matmul_mkldnn_fuse_pass.h @@ -17,8 +17,6 @@ #include #include "paddle/fluid/framework/ir/fuse_pass_base.h" -#include "paddle/fluid/framework/ir/graph.h" -#include "paddle/fluid/framework/ir/graph_pattern_detector.h" namespace paddle { namespace framework { @@ -26,11 +24,10 @@ namespace ir { /* * Fuse Reshape->Transpose->MatMul when MatMul uses mkldnn. */ -class Graph; class ReshapeTransposeMatmulMkldnnFusePass : public FusePassBase { public: - virtual ~ReshapeTransposeMatmulMkldnnFusePass() {} + ReshapeTransposeMatmulMkldnnFusePass(); protected: void ApplyImpl(ir::Graph* graph) const override; diff --git a/paddle/fluid/framework/ir/mkldnn/scale_matmul_fuse_pass.cc b/paddle/fluid/framework/ir/mkldnn/scale_matmul_fuse_pass.cc index a552e42619f368..13f1fa50d080a3 100644 --- a/paddle/fluid/framework/ir/mkldnn/scale_matmul_fuse_pass.cc +++ b/paddle/fluid/framework/ir/mkldnn/scale_matmul_fuse_pass.cc @@ -28,6 +28,45 @@ namespace ir { class Graph; using string::PrettyLogDetail; +ScaleMatmulFusePass::ScaleMatmulFusePass() { + AddOpCompat(OpCompat("matmul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("alpha") + .IsNumGT(0.0f) + .End() + .AddAttr("transpose_X") + .IsType() + .End() + .AddAttr("transpose_Y") + .IsType() + .End(); + + AddOpCompat(OpCompat("scale")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("scale") + .IsNumGT(0.0f) + .End() + .AddAttr("bias") + .IsNumEQ(0.0f) + .End() + .AddAttr("bias_after_scale") + .IsOptional() + .IsType() + .End(); +} void ScaleMatmulFusePass::ApplyImpl(ir::Graph* graph) const { PADDLE_ENFORCE_NOT_NULL(graph, @@ -43,6 +82,10 @@ void ScaleMatmulFusePass::ApplyImpl(ir::Graph* graph) const { int found_scale_matmul_fuse_count = 0; auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass in op compat failed."; + return; + } GET_IR_NODE_FROM_SUBGRAPH(scale_in, scale_in, scale_matmul_pattern); GET_IR_NODE_FROM_SUBGRAPH(scale_op, scale_op, scale_matmul_pattern); GET_IR_NODE_FROM_SUBGRAPH(scale_out, scale_out, scale_matmul_pattern); @@ -75,6 +118,11 @@ void ScaleMatmulFusePass::ApplyImpl(ir::Graph* graph) const { matmul_op->Op()->SetInput(matmul_op_input_name, std::vector({scale_in->Name()})); IR_NODE_LINK_TO(scale_in, matmul_op); + + if (!IsCompat(*matmul_op->Op())) { + LOG(WARNING) << "scale_matmul_fuse_pass in out fc op compat failed."; + return; + } GraphSafeRemoveNodes(graph, {scale_op, scale_out}); found_scale_matmul_fuse_count++; } diff --git a/paddle/fluid/framework/ir/mkldnn/scale_matmul_fuse_pass.h b/paddle/fluid/framework/ir/mkldnn/scale_matmul_fuse_pass.h index 32ff78d9a73683..acea8ba563dc05 100644 --- a/paddle/fluid/framework/ir/mkldnn/scale_matmul_fuse_pass.h +++ b/paddle/fluid/framework/ir/mkldnn/scale_matmul_fuse_pass.h @@ -24,6 +24,7 @@ class Graph; class ScaleMatmulFusePass : public FusePassBase { public: + ScaleMatmulFusePass(); virtual ~ScaleMatmulFusePass() {} protected: diff --git a/paddle/fluid/framework/ir/mkldnn/scale_matmul_fuse_pass_tester.cc b/paddle/fluid/framework/ir/mkldnn/scale_matmul_fuse_pass_tester.cc index d37d014a87b660..60f844ffc80cea 100644 --- a/paddle/fluid/framework/ir/mkldnn/scale_matmul_fuse_pass_tester.cc +++ b/paddle/fluid/framework/ir/mkldnn/scale_matmul_fuse_pass_tester.cc @@ -31,6 +31,8 @@ void SetOp(ProgramDesc* prog, const std::string& type, op->SetAttr("scale", scale); op->SetAttr("bias", bias); } else if (type == "matmul") { + op->SetAttr("transpose_X", false); + op->SetAttr("transpose_Y", false); op->SetInput("X", {inputs[0]}); if (inputs.size() > 1) op->SetInput("Y", {inputs[1]}); op->SetAttr("alpha", scale); diff --git a/paddle/fluid/framework/ir/multihead_matmul_fuse_pass.cc b/paddle/fluid/framework/ir/multihead_matmul_fuse_pass.cc index 57bee20247c964..5a97727da3b456 100644 --- a/paddle/fluid/framework/ir/multihead_matmul_fuse_pass.cc +++ b/paddle/fluid/framework/ir/multihead_matmul_fuse_pass.cc @@ -422,13 +422,335 @@ PDNode* MultiHeadMatmulPattern::operator()() { return transpose2_2_out_var; } -static int BuildFusionV2(Graph* graph, const std::string& name_scope, - Scope* scope) { +PDNode* MultiHeadMatmulV3Pattern::operator()() { + std::unordered_set matmul_ops{"matmul", "matmul_v2"}; + auto* input0 = pattern->NewNode(input0_repr()); + input0->assert_is_op_input("matmul"); + + // First path with scale + auto* mul0 = pattern->NewNode(mul0_repr())->assert_is_op("matmul"); + auto* mul0_w_var = pattern->NewNode(mul0_w_repr()) + ->AsInput() + ->assert_is_op_input("matmul", "Y"); + auto* mul0_out_var = + pattern->NewNode(mul0_out_repr())->assert_is_op_output("matmul"); + + decltype(mul0) eltadd0; + decltype(mul0) eltadd0_b_var; + decltype(mul0) eltadd0_out_var; + + mul0_out_var->AsIntermediate()->assert_is_op_input("elementwise_add"); + + eltadd0 = pattern->NewNode(eltadd0_repr())->assert_is_op("elementwise_add"); + eltadd0_b_var = pattern->NewNode(eltadd0_b_repr()) + ->AsInput() + ->assert_is_op_input("elementwise_add", "Y"); + + eltadd0_out_var = pattern->NewNode(eltadd0_out_repr()) + ->assert_is_op_output("elementwise_add"); + eltadd0_out_var->AsIntermediate()->assert_is_op_input("reshape2"); + + auto* reshape2_0 = + pattern->NewNode(reshape2_0_repr())->assert_is_op("reshape2"); + + auto* reshape2_0_out_var = + pattern->NewNode(reshape2_0_out_repr())->assert_is_op_output("reshape2"); + reshape2_0_out_var->AsIntermediate()->assert_is_op_input("transpose2"); + + auto* transpose2_0 = + pattern->NewNode(transpose2_0_repr())->assert_is_op("transpose2"); + auto* transpose2_0_out_var = pattern->NewNode(transpose2_0_out_repr()) + ->assert_is_op_output("transpose2"); + transpose2_0_out_var->AsIntermediate()->assert_is_op_input("matmul", "X"); + + auto* matmul_qk = pattern->NewNode(matmul_qk_repr())->assert_is_op("matmul"); + auto* matmul_qk_out_var = + pattern->NewNode(matmul_qk_out_repr())->assert_is_op_output("matmul"); + matmul_qk_out_var->AsIntermediate()->assert_is_op_input("elementwise_add"); + + auto* eltadd_qk = + pattern->NewNode(eltadd_qk_repr())->assert_is_op("elementwise_add"); + auto* eltadd_qk_b_var = pattern->NewNode(eltadd_qk_b_repr()) + ->AsInput() + ->assert_is_op_input("elementwise_add", "Y"); + auto* eltadd_qk_out_var = pattern->NewNode(eltadd_qk_out_repr()) + ->assert_is_op_output("elementwise_add"); + eltadd_qk_out_var->AsIntermediate()->assert_is_op_input("softmax"); + + auto* softmax_qk = + pattern->NewNode(softmax_qk_repr())->assert_is_op("softmax"); + auto* softmax_qk_out_var = + pattern->NewNode(softmax_qk_out_repr())->assert_is_op_output("softmax"); + softmax_qk_out_var->AsIntermediate()->assert_is_ops_input(matmul_ops); + + auto* matmul_qkv = + pattern->NewNode(matmul_qkv_repr())->assert_is_ops(matmul_ops); + auto* matmul_qkv_out_var = + pattern->NewNode(matmul_qkv_out_repr())->assert_is_ops_output(matmul_ops); + matmul_qkv_out_var->AsIntermediate()->assert_is_op_input("transpose2"); + + auto* transpose2_qkv = + pattern->NewNode(transpose2_qkv_repr())->assert_is_op("transpose2"); + auto* transpose2_qkv_out_var = pattern->NewNode(transpose2_qkv_out_repr()) + ->assert_is_op_output("transpose2"); + transpose2_qkv_out_var->AsIntermediate()->assert_is_op_input("reshape2"); + + auto* reshape2_qkv = + pattern->NewNode(reshape2_qkv_repr())->assert_is_op("reshape2"); + auto* reshape2_qkv_out_var = pattern->NewNode(reshape2_qkv_out_repr()) + ->assert_is_op_output("reshape2"); + reshape2_qkv_out_var->assert_is_op_input("matmul"); + + // Second path to matmul + auto* mul1 = pattern->NewNode(mul1_repr())->assert_is_op("matmul"); + auto* mul1_w_var = pattern->NewNode(mul1_w_repr()) + ->AsInput() + ->assert_is_op_input("matmul", "Y"); + auto* mul1_out_var = + pattern->NewNode(mul1_out_repr())->assert_is_op_output("matmul"); + + decltype(mul1) eltadd1; + decltype(mul1) eltadd1_b_var; + decltype(mul1) eltadd1_out_var; + + mul1_out_var->AsIntermediate()->assert_is_op_input("elementwise_add"); + eltadd1 = pattern->NewNode(eltadd1_repr())->assert_is_op("elementwise_add"); + eltadd1_b_var = pattern->NewNode(eltadd1_b_repr()) + ->AsInput() + ->assert_is_op_input("elementwise_add", "Y"); + + eltadd1_out_var = pattern->NewNode(eltadd1_out_repr()) + ->assert_is_op_output("elementwise_add"); + eltadd1_out_var->AsIntermediate()->assert_is_op_input("reshape2"); + + auto* reshape2_1 = + pattern->NewNode(reshape2_1_repr())->assert_is_op("reshape2"); + + auto* reshape2_1_out_var = + pattern->NewNode(reshape2_1_out_repr())->assert_is_op_output("reshape2"); + reshape2_1_out_var->AsIntermediate()->assert_is_op_input("transpose2"); + + auto* transpose2_1 = + pattern->NewNode(transpose2_1_repr())->assert_is_op("transpose2"); + auto* transpose2_1_out_var = pattern->NewNode(transpose2_1_out_repr()) + ->assert_is_op_output("transpose2"); + transpose2_1_out_var->AsIntermediate()->assert_is_op_input( + "matmul", "Y"); // link to matmul qk + + // Third path to matmul + auto* mul2 = pattern->NewNode(mul2_repr())->assert_is_op("matmul"); + auto* mul2_w_var = pattern->NewNode(mul2_w_repr()) + ->AsInput() + ->assert_is_op_input("matmul", "Y"); + auto* mul2_out_var = + pattern->NewNode(mul2_out_repr())->assert_is_op_output("matmul"); + + decltype(mul2) eltadd2; + decltype(mul2) eltadd2_b_var; + decltype(mul2) eltadd2_out_var; + + mul2_out_var->AsIntermediate()->assert_is_op_input("elementwise_add"); + eltadd2 = pattern->NewNode(eltadd2_repr())->assert_is_op("elementwise_add"); + eltadd2_b_var = pattern->NewNode(eltadd2_b_repr()) + ->AsInput() + ->assert_is_op_input("elementwise_add", "Y"); + + eltadd2_out_var = pattern->NewNode(eltadd2_out_repr()) + ->assert_is_op_output("elementwise_add"); + eltadd2_out_var->AsIntermediate()->assert_is_op_input("reshape2"); + + auto* reshape2_2 = + pattern->NewNode(reshape2_2_repr())->assert_is_op("reshape2"); + + auto* reshape2_2_out_var = + pattern->NewNode(reshape2_2_out_repr())->assert_is_op_output("reshape2"); + reshape2_2_out_var->AsIntermediate()->assert_is_op_input("transpose2"); + + auto* transpose2_2 = + pattern->NewNode(transpose2_2_repr())->assert_is_op("transpose2"); + auto* transpose2_2_out_var = pattern->NewNode(transpose2_2_out_repr()) + ->assert_is_op_output("transpose2"); + transpose2_2_out_var->AsIntermediate()->assert_is_ops_input( + matmul_ops); // link to matmul qkv + + // Q path + mul0->LinksFrom({input0, mul0_w_var}).LinksTo({mul0_out_var}); + eltadd0->LinksFrom({mul0_out_var, eltadd0_b_var}).LinksTo({eltadd0_out_var}); + + reshape2_0->LinksFrom({eltadd0_out_var}).LinksTo({reshape2_0_out_var}); + transpose2_0->LinksFrom({reshape2_0_out_var}).LinksTo({transpose2_0_out_var}); + // K path + mul1->LinksFrom({input0, mul1_w_var}).LinksTo({mul1_out_var}); + eltadd1->LinksFrom({mul1_out_var, eltadd1_b_var}).LinksTo({eltadd1_out_var}); + reshape2_1->LinksFrom({eltadd1_out_var}).LinksTo({reshape2_1_out_var}); + transpose2_1->LinksFrom({reshape2_1_out_var}).LinksTo({transpose2_1_out_var}); + // compute q*k + matmul_qk->LinksFrom({transpose2_0_out_var, transpose2_1_out_var}) + .LinksTo({matmul_qk_out_var}); + eltadd_qk->LinksFrom({matmul_qk_out_var, eltadd_qk_b_var}) + .LinksTo({eltadd_qk_out_var}); + softmax_qk->LinksFrom({eltadd_qk_out_var}).LinksTo({softmax_qk_out_var}); + // V path + mul2->LinksFrom({input0, mul2_w_var}).LinksTo({mul2_out_var}); + eltadd2->LinksFrom({mul2_out_var, eltadd2_b_var}).LinksTo({eltadd2_out_var}); + reshape2_2->LinksFrom({eltadd2_out_var}).LinksTo({reshape2_2_out_var}); + transpose2_2->LinksFrom({reshape2_2_out_var}).LinksTo({transpose2_2_out_var}); + // compute q*k*v + matmul_qkv->LinksFrom({softmax_qk_out_var, transpose2_2_out_var}) + .LinksTo({matmul_qkv_out_var}); + transpose2_qkv->LinksFrom({matmul_qkv_out_var}) + .LinksTo({transpose2_qkv_out_var}); + reshape2_qkv->LinksFrom({transpose2_qkv_out_var}) + .LinksTo({reshape2_qkv_out_var}); + + return transpose2_2_out_var; +} +} // namespace patterns + +void MultiHeadMatmulFusePass::ApplyImpl(Graph* graph) const { + FusePassBase::Init(name_scope_, graph); + + int fusion_count = patterns::BuildFusion(graph, name_scope_); + AddStatis(fusion_count); +} + +MultiHeadMatmulV2FusePass::MultiHeadMatmulV2FusePass() { + AddOpCompat(OpCompat("mul")) + .AddInput("X") // the shape shoule be (B, S, N*H) + .IsTensor() + .End() + .AddInput("Y") // the shape shoule be (N*H, N*H) + .IsTensor() + .End() + .AddOutput("Out") // the shape shoule be (B, S, N*H) + .IsTensor() + .End() + .AddAttr("x_num_col_dims") + .IsNumEQ(2) + .End() + .AddAttr("y_num_col_dims") + .IsNumEQ(1) + .End(); + + AddOpCompat(OpCompat("elementwise_add")) + .AddInput("X") + // in bias, shape is (B, S, N*H), + // in biasqk, shape is (B, H, S, S) + .IsTensor() + .End() + .AddInput("Y") + // in bias, shape is (N*H) + // in biasqk, shape is (B, H, S, S) + .IsTensor() + .End() + // in bias, shape is (B, S, N*H) + // in biasqk, shape is (B, H, S, S) + .AddOutput("Out") + .IsTensor() + .End() + // in bias, it equal to 2 + // in biasqk, it equal to -1 or 0 + .AddAttr("axis") + .IsIntIn({2, -1, 0}) + .End(); + + AddOpCompat(OpCompat("reshape2")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Shape") + .IsTensor() + .IsOptional() + .End() + .AddInput("ShapeTensor") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("XShape") + .IsTensor() + .End() + .AddAttr("shape") // -->(B, S, H, N) <--(B, S, N*H) + .IsType>() + .End(); + + // -->: (B, S, H, N) -> (B, H, S, N) + // <--: (B, H, S, N) -> (B, S, H, N) + AddOpCompat(OpCompat("transpose2")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("XShape") + .IsTensor() + .End() + .AddAttr("axis") // {0, 2, 1, 3} + .IsType>() + .End(); + + AddOpCompat(OpCompat("scale")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("scale") + .IsType() // copy to new op. so unconstrained. + .End() + .AddAttr("bias") + .IsNumEQ(0.f) + .End() + .AddAttr("bias_after_scale") // bias is 0, so unconstrained. + .IsType() + .End(); + + // QK (B, H, S, N)*(B, H, S, N) -> (B, H, S, S) + // QKV (B, H, S, S)*(B, H, S, N) -> (B, H, S, N) + AddOpCompat(OpCompat("matmul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("alpha") + .IsNumEQ(1.0f) + .End() + .AddAttr("transpose_X") + .IsBoolEQ(false) + .End() + .AddAttr("transpose_Y") // QK(true) QKV(false) + .IsType() + .End(); + + AddOpCompat(OpCompat("softmax")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("axis") + .IsIntIn({-1, 3}) // shape is (B, H, S, S), so axis is -1 or 3 + .End(); +} + +int MultiHeadMatmulV2FusePass::BuildFusionV2(Graph* graph, + const std::string& name_scope, + Scope* scope) const { GraphPatternDetector gpd; auto* pattern = gpd.mutable_pattern(); // Create pattern. - MultiHeadMatmulPattern multihead_pattern(pattern, name_scope); + patterns::MultiHeadMatmulPattern multihead_pattern(pattern, name_scope); multihead_pattern(); // Create New OpDesc @@ -580,6 +902,11 @@ static int BuildFusionV2(Graph* graph, const std::string& name_scope, int fusion_count{0}; auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) + << "Op compat check in multihead_matmul_fuse_pass_v2 failed."; + return; + } // GET_IR_NODE_FROM_SUBGRAPH(dropout_out, dropout_out, multihead_pattern); GET_IR_NODE_FROM_SUBGRAPH(input0, input0, multihead_pattern); @@ -714,197 +1041,141 @@ static int BuildFusionV2(Graph* graph, const std::string& name_scope, return fusion_count; } -PDNode* MultiHeadMatmulV3Pattern::operator()() { - std::unordered_set matmul_ops{"matmul", "matmul_v2"}; - auto* input0 = pattern->NewNode(input0_repr()); - input0->assert_is_op_input("matmul"); - - // First path with scale - auto* mul0 = pattern->NewNode(mul0_repr())->assert_is_op("matmul"); - auto* mul0_w_var = pattern->NewNode(mul0_w_repr()) - ->AsInput() - ->assert_is_op_input("matmul", "Y"); - auto* mul0_out_var = - pattern->NewNode(mul0_out_repr())->assert_is_op_output("matmul"); - - decltype(mul0) eltadd0; - decltype(mul0) eltadd0_b_var; - decltype(mul0) eltadd0_out_var; - - mul0_out_var->AsIntermediate()->assert_is_op_input("elementwise_add"); - - eltadd0 = pattern->NewNode(eltadd0_repr())->assert_is_op("elementwise_add"); - eltadd0_b_var = pattern->NewNode(eltadd0_b_repr()) - ->AsInput() - ->assert_is_op_input("elementwise_add", "Y"); - - eltadd0_out_var = pattern->NewNode(eltadd0_out_repr()) - ->assert_is_op_output("elementwise_add"); - eltadd0_out_var->AsIntermediate()->assert_is_op_input("reshape2"); - - auto* reshape2_0 = - pattern->NewNode(reshape2_0_repr())->assert_is_op("reshape2"); - - auto* reshape2_0_out_var = - pattern->NewNode(reshape2_0_out_repr())->assert_is_op_output("reshape2"); - reshape2_0_out_var->AsIntermediate()->assert_is_op_input("transpose2"); - - auto* transpose2_0 = - pattern->NewNode(transpose2_0_repr())->assert_is_op("transpose2"); - auto* transpose2_0_out_var = pattern->NewNode(transpose2_0_out_repr()) - ->assert_is_op_output("transpose2"); - transpose2_0_out_var->AsIntermediate()->assert_is_op_input("matmul", "X"); - - auto* matmul_qk = pattern->NewNode(matmul_qk_repr())->assert_is_op("matmul"); - auto* matmul_qk_out_var = - pattern->NewNode(matmul_qk_out_repr())->assert_is_op_output("matmul"); - matmul_qk_out_var->AsIntermediate()->assert_is_op_input("elementwise_add"); - - auto* eltadd_qk = - pattern->NewNode(eltadd_qk_repr())->assert_is_op("elementwise_add"); - auto* eltadd_qk_b_var = pattern->NewNode(eltadd_qk_b_repr()) - ->AsInput() - ->assert_is_op_input("elementwise_add", "Y"); - auto* eltadd_qk_out_var = pattern->NewNode(eltadd_qk_out_repr()) - ->assert_is_op_output("elementwise_add"); - eltadd_qk_out_var->AsIntermediate()->assert_is_op_input("softmax"); - - auto* softmax_qk = - pattern->NewNode(softmax_qk_repr())->assert_is_op("softmax"); - auto* softmax_qk_out_var = - pattern->NewNode(softmax_qk_out_repr())->assert_is_op_output("softmax"); - softmax_qk_out_var->AsIntermediate()->assert_is_ops_input(matmul_ops); - - auto* matmul_qkv = - pattern->NewNode(matmul_qkv_repr())->assert_is_ops(matmul_ops); - auto* matmul_qkv_out_var = - pattern->NewNode(matmul_qkv_out_repr())->assert_is_ops_output(matmul_ops); - matmul_qkv_out_var->AsIntermediate()->assert_is_op_input("transpose2"); - - auto* transpose2_qkv = - pattern->NewNode(transpose2_qkv_repr())->assert_is_op("transpose2"); - auto* transpose2_qkv_out_var = pattern->NewNode(transpose2_qkv_out_repr()) - ->assert_is_op_output("transpose2"); - transpose2_qkv_out_var->AsIntermediate()->assert_is_op_input("reshape2"); - - auto* reshape2_qkv = - pattern->NewNode(reshape2_qkv_repr())->assert_is_op("reshape2"); - auto* reshape2_qkv_out_var = pattern->NewNode(reshape2_qkv_out_repr()) - ->assert_is_op_output("reshape2"); - reshape2_qkv_out_var->assert_is_op_input("matmul"); - - // Second path to matmul - auto* mul1 = pattern->NewNode(mul1_repr())->assert_is_op("matmul"); - auto* mul1_w_var = pattern->NewNode(mul1_w_repr()) - ->AsInput() - ->assert_is_op_input("matmul", "Y"); - auto* mul1_out_var = - pattern->NewNode(mul1_out_repr())->assert_is_op_output("matmul"); - - decltype(mul1) eltadd1; - decltype(mul1) eltadd1_b_var; - decltype(mul1) eltadd1_out_var; - - mul1_out_var->AsIntermediate()->assert_is_op_input("elementwise_add"); - eltadd1 = pattern->NewNode(eltadd1_repr())->assert_is_op("elementwise_add"); - eltadd1_b_var = pattern->NewNode(eltadd1_b_repr()) - ->AsInput() - ->assert_is_op_input("elementwise_add", "Y"); - - eltadd1_out_var = pattern->NewNode(eltadd1_out_repr()) - ->assert_is_op_output("elementwise_add"); - eltadd1_out_var->AsIntermediate()->assert_is_op_input("reshape2"); - - auto* reshape2_1 = - pattern->NewNode(reshape2_1_repr())->assert_is_op("reshape2"); - - auto* reshape2_1_out_var = - pattern->NewNode(reshape2_1_out_repr())->assert_is_op_output("reshape2"); - reshape2_1_out_var->AsIntermediate()->assert_is_op_input("transpose2"); - - auto* transpose2_1 = - pattern->NewNode(transpose2_1_repr())->assert_is_op("transpose2"); - auto* transpose2_1_out_var = pattern->NewNode(transpose2_1_out_repr()) - ->assert_is_op_output("transpose2"); - transpose2_1_out_var->AsIntermediate()->assert_is_op_input( - "matmul", "Y"); // link to matmul qk - - // Third path to matmul - auto* mul2 = pattern->NewNode(mul2_repr())->assert_is_op("matmul"); - auto* mul2_w_var = pattern->NewNode(mul2_w_repr()) - ->AsInput() - ->assert_is_op_input("matmul", "Y"); - auto* mul2_out_var = - pattern->NewNode(mul2_out_repr())->assert_is_op_output("matmul"); - - decltype(mul2) eltadd2; - decltype(mul2) eltadd2_b_var; - decltype(mul2) eltadd2_out_var; - - mul2_out_var->AsIntermediate()->assert_is_op_input("elementwise_add"); - eltadd2 = pattern->NewNode(eltadd2_repr())->assert_is_op("elementwise_add"); - eltadd2_b_var = pattern->NewNode(eltadd2_b_repr()) - ->AsInput() - ->assert_is_op_input("elementwise_add", "Y"); - - eltadd2_out_var = pattern->NewNode(eltadd2_out_repr()) - ->assert_is_op_output("elementwise_add"); - eltadd2_out_var->AsIntermediate()->assert_is_op_input("reshape2"); - - auto* reshape2_2 = - pattern->NewNode(reshape2_2_repr())->assert_is_op("reshape2"); - - auto* reshape2_2_out_var = - pattern->NewNode(reshape2_2_out_repr())->assert_is_op_output("reshape2"); - reshape2_2_out_var->AsIntermediate()->assert_is_op_input("transpose2"); - - auto* transpose2_2 = - pattern->NewNode(transpose2_2_repr())->assert_is_op("transpose2"); - auto* transpose2_2_out_var = pattern->NewNode(transpose2_2_out_repr()) - ->assert_is_op_output("transpose2"); - transpose2_2_out_var->AsIntermediate()->assert_is_ops_input( - matmul_ops); // link to matmul qkv - - // Q path - mul0->LinksFrom({input0, mul0_w_var}).LinksTo({mul0_out_var}); - eltadd0->LinksFrom({mul0_out_var, eltadd0_b_var}).LinksTo({eltadd0_out_var}); +void MultiHeadMatmulV2FusePass::ApplyImpl(Graph* graph) const { + FusePassBase::Init(name_scope_, graph); + auto* scope = param_scope(); + PADDLE_ENFORCE_NOT_NULL( + scope, + platform::errors::Fatal( + "During the multiheadMatmul pass, The scope should not be null.")); - reshape2_0->LinksFrom({eltadd0_out_var}).LinksTo({reshape2_0_out_var}); - transpose2_0->LinksFrom({reshape2_0_out_var}).LinksTo({transpose2_0_out_var}); - // K path - mul1->LinksFrom({input0, mul1_w_var}).LinksTo({mul1_out_var}); - eltadd1->LinksFrom({mul1_out_var, eltadd1_b_var}).LinksTo({eltadd1_out_var}); - reshape2_1->LinksFrom({eltadd1_out_var}).LinksTo({reshape2_1_out_var}); - transpose2_1->LinksFrom({reshape2_1_out_var}).LinksTo({transpose2_1_out_var}); - // compute q*k - matmul_qk->LinksFrom({transpose2_0_out_var, transpose2_1_out_var}) - .LinksTo({matmul_qk_out_var}); - eltadd_qk->LinksFrom({matmul_qk_out_var, eltadd_qk_b_var}) - .LinksTo({eltadd_qk_out_var}); - softmax_qk->LinksFrom({eltadd_qk_out_var}).LinksTo({softmax_qk_out_var}); - // V path - mul2->LinksFrom({input0, mul2_w_var}).LinksTo({mul2_out_var}); - eltadd2->LinksFrom({mul2_out_var, eltadd2_b_var}).LinksTo({eltadd2_out_var}); - reshape2_2->LinksFrom({eltadd2_out_var}).LinksTo({reshape2_2_out_var}); - transpose2_2->LinksFrom({reshape2_2_out_var}).LinksTo({transpose2_2_out_var}); - // compute q*k*v - matmul_qkv->LinksFrom({softmax_qk_out_var, transpose2_2_out_var}) - .LinksTo({matmul_qkv_out_var}); - transpose2_qkv->LinksFrom({matmul_qkv_out_var}) - .LinksTo({transpose2_qkv_out_var}); - reshape2_qkv->LinksFrom({transpose2_qkv_out_var}) - .LinksTo({reshape2_qkv_out_var}); + int fusion_count = BuildFusionV2(graph, name_scope_, scope); + if (fusion_count > 0) { + graph->Set(kMultiheadMatmulPass, new bool(true)); + } + AddStatis(fusion_count); +} - return transpose2_2_out_var; +MultiHeadMatmulV3FusePass::MultiHeadMatmulV3FusePass() { + AddOpCompat(OpCompat("mul")) + .AddInput("X") // the shape shoule be (B, S, N*H) + .IsTensor() + .End() + .AddInput("Y") // the shape shoule be (N*H, N*H) + .IsTensor() + .End() + .AddOutput("Out") // the shape shoule be (B, S, N*H) + .IsTensor() + .End() + .AddAttr("x_num_col_dims") + .IsNumEQ(2) + .End() + .AddAttr("y_num_col_dims") + .IsNumEQ(1) + .End(); + + AddOpCompat(OpCompat("elementwise_add")) + .AddInput("X") + // in bias, shape is (B, S, N*H), + // in biasqk, shape is (B, H, S, S) + .IsTensor() + .End() + .AddInput("Y") + // in bias, shape is (N*H) + // in biasqk, shape is (B, H, S, S) + .IsTensor() + .End() + // in bias, shape is (B, S, N*H) + // in biasqk, shape is (B, H, S, S) + .AddOutput("Out") + .IsTensor() + .End() + // in bias, it equal to 2 + // in biasqk, it equal to -1 or 0 + .AddAttr("axis") + .IsIntIn({2, -1, 0}) + .End(); + + AddOpCompat(OpCompat("reshape2")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Shape") + .IsTensor() + .IsOptional() + .End() + .AddInput("ShapeTensor") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("XShape") + .IsTensor() + .End() + .AddAttr("shape") // -->(B, S, H, N) <--(B, S, N*H) + .IsType>() + .End(); + + // -->: (B, S, H, N) -> (B, H, S, N) + // <--: (B, H, S, N) -> (B, S, H, N) + AddOpCompat(OpCompat("transpose2")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("XShape") + .IsTensor() + .End() + .AddAttr("axis") // {0, 2, 1, 3} + .IsType>() + .End(); + + // QK (B, H, S, N)*(B, H, S, N) -> (B, H, S, S) + // QKV (B, H, S, S)*(B, H, S, N) -> (B, H, S, N) + AddOpCompat(OpCompat("matmul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("alpha") + .IsType() // QK(anyvalue, will copy to new op) QKV(1.0) + .End() + .AddAttr("transpose_X") + .IsBoolEQ(false) + .End() + .AddAttr("transpose_Y") // QK(true) QKV(false) + .IsType() + .End(); + + AddOpCompat(OpCompat("softmax")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("axis") + .IsIntIn({-1, 3}) // shape is (B, H, S, S), so axis is -1 or 3 + .End(); } -static int BuildFusionV3(Graph* graph, const std::string& name_scope, - Scope* scope) { +int MultiHeadMatmulV3FusePass::BuildFusionV3(Graph* graph, + const std::string& name_scope, + Scope* scope) const { GraphPatternDetector gpd; auto* pattern = gpd.mutable_pattern(); // Create pattern. - MultiHeadMatmulV3Pattern multihead_pattern(pattern, name_scope); + patterns::MultiHeadMatmulV3Pattern multihead_pattern(pattern, name_scope); multihead_pattern(); // Create New OpDesc @@ -1155,30 +1426,6 @@ static int BuildFusionV3(Graph* graph, const std::string& name_scope, return fusion_count; } -} // namespace patterns - -void MultiHeadMatmulFusePass::ApplyImpl(Graph* graph) const { - FusePassBase::Init(name_scope_, graph); - - int fusion_count = patterns::BuildFusion(graph, name_scope_); - AddStatis(fusion_count); -} - -void MultiHeadMatmulV2FusePass::ApplyImpl(Graph* graph) const { - FusePassBase::Init(name_scope_, graph); - auto* scope = param_scope(); - PADDLE_ENFORCE_NOT_NULL( - scope, - platform::errors::Fatal( - "During the multiheadMatmul pass, The scope should not be null.")); - - int fusion_count = patterns::BuildFusionV2(graph, name_scope_, scope); - if (fusion_count > 0) { - graph->Set(kMultiheadMatmulPass, new bool(true)); - } - AddStatis(fusion_count); -} - void MultiHeadMatmulV3FusePass::ApplyImpl(Graph* graph) const { FusePassBase::Init(name_scope_, graph); auto* scope = param_scope(); @@ -1187,7 +1434,7 @@ void MultiHeadMatmulV3FusePass::ApplyImpl(Graph* graph) const { platform::errors::Fatal( "During the multiheadMatmul pass, The scope should not be null.")); - int fusion_count = patterns::BuildFusionV3(graph, name_scope_, scope); + int fusion_count = BuildFusionV3(graph, name_scope_, scope); if (fusion_count > 0) { graph->Set(kMultiheadMatmulPass, new bool(true)); } diff --git a/paddle/fluid/framework/ir/multihead_matmul_fuse_pass.h b/paddle/fluid/framework/ir/multihead_matmul_fuse_pass.h index c7f1336211d346..c39823e7325c19 100644 --- a/paddle/fluid/framework/ir/multihead_matmul_fuse_pass.h +++ b/paddle/fluid/framework/ir/multihead_matmul_fuse_pass.h @@ -18,16 +18,6 @@ #include #include "paddle/fluid/framework/ir/fuse_pass_base.h" -#include "paddle/fluid/framework/ir/graph.h" -#include "paddle/fluid/framework/ir/graph_pattern_detector.h" - -namespace paddle { -namespace framework { -namespace ir { -class Graph; -} // namespace ir -} // namespace framework -} // namespace paddle namespace paddle { namespace framework { @@ -158,22 +148,30 @@ class MultiHeadMatmulFusePass : public FusePassBase { class MultiHeadMatmulV2FusePass : public FusePassBase { public: - virtual ~MultiHeadMatmulV2FusePass() {} + MultiHeadMatmulV2FusePass(); protected: void ApplyImpl(Graph* graph) const; const std::string name_scope_{"multihead_matmul_fuse_v2"}; + + private: + int BuildFusionV2(Graph* graph, const std::string& name_scope, + Scope* scope) const; }; class MultiHeadMatmulV3FusePass : public FusePassBase { public: - virtual ~MultiHeadMatmulV3FusePass() {} + MultiHeadMatmulV3FusePass(); protected: void ApplyImpl(Graph* graph) const; const std::string name_scope_{"multihead_matmul_fuse_v3"}; + + private: + int BuildFusionV3(Graph* graph, const std::string& name_scope, + Scope* scope) const; }; } // namespace ir diff --git a/paddle/fluid/framework/ir/multihead_matmul_fuse_pass_tester.cc b/paddle/fluid/framework/ir/multihead_matmul_fuse_pass_tester.cc index 2eda643d4e53aa..b121436ee870b3 100644 --- a/paddle/fluid/framework/ir/multihead_matmul_fuse_pass_tester.cc +++ b/paddle/fluid/framework/ir/multihead_matmul_fuse_pass_tester.cc @@ -64,7 +64,7 @@ TEST(MultiHeadMatmulFusePass, basic) { // (transpose_qkv) reshape -> reshape_qkv // (reshape_qkv) mul -> mul_qkv Layers layers; - auto* x = layers.data("x", {128, 768}); + auto* x = layers.data("x", {1, 128, 768}); auto out = layers.layer_norm(x); auto* layer_out = out[0]; @@ -72,41 +72,41 @@ TEST(MultiHeadMatmulFusePass, basic) { auto* weights_1 = layers.data("weights1", {768, 768}, true); auto* weights_2 = layers.data("weights2", {768, 768}, true); - auto* mul_out_0 = layers.mul(layer_out, weights_0); - auto* mul_out_1 = layers.mul(layer_out, weights_1); - auto* mul_out_2 = layers.mul(layer_out, weights_2); + auto* mul_out_0 = layers.mul(layer_out, weights_0, nullptr, 2); + auto* mul_out_1 = layers.mul(layer_out, weights_1, nullptr, 2); + auto* mul_out_2 = layers.mul(layer_out, weights_2, nullptr, 2); auto* b0 = layers.data("bias_0", {768}, true); auto* b1 = layers.data("bias_1", {768}, true); auto* b2 = layers.data("bias_2", {768}, true); - auto* elementwise_out_0 = layers.elementwise_add(mul_out_0, b0); - auto* elementwise_out_1 = layers.elementwise_add(mul_out_1, b1); - auto* elementwise_out_2 = layers.elementwise_add(mul_out_2, b2); + auto* elementwise_out_0 = layers.elementwise_add(mul_out_0, b0, nullptr, 2); + auto* elementwise_out_1 = layers.elementwise_add(mul_out_1, b1, nullptr, 2); + auto* elementwise_out_2 = layers.elementwise_add(mul_out_2, b2, nullptr, 2); - std::vector shape = {128, 12, 64}; - auto* reshape_0 = layers.reshape2(elementwise_out_0, shape); - auto* reshape_1 = layers.reshape2(elementwise_out_1, shape); - auto* reshape_2 = layers.reshape2(elementwise_out_2, shape); + std::vector shape = {1, 128, 12, 64}; + auto* reshape_0 = layers.reshape2(elementwise_out_0, shape, true); + auto* reshape_1 = layers.reshape2(elementwise_out_1, shape, true); + auto* reshape_2 = layers.reshape2(elementwise_out_2, shape, true); std::vector axis = {0, 2, 1, 3}; - auto* transpose_0 = layers.transpose2(reshape_0, axis); - auto* transpose_1 = layers.transpose2(reshape_1, axis); - auto* transpose_2 = layers.transpose2(reshape_2, axis); + auto* transpose_0 = layers.transpose2(reshape_0, axis, true); + auto* transpose_1 = layers.transpose2(reshape_1, axis, true); + auto* transpose_2 = layers.transpose2(reshape_2, axis, true); auto* scale_0 = layers.scale(transpose_0, 0.125, 0, false); - auto* matmul_qk = layers.matmul(scale_0, transpose_1); + auto* matmul_qk = layers.matmul(scale_0, transpose_1, nullptr, false, true); - auto* bqk = layers.data("biasqk", {768}, true); + auto* bqk = layers.data("biasqk", {1, 12, 128, 128}, true); auto* elementwise_qk = layers.elementwise_add(matmul_qk, bqk); auto* softmax_qk = layers.softmax(elementwise_qk, -1); auto* matmul_qkv = layers.matmul(softmax_qk, transpose_2); - auto* transpose_qkv = layers.transpose2(matmul_qkv, {0, 2, 1, 3}); - auto* reshape_qkv_out = layers.reshape2(transpose_qkv, {128, 768}); + auto* transpose_qkv = layers.transpose2(matmul_qkv, {0, 2, 1, 3}, true); + auto* reshape_qkv_out = layers.reshape2(transpose_qkv, {1, 128, 768}, true); auto* weights_l = layers.data("weightsl", {768, 768}, true); - layers.mul(reshape_qkv_out, weights_l); + layers.mul(reshape_qkv_out, weights_l, nullptr, 2); std::unique_ptr graph(new ir::Graph(layers.main_program())); graph->Set("__param_scope__", CreateParamScope()); diff --git a/paddle/fluid/framework/ir/op_compat_sensible_pass.cc b/paddle/fluid/framework/ir/op_compat_sensible_pass.cc index cbb12839362f38..c0f17af3160ccd 100644 --- a/paddle/fluid/framework/ir/op_compat_sensible_pass.cc +++ b/paddle/fluid/framework/ir/op_compat_sensible_pass.cc @@ -23,6 +23,13 @@ namespace paddle { namespace framework { namespace ir { +AttrCompat& AttrCompat::IsStringEQ(const std::string& value) { + conditions_.emplace_back([value](const Attribute& attr) -> bool { + return value == BOOST_GET_CONST(std::string, attr); + }); + return *this; +} + AttrCompat& AttrCompat::IsStringIn(const std::set& candidates) { conditions_.emplace_back([candidates](const Attribute& attr) -> bool { std::string value = BOOST_GET_CONST(std::string, attr); @@ -61,7 +68,7 @@ AttrCompat& AttrCompat::IsLeftDefault() { return *this; } const OpInfo& op_info = OpInfoMap::Instance().Get(op_name); - const AttributeMap attrs = op_info.Checker()->GetAttrsDefaultValuesMap(); + const AttributeMap attrs = op_info.Checker()->GetDefaultAttrsMap(); if (attrs.find(attr_name_) == attrs.end()) { LOG(WARNING) << "Op (" << op_name << ") has no default attr:" << attr_name_; conditions_.emplace_back([](const Attribute& attr) { return false; }); @@ -260,7 +267,7 @@ bool OpCompatSensiblePass::IsCompat( auto op_type = node_pair.second->Op()->Type(); if (!op_compat_judgers_.count(op_type)) { if (HasOpDef(op_type)) { - LOG(WARNING) << op_type << "compat not registered!"; + LOG(WARNING) << op_type << " compat not registered!"; return false; } continue; diff --git a/paddle/fluid/framework/ir/op_compat_sensible_pass.h b/paddle/fluid/framework/ir/op_compat_sensible_pass.h index 7346ca3756f361..cfec1f123e238e 100644 --- a/paddle/fluid/framework/ir/op_compat_sensible_pass.h +++ b/paddle/fluid/framework/ir/op_compat_sensible_pass.h @@ -31,8 +31,14 @@ class AttrCompat { AttrCompat(const std::string& attr_name, OpCompat* op_compat) : optional_(false), attr_name_(attr_name), op_compat_(op_compat) {} + //! Assert the attribute type is `T`. + template + AttrCompat& IsType(); + // @{ String-related methods //! Assert the attribute is an string in the `candidates` domain. + AttrCompat& IsStringEQ(const std::string& value); + //! Assert the attribute is an string in the `candidates` domain. AttrCompat& IsStringIn(const std::set& candidates); //! Assert the attribute is a string and match a custom judging function. AttrCompat& IsStringMatch( @@ -207,6 +213,13 @@ class OpCompatSensiblePass : public Pass { std::map> op_compat_judgers_; }; +template +AttrCompat& AttrCompat::IsType() { + conditions_.emplace_back( + [](const Attribute& attr) -> bool { return attr.type() == typeid(T); }); + return *this; +} + template AttrCompat& AttrCompat::IsNumGT(T v) { conditions_.emplace_back([v](const Attribute& attr) -> bool { diff --git a/paddle/fluid/framework/ir/pass_tester_helper.h b/paddle/fluid/framework/ir/pass_tester_helper.h index f5639e7bc9af9f..284e54b3cb9f30 100644 --- a/paddle/fluid/framework/ir/pass_tester_helper.h +++ b/paddle/fluid/framework/ir/pass_tester_helper.h @@ -293,13 +293,17 @@ struct Layers { return outs; } - VarDesc* matmul(VarDesc* x, VarDesc* y, VarDesc* alpha = nullptr) { + VarDesc* matmul(VarDesc* x, VarDesc* y, VarDesc* alpha = nullptr, + bool transpose_x = false, bool transpose_y = false) { VarDesc* out = lod_tensor(unique_name()); OpDesc* op = program_.MutableBlock(0)->AppendOp(); op->SetType("matmul"); op->SetInput("X", {x->Name()}); op->SetInput("Y", {y->Name()}); op->SetOutput("Out", {out->Name()}); + op->SetAttr("transpose_X", transpose_x); + op->SetAttr("transpose_Y", transpose_y); + op->SetAttr("alpha", 1.0f); return out; } diff --git a/paddle/fluid/framework/ir/quant_conv2d_dequant_fuse_pass.cc b/paddle/fluid/framework/ir/quant_conv2d_dequant_fuse_pass.cc index 2fc39fd25d56c1..60675bf8488639 100644 --- a/paddle/fluid/framework/ir/quant_conv2d_dequant_fuse_pass.cc +++ b/paddle/fluid/framework/ir/quant_conv2d_dequant_fuse_pass.cc @@ -21,11 +21,216 @@ namespace paddle { namespace framework { namespace ir { - +QuantDequantFusePass::QuantDequantFusePass() { + AddOpCompat(OpCompat("fake_quantize_range_abs_max")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("InScale") + .IsTensor() + .End() + .AddInput("Iter") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("OutScale") + .IsTensor() + .End() + .AddOutput("OutScales") + .IsTensor() + .End() + .AddAttr("window_size") + .IsType() + .IsNumGT(0) + .End() + .AddAttr("bit_length") + .IsIntIn({8, 16}) + .End(); + AddOpCompat(OpCompat("fake_quantize_moving_average_abs_max")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("InScale") + .IsTensor() + .End() + .AddInput("InAccum") + .IsTensor() + .IsOptional() + .End() + .AddInput("InState") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("OutScale") + .IsTensor() + .End() + .AddOutput("OutState") + .IsTensor() + .IsOptional() + .End() + .AddOutput("OutAccum") + .IsTensor() + .IsOptional() + .End() + .AddAttr("moving_rate") + .IsType() + .IsNumGT(0.0f) + .End() + .AddAttr("bit_length") + .IsIntIn({8, 16}) + .End(); + AddOpCompat(OpCompat("fake_dequantize_max_abs")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Scale") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("max_range") + .IsType() + .IsNumGT(0.0f) + .End(); + AddOpCompat(OpCompat("fake_channel_wise_dequantize_max_abs")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Scales") // "Scales" is a vector with at most two tensors + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("quant_bits") + .IsType>() + .End() + .AddAttr("quant_axis") + .IsIntIn({0, 1}) + .IsOptional() + .End(); + AddOpCompat(OpCompat("conv2d")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("Filter") + .IsTensor() + .End() + .AddInput("Bias") + .IsTensor() + .IsOptional() + .End() + .AddInput("ResidualData") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Output") + .IsTensor() + .End() + .AddAttr("strides") + .IsType>() + .End() + .AddAttr("paddings") + .IsType>() + .End() + .AddAttr("padding_algorithm") + .IsOptional() + .IsStringIn({"EXPLICIT", "SAME", "VALID"}) + .End() + .AddAttr("groups") + .IsNumGE(1) + .End() + .AddAttr("dilations") + .IsType>() + .End() + .AddAttr("data_format") + .IsStringIn({"NCHW", "NHWC", "AnyLayout"}) + .End(); + AddOpCompat(OpCompat("mul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("x_num_col_dims") + .IsNumGE(1) + .End() + .AddAttr("y_num_col_dims") + .IsNumEQ(1) + .End(); + AddOpCompat(OpCompat("fc")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("W") + .IsTensor() + .End() + .AddInput("Bias") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("in_num_col_dims") + .IsNumGE(1) + .End() + .AddAttr("activation_type") + .IsStringIn({"relu", ""}) + .End(); + AddOpCompat(OpCompat("conv2d_transpose")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("Filter") + .IsTensor() + .End() + .AddInput("Bias") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Output") + .IsTensor() + .End() + .AddAttr("output_padding") + .IsType>() + .IsOptional() + .End() + .AddAttr("output_size") + .IsType>() + .IsOptional() + .End() + .AddAttr("groups") + .IsNumGE(1) + .End() + .AddAttr("dilations") + .IsType>() + .End() + .AddAttr("strides") + .IsType>() + .End() + .AddAttr("paddings") + .IsType>() + .End() + .AddAttr("padding_algorithm") + .IsStringIn({"EXPLICIT", "SAME", "VALID"}) + .End() + .AddAttr("data_format") + .IsStringIn({"NCHW", "NHWC", "AnyLayout"}) + .End(); +} // Delete quant op before quantized ops, and set input scale in the attr of // quantized ops -void DeleteQuant(ir::Graph* graph, Scope* scope, - const std::string& quant_type) { +void QuantDequantFusePass::DeleteQuant(ir::Graph* graph, Scope* scope, + const std::string& quant_type) const { const std::string pattern_name = "delete_quant_fuse"; GraphPatternDetector gpd; auto* input_act_node = gpd.mutable_pattern() @@ -41,6 +246,10 @@ void DeleteQuant(ir::Graph* graph, Scope* scope, // ops linked from it auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass in op compat failed."; + return; + } PADDLE_ENFORCE_EQ( subgraph.count(input_act_node), true, platform::errors::NotFound( @@ -103,9 +312,9 @@ void DeleteQuant(ir::Graph* graph, Scope* scope, // Delete dequant op after quantized ops, and convert weight from fp32 range to // int8 range -void FuseDequant(ir::Graph* graph, Scope* scope, - const std::string& quantized_op_type, - const std::string& dequant_type) { +void QuantDequantFusePass::FuseDequant(ir::Graph* graph, Scope* scope, + const std::string& quantized_op_type, + const std::string& dequant_type) const { std::string weight_name = ""; std::string input_name = ""; if (quantized_op_type == "conv2d" || @@ -142,6 +351,10 @@ void FuseDequant(ir::Graph* graph, Scope* scope, // Create new op desc auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass in op compat failed."; + return; + } PADDLE_ENFORCE_EQ( subgraph.count(quantized_op_input), true, platform::errors::NotFound("Quantized op input node(%s) did not find " diff --git a/paddle/fluid/framework/ir/quant_conv2d_dequant_fuse_pass.h b/paddle/fluid/framework/ir/quant_conv2d_dequant_fuse_pass.h index a16dc7620b4285..521e186c2be416 100644 --- a/paddle/fluid/framework/ir/quant_conv2d_dequant_fuse_pass.h +++ b/paddle/fluid/framework/ir/quant_conv2d_dequant_fuse_pass.h @@ -16,7 +16,6 @@ #include #include "paddle/fluid/framework/ir/fuse_pass_base.h" -#include "paddle/fluid/framework/ir/graph_pattern_detector.h" namespace paddle { namespace framework { @@ -25,14 +24,20 @@ namespace ir { /// /// Fuse quant + conv2d/depthwise_conv2d/mul/fc + dequant /// -class Graph; - class QuantDequantFusePass : public FusePassBase { public: + QuantDequantFusePass(); virtual ~QuantDequantFusePass() {} protected: void ApplyImpl(ir::Graph* graph) const override; + + private: + void DeleteQuant(ir::Graph* graph, Scope* scope, + const std::string& quant_type) const; + void FuseDequant(ir::Graph* graph, Scope* scope, + const std::string& quantized_op_type, + const std::string& dequant_type) const; }; } // namespace ir diff --git a/paddle/fluid/framework/ir/repeated_fc_relu_fuse_pass.cc b/paddle/fluid/framework/ir/repeated_fc_relu_fuse_pass.cc index 4c87b63625c1f6..a03a6f5b2c72c6 100644 --- a/paddle/fluid/framework/ir/repeated_fc_relu_fuse_pass.cc +++ b/paddle/fluid/framework/ir/repeated_fc_relu_fuse_pass.cc @@ -31,6 +31,27 @@ namespace paddle { namespace framework { namespace ir { +RepeatedFCReluFusePass::RepeatedFCReluFusePass() { + AddOpCompat(OpCompat("fc")) + .AddInput("Input") + .IsTensor() + .End() + .AddInput("W") + .IsTensor() + .End() + .AddInput("Bias") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("in_num_col_dims") + .IsNumEQ(1) + .End() + .AddAttr("activation_type") + .IsStringEQ("relu") + .End(); +} static bool IsInputOfFC(Node* n) { if (n && n->IsVar() && VarLinksToOp(n, "fc")) { return true; @@ -295,8 +316,9 @@ void BuildRepeatedFCReluPattern(PDPattern* pattern, } } -static int BuildFusion(Graph* graph, const std::string& name_scope, - int num_fc) { +int RepeatedFCReluFusePass::BuildFusion(Graph* graph, + const std::string& name_scope, + int num_fc) const { GraphPatternDetector gpd; auto* pattern = gpd.mutable_pattern(); BuildRepeatedFCReluPattern(pattern, name_scope, num_fc); @@ -316,6 +338,10 @@ static int BuildFusion(Graph* graph, const std::string& name_scope, int fusion_count{0}; auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "repeated_fc_relu_fuse_pass failed in op compat."; + return; + } LOG(INFO) << "handle Repeated FC Act fuse"; std::vector weights_vars(num_fc); std::vector bias_vars(num_fc); diff --git a/paddle/fluid/framework/ir/repeated_fc_relu_fuse_pass.h b/paddle/fluid/framework/ir/repeated_fc_relu_fuse_pass.h index 0be217cc748a24..b2933d26e07ab7 100644 --- a/paddle/fluid/framework/ir/repeated_fc_relu_fuse_pass.h +++ b/paddle/fluid/framework/ir/repeated_fc_relu_fuse_pass.h @@ -31,12 +31,16 @@ class Graph; class RepeatedFCReluFusePass : public FusePassBase { public: - virtual ~RepeatedFCReluFusePass() {} + RepeatedFCReluFusePass(); protected: void ApplyImpl(ir::Graph* graph) const override; const std::string name_scope_{"repeated_fc_relu_fuse"}; + + private: + int BuildFusion(Graph* graph, const std::string& name_scope, + int num_fc) const; }; } // namespace ir diff --git a/paddle/fluid/framework/ir/seqpool_cvm_concat_fuse_pass.cc b/paddle/fluid/framework/ir/seqpool_cvm_concat_fuse_pass.cc index 6bff4a05627d38..effaa0814ea79e 100644 --- a/paddle/fluid/framework/ir/seqpool_cvm_concat_fuse_pass.cc +++ b/paddle/fluid/framework/ir/seqpool_cvm_concat_fuse_pass.cc @@ -52,6 +52,52 @@ static void GetConcatNodes(ir::Graph* graph, std::vector* concat_nodes) { } } // anonymous namespace +SeqPoolCVMConcatFusePass::SeqPoolCVMConcatFusePass() { + AddOpCompat(OpCompat("sequence_pool")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("MaxIndex") + .IsTensor() + .IsOptional() + .End() + .AddAttr("pooltype") + .IsStringIn({"AVERAGE", "SUM", "SQRT", "LAST", "FIRST", "MAX"}) + .End() + .AddAttr("pad_value") + .End(); + AddOpCompat(OpCompat("cvm")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("CVM") + .IsTensor() + .End() + .AddOutput("Y") + .IsTensor() + .End() + .AddAttr("use_cvm") + .IsBoolEQ(true) + .End(); + AddOpCompat(OpCompat("concat")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("AxisTensor") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("axis") + .IsNumGE(1) + .End(); +} + void SeqPoolCVMConcatFusePass::ApplyImpl(ir::Graph* graph) const { FusePassBase::Init("seqpool_cvm_concat_fuse", graph); std::vector concat_nodes; diff --git a/paddle/fluid/framework/ir/seqpool_cvm_concat_fuse_pass.h b/paddle/fluid/framework/ir/seqpool_cvm_concat_fuse_pass.h index b0a3573fb59f97..7680c30e485a8e 100644 --- a/paddle/fluid/framework/ir/seqpool_cvm_concat_fuse_pass.h +++ b/paddle/fluid/framework/ir/seqpool_cvm_concat_fuse_pass.h @@ -44,7 +44,7 @@ class Graph; class SeqPoolCVMConcatFusePass : public FusePassBase { public: - virtual ~SeqPoolCVMConcatFusePass() {} + SeqPoolCVMConcatFusePass(); protected: void ApplyImpl(ir::Graph* graph) const override; diff --git a/paddle/fluid/framework/ir/shuffle_channel_detect_pass.cc b/paddle/fluid/framework/ir/shuffle_channel_detect_pass.cc index b9bd660043bf1b..1e9598fff87a8e 100644 --- a/paddle/fluid/framework/ir/shuffle_channel_detect_pass.cc +++ b/paddle/fluid/framework/ir/shuffle_channel_detect_pass.cc @@ -30,6 +30,44 @@ namespace ir { GET_IR_NODE(reshape2_op); \ GET_IR_NODE(reshape2_out); +ShuffleChannelDetectPass::ShuffleChannelDetectPass() { + AddOpCompat(OpCompat("reshape2")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Shape") + .IsOptional() + .IsTensor() + .End() + .AddInput("ShapeTensor") + .IsOptional() + .IsTensor() + .End() + .AddOutput("XShape") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("shape") + .IsType>() + .End(); + + AddOpCompat(OpCompat("transpose2")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("XShape") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("axis") + .IsType>() + .End(); +} + void ShuffleChannelDetectPass::ApplyImpl(ir::Graph* graph) const { const std::string pattern_name = "shufflechannel_pattern"; FusePassBase::Init(pattern_name, graph); @@ -46,7 +84,10 @@ void ShuffleChannelDetectPass::ApplyImpl(ir::Graph* graph) const { auto handler = [&](const GraphPatternDetector::subgraph_t& subgraph, Graph* g) { GET_NODES; - + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "The Pass in op compat failed."; + return; + } PADDLE_ENFORCE_GT( subgraph.count(x), 0, platform::errors::NotFound("Detector did not find input X.")); diff --git a/paddle/fluid/framework/ir/shuffle_channel_detect_pass.h b/paddle/fluid/framework/ir/shuffle_channel_detect_pass.h index d0caba5629f003..4576cfd865bb33 100644 --- a/paddle/fluid/framework/ir/shuffle_channel_detect_pass.h +++ b/paddle/fluid/framework/ir/shuffle_channel_detect_pass.h @@ -26,6 +26,7 @@ class Graph; class ShuffleChannelDetectPass : public FusePassBase { public: + ShuffleChannelDetectPass(); virtual ~ShuffleChannelDetectPass() {} protected: diff --git a/paddle/fluid/framework/ir/transpose_flatten_concat_fuse_pass.cc b/paddle/fluid/framework/ir/transpose_flatten_concat_fuse_pass.cc index 50d6b97bbea8ef..523c2161326466 100644 --- a/paddle/fluid/framework/ir/transpose_flatten_concat_fuse_pass.cc +++ b/paddle/fluid/framework/ir/transpose_flatten_concat_fuse_pass.cc @@ -19,7 +19,50 @@ namespace paddle { namespace framework { namespace ir { -void RunTransposeFlattenConcatFuse(ir::Graph *graph, int times) { +TransposeFlattenConcatFusePass::TransposeFlattenConcatFusePass() { + AddOpCompat(OpCompat("transpose2")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("XShape") + .IsTensor() + .End() + .AddAttr("axis") + .IsType>() + .End(); + AddOpCompat(OpCompat("flatten2")) + .AddInput("X") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddOutput("XShape") + .IsTensor() + .End() + .AddAttr("axis") + .IsNumGE(0) + .End(); + AddOpCompat(OpCompat("concat")) + .AddInput("X") // Input("X"): vector + .End() + .AddInput("AxisTensor") + .IsTensor() + .IsOptional() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("axis") + .IsIntIn({0, 1}) + .End(); +} + +void TransposeFlattenConcatFusePass::RunTransposeFlattenConcatFuse( + ir::Graph *graph, int times) const { const std::string pattern_name = "transpose_flatten" + std::to_string(times) + "_concat_fuse"; @@ -37,6 +80,10 @@ void RunTransposeFlattenConcatFuse(ir::Graph *graph, int times) { auto handler = [&](const GraphPatternDetector::subgraph_t &subgraph, Graph *g) { + if (!IsCompat(subgraph, g)) { + LOG(WARNING) << "Pass in op compat failed."; + return; + } const int kNumFields = 5; const int kTransOffset = 1; const int kTransOutOffset = 2; diff --git a/paddle/fluid/framework/ir/transpose_flatten_concat_fuse_pass.h b/paddle/fluid/framework/ir/transpose_flatten_concat_fuse_pass.h index 939a8c31e5501e..7c3ef2986e27e0 100644 --- a/paddle/fluid/framework/ir/transpose_flatten_concat_fuse_pass.h +++ b/paddle/fluid/framework/ir/transpose_flatten_concat_fuse_pass.h @@ -16,7 +16,6 @@ #include #include "paddle/fluid/framework/ir/fuse_pass_base.h" -#include "paddle/fluid/framework/ir/graph_pattern_detector.h" namespace paddle { namespace framework { @@ -28,10 +27,14 @@ namespace ir { // structure. class TransposeFlattenConcatFusePass : public FusePassBase { public: + TransposeFlattenConcatFusePass(); virtual ~TransposeFlattenConcatFusePass() {} protected: void ApplyImpl(ir::Graph* graph) const override; + + private: + void RunTransposeFlattenConcatFuse(ir::Graph* graph, int times) const; }; } // namespace ir diff --git a/paddle/fluid/framework/ir/unsqueeze2_eltwise_fuse_pass.cc b/paddle/fluid/framework/ir/unsqueeze2_eltwise_fuse_pass.cc index dc97e8c0233a60..d53431d260eaff 100644 --- a/paddle/fluid/framework/ir/unsqueeze2_eltwise_fuse_pass.cc +++ b/paddle/fluid/framework/ir/unsqueeze2_eltwise_fuse_pass.cc @@ -73,6 +73,46 @@ PDNode *UnsqueezeEltwise::operator()(PDNode *x, PDNode *y) { } // namespace patterns +UnsqueezeEltwiseFusePass::UnsqueezeEltwiseFusePass() { + AddOpCompat(OpCompat("unsqueeze2")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("AxesTensor") + .IsOptional() + .IsTensor() + .End() + .AddInput("AxesTensorList") + .IsOptional() + .IsTensor() + .End() + .AddOutput("XShape") + .IsOptional() + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + .AddAttr("axes") + .IsType>() + .End(); + + AddOpCompat(OpCompat("elementwise_mul")) + .AddInput("X") + .IsTensor() + .End() + .AddInput("Y") + .IsTensor() + .End() + .AddOutput("Out") + .IsTensor() + .End() + // The attribute value is - 1 before fusion and 0 after fusion + .AddAttr("axis") + .IsIntIn({-1, 0}) + .End(); +} + void UnsqueezeEltwiseFusePass::ApplyImpl(ir::Graph *graph) const { PADDLE_ENFORCE_NOT_NULL( graph, platform::errors::PreconditionNotMet("graph should not be null.")); @@ -100,7 +140,10 @@ void UnsqueezeEltwiseFusePass::ApplyImpl(ir::Graph *graph) const { LOG(WARNING) << "The subgraph is empty."; return; } - + if (!IsCompat(subgraph, graph)) { + LOG(WARNING) << "Pass in op compat failed."; + return; + } VLOG(4) << "handle UnsqueezeEltwise fuse"; GET_IR_NODE_FROM_SUBGRAPH(eltwise_op, elementwise, fused_pattern); GET_IR_NODE_FROM_SUBGRAPH(eltwise_out, eltwise_out, fused_pattern); @@ -123,6 +166,10 @@ void UnsqueezeEltwiseFusePass::ApplyImpl(ir::Graph *graph) const { IR_NODE_LINK_TO(eltwise_op, eltwise_out); GraphSafeRemoveNodes(graph, {unsqz_op, unsqz_out}); found_subgraph_count++; + if (!IsCompat(*eltwise_op->Op())) { + LOG(WARNING) << "unsqueeze2_eltwise_fuse_pass op compat failed."; + return; + } } }; diff --git a/paddle/fluid/framework/ir/unsqueeze2_eltwise_fuse_pass.h b/paddle/fluid/framework/ir/unsqueeze2_eltwise_fuse_pass.h index 3be29f0e028885..0410e5b3f330cd 100644 --- a/paddle/fluid/framework/ir/unsqueeze2_eltwise_fuse_pass.h +++ b/paddle/fluid/framework/ir/unsqueeze2_eltwise_fuse_pass.h @@ -34,6 +34,7 @@ class Graph; // it maybe change in runtime. class UnsqueezeEltwiseFusePass : public FusePassBase { public: + UnsqueezeEltwiseFusePass(); virtual ~UnsqueezeEltwiseFusePass() {} protected: diff --git a/paddle/fluid/framework/op_def_api.cc b/paddle/fluid/framework/op_def_api.cc index b950f000bb8e50..73f1409ae690e1 100644 --- a/paddle/fluid/framework/op_def_api.cc +++ b/paddle/fluid/framework/op_def_api.cc @@ -29,7 +29,6 @@ #include #include #include "glog/logging.h" -#include "io/fs.h" #include "paddle/fluid/framework/op_def.pb.h" /* diff --git a/paddle/fluid/framework/op_proto_maker.cc b/paddle/fluid/framework/op_proto_maker.cc index 0b9fd0a47e22c7..8fbea51584d3ca 100644 --- a/paddle/fluid/framework/op_proto_maker.cc +++ b/paddle/fluid/framework/op_proto_maker.cc @@ -66,6 +66,7 @@ void OpProtoAndCheckerMaker::operator()(proto::OpProto* proto, op_checker_ = attr_checker; Make(); op_checker_->RecordExplicitCheckerNum(); + op_checker_->InitDefaultAttributeMap(); AddAttr(OpRoleAttrName(), "The role of this operator") .InEnum( diff --git a/paddle/fluid/framework/tensor.cc b/paddle/fluid/framework/tensor.cc index b304a45be3cdcc..4f6eb803d1c26e 100644 --- a/paddle/fluid/framework/tensor.cc +++ b/paddle/fluid/framework/tensor.cc @@ -135,6 +135,49 @@ Tensor Tensor::Slice(int64_t begin_idx, int64_t end_idx) const { } } +std::vector Tensor::Split(int64_t split_size, int64_t axis) const { + check_memory_size(); + PADDLE_ENFORCE_GE(dims_.size(), 0, + platform::errors::OutOfRange( + "split expects at least a 1-dimensional tensor")); + PADDLE_ENFORCE_GE( + split_size, 0, + platform::errors::OutOfRange( + "split expects split_size be non-negative, but got split_size is %d", + split_size)); + int64_t numel_size = dims_[axis]; + + int64_t num_splits = 1; + if (split_size != 0) { + num_splits = + std::max((numel_size + split_size - 1) / split_size, 1); + } + + std::vector splits(num_splits); + int64_t last_split_size = split_size - (split_size * num_splits - numel_size); + + for (int64_t i = 0; i < num_splits; ++i) { + int64_t length = i < num_splits - 1 ? split_size : last_split_size; + splits[i] = Slice(i * split_size, i * split_size + length); + } + return splits; +} + +std::vector Tensor::Chunk(int64_t chunks, int64_t axis) const { + check_memory_size(); + PADDLE_ENFORCE_GE(dims_.size(), 0, + platform::errors::OutOfRange( + "split expects at least a 1-dimensional tensor")); + PADDLE_ENFORCE_GE( + chunks, 0, + platform::errors::OutOfRange( + "chunks expects to be greater than 0, but got chunks is %d", chunks)); + + int64_t numel_size = dims_[axis]; + int64_t split_size = (numel_size + chunks - 1) / chunks; + return Split(split_size, axis); +} + Tensor& Tensor::Resize(const DDim& dims) { dims_ = dims; return *this; diff --git a/paddle/fluid/framework/tensor.h b/paddle/fluid/framework/tensor.h index 0747321bcfa492..539859c45c9076 100644 --- a/paddle/fluid/framework/tensor.h +++ b/paddle/fluid/framework/tensor.h @@ -187,6 +187,22 @@ class Tensor { */ Tensor Slice(int64_t begin_idx, int64_t end_idx) const; + /** + * @brief Return a tensor list of the given tensor. + * + * @param[in] split_size The size of tensor to be split along axis. + * @param[in] axis The axis along which to split. + */ + std::vector Split(int64_t split_size, int64_t axis) const; + + /** + * @brief Return a tensor list of the given tensor. + * + * @param[in] chunks The number of tensor to be split along axis. + * @param[in] axis The axis along which to split. + */ + std::vector Chunk(int64_t chunks, int64_t axis) const; + const platform::Place& place() const { PADDLE_ENFORCE_NOT_NULL( holder_, diff --git a/paddle/fluid/framework/tensor_test.cc b/paddle/fluid/framework/tensor_test.cc index 101463756c0a51..71ff50c92ca59f 100644 --- a/paddle/fluid/framework/tensor_test.cc +++ b/paddle/fluid/framework/tensor_test.cc @@ -337,3 +337,129 @@ TEST(Tensor, FP16) { // Tensor holds the wrong type, it holds N6paddle8platform7float16E at // [/paddle/Paddle/paddle/fluid/framework/tensor_impl.h:43] } + +TEST(Tensor, Split) { + { + framework::Tensor src_tensor; + src_tensor.mutable_data(framework::make_ddim({6, 2}), + platform::CPUPlace()); + std::vector split_tensor_list = src_tensor.Split(2, 0); + ASSERT_EQ(split_tensor_list.size(), 3UL); + EXPECT_EQ(split_tensor_list[0].dims()[0], 2); + EXPECT_EQ(split_tensor_list[1].dims()[0], 2); + EXPECT_EQ(split_tensor_list[2].dims()[0], 2); + EXPECT_EQ(split_tensor_list[0].dims()[1], 2); + EXPECT_EQ(split_tensor_list[1].dims()[1], 2); + EXPECT_EQ(split_tensor_list[2].dims()[1], 2); + + uintptr_t src_data_address = + reinterpret_cast(src_tensor.data()); + uintptr_t src_mutable_data_address = reinterpret_cast( + src_tensor.mutable_data(src_tensor.dims(), platform::CPUPlace())); + EXPECT_EQ(src_data_address, src_mutable_data_address); + for (int i = 0; i < 3; ++i) { + uintptr_t split_data_address = + reinterpret_cast(split_tensor_list[i].data()); + uintptr_t split_mutable_data_address = + reinterpret_cast(split_tensor_list[i].mutable_data( + split_tensor_list[i].dims(), platform::CPUPlace())); + EXPECT_EQ(split_data_address, split_mutable_data_address); + EXPECT_EQ(src_data_address + 2 * 2 * i * sizeof(int), split_data_address); + } + } +#if defined(PADDLE_WITH_CUDA) || defined(PADDLE_WITH_HIP) + { + framework::Tensor src_tensor; + src_tensor.mutable_data(framework::make_ddim({6, 4}), + platform::CUDAPlace(0)); + std::vector split_tensor_list = src_tensor.Split(2, 0); + ASSERT_EQ(split_tensor_list.size(), 3UL); + EXPECT_EQ(split_tensor_list[0].dims()[0], 2); + EXPECT_EQ(split_tensor_list[1].dims()[0], 2); + EXPECT_EQ(split_tensor_list[2].dims()[0], 2); + EXPECT_EQ(split_tensor_list[0].dims()[1], 4); + EXPECT_EQ(split_tensor_list[1].dims()[1], 4); + EXPECT_EQ(split_tensor_list[2].dims()[1], 4); + + uintptr_t src_data_address = + reinterpret_cast(src_tensor.data()); + uintptr_t src_mutable_data_address = + reinterpret_cast(src_tensor.mutable_data( + src_tensor.dims(), platform::CUDAPlace(0))); + EXPECT_EQ(src_data_address, src_mutable_data_address); + for (int i = 0; i < 3; ++i) { + uintptr_t split_data_address = + reinterpret_cast(split_tensor_list[i].data()); + uintptr_t split_mutable_data_address = + reinterpret_cast(split_tensor_list[i].mutable_data( + split_tensor_list[i].dims(), platform::CUDAPlace(0))); + EXPECT_EQ(split_data_address, split_mutable_data_address); + EXPECT_EQ(src_data_address + 2 * 4 * i * sizeof(double), + split_data_address); + } + } +#endif +} + +TEST(Tensor, Chunk) { + { + framework::Tensor src_tensor; + src_tensor.mutable_data(framework::make_ddim({6, 2}), + platform::CPUPlace()); + std::vector split_tensor_list = src_tensor.Chunk(3, 0); + ASSERT_EQ(split_tensor_list.size(), 3UL); + EXPECT_EQ(split_tensor_list[0].dims()[0], 2); + EXPECT_EQ(split_tensor_list[1].dims()[0], 2); + EXPECT_EQ(split_tensor_list[2].dims()[0], 2); + EXPECT_EQ(split_tensor_list[0].dims()[1], 2); + EXPECT_EQ(split_tensor_list[1].dims()[1], 2); + EXPECT_EQ(split_tensor_list[2].dims()[1], 2); + + uintptr_t src_data_address = + reinterpret_cast(src_tensor.data()); + uintptr_t src_mutable_data_address = reinterpret_cast( + src_tensor.mutable_data(src_tensor.dims(), platform::CPUPlace())); + for (int i = 0; i < 3; ++i) { + uintptr_t split_data_address = + reinterpret_cast(split_tensor_list[i].data()); + uintptr_t split_mutable_data_address = + reinterpret_cast(split_tensor_list[i].mutable_data( + split_tensor_list[i].dims(), platform::CPUPlace())); + EXPECT_EQ(src_data_address, src_mutable_data_address); + EXPECT_EQ(split_data_address, split_mutable_data_address); + EXPECT_EQ(src_data_address + 2 * 2 * i * sizeof(int), split_data_address); + } + } +#if defined(PADDLE_WITH_CUDA) || defined(PADDLE_WITH_HIP) + { + framework::Tensor src_tensor; + src_tensor.mutable_data(framework::make_ddim({6, 4}), + platform::CUDAPlace(0)); + std::vector split_tensor_list = src_tensor.Chunk(3, 0); + ASSERT_EQ(split_tensor_list.size(), 3UL); + EXPECT_EQ(split_tensor_list[0].dims()[0], 2); + EXPECT_EQ(split_tensor_list[1].dims()[0], 2); + EXPECT_EQ(split_tensor_list[2].dims()[0], 2); + EXPECT_EQ(split_tensor_list[0].dims()[1], 4); + EXPECT_EQ(split_tensor_list[1].dims()[1], 4); + EXPECT_EQ(split_tensor_list[2].dims()[1], 4); + + uintptr_t src_data_address = + reinterpret_cast(src_tensor.data()); + uintptr_t src_mutable_data_address = + reinterpret_cast(src_tensor.mutable_data( + src_tensor.dims(), platform::CUDAPlace(0))); + EXPECT_EQ(src_data_address, src_mutable_data_address); + for (int i = 0; i < 3; ++i) { + uintptr_t split_data_address = + reinterpret_cast(split_tensor_list[i].data()); + uintptr_t split_mutable_data_address = + reinterpret_cast(split_tensor_list[i].mutable_data( + split_tensor_list[i].dims(), platform::CUDAPlace(0))); + EXPECT_EQ(split_data_address, split_mutable_data_address); + EXPECT_EQ(src_data_address + 2 * 4 * i * sizeof(double), + split_data_address); + } + } +#endif +} diff --git a/paddle/fluid/framework/tensor_util.cc b/paddle/fluid/framework/tensor_util.cc index d8f6df3e0bacf9..7cd62e3e2a785d 100644 --- a/paddle/fluid/framework/tensor_util.cc +++ b/paddle/fluid/framework/tensor_util.cc @@ -278,7 +278,7 @@ void TensorCopy(const Tensor& src, const platform::Place& dst_place, Tensor* dst) { platform::DeviceContextPool& pool = platform::DeviceContextPool::Instance(); const platform::DeviceContext* dev_ctx; - if (platform::is_gpu_place(dst_place)) { + if (platform::is_gpu_place(dst_place) || platform::is_npu_place(dst_place)) { dev_ctx = pool.Get(dst_place); } else { dev_ctx = pool.Get(src.place()); diff --git a/paddle/fluid/framework/trainer.h b/paddle/fluid/framework/trainer.h index 636760029fedc4..fc8fb9327d5bb2 100644 --- a/paddle/fluid/framework/trainer.h +++ b/paddle/fluid/framework/trainer.h @@ -243,55 +243,6 @@ class HeterXpuTrainer : public TrainerBase { #endif }; -class HeterBoxTrainer : public TrainerBase { - public: - HeterBoxTrainer() {} - virtual ~HeterBoxTrainer() {} - virtual void Initialize(const TrainerDesc& trainer_desc, Dataset* data_set); - virtual void InitTrainerEnv(const ProgramDesc& main_program, - const platform::Place& place); - virtual void InitOtherEnv(const ProgramDesc& main_program); - virtual void Run(); - virtual void Finalize(); - virtual void RegisterHeterCallback(); - virtual void DumpWork(int tid); - virtual Scope* GetWorkerScope(int thread_id); - virtual void CacheProgram(const ProgramDesc& main_program) { - new (&program_) ProgramDesc(main_program); - } - virtual std::string GetDumpPath(int tid) { return ""; } - virtual void InitDumpEnv() {} - template -#if defined(PADDLE_WITH_CUDA) || defined(PADDLE_WITH_HIP) - void HeterMemCpy(LoDTensor* tensor, LoDTensor* root_tensor, - const paddle::platform::Place& thread_place, - gpuStream_t stream); -#endif - void CreateThreadParam(const ProgramDesc& program, int num); - template - void MergeToRootScope(LoDTensor* root_tensor, LoDTensor* thread_tensor); - - protected: - DownpourWorkerParameter param_; - std::map> dense_grad_names_; - std::vector need_merge_var_names_; - float scale_datanorm_; - paddle::platform::Place place_; - ProgramDesc program_; - std::shared_ptr fleet_ptr_; - std::shared_ptr pull_dense_worker_; - std::vector> workers_; - std::vector places_; - // ps-gpu - std::vector pull_threads_; - std::vector threads_; - int use_ps_gpu_; - int thread_num_; -#if defined(PADDLE_WITH_CUDA) || defined(PADDLE_WITH_HIP) - std::vector copy_streams_; - std::vector events_; -#endif -}; #endif #if (defined PADDLE_WITH_NCCL || defined PADDLE_WITH_RCCL) && \ diff --git a/paddle/fluid/framework/trainer_factory.cc b/paddle/fluid/framework/trainer_factory.cc index 15073b6f78c5b3..660511b1f268d9 100644 --- a/paddle/fluid/framework/trainer_factory.cc +++ b/paddle/fluid/framework/trainer_factory.cc @@ -70,7 +70,6 @@ REGISTER_TRAINER_CLASS(DistMultiTrainer); defined PADDLE_WITH_XPU) && \ (defined PADDLE_WITH_PSLIB) REGISTER_TRAINER_CLASS(HeterXpuTrainer); -REGISTER_TRAINER_CLASS(HeterBoxTrainer); #endif #if (defined PADDLE_WITH_NCCL || defined PADDLE_WITH_RCCL) && \ (defined PADDLE_WITH_PSLIB) diff --git a/paddle/fluid/framework/type_defs.h b/paddle/fluid/framework/type_defs.h index e43cccfe648165..951daea47bde3b 100644 --- a/paddle/fluid/framework/type_defs.h +++ b/paddle/fluid/framework/type_defs.h @@ -71,6 +71,7 @@ using DygraphGradOpMakerFN = const imperative::NameVarBaseMap& /*var_base_map_in*/, const imperative::NameVarBaseMap& /*var_base_map_out*/, const framework::AttributeMap& /*attributes*/, + const framework::AttributeMap& /*default attributes*/, const std::map& /*inplace_map*/)>; using InferVarTypeFN = diff --git a/paddle/fluid/imperative/amp_auto_cast.cc b/paddle/fluid/imperative/amp_auto_cast.cc index 647b7cb34f6598..eba30ff8edebf9 100644 --- a/paddle/fluid/imperative/amp_auto_cast.cc +++ b/paddle/fluid/imperative/amp_auto_cast.cc @@ -33,7 +33,8 @@ AmpOperators::AmpOperators() for (auto it = all_kernels.begin(); it != all_kernels.end(); it++) { bool supported = false; for (auto& kernel_type : it->second) { - if (platform::is_gpu_place(kernel_type.first.place_) && + if ((platform::is_gpu_place(kernel_type.first.place_) || + platform::is_xpu_place(kernel_type.first.place_)) && kernel_type.first.data_type_ == fp16_dtype) { supported = true; } @@ -91,7 +92,8 @@ inline std::string GetDtypeStr( inline bool NeedCast(const std::shared_ptr& var) { if (platform::is_gpu_place(var->Place()) || - platform::is_cuda_pinned_place(var->Place())) { + platform::is_cuda_pinned_place(var->Place()) || + platform::is_xpu_place(var->Place())) { // CudaPinndePlace is added for varbase created by dataloader if (var->DataType() == framework::proto::VarType::FP32 || var->DataType() == framework::proto::VarType::FP16) { diff --git a/paddle/fluid/imperative/basic_engine.cc b/paddle/fluid/imperative/basic_engine.cc index 7bcc3d6c608c94..84ee1fbe5df96a 100644 --- a/paddle/fluid/imperative/basic_engine.cc +++ b/paddle/fluid/imperative/basic_engine.cc @@ -474,10 +474,11 @@ void BasicEngine::Execute() { try { if (tmp_ins_ptr == nullptr) { OpBase::Run(cur_op.InnerOp(), bwd_ins, tmp_outs, cur_op.Attrs(), - cur_op.place()); + cur_op.DefaultAttrsMap(), cur_op.place()); } else { OpBase::Run(cur_op.InnerOp(), *tmp_ins_ptr, tmp_outs, - cur_op.Attrs(), cur_op.place()); + cur_op.Attrs(), cur_op.DefaultAttrsMap(), + cur_op.place()); } } catch (platform::EnforceNotMet& exception) { Clear(); diff --git a/paddle/fluid/imperative/dygraph_grad_maker.h b/paddle/fluid/imperative/dygraph_grad_maker.h index 7fefc9ccc67b52..f1eb8aa62c9271 100644 --- a/paddle/fluid/imperative/dygraph_grad_maker.h +++ b/paddle/fluid/imperative/dygraph_grad_maker.h @@ -113,9 +113,18 @@ class GradOpBaseMakerBase { return vec_temp; } + // Only for dygraph + void SetDygraphDefaultAttrsMap(const framework::AttributeMap& default_attrs) { + default_attrs_ = &default_attrs; + } + + const framework::AttributeMap& DefaultAttrsMap() const { + return *default_attrs_; + } + const framework::AttributeMap& Attrs() const { return attrs_; } - const framework::Attribute& GetAttr(const std::string& name) const { + virtual const framework::Attribute& GetAttr(const std::string& name) const { auto it = attrs_.find(name); PADDLE_ENFORCE_EQ( it != attrs_.end(), true, @@ -199,6 +208,7 @@ class GradOpBaseMakerBase { const NameVarBaseMap& var_base_map_in_; const NameVarBaseMap& var_base_map_out_; const framework::AttributeMap& attrs_; + const framework::AttributeMap* default_attrs_; const std::map& inplace_map_; }; @@ -285,6 +295,10 @@ class TracedGradOp { return op_->SetAttrMap(attrs); } + void SetDefaultAttrsMap(const framework::AttributeMap& attrs) { + return op_->SetDefaultAttrsMap(attrs); + } + void SetAttr(const std::string& name, const framework::Attribute& v) { op_->SetAttr(name, v); } diff --git a/paddle/fluid/imperative/execution_context.h b/paddle/fluid/imperative/execution_context.h index 398b1292e2ffe0..5446add86788b2 100644 --- a/paddle/fluid/imperative/execution_context.h +++ b/paddle/fluid/imperative/execution_context.h @@ -35,11 +35,13 @@ class DygraphExecutionContext : public framework::ExecutionContext { const framework::RuntimeContext& ctx, const NameVarMap& var_base_map_in, const NameVarMap& var_base_map_out, - const framework::AttributeMap& attrs) + const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs) : ExecutionContext(op, scope, device_context, ctx), var_base_map_in_(var_base_map_in), var_base_map_out_(var_base_map_out), - attrs_(attrs) {} + attrs_(attrs), + default_attrs_(default_attrs) {} std::string InputName(const std::string& name) const override { auto it = var_base_map_in_.find(name); @@ -92,7 +94,7 @@ class DygraphExecutionContext : public framework::ExecutionContext { } bool HasAttr(const std::string& name) const override { - return attrs_.count(name) != 0; + return attrs_.count(name) != 0 || default_attrs_.count(name) != 0; } const framework::AttributeMap& Attrs() const override { return attrs_; } @@ -100,9 +102,14 @@ class DygraphExecutionContext : public framework::ExecutionContext { const framework::Attribute& GetAttr(const std::string& name) const override { auto it = attrs_.find(name); - PADDLE_ENFORCE_NE( - it, attrs_.end(), - platform::errors::NotFound("can not find [%s] in attrs", name)); + if (it == attrs_.end()) { + it = default_attrs_.find(name); + if (it == default_attrs_.end()) { + PADDLE_THROW(platform::errors::NotFound( + "Can not find [%s] in attributes of op %s.", name, + this->GetOp().Type())); + } + } return it->second; } @@ -192,6 +199,7 @@ class DygraphExecutionContext : public framework::ExecutionContext { const NameVarMap& var_base_map_in_; const NameVarMap& var_base_map_out_; const framework::AttributeMap& attrs_; + const framework::AttributeMap& default_attrs_; }; } // namespace imperative diff --git a/paddle/fluid/imperative/infer_shape_context.h b/paddle/fluid/imperative/infer_shape_context.h index fcd4545a2c82d3..7efe1177f5dc78 100644 --- a/paddle/fluid/imperative/infer_shape_context.h +++ b/paddle/fluid/imperative/infer_shape_context.h @@ -35,10 +35,12 @@ class DygraphInferShapeContext : public framework::InferShapeContext { DygraphInferShapeContext(const NameVarMap* in, const NameVarMap* out, const framework::AttributeMap* attr, + const framework::AttributeMap* default_attr, const std::string op_type) : var_base_map_in_(in), var_base_map_out_(out), attrs_(attr), + default_attrs_(default_attr), op_type_(op_type) {} bool HasInput(const std::string& name) const override { @@ -101,7 +103,7 @@ class DygraphInferShapeContext : public framework::InferShapeContext { } framework::AttrReader Attrs() const override { - return framework::AttrReader(*attrs_); + return framework::AttrReader(*attrs_, *default_attrs_); } std::vector Inputs(const std::string& name) const override { @@ -395,6 +397,7 @@ class DygraphInferShapeContext : public framework::InferShapeContext { const NameVarMap* var_base_map_in_; const NameVarMap* var_base_map_out_; const framework::AttributeMap* attrs_; + const framework::AttributeMap* default_attrs_; const std::string op_type_; }; diff --git a/paddle/fluid/imperative/infer_var_type_context.h b/paddle/fluid/imperative/infer_var_type_context.h index f740507fa50860..7defc339f4f81d 100644 --- a/paddle/fluid/imperative/infer_var_type_context.h +++ b/paddle/fluid/imperative/infer_var_type_context.h @@ -32,20 +32,28 @@ class RuntimeInferVarTypeContext : public framework::InferVarTypeContext { public: RuntimeInferVarTypeContext(const NameVarMap& inputs, const NameVarMap& outputs, - const framework::AttributeMap& attrs_map) + const framework::AttributeMap& attrs_map, + const framework::AttributeMap& default_attrs_map) : InferVarTypeContext(nullptr, nullptr), inputs_(inputs), outputs_(outputs), - attrs_(attrs_map) {} + attrs_(attrs_map), + default_attrs_(default_attrs_map) {} virtual ~RuntimeInferVarTypeContext() {} framework::Attribute GetAttr(const std::string& name) const override { - auto iter = attrs_.find(name); - PADDLE_ENFORCE_EQ( - iter != attrs_.end(), true, - platform::errors::NotFound("Cannot find attribute %s", name)); - return iter->second; + auto it = attrs_.find(name); + + if (it == attrs_.end()) { + it = default_attrs_.find(name); + if (it == default_attrs_.end()) { + PADDLE_THROW(platform::errors::NotFound( + "Can not find [%s] in attributes.", name)); + } + } + + return it->second; } bool HasInput(const std::string& name) const override { @@ -233,6 +241,7 @@ class RuntimeInferVarTypeContext : public framework::InferVarTypeContext { const NameVarMap& inputs_; const NameVarMap& outputs_; const framework::AttributeMap& attrs_; + const framework::AttributeMap& default_attrs_; }; } // namespace imperative diff --git a/paddle/fluid/imperative/layer.cc b/paddle/fluid/imperative/layer.cc index a4af3117d3e32e..6e28ecd9971abc 100644 --- a/paddle/fluid/imperative/layer.cc +++ b/paddle/fluid/imperative/layer.cc @@ -329,6 +329,7 @@ static void OpBaseRunImpl(const framework::OperatorBase& op, const NameVarMap& ins, const NameVarMap& outs, const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs, const platform::Place& place) { auto* op_kernel = dynamic_cast(&op); PADDLE_ENFORCE_NOT_NULL( @@ -336,7 +337,8 @@ static void OpBaseRunImpl(const framework::OperatorBase& op, "Only support operator with kernel in Dygraph mode.")); auto& info = op.Info(); if (info.infer_var_type_) { - RuntimeInferVarTypeContext infer_var_type_ctx(ins, outs, attrs); + RuntimeInferVarTypeContext infer_var_type_ctx(ins, outs, attrs, + default_attrs); info.infer_var_type_(&infer_var_type_ctx); } @@ -369,13 +371,14 @@ static void OpBaseRunImpl(const framework::OperatorBase& op, * after the execution of op, but the original input is directly * overwritten in the previous dynamic graph implemention. */ - auto prepared_op = PreparedOp::Prepare(ins, outs, *op_kernel, place, attrs); + auto prepared_op = + PreparedOp::Prepare(ins, outs, *op_kernel, place, attrs, default_attrs); auto tmp_ins_ptr = PrepareData(*op_kernel, ins, prepared_op.kernel_type()); if (tmp_ins_ptr == nullptr) { - prepared_op.Run(ins, outs, attrs); + prepared_op.Run(ins, outs, attrs, default_attrs); } else { - prepared_op.Run(*tmp_ins_ptr, outs, attrs); + prepared_op.Run(*tmp_ins_ptr, outs, attrs, default_attrs); } VLOG(4) << LayerDebugString(op.Type(), ins, outs); @@ -395,16 +398,18 @@ void OpBase::Run(const framework::OperatorBase& op, const NameVarMap& ins, const NameVarMap& outs, const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs, const platform::Place& place) { - OpBaseRunImpl(op, ins, outs, attrs, place); + OpBaseRunImpl(op, ins, outs, attrs, default_attrs, place); } void OpBase::Run(const framework::OperatorBase& op, const NameVarMap& ins, const NameVarMap& outs, const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs, const platform::Place& place) { - OpBaseRunImpl(op, ins, outs, attrs, place); + OpBaseRunImpl(op, ins, outs, attrs, default_attrs, place); } void ClearNoNeedBufferInputs(OpBase* op) { @@ -446,15 +451,15 @@ void ClearNoNeedBufferInputs(OpBase* op) { std::shared_ptr CreateGradOpNode( const framework::OperatorBase& op, const NameVarBaseMap& ins, const NameVarBaseMap& outs, const framework::AttributeMap& attrs, - const platform::Place& place, + const framework::AttributeMap& default_attrs, const platform::Place& place, const std::map& inplace_map) { const auto& info = op.Info(); if (!info.dygraph_grad_op_maker_) { return nullptr; } - auto grad_node = - info.dygraph_grad_op_maker_(op.Type(), ins, outs, attrs, inplace_map); + auto grad_node = info.dygraph_grad_op_maker_(op.Type(), ins, outs, attrs, + default_attrs, inplace_map); if (grad_node && !grad_node->empty()) { for (auto& grad_op : *grad_node) { grad_op.SetId(OpBase::GenerateUniqueId()); diff --git a/paddle/fluid/imperative/layer.h b/paddle/fluid/imperative/layer.h index bbede47e364298..56e16ba199707c 100644 --- a/paddle/fluid/imperative/layer.h +++ b/paddle/fluid/imperative/layer.h @@ -108,7 +108,7 @@ class VarBase { void ClearGradVarBase() { grad_var_ = nullptr; } - void SetGradVarBase(VarBase& grad_var) { + void SetGradVarBase(const VarBase& grad_var) { MutableGradVarBase()->CopyFrom(grad_var, true); } @@ -283,7 +283,7 @@ class Layer { std::shared_ptr CreateGradOpNode( const framework::OperatorBase& op, const NameVarBaseMap& ins, const NameVarBaseMap& outs, const framework::AttributeMap& attrs, - const platform::Place& place, + const framework::AttributeMap& default_attrs, const platform::Place& place, const std::map& inplace_map); void ClearNoNeedBufferInputs(OpBase* op); diff --git a/paddle/fluid/imperative/op_base.h b/paddle/fluid/imperative/op_base.h index 0164ff9313cdfe..acb125a82925d7 100644 --- a/paddle/fluid/imperative/op_base.h +++ b/paddle/fluid/imperative/op_base.h @@ -50,6 +50,10 @@ class OpBase { const framework::AttributeMap& Attrs() const { return attrs_; } + const framework::AttributeMap& DefaultAttrsMap() const { + return *default_attrs_; + } + const framework::OpInfo& Info() const { PADDLE_ENFORCE_NOT_NULL(op_, platform::errors::PreconditionNotMet( "OpBase::Info() should be called after " @@ -99,6 +103,10 @@ class OpBase { void SetAttrMap(const framework::AttributeMap& attrs) { attrs_ = attrs; } + void SetDefaultAttrsMap(const framework::AttributeMap& default_attrs) { + default_attrs_ = &default_attrs; + } + void SetAttr(const std::string& name, const framework::Attribute& v) { attrs_[name] = v; } @@ -110,14 +118,23 @@ class OpBase { const framework::AttributeMap& Attrs() { return attrs_; } - bool HasAttr(const std::string& name) const { return attrs_.count(name) > 0; } + const framework::AttributeMap& DefaultAttrsMap() { return *default_attrs_; } + + bool HasAttr(const std::string& name) const { + return attrs_.count(name) > 0 || default_attrs_->count(name) > 0; + } const framework::Attribute& GetAttr(const std::string& name) const { auto it = attrs_.find(name); - PADDLE_ENFORCE_NE( - it, attrs_.end(), - platform::errors::NotFound("can not find attribute [%s]", name)); - return it->second; + if (it != attrs_.end()) { + return it->second; + } else { + auto it_default = default_attrs_->find(name); + PADDLE_ENFORCE_NE( + it_default, default_attrs_->end(), + platform::errors::NotFound("can not find attribute [%s]", name)); + return it_default->second; + } } template @@ -156,12 +173,14 @@ class OpBase { const NameVarMap& ins, const NameVarMap& outs, const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs, const platform::Place& place); static void Run(const framework::OperatorBase& op, const NameVarMap& ins, const NameVarMap& outs, const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs, const platform::Place& place); private: @@ -174,6 +193,7 @@ class OpBase { NameVarMap ins_; NameVarMap outs_; framework::AttributeMap attrs_; + const framework::AttributeMap* default_attrs_; std::unique_ptr op_; platform::Place place_; size_t id_{-1UL}; diff --git a/paddle/fluid/imperative/partial_grad_engine.cc b/paddle/fluid/imperative/partial_grad_engine.cc index 3da3a05ed1071c..d905b1350821c4 100644 --- a/paddle/fluid/imperative/partial_grad_engine.cc +++ b/paddle/fluid/imperative/partial_grad_engine.cc @@ -884,11 +884,13 @@ void PartialGradTask::RunEachOp(OpBase *op) { } // Run op - OpBase::Run(op->InnerOp(), tmp_ins, tmp_outs, op->Attrs(), op->place()); + OpBase::Run(op->InnerOp(), tmp_ins, tmp_outs, op->Attrs(), + op->DefaultAttrsMap(), op->place()); if (create_graph_) { - auto double_grad_node = CreateGradOpNode(op->InnerOp(), tmp_ins, tmp_outs, - op->Attrs(), op->place(), {}); + auto double_grad_node = + CreateGradOpNode(op->InnerOp(), tmp_ins, tmp_outs, op->Attrs(), + op->DefaultAttrsMap(), op->place(), {}); PADDLE_ENFORCE_NOT_NULL( double_grad_node, platform::errors::NotFound("The Op %s doesn't have any grad op. If you " diff --git a/paddle/fluid/imperative/prepared_operator.cc b/paddle/fluid/imperative/prepared_operator.cc index 4a42751b1c4d5b..57c6ae3cbb0a13 100644 --- a/paddle/fluid/imperative/prepared_operator.cc +++ b/paddle/fluid/imperative/prepared_operator.cc @@ -91,7 +91,8 @@ PreparedOp PrepareImpl(const NameVarMap& ins, const NameVarMap& outs, const framework::OperatorWithKernel& op, const platform::Place& place, - const framework::AttributeMap& attrs) { + const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs) { platform::DeviceContextPool& pool = platform::DeviceContextPool::Instance(); auto* dev_ctx = pool.Get(place); @@ -108,9 +109,9 @@ PreparedOp PrepareImpl(const NameVarMap& ins, #endif // 1. get expected kernel key - auto expected_kernel_key = - op.GetExpectedKernelType(DygraphExecutionContext( - op, framework::Scope(), *dev_ctx, ctx, ins, outs, attrs)); + auto expected_kernel_key = op.GetExpectedKernelType( + DygraphExecutionContext(op, framework::Scope(), *dev_ctx, ctx, + ins, outs, attrs, default_attrs)); VLOG(3) << "expected_kernel_key:" << expected_kernel_key; // 2. check if op[type] has kernel registered. @@ -127,6 +128,19 @@ PreparedOp PrepareImpl(const NameVarMap& ins, #ifdef PADDLE_WITH_XPU if (kernel_iter == kernels.end() && is_xpu_place(expected_kernel_key.place_)) { + VLOG(3) << "missing XPU kernel: " << op.Type() + << ", expected_kernel_key:" << expected_kernel_key + << ", fallbacking to CPU one!"; + expected_kernel_key.place_ = platform::CPUPlace(); + kernel_iter = kernels.find(expected_kernel_key); + } +#endif +#ifdef PADDLE_WITH_ASCEND_CL + if (kernel_iter == kernels.end() && + is_npu_place(expected_kernel_key.place_)) { + VLOG(3) << "missing NPU kernel: " << op.Type() + << ", expected_kernel_key:" << expected_kernel_key + << ", fallbacking to CPU one!"; expected_kernel_key.place_ = platform::CPUPlace(); kernel_iter = kernels.find(expected_kernel_key); } @@ -148,16 +162,19 @@ PreparedOp PreparedOp::Prepare(const NameVarMap& ins, const NameVarMap& outs, const framework::OperatorWithKernel& op, const platform::Place& place, - const framework::AttributeMap& attrs) { - return PrepareImpl(ins, outs, op, place, attrs); + const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs) { + return PrepareImpl(ins, outs, op, place, attrs, default_attrs); } PreparedOp PreparedOp::Prepare(const NameVarMap& ins, const NameVarMap& outs, const framework::OperatorWithKernel& op, const platform::Place& place, - const framework::AttributeMap& attrs) { - return PrepareImpl(ins, outs, op, place, attrs); + const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs) { + return PrepareImpl(ins, outs, op, place, attrs, + default_attrs); } template @@ -166,17 +183,18 @@ static void PreparedOpRunImpl( const framework::OpKernelType& kernel_type, const framework::OperatorWithKernel::OpKernelFunc& func, platform::DeviceContext* dev_ctx, const NameVarMap& ins, - const NameVarMap& outs, const framework::AttributeMap& attrs) { + const NameVarMap& outs, const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs) { // TODO(zjl): remove scope in dygraph framework::Scope scope; DygraphInferShapeContext infer_shape_ctx(&ins, &outs, &attrs, - op.Type()); + &default_attrs, op.Type()); static_cast(op).InferShape( &infer_shape_ctx); func(DygraphExecutionContext(op, scope, *dev_ctx, ctx, ins, outs, - attrs)); + attrs, default_attrs)); if (FLAGS_check_nan_inf) { framework::details::CheckOpHasNanOrInfInDygraph( @@ -202,16 +220,18 @@ static void PreparedOpRunImpl( void PreparedOp::Run(const NameVarMap& ins, const NameVarMap& outs, - const framework::AttributeMap& attrs) { + const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs) { PreparedOpRunImpl(op_, ctx_, kernel_type_, func_, dev_ctx_, ins, - outs, attrs); + outs, attrs, default_attrs); } void PreparedOp::Run(const NameVarMap& ins, const NameVarMap& outs, - const framework::AttributeMap& attrs) { + const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs) { PreparedOpRunImpl(op_, ctx_, kernel_type_, func_, dev_ctx_, - ins, outs, attrs); + ins, outs, attrs, default_attrs); } } // namespace imperative diff --git a/paddle/fluid/imperative/prepared_operator.h b/paddle/fluid/imperative/prepared_operator.h index 1f6be5483be30b..53f876c498cd04 100644 --- a/paddle/fluid/imperative/prepared_operator.h +++ b/paddle/fluid/imperative/prepared_operator.h @@ -151,20 +151,24 @@ class PreparedOp { const NameVarMap& outs, const framework::OperatorWithKernel& op, const platform::Place& place, - const framework::AttributeMap& attrs); + const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs); static PreparedOp Prepare(const NameVarMap& ins, const NameVarMap& outs, const framework::OperatorWithKernel& op, const platform::Place& place, - const framework::AttributeMap& attrs); + const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs); void Run(const NameVarMap& in, const NameVarMap& out, - const framework::AttributeMap& attrs); + const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs); void Run(const NameVarMap& ins, const NameVarMap& outs, - const framework::AttributeMap& attrs); + const framework::AttributeMap& attrs, + const framework::AttributeMap& default_attrs); const framework::OpKernelType& kernel_type() const { return kernel_type_; } diff --git a/paddle/fluid/imperative/tests/test_layer.cc b/paddle/fluid/imperative/tests/test_layer.cc index 4a30ffb7e3d01f..064f47f54979a1 100644 --- a/paddle/fluid/imperative/tests/test_layer.cc +++ b/paddle/fluid/imperative/tests/test_layer.cc @@ -43,10 +43,12 @@ template class TestRuntimeInferVarTypeContext : public RuntimeInferVarTypeContext { public: - TestRuntimeInferVarTypeContext(const NameVarMap& inputs, - const NameVarMap& outputs, - const framework::AttributeMap& attrs_map) - : RuntimeInferVarTypeContext(inputs, outputs, attrs_map) {} + TestRuntimeInferVarTypeContext( + const NameVarMap& inputs, const NameVarMap& outputs, + const framework::AttributeMap& attrs_map, + const framework::AttributeMap& default_attrs_map) + : RuntimeInferVarTypeContext(inputs, outputs, attrs_map, + default_attrs_map) {} bool HasVar(const std::string& name) const { return RuntimeInferVarTypeContext::HasVar(name); @@ -125,7 +127,7 @@ TEST(test_layer, test_runtime_context) { auto* ctx = new imperative::TestRuntimeInferVarTypeContext( - ins, outs, attrs); + ins, outs, attrs, {}); ASSERT_TRUE(ctx->HasInput("X")); ASSERT_TRUE(ctx->HasOutput("Out")); @@ -358,7 +360,7 @@ TEST(test_layer, test_dygraph_execution_context) { framework::Scope scope; DygraphExecutionContext dy_exe_context( - *(op.get()), scope, *dev_ctx, ctx, ins, outs, concat_att_map); + *(op.get()), scope, *dev_ctx, ctx, ins, outs, concat_att_map, {}); ASSERT_EQ(dy_exe_context.InputSize("X"), 1u); ASSERT_EQ(dy_exe_context.InputName("X"), "vin"); @@ -386,7 +388,7 @@ TEST(test_layer, test_dygraph_infershape_context) { concat_att_map["axis"] = 1; DygraphInferShapeContext infer_shape_ctx( - &ins, &outs, &concat_att_map, "dummy"); + &ins, &outs, &concat_att_map, {}, "dummy"); bool have_x = infer_shape_ctx.HasOutputs("Out"); ASSERT_EQ(have_x, true); diff --git a/paddle/fluid/imperative/tests/test_prepare_op.cc b/paddle/fluid/imperative/tests/test_prepare_op.cc index 7d6882a4ee7d00..5e269d74044d24 100644 --- a/paddle/fluid/imperative/tests/test_prepare_op.cc +++ b/paddle/fluid/imperative/tests/test_prepare_op.cc @@ -93,7 +93,7 @@ TEST(test_prepare_op, test_prepare_op) { ASSERT_NO_FATAL_FAILURE(PreparedOp preparedOp = PreparedOp::Prepare( ins, outs, dynamic_cast(*op), - place, split_attr_map)); + place, split_attr_map, {})); } const framework::Tensor* GetTensorFromVar(const framework::Variable& var); @@ -144,7 +144,7 @@ TEST(test_prepare_op, test_prepare_data) { // test if it can be transformed to GPU place auto prepared_op = PreparedOp::Prepare( ins, outs, dynamic_cast(*op), gpu_place, - attr_map); + attr_map, {}); PrepareData( dynamic_cast(*op), ins, prepared_op.kernel_type()); @@ -193,7 +193,7 @@ void TestPrepareDataSamePlace(framework::AttributeMap attr_map) { // test if it never transferred on GPU place auto prepared_op = PreparedOp::Prepare( ins, outs, dynamic_cast(*op), cpu_place, - attr_map); + attr_map, {}); PrepareData( dynamic_cast(*op), ins, prepared_op.kernel_type()); diff --git a/paddle/fluid/imperative/tracer.cc b/paddle/fluid/imperative/tracer.cc index 41ad70e5a5741b..3d97d68b5c7dfd 100644 --- a/paddle/fluid/imperative/tracer.cc +++ b/paddle/fluid/imperative/tracer.cc @@ -120,6 +120,17 @@ paddle::framework::GarbageCollector* Tracer::MutableGarbageCollectorIfNotExists( gc.reset(new framework::CPUGarbageCollector( BOOST_GET_CONST(platform::CPUPlace, place), 0)); VLOG(10) << "Created GarbageCollector at " << place; + } else if (platform::is_npu_place(place)) { +#if defined(PADDLE_WITH_ASCEND_CL) + // TODO(zhiqiu): fix bugs and enable NPUDefaultStreamGarbageCollector. + gc.reset(new framework::NPUUnsafeFastGarbageCollector( + BOOST_GET_CONST(platform::NPUPlace, place), 0)); + VLOG(10) << "Created GarbageCollector at " << place; +#else + PADDLE_THROW(platform::errors::PermissionDenied( + "Paddle can't use NPU device since it's not compiled with NPU," + "Please recompile or reinstall Paddle with NPU support.")); +#endif } else { PADDLE_THROW(platform::errors::PreconditionNotMet( "Unsupported place for garbage collection")); @@ -154,9 +165,14 @@ void Tracer::TraceOp(const std::string& type, const NameVarBaseMap& ins, const auto& op_info = op->Info(); auto* attr_checker = op_info.Checker(); if (attr_checker) { - attr_checker->Check(&attrs, true); + attr_checker->Check(&attrs, true, /*only_check_exist_value=*/true); } + static paddle::framework::AttributeMap empty_attrs_map = {}; + const paddle::framework::AttributeMap& default_attrs = + attr_checker == nullptr ? empty_attrs_map + : attr_checker->GetDefaultAttrMap(); + NameVarBaseMap new_ins = ins; if (enable_autocast_) { VLOG(5) << "Auto mixed precision run operator: " << type; @@ -178,10 +194,18 @@ void Tracer::TraceOp(const std::string& type, const NameVarBaseMap& ins, #else PADDLE_THROW(platform::errors::PreconditionNotMet( "PaddlePaddle should compile with XPU if use XPUPlace.")); +#endif + } else if (platform::is_npu_place(place)) { +#ifdef PADDLE_WITH_ASCEND_CL + platform::SetNPUDeviceId( + BOOST_GET_CONST(platform::NPUPlace, place).device); +#else + PADDLE_THROW(platform::errors::PreconditionNotMet( + "PaddlePaddle should compile with NPU if use NPUPlace.")); #endif } - OpBase::Run(*op, new_ins, outs, attrs, place); + OpBase::Run(*op, new_ins, outs, attrs, default_attrs, place); } catch (platform::EnforceNotMet& exception) { framework::AppendErrorOpHint(type, &exception); throw std::move(exception); @@ -204,7 +228,8 @@ void Tracer::TraceOp(const std::string& type, const NameVarBaseMap& ins, } if (ComputeRequiredGrad(new_ins, outs, trace_backward)) { - CreateGradOpNode(*op, new_ins, outs, attrs, place, inplace_map); + CreateGradOpNode(*op, new_ins, outs, attrs, default_attrs, place, + inplace_map); } else { VLOG(3) << "No Grad to track for Op: " << type; } diff --git a/paddle/fluid/inference/analysis/helper.h b/paddle/fluid/inference/analysis/helper.h index cace420d87c9df..ebea4d0386090c 100644 --- a/paddle/fluid/inference/analysis/helper.h +++ b/paddle/fluid/inference/analysis/helper.h @@ -182,15 +182,16 @@ static bool PathExists(const std::string &path) { } static std::string GetDirRoot(const std::string &path) { - char sep = '/'; - -#ifdef _WIN32 - sep = '\\'; -#endif - - size_t i = path.rfind(sep, path.length()); - if (i != std::string::npos) { - return (path.substr(0, i)); + char sep_1 = '/', sep_2 = '\\'; + + size_t i_1 = path.rfind(sep_1, path.length()); + size_t i_2 = path.rfind(sep_2, path.length()); + if (i_1 != std::string::npos && i_2 != std::string::npos) { + return path.substr(0, std::max(i_1, i_2)); + } else if (i_1 != std::string::npos) { + return path.substr(0, i_1); + } else if (i_2 != std::string::npos) { + return path.substr(0, i_2); } return path; } diff --git a/paddle/fluid/inference/api/paddle_analysis_config.h b/paddle/fluid/inference/api/paddle_analysis_config.h index ae29b4ff64cf4c..81e742e8a6f685 100644 --- a/paddle/fluid/inference/api/paddle_analysis_config.h +++ b/paddle/fluid/inference/api/paddle_analysis_config.h @@ -699,7 +699,7 @@ struct PD_INFER_DECL AnalysisConfig { bool xpu_adaptive_seqlen_; // mkldnn related. - int mkldnn_cache_capacity_{0}; + int mkldnn_cache_capacity_{10}; bool use_mkldnn_quantizer_{false}; std::shared_ptr mkldnn_quantizer_config_; bool use_mkldnn_bfloat16_{false}; diff --git a/paddle/fluid/inference/goapi/go.mod b/paddle/fluid/inference/goapi/go.mod index 1036a2e3281901..96e04486f0ffbf 100644 --- a/paddle/fluid/inference/goapi/go.mod +++ b/paddle/fluid/inference/goapi/go.mod @@ -1,3 +1,3 @@ -module github.com/jiweibo/paddle/paddle/fluid/inference/goapi +module github.com/paddlepaddle/paddle/paddle/fluid/inference/goapi go 1.15 diff --git a/paddle/fluid/inference/tensorrt/convert/flatten_op.cc b/paddle/fluid/inference/tensorrt/convert/flatten_op.cc index 322b42667fa30f..25351cc10ec11b 100644 --- a/paddle/fluid/inference/tensorrt/convert/flatten_op.cc +++ b/paddle/fluid/inference/tensorrt/convert/flatten_op.cc @@ -53,10 +53,19 @@ class FlattenOpConverter : public OpConverter { layer->setReshapeDimensions(flatten_dim); } else { auto* shape_layer = TRT_ENGINE_ADD_LAYER(engine_, Shape, *input); + nvinfer1::Dims start_dim, size_dim, stride_dim; + start_dim.nbDims = 1; + size_dim.nbDims = 1; + stride_dim.nbDims = 1; + start_dim.d[0] = 1; + size_dim.d[0] = dims - 1; + stride_dim.d[0] = 1; + auto* slice_layer = + TRT_ENGINE_ADD_LAYER(engine_, Slice, *(shape_layer->getOutput(0)), + start_dim, size_dim, stride_dim); uint32_t reduce_dim = 1; - auto* reduce_prod_layer = TRT_ENGINE_ADD_LAYER( - engine_, Reduce, *(shape_layer->getOutput(0)), + engine_, Reduce, *(slice_layer->getOutput(0)), nvinfer1::ReduceOperation::kPROD, reduce_dim, true); int32_t* constant_weight_data = new int32_t[1]; constant_weight_data[0] = -1; diff --git a/paddle/fluid/inference/tensorrt/convert/matmul_op.cc b/paddle/fluid/inference/tensorrt/convert/matmul_op.cc index a182119776edd9..0358c86926bec2 100644 --- a/paddle/fluid/inference/tensorrt/convert/matmul_op.cc +++ b/paddle/fluid/inference/tensorrt/convert/matmul_op.cc @@ -45,9 +45,16 @@ class MatMulOpConverter : public OpConverter { bool transpose_X = BOOST_GET_CONST(bool, op_desc.GetAttr("transpose_X")); bool transpose_Y = BOOST_GET_CONST(bool, op_desc.GetAttr("transpose_Y")); - auto* layer = TRT_ENGINE_ADD_LAYER( - engine_, MatrixMultiply, *const_cast(input1), - transpose_X, *const_cast(input2), transpose_Y); + nvinfer1::MatrixOperation matrix_operation_X = + transpose_X ? nvinfer1::MatrixOperation::kTRANSPOSE + : nvinfer1::MatrixOperation::kNONE; + nvinfer1::MatrixOperation matrix_operation_Y = + transpose_Y ? nvinfer1::MatrixOperation::kTRANSPOSE + : nvinfer1::MatrixOperation::kNONE; + + auto* layer = + TRT_ENGINE_ADD_LAYER(engine_, MatrixMultiply, *input1, + matrix_operation_X, *input2, matrix_operation_Y); float alpha = BOOST_GET_CONST(float, op_desc.GetAttr("alpha")); auto output_name = op_desc.Output("Out")[0]; diff --git a/paddle/fluid/inference/tensorrt/convert/shuffle_channel_op.cc b/paddle/fluid/inference/tensorrt/convert/shuffle_channel_op.cc index 0fdc262f7e740b..1da44c98f36a04 100644 --- a/paddle/fluid/inference/tensorrt/convert/shuffle_channel_op.cc +++ b/paddle/fluid/inference/tensorrt/convert/shuffle_channel_op.cc @@ -57,7 +57,7 @@ class ShuffleChannelOpConverter : public OpConverter { auto* output = layer->getOutput(0); auto* reshape_layer = TRT_ENGINE_ADD_LAYER(engine_, Shuffle, *output); - nvinfer1::DimsCHW reshape_dim2(c, h, w); + nvinfer1::Dims3 reshape_dim2(c, h, w); reshape_layer->setReshapeDimensions(reshape_dim2); auto output_name = op_desc.Output("Out")[0]; diff --git a/paddle/fluid/inference/tensorrt/convert/test_batch_norm_op.cc b/paddle/fluid/inference/tensorrt/convert/test_batch_norm_op.cc index 41412cb079540d..92e34e48bdb295 100644 --- a/paddle/fluid/inference/tensorrt/convert/test_batch_norm_op.cc +++ b/paddle/fluid/inference/tensorrt/convert/test_batch_norm_op.cc @@ -28,12 +28,12 @@ TEST(batch_norm_op, test) { TRTConvertValidation validator(5, parameters, scope, 1 << 15); std::vector param_shape{2}; - validator.DeclInputVar("batch_norm_X", nvinfer1::DimsCHW(2, 5, 5)); + validator.DeclInputVar("batch_norm_X", nvinfer1::Dims3(2, 5, 5)); validator.DeclParamVar("batch_norm_scale", param_shape); validator.DeclParamVar("batch_norm_bias", param_shape); validator.DeclParamVar("batch_norm_mean", param_shape); validator.DeclParamVar("batch_norm_variance", param_shape); - validator.DeclOutputVar("batch_norm_Y", nvinfer1::DimsCHW(2, 5, 5)); + validator.DeclOutputVar("batch_norm_Y", nvinfer1::Dims3(2, 5, 5)); validator.DeclOutputVar("batch_norm_save_mean", param_shape); validator.DeclOutputVar("batch_norm_save_variance", param_shape); diff --git a/paddle/fluid/inference/tensorrt/convert/test_concat_op.cc b/paddle/fluid/inference/tensorrt/convert/test_concat_op.cc index 4f284a4db5758e..6c876964297f94 100644 --- a/paddle/fluid/inference/tensorrt/convert/test_concat_op.cc +++ b/paddle/fluid/inference/tensorrt/convert/test_concat_op.cc @@ -24,10 +24,10 @@ TEST(concat_op, test) { std::unordered_set parameters({""}); framework::Scope scope; TRTConvertValidation validator(10, parameters, scope, 1000); - validator.DeclInputVar("concat_x1", nvinfer1::DimsCHW(10, 3, 1)); - validator.DeclInputVar("concat_x2", nvinfer1::DimsCHW(3, 3, 1)); - validator.DeclInputVar("concat_x3", nvinfer1::DimsCHW(7, 3, 1)); - validator.DeclOutputVar("concat_out", nvinfer1::DimsCHW(20, 3, 1)); + validator.DeclInputVar("concat_x1", nvinfer1::Dims3(10, 3, 1)); + validator.DeclInputVar("concat_x2", nvinfer1::Dims3(3, 3, 1)); + validator.DeclInputVar("concat_x3", nvinfer1::Dims3(7, 3, 1)); + validator.DeclOutputVar("concat_out", nvinfer1::Dims3(20, 3, 1)); // Prepare Op description framework::OpDesc desc; diff --git a/paddle/fluid/inference/tensorrt/convert/test_dropout_op.cc b/paddle/fluid/inference/tensorrt/convert/test_dropout_op.cc index 81e905b9753271..474fd92071fb07 100644 --- a/paddle/fluid/inference/tensorrt/convert/test_dropout_op.cc +++ b/paddle/fluid/inference/tensorrt/convert/test_dropout_op.cc @@ -25,10 +25,9 @@ TEST(DropoutOpConverter, main) { TRTConvertValidation validator(8, parameters, scope, 1000); std::vector tensor_shape{8, 10}; - validator.DeclInputVar("dropout-X", tensor_shape, - nvinfer1::DimsCHW(10, 1, 1)); - validator.DeclOutputVar("dropout-Out", nvinfer1::DimsCHW(10, 1, 1)); - validator.DeclOutputVar("mask-Out", nvinfer1::DimsCHW(10, 1, 1)); + validator.DeclInputVar("dropout-X", tensor_shape, nvinfer1::Dims3(10, 1, 1)); + validator.DeclOutputVar("dropout-Out", nvinfer1::Dims3(10, 1, 1)); + validator.DeclOutputVar("mask-Out", nvinfer1::Dims3(10, 1, 1)); // Prepare Op description framework::OpDesc desc; diff --git a/paddle/fluid/inference/tensorrt/convert/test_elementwise_op.cc b/paddle/fluid/inference/tensorrt/convert/test_elementwise_op.cc index cc967464a5f291..17adf957f64a76 100644 --- a/paddle/fluid/inference/tensorrt/convert/test_elementwise_op.cc +++ b/paddle/fluid/inference/tensorrt/convert/test_elementwise_op.cc @@ -24,9 +24,9 @@ TEST(elementwise_op, add_weight) { std::unordered_set parameters({"elementwise_add-Y"}); framework::Scope scope; TRTConvertValidation validator(10, parameters, scope, 1 << 15); - validator.DeclInputVar("elementwise_add-X", nvinfer1::DimsCHW(10, 3, 3)); + validator.DeclInputVar("elementwise_add-X", nvinfer1::Dims3(10, 3, 3)); validator.DeclParamVar("elementwise_add-Y", nvinfer1::Dims3(10, 1, 1)); - validator.DeclOutputVar("elementwise_add-Out", nvinfer1::DimsCHW(10, 3, 3)); + validator.DeclOutputVar("elementwise_add-Out", nvinfer1::Dims3(10, 3, 3)); // Prepare Op description framework::OpDesc desc; @@ -50,11 +50,11 @@ TEST(elementwise_op, native) { framework::Scope scope; TRTConvertValidation validator(batch_size, parameters, scope, 1 << 15); validator.DeclInputVar("elementwise_" + type + "-X", - nvinfer1::DimsCHW(10, 3, 3)); + nvinfer1::Dims3(10, 3, 3)); validator.DeclInputVar("elementwise_" + type + "-Y", nvinfer1::Dims3(10, 3, 3)); validator.DeclOutputVar("elementwise_" + type + "-Out", - nvinfer1::DimsCHW(10, 3, 3)); + nvinfer1::Dims3(10, 3, 3)); // Prepare Op description framework::OpDesc desc; @@ -78,11 +78,11 @@ TEST(elementwise_op, plugin) { framework::Scope scope; TRTConvertValidation validator(batch_size, parameters, scope, 1 << 15); validator.DeclInputVar("elementwise_" + type + "-X", - nvinfer1::DimsCHW(10, 3, 3)); + nvinfer1::Dims3(10, 3, 3)); validator.DeclInputVar("elementwise_" + type + "-Y", nvinfer1::Dims3(10, 1, 1)); validator.DeclOutputVar("elementwise_" + type + "-Out", - nvinfer1::DimsCHW(10, 3, 3)); + nvinfer1::Dims3(10, 3, 3)); // Prepare Op description framework::OpDesc desc; diff --git a/paddle/fluid/inference/tensorrt/convert/test_leaky_relu_op.cc b/paddle/fluid/inference/tensorrt/convert/test_leaky_relu_op.cc index d00826af075159..1725888abc379b 100644 --- a/paddle/fluid/inference/tensorrt/convert/test_leaky_relu_op.cc +++ b/paddle/fluid/inference/tensorrt/convert/test_leaky_relu_op.cc @@ -24,8 +24,8 @@ TEST(leaky_relu_op, test_leaky_relu) { std::unordered_set parameters; framework::Scope scope; TRTConvertValidation validator(10, parameters, scope, 1000); - validator.DeclInputVar("leaky_relu_input", nvinfer1::DimsCHW(3, 2, 2)); - validator.DeclOutputVar("leaky_relu_out", nvinfer1::DimsCHW(3, 2, 2)); + validator.DeclInputVar("leaky_relu_input", nvinfer1::Dims3(3, 2, 2)); + validator.DeclOutputVar("leaky_relu_out", nvinfer1::Dims3(3, 2, 2)); // Prepare Op description framework::OpDesc desc; diff --git a/paddle/fluid/inference/tensorrt/convert/test_prelu_op.cc b/paddle/fluid/inference/tensorrt/convert/test_prelu_op.cc index b086c910d38a24..f2541ff7c0b5e5 100644 --- a/paddle/fluid/inference/tensorrt/convert/test_prelu_op.cc +++ b/paddle/fluid/inference/tensorrt/convert/test_prelu_op.cc @@ -24,9 +24,9 @@ TEST(prelu_op, test_channel_wise) { std::unordered_set parameters({"prelu_alpha"}); framework::Scope scope; TRTConvertValidation validator(10, parameters, scope, 1000); - validator.DeclInputVar("prelu_input", nvinfer1::DimsCHW(3, 2, 2)); + validator.DeclInputVar("prelu_input", nvinfer1::Dims3(3, 2, 2)); validator.DeclParamVar("prelu_alpha", nvinfer1::Dims3(3, 1, 1)); - validator.DeclOutputVar("prelu_out", nvinfer1::DimsCHW(3, 2, 2)); + validator.DeclOutputVar("prelu_out", nvinfer1::Dims3(3, 2, 2)); // Prepare Op description framework::OpDesc desc; @@ -46,9 +46,9 @@ TEST(prelu_op, test_element_wise) { std::unordered_set parameters({"prelu_alpha"}); framework::Scope scope; TRTConvertValidation validator(10, parameters, scope, 1000); - validator.DeclInputVar("prelu_input", nvinfer1::DimsCHW(3, 2, 2)); + validator.DeclInputVar("prelu_input", nvinfer1::Dims3(3, 2, 2)); validator.DeclParamVar("prelu_alpha", nvinfer1::Dims4(10, 3, 2, 2)); - validator.DeclOutputVar("prelu_out", nvinfer1::DimsCHW(3, 2, 2)); + validator.DeclOutputVar("prelu_out", nvinfer1::Dims3(3, 2, 2)); // Prepare Op description framework::OpDesc desc; @@ -68,9 +68,9 @@ TEST(prelu_op, test_scalar) { std::unordered_set parameters({"prelu_alpha"}); framework::Scope scope; TRTConvertValidation validator(10, parameters, scope, 1000); - validator.DeclInputVar("prelu_input", nvinfer1::DimsCHW(3, 2, 2)); + validator.DeclInputVar("prelu_input", nvinfer1::Dims3(3, 2, 2)); validator.DeclParamVar("prelu_alpha", nvinfer1::Dims3(1, 1, 1)); - validator.DeclOutputVar("prelu_out", nvinfer1::DimsCHW(3, 2, 2)); + validator.DeclOutputVar("prelu_out", nvinfer1::Dims3(3, 2, 2)); // Prepare Op description framework::OpDesc desc; diff --git a/paddle/fluid/inference/tensorrt/convert/test_shuffle_channel_op.cc b/paddle/fluid/inference/tensorrt/convert/test_shuffle_channel_op.cc index e3cc5273734e02..3ebb51afdf44f4 100644 --- a/paddle/fluid/inference/tensorrt/convert/test_shuffle_channel_op.cc +++ b/paddle/fluid/inference/tensorrt/convert/test_shuffle_channel_op.cc @@ -24,8 +24,8 @@ TEST(leaky_relu_op, test_leaky_relu) { std::unordered_set parameters; framework::Scope scope; TRTConvertValidation validator(10, parameters, scope, 1000); - validator.DeclInputVar("sc_input", nvinfer1::DimsCHW(4, 2, 2)); - validator.DeclOutputVar("sc_out", nvinfer1::DimsCHW(4, 2, 2)); + validator.DeclInputVar("sc_input", nvinfer1::Dims3(4, 2, 2)); + validator.DeclOutputVar("sc_out", nvinfer1::Dims3(4, 2, 2)); // Prepare Op description framework::OpDesc desc; diff --git a/paddle/fluid/inference/tensorrt/convert/test_softmax_op.cc b/paddle/fluid/inference/tensorrt/convert/test_softmax_op.cc index 503ce71f7fb437..b6fdcddf309d85 100644 --- a/paddle/fluid/inference/tensorrt/convert/test_softmax_op.cc +++ b/paddle/fluid/inference/tensorrt/convert/test_softmax_op.cc @@ -25,9 +25,8 @@ TEST(SoftMaxOpConverter, main) { TRTConvertValidation validator(8, parameters, scope, 1000); std::vector tensor_shape{8, 10}; - validator.DeclInputVar("softmax-X", tensor_shape, - nvinfer1::DimsCHW(10, 1, 1)); - validator.DeclOutputVar("softmax-Out", nvinfer1::DimsCHW(10, 1, 1)); + validator.DeclInputVar("softmax-X", tensor_shape, nvinfer1::Dims3(10, 1, 1)); + validator.DeclOutputVar("softmax-Out", nvinfer1::Dims3(10, 1, 1)); // Prepare Op description framework::OpDesc desc; diff --git a/paddle/fluid/inference/tensorrt/convert/test_split_op.cc b/paddle/fluid/inference/tensorrt/convert/test_split_op.cc index 5aacc5c600dd13..3b6a4a80044eb6 100644 --- a/paddle/fluid/inference/tensorrt/convert/test_split_op.cc +++ b/paddle/fluid/inference/tensorrt/convert/test_split_op.cc @@ -28,7 +28,7 @@ void TensorRTSplitTest(const std::vector &in_shape, TRTConvertValidation validator(BatchSize + 1, parameters, scope, 10000); auto make_dim = [](const std::vector &shape) { - nvinfer1::DimsCHW dim; + nvinfer1::Dims3 dim; dim.c() = shape[0]; dim.h() = shape[1]; dim.w() = shape[2]; diff --git a/paddle/fluid/inference/tensorrt/convert/test_swish_op.cc b/paddle/fluid/inference/tensorrt/convert/test_swish_op.cc index c15c79bb13fad4..7a5a886affed33 100644 --- a/paddle/fluid/inference/tensorrt/convert/test_swish_op.cc +++ b/paddle/fluid/inference/tensorrt/convert/test_swish_op.cc @@ -24,8 +24,8 @@ TEST(swish_op, test_swish) { std::unordered_set parameters; framework::Scope scope; TRTConvertValidation validator(10, parameters, scope, 1000); - validator.DeclInputVar("sw_input", nvinfer1::DimsCHW(3, 2, 2)); - validator.DeclOutputVar("sw_out", nvinfer1::DimsCHW(3, 2, 2)); + validator.DeclInputVar("sw_input", nvinfer1::Dims3(3, 2, 2)); + validator.DeclOutputVar("sw_out", nvinfer1::Dims3(3, 2, 2)); // Prepare Op description framework::OpDesc desc; diff --git a/paddle/fluid/inference/tensorrt/engine.cc b/paddle/fluid/inference/tensorrt/engine.cc index 99549fd6b5cbf9..68cd3c0b67eab9 100644 --- a/paddle/fluid/inference/tensorrt/engine.cc +++ b/paddle/fluid/inference/tensorrt/engine.cc @@ -34,17 +34,15 @@ void TensorRTEngine::InitNetwork() { infer_builder_.reset(createInferBuilder(&logger_)); if (with_dynamic_shape_) { -#if IS_TRT_VERSION_GE(6000) - infer_networkv2_.reset(infer_builder_->createNetworkV2( + infer_network_.reset(infer_builder_->createNetworkV2( 1U << static_cast( nvinfer1::NetworkDefinitionCreationFlag::kEXPLICIT_BATCH))); - infer_builder_config_.reset(infer_builder_->createBuilderConfig()); - infer_ptr infer_builder_config_; - optim_profile_ = infer_builder_->createOptimizationProfile(); -#endif } else { - infer_network_.reset(infer_builder_->createNetwork()); + infer_network_.reset(infer_builder_->createNetworkV2(0U)); } + + infer_builder_config_.reset(infer_builder_->createBuilderConfig()); + optim_profile_ = infer_builder_->createOptimizationProfile(); } void TensorRTEngine::Execute(int batch_size, std::vector *buffers, @@ -73,12 +71,12 @@ void TensorRTEngine::FreezeNetwork() { "Call InitNetwork first to initialize network.")); // build engine. infer_builder_->setMaxBatchSize(max_batch_); - infer_builder_->setMaxWorkspaceSize(max_workspace_); + infer_builder_config_->setMaxWorkspaceSize(max_workspace_); + bool enable_fp16 = (precision_ == AnalysisConfig::Precision::kHalf); -#if IS_TRT_VERSION_GE(5000) if (enable_fp16) { bool support_fp16 = infer_builder_->platformHasFastFp16(); - infer_builder_->setFp16Mode(support_fp16); + infer_builder_config_->setFlag(nvinfer1::BuilderFlag::kFP16); if (!support_fp16) { LOG(INFO) << "You specify FP16 mode, but the hardware do not support " "FP16 speed up, use FP32 instead."; @@ -86,23 +84,19 @@ void TensorRTEngine::FreezeNetwork() { LOG(INFO) << "Run Paddle-TRT FP16 mode"; } } -#else - if (enable_fp16) - LOG(INFO) << "Using FP16 in Paddle-TRT must ensure that the version of TRT " - "is at least 5." - "So, use FP32 to run."; -#endif - bool enable_int8 = (precision_ == AnalysisConfig::Precision::kInt8); + bool enable_int8 = (precision_ == AnalysisConfig::Precision::kInt8); if (enable_int8) { - infer_builder_->setInt8Mode(true); + infer_builder_config_->setFlag(nvinfer1::BuilderFlag::kFP16); + infer_builder_config_->setFlag(nvinfer1::BuilderFlag::kINT8); + infer_builder_config_->setFlag(nvinfer1::BuilderFlag::kSTRICT_TYPES); + if (calibrator_) { - infer_builder_->setInt8Calibrator(calibrator_); + infer_builder_config_->setInt8Calibrator(calibrator_); } else { - infer_builder_->setInt8Calibrator(nullptr); + infer_builder_config_->setInt8Calibrator(nullptr); #if IS_TRT_VERSION_GE(5000) - infer_builder_->setStrictTypeConstraints(true); for (auto &quant_range : quant_dynamic_range_) { auto tensor = quant_range.first; float range = quant_range.second; @@ -116,6 +110,7 @@ void TensorRTEngine::FreezeNetwork() { all_t.insert(layer->getOutput(j)); } } + for (int i = 0; i < network()->getNbInputs(); i++) { all_t.insert(network()->getInput(i)); } @@ -127,6 +122,7 @@ void TensorRTEngine::FreezeNetwork() { << ", this might be ok when trt does not need this range"; } } + #if IS_TRT_VERSION_GE(5122) auto is_layer_int8 = [&](nvinfer1::ILayer *layer) -> bool { for (int j = 0; j < layer->getNbInputs(); j++) { @@ -189,9 +185,9 @@ void TensorRTEngine::FreezeNetwork() { << infer_builder_->getNbDLACores() << ", but got " << dla_core_ << ", so use use 0 as default."; } - infer_builder_->setDefaultDeviceType(nvinfer1::DeviceType::kDLA); - infer_builder_->setDLACore(dla_core_); - infer_builder_->allowGPUFallback(true); + infer_builder_config_->setDefaultDeviceType(nvinfer1::DeviceType::kDLA); + infer_builder_config_->setDLACore(dla_core_); + infer_builder_config_->setFlag(nvinfer1::BuilderFlag::kGPU_FALLBACK); LOG(INFO) << "TensorRT DLA enabled in FreezeNetwork(), DLACore " << dla_core_; } @@ -212,30 +208,18 @@ void TensorRTEngine::FreezeNetwork() { Vec2TRT_Dims(optim_input_shape_[input.first], input.first, true)); } infer_builder_config_->addOptimizationProfile(optim_profile_); - infer_builder_config_->setMaxWorkspaceSize(max_workspace_); - if (enable_int8) { - // Due to a bug of TRT, we must set precision BuilderFlag to kFP16 before - // kINT8 here to perform INT8 inference. - infer_builder_config_->setFlag(nvinfer1::BuilderFlag::kFP16); - infer_builder_config_->setFlag(nvinfer1::BuilderFlag::kINT8); - infer_builder_config_->setFlag(nvinfer1::BuilderFlag::kSTRICT_TYPES); + if (WithFp16() && disable_trt_plugin_fp16()) { + LOG(INFO) << "NOTE: In order to achieve higher accuracy, you have " + "disabled the fp16 mode of TRT Plugin,\n" + << "you can reopen it with " + "'config.SetDynamicShapeInfo(min_shape, max_shape, " + "opt_shape, false /*disable_trt_plugin_fp16*/)'"; } - if (WithFp16()) { - infer_builder_config_->setFlag(nvinfer1::BuilderFlag::kFP16); - if (disable_trt_plugin_fp16()) { - LOG(INFO) << "NOTE: In order to achieve higher accuracy, you have " - "disabled the fp16 mode of TRT Plugin,\n" - << "you can reopen it with " - "'config.SetDynamicShapeInfo(min_shape, max_shape, " - "opt_shape, false /*disable_trt_plugin_fp16*/)'"; - } - } - infer_engine_.reset(infer_builder_->buildEngineWithConfig( - *network(), *infer_builder_config_)); #endif - } else { - infer_engine_.reset(infer_builder_->buildCudaEngine(*network())); } + infer_engine_.reset(infer_builder_->buildEngineWithConfig( + *network(), *infer_builder_config_)); + PADDLE_ENFORCE_NOT_NULL( infer_engine_, platform::errors::Fatal( "Build TensorRT cuda engine failed! Please recheck " diff --git a/paddle/fluid/inference/tensorrt/engine.h b/paddle/fluid/inference/tensorrt/engine.h index 7e5707269782ed..773615beb12370 100644 --- a/paddle/fluid/inference/tensorrt/engine.h +++ b/paddle/fluid/inference/tensorrt/engine.h @@ -102,7 +102,7 @@ nvinfer1::Dims Vec2TRT_Dims(const std::vector& shape, std::string input, "trt dynamic_shape mode by SetTRTDynamicShapeInfo.", input, ShapeStr(shape))); } - return nvinfer1::DimsCHW(shape[1], shape[2], shape[3]); + return nvinfer1::Dims3(shape[1], shape[2], shape[3]); } else if (shape.size() == 3UL) { if (shape[1] == -1 || shape[2] == -1) { PADDLE_THROW(platform::errors::InvalidArgument( @@ -112,10 +112,10 @@ nvinfer1::Dims Vec2TRT_Dims(const std::vector& shape, std::string input, } return nvinfer1::Dims2(shape[1], shape[2]); } - return nvinfer1::DimsCHW(shape[1], 1, 1); + return nvinfer1::Dims3(shape[1], 1, 1); } else { if (shape.size() == 4UL) { - return nvinfer1::DimsNCHW(shape[0], shape[1], shape[2], shape[3]); + return nvinfer1::Dims4(shape[0], shape[1], shape[2], shape[3]); } else if (shape.size() == 3UL) { return nvinfer1::Dims3(shape[0], shape[1], shape[2]); } @@ -277,22 +277,19 @@ class TensorRTEngine { } if (with_dynamic_shape_) { -#if IS_TRT_VERSION_GE(6000) infer_engine_.reset(runtime->deserializeCudaEngine( - engine_serialized_data.c_str(), engine_serialized_data.size(), - nullptr)); -#else - - PADDLE_THROW(platform::errors::PreconditionNotMet( - "To enable dynamic shape support, the TensorRT version should be " - "greater than 6.0.0")); - -#endif + engine_serialized_data.c_str(), engine_serialized_data.size())); } else { +#if IS_TRT_VERSION_LT(8000) infer_engine_.reset(runtime->deserializeCudaEngine( engine_serialized_data.c_str(), engine_serialized_data.size(), &inference::Singleton::Global())); +#else + infer_engine_.reset(runtime->deserializeCudaEngine( + engine_serialized_data.c_str(), engine_serialized_data.size())); +#endif } + PADDLE_ENFORCE_NOT_NULL( infer_engine_, platform::errors::Fatal( @@ -369,13 +366,7 @@ class TensorRTEngine { void Execute(int batch_size, std::vector* buffers, cudaStream_t stream = nullptr); - nvinfer1::INetworkDefinition* network() { - if (with_dynamic_shape_) { - return infer_networkv2_.get(); - } else { - return infer_network_.get(); - } - } + nvinfer1::INetworkDefinition* network() { return infer_network_.get(); } ShapeMapType min_input_shape() { return min_input_shape_; } ShapeMapType max_input_shape() { return max_input_shape_; } @@ -530,7 +521,6 @@ class TensorRTEngine { // For dynamic shape bool with_dynamic_shape_{false}; - infer_ptr infer_networkv2_; #if IS_TRT_VERSION_GE(6000) infer_ptr infer_builder_config_; nvinfer1::IOptimizationProfile* optim_profile_; diff --git a/paddle/fluid/inference/tensorrt/helper.h b/paddle/fluid/inference/tensorrt/helper.h index 6158fd130bad8d..e3c7d8b10333c3 100644 --- a/paddle/fluid/inference/tensorrt/helper.h +++ b/paddle/fluid/inference/tensorrt/helper.h @@ -31,6 +31,10 @@ namespace tensorrt { ((NV_TENSORRT_MAJOR * 1000 + NV_TENSORRT_MINOR * 100 + \ NV_TENSORRT_PATCH * 10 + NV_TENSORRT_BUILD) >= version) +#define IS_TRT_VERSION_LT(version) \ + ((NV_TENSORRT_MAJOR * 1000 + NV_TENSORRT_MINOR * 100 + \ + NV_TENSORRT_PATCH * 10 + NV_TENSORRT_BUILD) < version) + #define TRT_VERSION \ NV_TENSORRT_MAJOR * 1000 + NV_TENSORRT_MINOR * 100 + \ NV_TENSORRT_PATCH * 10 + NV_TENSORRT_BUILD @@ -130,6 +134,19 @@ inline size_t ProductDim(const nvinfer1::Dims& dims) { return v; } +inline void PrintITensorShape(nvinfer1::ITensor* X) { + auto dims = X->getDimensions(); + auto name = X->getName(); + std::cout << "ITensor " << name << " shape: ["; + for (int i = 0; i < dims.nbDims; i++) { + if (i == dims.nbDims - 1) + std::cout << dims.d[i]; + else + std::cout << dims.d[i] << ", "; + } + std::cout << "]\n"; +} + } // namespace tensorrt } // namespace inference } // namespace paddle diff --git a/paddle/fluid/inference/tensorrt/plugin/anchor_generator_op_plugin.cu b/paddle/fluid/inference/tensorrt/plugin/anchor_generator_op_plugin.cu index 01ee86ceb48a9e..8e9845183b3fe7 100644 --- a/paddle/fluid/inference/tensorrt/plugin/anchor_generator_op_plugin.cu +++ b/paddle/fluid/inference/tensorrt/plugin/anchor_generator_op_plugin.cu @@ -166,7 +166,11 @@ int AnchorGeneratorPlugin::enqueue_impl(int batch_size, } int AnchorGeneratorPlugin::enqueue(int batch_size, const void* const* inputs, +#if IS_TRT_VERSION_LT(8000) void** outputs, void* workspace, +#else + void* const* outputs, void* workspace, +#endif cudaStream_t stream) { return enqueue_impl(batch_size, inputs, outputs, workspace, stream); } diff --git a/paddle/fluid/inference/tensorrt/plugin/anchor_generator_op_plugin.h b/paddle/fluid/inference/tensorrt/plugin/anchor_generator_op_plugin.h index aff0b6a6802f11..458326d0679ca9 100644 --- a/paddle/fluid/inference/tensorrt/plugin/anchor_generator_op_plugin.h +++ b/paddle/fluid/inference/tensorrt/plugin/anchor_generator_op_plugin.h @@ -42,7 +42,11 @@ class AnchorGeneratorPlugin : public nvinfer1::IPluginV2Ext { bool supportsFormat(nvinfer1::DataType type, nvinfer1::TensorFormat format) const override; size_t getWorkspaceSize(int max_batch_size) const override; +#if IS_TRT_VERSION_LT(8000) int enqueue(int batch_size, const void* const* inputs, void** outputs, +#else + int enqueue(int batch_size, const void* const* inputs, void* const* outputs, +#endif void* workspace, cudaStream_t stream) override; int initialize() override; void terminate() override; diff --git a/paddle/fluid/inference/tensorrt/plugin/elementwise_op_plugin.cu b/paddle/fluid/inference/tensorrt/plugin/elementwise_op_plugin.cu index cc17f8aa248170..687e564e8a8360 100644 --- a/paddle/fluid/inference/tensorrt/plugin/elementwise_op_plugin.cu +++ b/paddle/fluid/inference/tensorrt/plugin/elementwise_op_plugin.cu @@ -122,7 +122,11 @@ int ElementWisePlugin::initialize() { } int ElementWisePlugin::enqueue(int batch_size, const void *const *inputs, +#if IS_TRT_VERSION_LT(8000) void **outputs, void *workspace, +#else + void *const *outputs, void *workspace, +#endif cudaStream_t stream) { const float *x = reinterpret_cast(inputs[0]); const float *y = reinterpret_cast(inputs[1]); diff --git a/paddle/fluid/inference/tensorrt/plugin/elementwise_op_plugin.h b/paddle/fluid/inference/tensorrt/plugin/elementwise_op_plugin.h index 75a1dd85f0f2c4..946e327e355798 100644 --- a/paddle/fluid/inference/tensorrt/plugin/elementwise_op_plugin.h +++ b/paddle/fluid/inference/tensorrt/plugin/elementwise_op_plugin.h @@ -58,8 +58,11 @@ class ElementWisePlugin : public PluginTensorRT { int initialize() override; - // execute the layer +#if IS_TRT_VERSION_LT(8000) int enqueue(int batch_size, const void* const* inputs, void** outputs, +#else + int enqueue(int batch_size, const void* const* inputs, void* const* outputs, +#endif void* workspace, cudaStream_t stream); protected: diff --git a/paddle/fluid/inference/tensorrt/plugin/gelu_op_plugin.cu b/paddle/fluid/inference/tensorrt/plugin/gelu_op_plugin.cu index deda2e2cc7247f..62cf059de492a1 100644 --- a/paddle/fluid/inference/tensorrt/plugin/gelu_op_plugin.cu +++ b/paddle/fluid/inference/tensorrt/plugin/gelu_op_plugin.cu @@ -42,10 +42,10 @@ bool GeluPlugin::supportsFormat(nvinfer1::DataType type, if (with_fp16_) { return ((type == nvinfer1::DataType::kFLOAT || type == nvinfer1::DataType::kHALF) && - (format == nvinfer1::PluginFormat::kNCHW)); + (format == nvinfer1::PluginFormat::kLINEAR)); } else { return ((type == nvinfer1::DataType::kFLOAT) && - (format == nvinfer1::PluginFormat::kNCHW)); + (format == nvinfer1::PluginFormat::kLINEAR)); } } @@ -100,7 +100,11 @@ __global__ void no_exact_gelu_kernel(const T a, const T b, const T c, int n, } int GeluPlugin::enqueue(int batch_size, const void* const* inputs, +#if IS_TRT_VERSION_LT(8000) void** outputs, void*, cudaStream_t stream) { +#else + void* const* outputs, void*, cudaStream_t stream) { +#endif const auto& input_dims = this->getInputDims(0); int num = batch_size; for (int i = 0; i < input_dims.nbDims; i++) { diff --git a/paddle/fluid/inference/tensorrt/plugin/gelu_op_plugin.h b/paddle/fluid/inference/tensorrt/plugin/gelu_op_plugin.h index 23e507ee477e1a..98c05e9792af45 100644 --- a/paddle/fluid/inference/tensorrt/plugin/gelu_op_plugin.h +++ b/paddle/fluid/inference/tensorrt/plugin/gelu_op_plugin.h @@ -44,7 +44,11 @@ class GeluPlugin : public PluginTensorRT { nvinfer1::PluginFormat format) const override; nvinfer1::Dims getOutputDimensions(int index, const nvinfer1::Dims* inputs, int nb_input_dims) override; +#if IS_TRT_VERSION_LT(8000) int enqueue(int batch_size, const void* const* inputs, void** outputs, +#else + int enqueue(int batch_size, const void* const* inputs, void* const* outputs, +#endif void* workspace, cudaStream_t stream) override; protected: diff --git a/paddle/fluid/inference/tensorrt/plugin/hard_swish_op_plugin.cu b/paddle/fluid/inference/tensorrt/plugin/hard_swish_op_plugin.cu index 8b2d0ac3cf70f7..df25b5ba927974 100644 --- a/paddle/fluid/inference/tensorrt/plugin/hard_swish_op_plugin.cu +++ b/paddle/fluid/inference/tensorrt/plugin/hard_swish_op_plugin.cu @@ -59,7 +59,11 @@ __global__ void hard_swish_kernel(float threshold, float scale, float offset, } int HardSwishPlugin::enqueue(int batch_size, const void* const* inputs, +#if IS_TRT_VERSION_LT(8000) void** outputs, void*, cudaStream_t stream) { +#else + void* const* outputs, void*, cudaStream_t stream) { +#endif const auto& input_dims = this->getInputDims(0); int num = batch_size; for (int i = 0; i < input_dims.nbDims; i++) { diff --git a/paddle/fluid/inference/tensorrt/plugin/hard_swish_op_plugin.h b/paddle/fluid/inference/tensorrt/plugin/hard_swish_op_plugin.h index 2e1e1d03baf7e1..ad1952c246a80f 100644 --- a/paddle/fluid/inference/tensorrt/plugin/hard_swish_op_plugin.h +++ b/paddle/fluid/inference/tensorrt/plugin/hard_swish_op_plugin.h @@ -49,7 +49,11 @@ class HardSwishPlugin : public PluginTensorRT { int initialize() override { return 0; } nvinfer1::Dims getOutputDimensions(int index, const nvinfer1::Dims* inputs, int nbInputDims) override; +#if IS_TRT_VERSION_LT(8000) int enqueue(int batchSize, const void* const* inputs, void** outputs, +#else + int enqueue(int batchSize, const void* const* inputs, void* const* outputs, +#endif void* workspace, cudaStream_t stream) override; protected: diff --git a/paddle/fluid/inference/tensorrt/plugin/instance_norm_op_plugin.cu b/paddle/fluid/inference/tensorrt/plugin/instance_norm_op_plugin.cu index a579743ee8ad1a..af063c61c5a568 100644 --- a/paddle/fluid/inference/tensorrt/plugin/instance_norm_op_plugin.cu +++ b/paddle/fluid/inference/tensorrt/plugin/instance_norm_op_plugin.cu @@ -59,7 +59,11 @@ nvinfer1::Dims InstanceNormPlugin::getOutputDimensions( } int InstanceNormPlugin::enqueue(int batch_size, const void *const *inputs, +#if IS_TRT_VERSION_LT(8000) void **outputs, void *workspace, +#else + void *const *outputs, void *workspace, +#endif cudaStream_t stream) { const auto &input_dims = this->getInputDims(0); diff --git a/paddle/fluid/inference/tensorrt/plugin/instance_norm_op_plugin.h b/paddle/fluid/inference/tensorrt/plugin/instance_norm_op_plugin.h index 83422708f593d8..421c4c7970ec68 100644 --- a/paddle/fluid/inference/tensorrt/plugin/instance_norm_op_plugin.h +++ b/paddle/fluid/inference/tensorrt/plugin/instance_norm_op_plugin.h @@ -101,14 +101,18 @@ class InstanceNormPlugin : public PluginTensorRT { int getNbOutputs() const override { return 1; } nvinfer1::Dims getOutputDimensions(int index, const nvinfer1::Dims *inputs, int nbInputDims) override; +#if IS_TRT_VERSION_LT(8000) int enqueue(int batchSize, const void *const *inputs, void **outputs, +#else + int enqueue(int batchSize, const void *const *inputs, void *const *outputs, +#endif void *workspace, cudaStream_t stream) override; bool supportsFormat(nvinfer1::DataType type, nvinfer1::PluginFormat format) const override { return ((type == nvinfer1::DataType::kFLOAT || type == nvinfer1::DataType::kHALF) && - (format == nvinfer1::PluginFormat::kNCHW)); + (format == nvinfer1::PluginFormat::kLINEAR)); } }; diff --git a/paddle/fluid/inference/tensorrt/plugin/layer_norm_op_plugin.cu b/paddle/fluid/inference/tensorrt/plugin/layer_norm_op_plugin.cu index f9341613a0f55e..4d55aea316a358 100644 --- a/paddle/fluid/inference/tensorrt/plugin/layer_norm_op_plugin.cu +++ b/paddle/fluid/inference/tensorrt/plugin/layer_norm_op_plugin.cu @@ -43,7 +43,11 @@ nvinfer1::Dims LayerNormPlugin::getOutputDimensions( } int LayerNormPlugin::enqueue(int batch_size, const void *const *inputs, +#if IS_TRT_VERSION_LT(8000) void **outputs, void *workspace, +#else + void *const *outputs, void *workspace, +#endif cudaStream_t stream) { const auto &input_dims = this->getInputDims(0); const float *input = reinterpret_cast(inputs[0]); diff --git a/paddle/fluid/inference/tensorrt/plugin/layer_norm_op_plugin.h b/paddle/fluid/inference/tensorrt/plugin/layer_norm_op_plugin.h index 9c4c31b61e128d..a16c5191f88644 100644 --- a/paddle/fluid/inference/tensorrt/plugin/layer_norm_op_plugin.h +++ b/paddle/fluid/inference/tensorrt/plugin/layer_norm_op_plugin.h @@ -100,7 +100,11 @@ class LayerNormPlugin : public PluginTensorRT { int getNbOutputs() const override { return 1; } nvinfer1::Dims getOutputDimensions(int index, const nvinfer1::Dims* inputs, int nbInputDims) override; +#if IS_TRT_VERSION_LT(8000) int enqueue(int batchSize, const void* const* inputs, void** outputs, +#else + int enqueue(int batchSize, const void* const* inputs, void* const* outputs, +#endif void* workspace, cudaStream_t stream) override; }; diff --git a/paddle/fluid/inference/tensorrt/plugin/pool_op_plugin.cu b/paddle/fluid/inference/tensorrt/plugin/pool_op_plugin.cu index 154f61a2b7cd3f..0d3b8ca1b4244a 100644 --- a/paddle/fluid/inference/tensorrt/plugin/pool_op_plugin.cu +++ b/paddle/fluid/inference/tensorrt/plugin/pool_op_plugin.cu @@ -42,7 +42,12 @@ nvinfer1::Dims PoolPlugin::getOutputDimensions(int index, } int PoolPlugin::enqueue(int batchSize, const void *const *inputs, +#if IS_TRT_VERSION_LT(8000) void **outputs, void *workspace, cudaStream_t stream) { +#else + void *const *outputs, void *workspace, + cudaStream_t stream) { +#endif auto const &input_dims = this->getInputDims(0); int input_size = 0; float const *idata = reinterpret_cast(inputs[0]); @@ -169,7 +174,7 @@ bool PoolPluginDynamic::supportsFormatCombination( (in_out && pos < (nb_inputs + nb_outputs)); return ((in_out[pos].type == nvinfer1::DataType::kFLOAT) && - in_out[pos].format == nvinfer1::PluginFormat::kNCHW); + in_out[pos].format == nvinfer1::PluginFormat::kLINEAR); } nvinfer1::DataType PoolPluginDynamic::getOutputDataType( diff --git a/paddle/fluid/inference/tensorrt/plugin/pool_op_plugin.h b/paddle/fluid/inference/tensorrt/plugin/pool_op_plugin.h index 6693a1fae4d430..90ce44e6822565 100644 --- a/paddle/fluid/inference/tensorrt/plugin/pool_op_plugin.h +++ b/paddle/fluid/inference/tensorrt/plugin/pool_op_plugin.h @@ -128,7 +128,11 @@ class PoolPlugin : public PluginTensorRT { nvinfer1::Dims getOutputDimensions(int index, const nvinfer1::Dims* inputs, int nbInputDims) override; int initialize() override { return 0; } +#if IS_TRT_VERSION_LT(8000) int enqueue(int batchSize, const void* const* inputs, void** outputs, +#else + int enqueue(int batchSize, const void* const* inputs, void* const* outputs, +#endif void* workspace, cudaStream_t stream) override; private: diff --git a/paddle/fluid/inference/tensorrt/plugin/prelu_op_plugin.cu b/paddle/fluid/inference/tensorrt/plugin/prelu_op_plugin.cu index 00182b87e984fc..09e39a3b9876f0 100644 --- a/paddle/fluid/inference/tensorrt/plugin/prelu_op_plugin.cu +++ b/paddle/fluid/inference/tensorrt/plugin/prelu_op_plugin.cu @@ -57,7 +57,12 @@ nvinfer1::Dims PReluPlugin::getOutputDimensions(int index, } int PReluPlugin::enqueue(int batch_size, const void *const *inputs, +#if IS_TRT_VERSION_LT(8000) void **outputs, void *workspace, cudaStream_t stream) { +#else + void *const *outputs, void *workspace, + cudaStream_t stream) { +#endif // input dims is CHW. const auto &input_dims = this->getInputDims(0); const float *input = reinterpret_cast(inputs[0]); @@ -124,7 +129,7 @@ bool PReluPluginDynamic::supportsFormatCombination( (in_out && pos < (nb_inputs + nb_outputs)); return ((in_out[pos].type == nvinfer1::DataType::kFLOAT) && - in_out[pos].format == nvinfer1::PluginFormat::kNCHW); + in_out[pos].format == nvinfer1::PluginFormat::kLINEAR); } nvinfer1::DataType PReluPluginDynamic::getOutputDataType( diff --git a/paddle/fluid/inference/tensorrt/plugin/prelu_op_plugin.h b/paddle/fluid/inference/tensorrt/plugin/prelu_op_plugin.h index a0a24e70a01ef4..313272823d4a6d 100644 --- a/paddle/fluid/inference/tensorrt/plugin/prelu_op_plugin.h +++ b/paddle/fluid/inference/tensorrt/plugin/prelu_op_plugin.h @@ -80,7 +80,11 @@ class PReluPlugin : public PluginTensorRT { int getNbOutputs() const override { return 1; } nvinfer1::Dims getOutputDimensions(int index, const nvinfer1::Dims* inputs, int nbInputDims) override; +#if IS_TRT_VERSION_LT(8000) int enqueue(int batchSize, const void* const* inputs, void** outputs, +#else + int enqueue(int batchSize, const void* const* inputs, void* const* outputs, +#endif void* workspace, cudaStream_t stream) override; }; diff --git a/paddle/fluid/inference/tensorrt/plugin/qkv_to_context_plugin.cu b/paddle/fluid/inference/tensorrt/plugin/qkv_to_context_plugin.cu index 214e1a81e7dc04..5f10e5821c4f7e 100644 --- a/paddle/fluid/inference/tensorrt/plugin/qkv_to_context_plugin.cu +++ b/paddle/fluid/inference/tensorrt/plugin/qkv_to_context_plugin.cu @@ -299,7 +299,7 @@ int QkvToContextPluginDynamic::enqueue( platform::DeviceContextPool::Instance().Get( platform::CUDAPlace(device_id))); - int n_q = seq_len * head_number_ * head_size_; + int n_q = seq_len * head_number_ * head_size_ * batch; constexpr int threads = 128; int blocks = (n_q + threads - 1) / threads; diff --git a/paddle/fluid/inference/tensorrt/plugin/slice_op_plugin.cu b/paddle/fluid/inference/tensorrt/plugin/slice_op_plugin.cu index b44b3face92e14..e976496ec44ca8 100644 --- a/paddle/fluid/inference/tensorrt/plugin/slice_op_plugin.cu +++ b/paddle/fluid/inference/tensorrt/plugin/slice_op_plugin.cu @@ -90,10 +90,10 @@ bool SlicePlugin::supportsFormat(nvinfer1::DataType type, if (with_fp16_) { return ((type == nvinfer1::DataType::kFLOAT || type == nvinfer1::DataType::kHALF) && - (format == nvinfer1::PluginFormat::kNCHW)); + (format == nvinfer1::PluginFormat::kLINEAR)); } else { return ((type == nvinfer1::DataType::kFLOAT) && - (format == nvinfer1::PluginFormat::kNCHW)); + (format == nvinfer1::PluginFormat::kLINEAR)); } } @@ -111,7 +111,12 @@ nvinfer1::Dims SlicePlugin::getOutputDimensions(int index, } int SlicePlugin::enqueue(int batch_size, const void *const *inputs, +#if IS_TRT_VERSION_LT(8000) void **outputs, void *workspace, cudaStream_t stream) { +#else + void *const *outputs, void *workspace, + cudaStream_t stream) { +#endif auto input_dims = getInputDims(0); // notice input dims is [C, H, W], add input batch dim here diff --git a/paddle/fluid/inference/tensorrt/plugin/slice_op_plugin.h b/paddle/fluid/inference/tensorrt/plugin/slice_op_plugin.h index 9d4f9a35c3b6fe..015a6b116f60a1 100644 --- a/paddle/fluid/inference/tensorrt/plugin/slice_op_plugin.h +++ b/paddle/fluid/inference/tensorrt/plugin/slice_op_plugin.h @@ -44,7 +44,11 @@ class SlicePlugin : public PluginTensorRT { nvinfer1::PluginFormat format) const override; nvinfer1::Dims getOutputDimensions(int index, const nvinfer1::Dims* inputs, int nb_input_dims) override; +#if IS_TRT_VERSION_LT(8000) int enqueue(int batch_size, const void* const* inputs, void** outputs, +#else + int enqueue(int batch_size, const void* const* inputs, void* const* outputs, +#endif void* workspace, cudaStream_t stream) override; protected: diff --git a/paddle/fluid/inference/tensorrt/plugin/split_op_plugin.cu b/paddle/fluid/inference/tensorrt/plugin/split_op_plugin.cu index 1b5c39f8fff855..24d4715e0312dc 100644 --- a/paddle/fluid/inference/tensorrt/plugin/split_op_plugin.cu +++ b/paddle/fluid/inference/tensorrt/plugin/split_op_plugin.cu @@ -126,7 +126,12 @@ __global__ void split_kernel(int nsegment, } int SplitPlugin::enqueue(int batchSize, const void* const* inputs, +#if IS_TRT_VERSION_LT(8000) void** outputs, void* workspace, cudaStream_t stream) { +#else + void* const* outputs, void* workspace, + cudaStream_t stream) { +#endif const int* d_segment_offsets_ptr = thrust::raw_pointer_cast(&d_segment_offsets_[0]); float const* input_ptr = reinterpret_cast(inputs[0]); diff --git a/paddle/fluid/inference/tensorrt/plugin/split_op_plugin.h b/paddle/fluid/inference/tensorrt/plugin/split_op_plugin.h index 1ee895154d6b04..a791395f4a3d38 100644 --- a/paddle/fluid/inference/tensorrt/plugin/split_op_plugin.h +++ b/paddle/fluid/inference/tensorrt/plugin/split_op_plugin.h @@ -60,7 +60,11 @@ class SplitPlugin : public PluginTensorRTV2Ext { int initialize() override; void terminate() override; +#if IS_TRT_VERSION_LT(8000) int enqueue(int batch_size, const void* const* inputs, void** outputs, +#else + int enqueue(int batch_size, const void* const* inputs, void* const* outputs, +#endif void* workspace, cudaStream_t stream) override; void destroy() override { delete this; } diff --git a/paddle/fluid/inference/tensorrt/plugin/swish_op_plugin.cu b/paddle/fluid/inference/tensorrt/plugin/swish_op_plugin.cu index 3847d999446e99..52e5af01822fac 100644 --- a/paddle/fluid/inference/tensorrt/plugin/swish_op_plugin.cu +++ b/paddle/fluid/inference/tensorrt/plugin/swish_op_plugin.cu @@ -85,7 +85,12 @@ __global__ void swish_kernel(int num, const half *input, half *output, } int SwishPlugin::enqueue(int batch_size, const void *const *inputs, +#if IS_TRT_VERSION_LT(8000) void **outputs, void *workspace, cudaStream_t stream) { +#else + void *const *outputs, void *workspace, + cudaStream_t stream) { +#endif // input dims is CHW. const auto &input_dims = this->getInputDims(0); const float *input = reinterpret_cast(inputs[0]); diff --git a/paddle/fluid/inference/tensorrt/plugin/swish_op_plugin.h b/paddle/fluid/inference/tensorrt/plugin/swish_op_plugin.h index 11579aadcc4573..2a8b637730b516 100644 --- a/paddle/fluid/inference/tensorrt/plugin/swish_op_plugin.h +++ b/paddle/fluid/inference/tensorrt/plugin/swish_op_plugin.h @@ -67,7 +67,11 @@ class SwishPlugin : public PluginTensorRT { int getNbOutputs() const override { return 1; } nvinfer1::Dims getOutputDimensions(int index, const nvinfer1::Dims* inputs, int nbInputDims) override; +#if IS_TRT_VERSION_LT(8000) int enqueue(int batchSize, const void* const* inputs, void** outputs, +#else + int enqueue(int batchSize, const void* const* inputs, void* const* outputs, +#endif void* workspace, cudaStream_t stream) override; }; diff --git a/paddle/fluid/inference/tensorrt/plugin/test_split_plugin.cc b/paddle/fluid/inference/tensorrt/plugin/test_split_plugin.cc index 6636513a555f9e..46f585e6557460 100644 --- a/paddle/fluid/inference/tensorrt/plugin/test_split_plugin.cc +++ b/paddle/fluid/inference/tensorrt/plugin/test_split_plugin.cc @@ -33,7 +33,7 @@ TEST(split_op_plugin, test_plugin) { input_dims.push_back(in_dims); sp_plugin.configurePlugin(input_dims.data(), 1, nullptr, 2, input_types.data(), nullptr, nullptr, nullptr, - nvinfer1::PluginFormat::kNCHW, 4); + nvinfer1::PluginFormat::kLINEAR, 4); sp_plugin.initialize(); sp_plugin.getPluginType(); sp_plugin.canBroadcastInputAcrossBatch(0); diff --git a/paddle/fluid/inference/tensorrt/plugin/trt_plugin.cc b/paddle/fluid/inference/tensorrt/plugin/trt_plugin.cc index 55bc786746beaf..e2f3810cc34e01 100644 --- a/paddle/fluid/inference/tensorrt/plugin/trt_plugin.cc +++ b/paddle/fluid/inference/tensorrt/plugin/trt_plugin.cc @@ -68,7 +68,7 @@ size_t PluginTensorRT::getBaseSerializationSize() { bool PluginTensorRT::supportsFormat(nvinfer1::DataType type, nvinfer1::PluginFormat format) const { return ((type == nvinfer1::DataType::kFLOAT) && - (format == nvinfer1::PluginFormat::kNCHW)); + (format == nvinfer1::PluginFormat::kLINEAR)); } void PluginTensorRT::configureWithFormat( diff --git a/paddle/fluid/inference/tensorrt/plugin/trt_plugin.h b/paddle/fluid/inference/tensorrt/plugin/trt_plugin.h index ce3133ae99e94c..9c4add0688987d 100644 --- a/paddle/fluid/inference/tensorrt/plugin/trt_plugin.h +++ b/paddle/fluid/inference/tensorrt/plugin/trt_plugin.h @@ -82,8 +82,13 @@ class PluginTensorRT : public nvinfer1::IPluginExt { int initialize() override { return 0; } // Shutdown the layer. This is called when the engine is destroyed void terminate() override {} - // Execute the layer +// Execute the layer +#if IS_TRT_VERSION_LT(8000) virtual int enqueue(int batch_size, const void* const* inputs, void** outputs, +#else + virtual int enqueue(int batch_size, const void* const* inputs, + void* const* outputs, +#endif void* workspace, cudaStream_t stream) = 0; // Find the size of the serialization buffer required @@ -176,7 +181,7 @@ class PluginTensorRTV2Ext : public nvinfer1::IPluginV2Ext { bool supportsFormat(nvinfer1::DataType type, nvinfer1::PluginFormat format) const override { return ((type == nvinfer1::DataType::kFLOAT) && - (format == nvinfer1::PluginFormat::kNCHW)); + (format == nvinfer1::PluginFormat::kLINEAR)); } // Initialize the layer for execution. // This is called when the engine is created. @@ -188,8 +193,13 @@ class PluginTensorRTV2Ext : public nvinfer1::IPluginV2Ext { // Find the workspace size required by the layer size_t getWorkspaceSize(int) const override { return 0; } - // Execute the layer +// Execute the layer +#if IS_TRT_VERSION_LT(8000) virtual int enqueue(int batch_size, const void* const* inputs, void** outputs, +#else + virtual int enqueue(int batch_size, const void* const* inputs, + void* const* outputs, +#endif void* workspace, cudaStream_t stream) = 0; // Find the size of the serialization buffer required diff --git a/paddle/fluid/inference/tensorrt/plugin/yolo_box_op_plugin.cu b/paddle/fluid/inference/tensorrt/plugin/yolo_box_op_plugin.cu index 13d07e774036a4..f9767f38559482 100644 --- a/paddle/fluid/inference/tensorrt/plugin/yolo_box_op_plugin.cu +++ b/paddle/fluid/inference/tensorrt/plugin/yolo_box_op_plugin.cu @@ -243,7 +243,11 @@ int YoloBoxPlugin::enqueue_impl(int batch_size, const void* const* inputs, } int YoloBoxPlugin::enqueue(int batch_size, const void* const* inputs, +#if IS_TRT_VERSION_LT(8000) void** outputs, void* workspace, +#else + void* const* outputs, void* workspace, +#endif cudaStream_t stream) { if (data_type_ == nvinfer1::DataType::kFLOAT) { return enqueue_impl(batch_size, inputs, outputs, workspace, stream); diff --git a/paddle/fluid/inference/tensorrt/plugin/yolo_box_op_plugin.h b/paddle/fluid/inference/tensorrt/plugin/yolo_box_op_plugin.h index 8ca21da7ae0377..4cd6a383336e23 100644 --- a/paddle/fluid/inference/tensorrt/plugin/yolo_box_op_plugin.h +++ b/paddle/fluid/inference/tensorrt/plugin/yolo_box_op_plugin.h @@ -43,7 +43,11 @@ class YoloBoxPlugin : public nvinfer1::IPluginV2Ext { bool supportsFormat(nvinfer1::DataType type, nvinfer1::TensorFormat format) const override; size_t getWorkspaceSize(int max_batch_size) const override; +#if IS_TRT_VERSION_LT(8000) int enqueue(int batch_size, const void* const* inputs, void** outputs, +#else + int enqueue(int batch_size, const void* const* inputs, void* const* outputs, +#endif void* workspace, cudaStream_t stream) override; template int enqueue_impl(int batch_size, const void* const* inputs, void** outputs, diff --git a/paddle/fluid/inference/tensorrt/test_engine.cc b/paddle/fluid/inference/tensorrt/test_engine.cc index 5c61bec55ba71b..c627075bfe95d9 100644 --- a/paddle/fluid/inference/tensorrt/test_engine.cc +++ b/paddle/fluid/inference/tensorrt/test_engine.cc @@ -68,7 +68,7 @@ TEST_F(TensorRTEngineTest, add_layer) { TensorRTEngine::Weight weight(nvinfer1::DataType::kFLOAT, raw_weight, size); TensorRTEngine::Weight bias(nvinfer1::DataType::kFLOAT, raw_bias, size); auto *x = engine_->DeclareInput("x", nvinfer1::DataType::kFLOAT, - nvinfer1::DimsCHW{1, 1, 1}); + nvinfer1::Dims3{1, 1, 1}); auto *fc_layer = TRT_ENGINE_ADD_LAYER(engine_, FullyConnected, *x, size, weight.get(), bias.get()); PADDLE_ENFORCE_NOT_NULL(fc_layer, @@ -123,7 +123,7 @@ TEST_F(TensorRTEngineTest, add_layer_multi_dim) { TensorRTEngine::Weight weight(nvinfer1::DataType::kFLOAT, raw_weight, 4); TensorRTEngine::Weight bias(nvinfer1::DataType::kFLOAT, raw_bias, 2); auto *x = engine_->DeclareInput("x", nvinfer1::DataType::kFLOAT, - nvinfer1::DimsCHW{1, 2, 1}); + nvinfer1::Dims3{1, 2, 1}); auto *fc_layer = TRT_ENGINE_ADD_LAYER(engine_, FullyConnected, *x, 2, weight.get(), bias.get()); PADDLE_ENFORCE_NOT_NULL(fc_layer, diff --git a/paddle/fluid/inference/tensorrt/test_tensorrt.cc b/paddle/fluid/inference/tensorrt/test_tensorrt.cc index 5f8ddcc94235f3..36a25e27d78f5b 100644 --- a/paddle/fluid/inference/tensorrt/test_tensorrt.cc +++ b/paddle/fluid/inference/tensorrt/test_tensorrt.cc @@ -80,7 +80,7 @@ nvinfer1::IHostMemory* CreateNetwork() { nvinfer1::INetworkDefinition* network = builder->createNetwork(); // Add the input auto input = network->addInput(kInputTensor, nvinfer1::DataType::kFLOAT, - nvinfer1::DimsCHW{1, 1, 1}); + nvinfer1::Dims3{1, 1, 1}); EXPECT_NE(input, nullptr); // Add the hidden layer. auto layer = network->addFullyConnected(*input, 1, weights.get(), bias.get()); diff --git a/paddle/fluid/inference/tests/api/trt_dynamic_shape_ernie_test.cc b/paddle/fluid/inference/tests/api/trt_dynamic_shape_ernie_test.cc index a45b78f05e73c4..e449fb5096e6e0 100644 --- a/paddle/fluid/inference/tests/api/trt_dynamic_shape_ernie_test.cc +++ b/paddle/fluid/inference/tests/api/trt_dynamic_shape_ernie_test.cc @@ -22,51 +22,60 @@ limitations under the License. */ namespace paddle { namespace inference { -void run(const AnalysisConfig& config, std::vector* out_data) { +void run(const AnalysisConfig& config, std::vector* out_data, int bs) { auto predictor = CreatePaddlePredictor(config); auto input_names = predictor->GetInputNames(); - int run_batch = 1; + int run_batch = bs; const int run_seq_len = 128; + size_t len = run_batch * run_seq_len; - int64_t i0[run_seq_len] = { + int64_t i0_bs1[run_seq_len] = { 1, 3558, 4, 75, 491, 89, 340, 313, 93, 4, 255, 10, 75, 321, 4095, 1902, 4, 134, 49, 75, 311, 14, 44, 178, 543, 15, 12043, 2, 75, 201, 340, 9, 14, 44, 486, 218, 1140, 279, 12043, 2}; - int64_t i1[run_seq_len] = { + int64_t i1_bs1[run_seq_len] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}; - int64_t i2[run_seq_len] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, - 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, - 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, - 30, 31, 32, 33, 34, 35, 36, 37, 38, 39}; - float i3[run_seq_len] = {1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, - 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, - 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, - 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0}; - + int64_t i2_bs1[run_seq_len] = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9, + 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, + 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, + 30, 31, 32, 33, 34, 35, 36, 37, 38, 39}; + float i3_bs1[run_seq_len] = { + 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, + 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, + 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0}; + std::vector i0_data(len), i1_data(len), i2_data(len); + std::vector i3_data(len); + + for (size_t i = 0; i < len; i++) { + i0_data[i] = i0_bs1[i % run_seq_len]; + i1_data[i] = i1_bs1[i % run_seq_len]; + i2_data[i] = i2_bs1[i % run_seq_len]; + i3_data[i] = i3_bs1[i % run_seq_len]; + } // first input auto input_t = predictor->GetInputTensor(input_names[0]); input_t->Reshape({run_batch, run_seq_len, 1}); - input_t->copy_from_cpu(i0); + input_t->copy_from_cpu(i0_data.data()); // second input auto input_t2 = predictor->GetInputTensor(input_names[1]); input_t2->Reshape({run_batch, run_seq_len, 1}); - input_t2->copy_from_cpu(i1); + input_t2->copy_from_cpu(i1_data.data()); // third input. auto input_t3 = predictor->GetInputTensor(input_names[2]); input_t3->Reshape({run_batch, run_seq_len, 1}); - input_t3->copy_from_cpu(i2); + input_t3->copy_from_cpu(i2_data.data()); auto input_t4 = predictor->GetInputTensor(input_names[3]); input_t4->Reshape({run_batch, run_seq_len, 1}); - input_t4->copy_from_cpu(i3); + input_t4->copy_from_cpu(i3_data.data()); ASSERT_TRUE(predictor->ZeroCopyRun()); @@ -79,8 +88,8 @@ void run(const AnalysisConfig& config, std::vector* out_data) { output_t->copy_to_cpu(out_data->data()); } -void trt_ernie(bool with_fp16, std::vector result, - float near_tolerance) { +void trt_ernie(bool with_fp16, std::vector result, float near_tolerance, + int batch_size = 1) { AnalysisConfig config; std::string model_dir = FLAGS_infer_model; SetConfig(&config, model_dir, true); @@ -120,7 +129,7 @@ void trt_ernie(bool with_fp16, std::vector result, config.SetTRTDynamicShapeInfo(min_input_shape, max_input_shape, opt_input_shape); std::vector out_data; - run(config, &out_data); + run(config, &out_data, batch_size); for (size_t i = 0; i < out_data.size(); i++) { EXPECT_NEAR(result[i], out_data[i], near_tolerance); @@ -139,6 +148,19 @@ TEST(AnalysisPredictor, fp16) { #endif } +TEST(AnalysisPredictor, no_fp16_bs2) { + std::vector result = {0.597841, 0.219972, 0.182187, + 0.597841, 0.219972, 0.182187}; + trt_ernie(false, result, 1e-5, 2); +} + +TEST(AnalysisPredictor, fp16_bs2) { +#ifdef TRT_PLUGIN_FP16_AVALIABLE + std::vector result = {0.598, 0.219, 0.182, 0.598, 0.219, 0.182}; + trt_ernie(true, result, 4e-3, 2); +#endif +} + // ernie_varlen std::shared_ptr InitPredictor() { paddle_infer::Config config; diff --git a/paddle/fluid/memory/detail/system_allocator.cc b/paddle/fluid/memory/detail/system_allocator.cc index d6dc303ebc789e..9f39c3a823f862 100644 --- a/paddle/fluid/memory/detail/system_allocator.cc +++ b/paddle/fluid/memory/detail/system_allocator.cc @@ -192,7 +192,7 @@ void* CUDAPinnedAllocator::Alloc(size_t* index, size_t size) { void* p; // PINNED memory is visible to all CUDA contexts. #ifdef PADDLE_WITH_HIP - hipError_t result = hipHostMalloc(&p, size); + hipError_t result = hipHostMalloc(&p, size, hipHostMallocPortable); #else cudaError_t result = cudaHostAlloc(&p, size, cudaHostAllocPortable); #endif diff --git a/paddle/fluid/operators/batch_norm_op.cu b/paddle/fluid/operators/batch_norm_op.cu index 6fc78732b1063a..1758463141cb8f 100644 --- a/paddle/fluid/operators/batch_norm_op.cu +++ b/paddle/fluid/operators/batch_norm_op.cu @@ -382,8 +382,8 @@ class BatchNormKernel } // Run training mode. - // obtain running mean and running inv var, and see if we need to - // initialize them. + // obtain running mean and running inv var, and there is no need + // to initialize them. auto *mean_out = ctx.Output("MeanOut"); auto *variance_out = ctx.Output("VarianceOut"); @@ -394,10 +394,6 @@ class BatchNormKernel auto *saved_variance = ctx.Output("SavedVariance"); saved_mean->mutable_data>(ctx.GetPlace()); saved_variance->mutable_data>(ctx.GetPlace()); - math::SetConstant> - functor; - functor(dev_ctx, saved_mean, static_cast>(0)); - functor(dev_ctx, saved_variance, static_cast>(0)); if ((N * H * W * D) == 1) { // Only 1 element in normalization dimension, diff --git a/paddle/fluid/operators/broadcast_tensors_op.cc b/paddle/fluid/operators/broadcast_tensors_op.cc new file mode 100644 index 00000000000000..074607e05ea7d5 --- /dev/null +++ b/paddle/fluid/operators/broadcast_tensors_op.cc @@ -0,0 +1,253 @@ +/* Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. */ + +#include "paddle/fluid/operators/broadcast_tensors_op.h" + +#include +#include +#include +#include +#include + +#include "paddle/fluid/framework/var_type_inference.h" + +namespace paddle { +namespace operators { +using framework::Tensor; +using framework::DDim; + +class BroadcastTensorsOp : public framework::OperatorWithKernel { + public: + using framework::OperatorWithKernel::OperatorWithKernel; + + void InferShape(framework::InferShapeContext* ctx) const override { + OP_INOUT_CHECK(ctx->HasInputs("X"), "Input", "X", "broadcast_tensors"); + OP_INOUT_CHECK(ctx->HasOutputs("Out"), "Output", "Out", + "broadcast_tensors"); + + int target_rank = 0; + const auto& input_dims = ctx->GetInputsDim("X"); + // 1. Find Output rank = max(Inputs rank) + for (const auto& input_ddim : input_dims) { + target_rank = std::max(target_rank, input_ddim.size()); + } + + PADDLE_ENFORCE_GT( + target_rank, 0, + platform::errors::InvalidArgument( + "BroadcastTensorsOp requires at least one input tensor" + "to have rank greater than zero")); + + std::vector target_dims(target_rank, 0); + // 2. Output dim(axis=x) = max(Inputs dim(axis=x)) + for (int index = 0; index < target_rank; index++) { + // Loop axes in reverse order, + // For each axis, take the maximum as target size + // Fill size = 1 if shape vector exhausts + int target_dim_size = 1; + for (const auto& input_ddim : input_dims) { + // Reversed order + int axis = static_cast(input_ddim.size()) - index - 1; + int dim_size = 1; + if (axis >= 0) { + dim_size = input_ddim[axis]; + } + + // We performed bcast semantics check at python level + // So input tensors should all have legal shape + target_dim_size = std::max(target_dim_size, dim_size); + } + target_dims[target_rank - index - 1] = target_dim_size; + } + + // 3. Set Output Dim + std::vector output_ddims; + for (size_t i = 0; i < input_dims.size(); i++) { + output_ddims.emplace_back(framework::make_ddim(target_dims)); + } + ctx->SetOutputsDim("Out", output_ddims); + ctx->ShareAllLoD("X", /*->*/ "Out"); + } + + protected: + framework::OpKernelType GetExpectedKernelType( + const framework::ExecutionContext& ctx) const override { + // Broadcast semantics enforces all input variables having the same + // DataType/VarType + // This condition is also checked during VarType Inference + // Here we simply copy input type to output + return framework::OpKernelType( + OperatorWithKernel::IndicateVarDataType(ctx, "X"), ctx.GetPlace()); + } +}; + +class BroadcastTensorsOpMaker : public framework::OpProtoAndCheckerMaker { + public: + void Make() override { + AddInput("X", + "A Varaible list. The shape and data type of the list elements" + "should be consistent. Variable can be multi-dimensional Tensor" + "or LoDTensor, and data types can be: bool, float16, float32, " + "float64, int32, " + "int64.") + .AsDuplicable(); + AddOutput("Out", + "the sum of input :code:`x`. its shape and data types are " + "consistent with :code:`x`.") + .AsDuplicable(); + AddComment( + R"DOC(This OP is used to broadcast a vector of inputs + with Tensor or LoDTensor type, following broadcast semantics.)DOC"); + } +}; + +class BroadcastTensorsOpVarTypeInference : public framework::VarTypeInference { + public: + void operator()(framework::InferVarTypeContext* ctx) const override { + // We need at least two tensors to satisfy broadcast semantics + size_t input_size = ctx->InputSize("X"); + PADDLE_ENFORCE_GT( + input_size, 0, + platform::errors::InvalidArgument( + "BroadcastTensorsOp should have at least one input variables," + "but only received %d ", + input_size)); + + // BroadcastTensorsOp takes a vector of variables named "X" + // Here we loop through input variables, + // and check if their DataType/VarType are the same + auto var_type = ctx->GetInputType("X", 0); + auto data_type = ctx->GetInputDataType("X", 0); + for (size_t ind = 1; ind < input_size; ind++) { + auto cur_var_type = ctx->GetInputType("X", ind); + PADDLE_ENFORCE_EQ( + var_type, cur_var_type, + platform::errors::InvalidArgument( + "inputs to BroadcastTensorsOp should have the same variable type," + "but detected %d v.s %d ", + framework::ToTypeName(var_type), + framework::ToTypeName(cur_var_type))); + + auto cur_data_type = ctx->GetInputDataType("X", ind); + PADDLE_ENFORCE_EQ( + data_type, cur_data_type, + platform::errors::InvalidArgument( + "inputs to BroadcastTensorsOp should have the same data type," + "but detected %d v.s %d ", + framework::ToTypeName(var_type), + framework::ToTypeName(cur_var_type))); + } + + // Outputs having the same DataType/VarType as inputs + ctx->SetOutputType("Out", var_type, framework::ALL_ELEMENTS); + ctx->SetOutputDataType("Out", data_type, framework::ALL_ELEMENTS); + } +}; + +/* ------ BroadcastTensorsGradOp ------ */ +class BroadcastTensorsGradOp : public framework::OperatorWithKernel { + public: + using framework::OperatorWithKernel::OperatorWithKernel; + + void InferShape(framework::InferShapeContext* ctx) const override { + OP_INOUT_CHECK(ctx->HasOutputs(framework::GradVarName("X")), "Output", + "X@grad", "broadcast_tensors"); + OP_INOUT_CHECK(ctx->HasInputs("X"), "Input", "X", "broadcast_tensors"); + OP_INOUT_CHECK(ctx->HasInputs(framework::GradVarName("Out")), "Input", + "Out@grad", "broadcast_tensors"); + + const auto& forward_input_dims = ctx->GetInputsDim("X"); + ctx->SetOutputsDim(framework::GradVarName("X"), forward_input_dims); + ctx->ShareAllLoD("X", /*->*/ framework::GradVarName("X")); + } + + protected: + framework::OpKernelType GetExpectedKernelType( + const framework::ExecutionContext& ctx) const override { + return framework::OpKernelType(OperatorWithKernel::IndicateVarDataType( + ctx, framework::GradVarName("Out")), + ctx.device_context()); + } +}; + +template +class BroadcastTensorsGradOpMaker : public framework::SingleGradOpMaker { + public: + using framework::SingleGradOpMaker::SingleGradOpMaker; + + void Apply(GradOpPtr grad_op) const override { + grad_op->SetType("broadcast_tensors_grad"); + // We need "X" only for backward shape inference + grad_op->SetInput("X", this->Input("X")); + grad_op->SetInput(framework::GradVarName("Out"), this->OutputGrad("Out")); + grad_op->SetOutput(framework::GradVarName("X"), + this->InputGrad("X", /* drop_empty_grad */ false)); + grad_op->SetAttrMap(this->Attrs()); + } +}; + +class BroadcastTensorsGradOpVarTypeInference + : public framework::VarTypeInference { + public: + void operator()(framework::InferVarTypeContext* ctx) const override { + auto var_type = ctx->GetInputType("X", 0); + auto data_type = ctx->GetInputDataType("X", 0); + + ctx->SetOutputType(framework::GradVarName("X"), var_type, + framework::ALL_ELEMENTS); + ctx->SetOutputDataType(framework::GradVarName("X"), data_type, + framework::ALL_ELEMENTS); + } +}; + +DECLARE_NO_NEED_BUFFER_VARS_INFERER(BroadcastTensorsGradNoNeedBufVarsInferer, + "X"); + +} // namespace operators +} // namespace paddle + +namespace ops = paddle::operators; +namespace plat = paddle::platform; + +REGISTER_OPERATOR(broadcast_tensors, ops::BroadcastTensorsOp, + ops::BroadcastTensorsOpMaker, + ops::BroadcastTensorsGradOpMaker, + ops::BroadcastTensorsGradOpMaker, + ops::BroadcastTensorsOpVarTypeInference); + +REGISTER_OPERATOR(broadcast_tensors_grad, ops::BroadcastTensorsGradOp, + ops::BroadcastTensorsGradOpVarTypeInference, + ops::BroadcastTensorsGradNoNeedBufVarsInferer); + +REGISTER_OP_CPU_KERNEL( + broadcast_tensors, + ops::BroadcastTensorsOpKernel, + ops::BroadcastTensorsOpKernel, + ops::BroadcastTensorsOpKernel, + ops::BroadcastTensorsOpKernel, + ops::BroadcastTensorsOpKernel, + ops::BroadcastTensorsOpKernel); + +REGISTER_OP_CPU_KERNEL( + broadcast_tensors_grad, + ops::BroadcastTensorsGradOpKernel, + ops::BroadcastTensorsGradOpKernel, + ops::BroadcastTensorsGradOpKernel, + ops::BroadcastTensorsGradOpKernel, + ops::BroadcastTensorsGradOpKernel); diff --git a/paddle/fluid/operators/broadcast_tensors_op.cu b/paddle/fluid/operators/broadcast_tensors_op.cu new file mode 100644 index 00000000000000..d670e1b333d411 --- /dev/null +++ b/paddle/fluid/operators/broadcast_tensors_op.cu @@ -0,0 +1,132 @@ +/* Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. */ + +#include "paddle/fluid/operators/broadcast_tensors_op.h" + +#include +#include +#include +#include +#include + +#include "paddle/fluid/operators/reduce_ops/cub_reduce.h" + +namespace paddle { +namespace operators { + +using framework::Tensor; +using framework::DDim; + +template +struct IdentityFunctor { + HOSTDEVICE explicit inline IdentityFunctor() {} + + template + HOSTDEVICE inline Tout operator()(const U& x) const { + return static_cast(x); + } +}; + +template +class CUDABroadcastTensorsGradOpKernel : public framework::OpKernel { + public: + void Compute(const framework::ExecutionContext& context) const override { + // Find reduce dimensions + const auto& in_tensors = + context.MultiInput(framework::GradVarName("Out")); + auto out_tensors = context.MultiOutput(framework::GradVarName("X")); + + size_t num_ins = in_tensors.size(); + + PADDLE_ENFORCE_GT( + num_ins, 1, + platform::errors::InvalidArgument( + "Expected at least 2 input tensors, but only received d%.", + in_tensors.size())); + + PADDLE_ENFORCE_EQ( + num_ins, out_tensors.size(), + platform::errors::InvalidArgument( + "BroadcastTensorsOp expects equal number of inputs and outputs," + "but received: %d inputs v.s %d outputs", + num_ins, out_tensors.size())); + + // For each In-Out tensor pair, + // Prepare and apply broadcast dims array + for (size_t i = 0; i < num_ins; i++) { + auto* input_tensor = in_tensors[i]; + auto* output_tensor = out_tensors[i]; + + const DDim& input_dims = input_tensor->dims(); + const DDim& output_dims = output_tensor->dims(); + + int in_rank = input_dims.size(); + int out_rank = output_dims.size(); + + // Collect reduce_dims + // Example: + // dX = [1,1,1,1] + // dOut = [1,1,1,4] + // + // reduce_dims = [3] // reduce along the broadcasted axis + std::vector reduce_dims_vec; + for (int j = 0; j < in_rank; j++) { + int out_axis = out_rank - j - 1; + int in_axis = in_rank - j - 1; + + if (out_axis < 0 || output_dims[out_axis] != input_dims[in_axis]) { + reduce_dims_vec.push_back(in_axis); + } + } + + bool just_copy = (reduce_dims_vec.size() == 0); + output_tensor->mutable_data(context.GetPlace()); + if (just_copy) { + // Turns out to be a No-Op, simply copy tensors + framework::TensorCopy(*input_tensor, context.GetPlace(), + context.device_context(), output_tensor); + } else { + // reduce_sum implementation on CUDA + auto stream = context.cuda_device_context().stream(); + TensorReduce>( + *input_tensor, output_tensor, reduce_dims_vec, static_cast(0), + cub::Sum(), IdentityFunctor(), stream); + } + } + } +}; + +} // namespace operators +} // namespace paddle + +namespace ops = paddle::operators; +namespace plat = paddle::platform; + +REGISTER_OP_CUDA_KERNEL( + broadcast_tensors, + ops::BroadcastTensorsOpKernel, + ops::BroadcastTensorsOpKernel, + ops::BroadcastTensorsOpKernel, + ops::BroadcastTensorsOpKernel, + ops::BroadcastTensorsOpKernel, + ops::BroadcastTensorsOpKernel); + +REGISTER_OP_CUDA_KERNEL(broadcast_tensors_grad, + ops::CUDABroadcastTensorsGradOpKernel, + ops::CUDABroadcastTensorsGradOpKernel, + ops::CUDABroadcastTensorsGradOpKernel, + ops::CUDABroadcastTensorsGradOpKernel, + ops::CUDABroadcastTensorsGradOpKernel); diff --git a/paddle/fluid/operators/broadcast_tensors_op.h b/paddle/fluid/operators/broadcast_tensors_op.h new file mode 100644 index 00000000000000..0eeb9234df0fee --- /dev/null +++ b/paddle/fluid/operators/broadcast_tensors_op.h @@ -0,0 +1,282 @@ +/* Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. */ + +#pragma once +#include + +#include "paddle/fluid/framework/eigen.h" +#include "paddle/fluid/framework/lod_tensor_array.h" +#include "paddle/fluid/framework/op_registry.h" +#include "paddle/fluid/operators/eigen/eigen_function.h" +#include "paddle/fluid/operators/math/math_function.h" + +#define SWITCH_OUT_RANK_CASE(n) \ + case n: { \ + ApplyBroadcast(context, in_tensors[i], out_tensors[i]); \ + break; \ + } + +namespace paddle { +namespace operators { + +using framework::Tensor; +using framework::DDim; +using framework::EigenTensor; + +template +class BroadcastTensorsOpKernel : public framework::OpKernel { + public: + void Compute(const framework::ExecutionContext& context) const override { + const auto& in_tensors = context.MultiInput("X"); + auto out_tensors = context.MultiOutput("Out"); + + size_t num_ins = in_tensors.size(); + + PADDLE_ENFORCE_GT( + num_ins, 1, + platform::errors::InvalidArgument( + "Expected at least 2 input tensors, but only received d%.", + in_tensors.size())); + + PADDLE_ENFORCE_EQ( + num_ins, out_tensors.size(), + platform::errors::InvalidArgument( + "BroadcastTensorsOp expects equal number of inputs and outputs," + "but received: %d inputs v.s %d outputs", + num_ins, out_tensors.size())); + + // Eigen has no support for dynamic ranked tensor + // Thus we perform static expansion for each possible ranks + for (size_t i = 0; i < num_ins; i++) { + int out_rank = out_tensors[i]->dims().size(); + switch (out_rank) { + SWITCH_OUT_RANK_CASE(1) + SWITCH_OUT_RANK_CASE(2) + SWITCH_OUT_RANK_CASE(3) + SWITCH_OUT_RANK_CASE(4) + SWITCH_OUT_RANK_CASE(5) + default: { + PADDLE_THROW(platform::errors::InvalidArgument( + "Target tensor rank out of range" + "Maximum supported rank for broadcast is: 5")); + } + } + } + } + + template + void ApplyBroadcast(const framework::ExecutionContext& context, + const Tensor* input_tensor, Tensor* output_tensor) const { + const auto& input_dims = input_tensor->dims(); + const auto& output_dims = output_tensor->dims(); + + int in_rank = input_dims.size(); + int out_rank = output_dims.size(); + + // 1. Collect bcast_dims, each element of which indicates how many + // times we need to replicate along the corresponding dimension + // 2. Collect new_input_dims_vec. Eigen::broadcast requires same rank for + // both input and output tensors, so we need to initialize input X with + // expanded dims: "new_input_dims_vec" + Eigen::DSizes bcast_dims; + std::vector new_input_dims_vec(out_rank); + for (int j = 0; j < out_rank; j++) { + int out_axis = out_rank - j - 1; + int in_axis = in_rank - j - 1; + + bcast_dims[out_axis] = output_dims[out_axis]; + new_input_dims_vec[out_axis] = 1; + if (in_axis >= 0 && input_dims[in_axis] == output_dims[out_axis]) { + bcast_dims[out_axis] = 1; + new_input_dims_vec[out_axis] = input_dims[in_axis]; + } + } + auto new_input_dims = framework::make_ddim(new_input_dims_vec); + + // Initialize input X with new_input_dims_vec, so it's rank-aligned with the + // output + auto x = EigenTensor::From(*input_tensor, new_input_dims); + + output_tensor->mutable_data(context.GetPlace()); + auto y = EigenTensor::From(*output_tensor, output_dims); + + auto& place = + *context.template device_context().eigen_device(); + EigenBroadcast, T, OutRank>::Eval(place, y, x, + bcast_dims); + } +}; + +#define SWITCH_RESHAPE_DIMS(n) \ + case n: { \ + Eigen::DSizes reshape_dims; \ + for (size_t i = 0; i < reshape_dims_vec.size(); ++i) { \ + reshape_dims[i] = reshape_dims_vec[i]; \ + } \ + dX.device(place) = \ + dOut.reshape(reshape_dims).sum(reduce_dims).reshape(dX.dimensions()); \ + break; \ + } + +#define UPPER_SWITCH_REDUCE_DIMS(m) \ + case m: { \ + Eigen::DSizes reduce_dims; \ + for (size_t i = 0; i < reduce_dims_vec.size(); ++i) { \ + reduce_dims[i] = reduce_dims_vec[i]; \ + } \ + switch (reshape_size) { +#define LOWER_SWITCH_REDUCE_DIMS \ + default: { \ + PADDLE_THROW(platform::errors::InvalidArgument( \ + "Detected reshape size: %d out of range" \ + "Minimum value should be larger than reduce size %d" \ + "While maximum supported is: 5", \ + reshape_size, reduce_size)); \ + } \ + } \ + break; \ + } + +/* ----- GradOpKernel ----- */ +template +class BroadcastTensorsGradOpKernel : public framework::OpKernel { + public: + void Compute(const framework::ExecutionContext& context) const override { + // Find reduce dimensions + const auto& in_tensors = + context.MultiInput(framework::GradVarName("Out")); + auto out_tensors = context.MultiOutput(framework::GradVarName("X")); + + size_t num_ins = in_tensors.size(); + + PADDLE_ENFORCE_GT( + num_ins, 1, + platform::errors::InvalidArgument( + "Expected at least 2 input tensors, but only received d%.", + in_tensors.size())); + + PADDLE_ENFORCE_EQ( + num_ins, out_tensors.size(), + platform::errors::InvalidArgument( + "BroadcastTensorsOp expects equal number of inputs and outputs," + "but received: %d inputs v.s %d outputs", + num_ins, out_tensors.size())); + + // For each In-Out tensor pair, + // Prepare and apply broadcast dims array + for (size_t i = 0; i < num_ins; i++) { + const auto* input_tensor = in_tensors[i]; + auto* output_tensor = out_tensors[i]; + + const auto& input_dims = input_tensor->dims(); + const auto& output_dims = output_tensor->dims(); + + int in_rank = input_dims.size(); + int out_rank = output_dims.size(); + + // BroadcastTensorsGrad is simply a reduce_sum along broadcasted axes + // Here we perform the following Eigen operations: + // dOut(Flattened) -> reshape(reshape_dims) -> reduce(reduce_dims) -> + // reshape(dX_shape) -> dX + // Note the last "reshape(dX_shape)" will be performed implicitly, + // and we only need to collect reduce_dims and reshape_dims + std::vector reduce_dims_vec; + std::vector reshape_dims_vec; + for (int j = 0; j < in_rank; j++) { + int out_axis = out_rank - j - 1; + int in_axis = in_rank - j - 1; + + reshape_dims_vec.push_back(input_dims[j]); + if (out_axis < 0 || output_dims[out_axis] != input_dims[in_axis]) { + reduce_dims_vec.push_back(in_axis); + } + } + + size_t reduce_size = reduce_dims_vec.size(); + size_t reshape_size = reshape_dims_vec.size(); + bool just_copy = (reduce_dims_vec.size() == 0); + output_tensor->mutable_data(context.GetPlace()); + if (just_copy) { + // If this turns out to be a No-Op, simply perform a tensor copy + framework::TensorCopy(*input_tensor, context.GetPlace(), + context.device_context(), output_tensor); + } else { + PADDLE_ENFORCE_GE(reduce_dims_vec.size(), 1, + platform::errors::InvalidArgument( + "The number of dimensions of the input " + "'Out@GRAD' for Op(broadcast_tensors)" + " must be greater than or equal to 1, but " + "the value received is %d.", + reduce_dims_vec.size())); + PADDLE_ENFORCE_LE( + reduce_dims_vec.size(), 5, + platform::errors::InvalidArgument( + "The number of dimensions of the input 'Out@GRAD' " + "for Op(broadcast_tensors) must be less than or equal " + "to 5, but the value received is %d.", + reduce_dims_vec.size())); + + // Overall: + // dOut(Flattened) -> reshape(reshape_dims) -> reduce(reduce_dims) -> + // reshape(dX_shape) -> dX + auto dX = framework::EigenVector::Flatten(*output_tensor); + auto dOut = framework::EigenVector::Flatten(*input_tensor); + auto& place = + *context.template device_context().eigen_device(); + + // Expand ReduceSize and ReshapeSize into static values + switch (reduce_size) { + UPPER_SWITCH_REDUCE_DIMS(1) + SWITCH_RESHAPE_DIMS(1) + SWITCH_RESHAPE_DIMS(2) + SWITCH_RESHAPE_DIMS(3) + SWITCH_RESHAPE_DIMS(4) + SWITCH_RESHAPE_DIMS(5) + LOWER_SWITCH_REDUCE_DIMS + + UPPER_SWITCH_REDUCE_DIMS(2) + SWITCH_RESHAPE_DIMS(2) + SWITCH_RESHAPE_DIMS(3) + SWITCH_RESHAPE_DIMS(4) + SWITCH_RESHAPE_DIMS(5) + LOWER_SWITCH_REDUCE_DIMS + + UPPER_SWITCH_REDUCE_DIMS(3) + SWITCH_RESHAPE_DIMS(3) + SWITCH_RESHAPE_DIMS(4) + SWITCH_RESHAPE_DIMS(5) + LOWER_SWITCH_REDUCE_DIMS + + UPPER_SWITCH_REDUCE_DIMS(4) + SWITCH_RESHAPE_DIMS(4) + SWITCH_RESHAPE_DIMS(5) + LOWER_SWITCH_REDUCE_DIMS + + UPPER_SWITCH_REDUCE_DIMS(5) + SWITCH_RESHAPE_DIMS(5) + LOWER_SWITCH_REDUCE_DIMS + + default: { + PADDLE_THROW(platform::errors::InvalidArgument( + "Detected reduce size: %d out of range" + "While maximum supported is: 5", + reduce_size)); + } + } + } + } + } +}; +} // namespace operators +} // namespace paddle diff --git a/paddle/fluid/operators/cast_op_xpu.cc b/paddle/fluid/operators/cast_op_xpu.cc index ca15858cf67d75..c7c0f81f2131f7 100644 --- a/paddle/fluid/operators/cast_op_xpu.cc +++ b/paddle/fluid/operators/cast_op_xpu.cc @@ -23,21 +23,9 @@ limitations under the License. */ namespace paddle { namespace operators { -template -class XPUFPTypeTrait { - public: - using Type = T; -}; - -template <> -class XPUFPTypeTrait { - public: - using Type = float16; -}; - template class CastXPUKernel : public framework::OpKernel { - using XPUInTDType = typename XPUFPTypeTrait::Type; + using XPUInTDType = typename XPUTypeTrait::Type; public: void Compute(const framework::ExecutionContext& context) const override { @@ -49,7 +37,6 @@ class CastXPUKernel : public framework::OpKernel { context.Attr("out_dtype")); auto* in_data = in->data(); - // using XPUOutTDType = typename XPUFPTypeTrait::Type; auto numel = in->numel(); auto& dev_ctx = context.template device_context(); int r = -1; diff --git a/paddle/fluid/operators/collective/c_comm_init_hccl_op.cc b/paddle/fluid/operators/collective/c_comm_init_hccl_op.cc index 7817f19bacb187..3df0595525941a 100644 --- a/paddle/fluid/operators/collective/c_comm_init_hccl_op.cc +++ b/paddle/fluid/operators/collective/c_comm_init_hccl_op.cc @@ -22,7 +22,11 @@ class Scope; } // namespace framework } // namespace paddle #if defined(PADDLE_WITH_ASCEND_CL) +#include "acl/acl.h" +#include "hccl/hccl.h" +#include "hccl/hccl_types.h" #include "paddle/fluid/platform/collective_helper.h" +#include "paddle/fluid/platform/hccl_helper.h" #endif namespace paddle { @@ -57,6 +61,33 @@ class CCommInitOpAscend : public framework::OperatorBase { } platform::HCCLCommContext::Instance().CreateHCCLComm( hccl_id, rank_ids, rank_id, device_id, rid); + + // Build comm + float* buff; + int32_t size = 20; + std::vector input(size, 0); + for (int32_t idx = 0; idx < size; idx++) { + input[idx] = 1.0; + } + PADDLE_ENFORCE_NPU_SUCCESS(aclrtMalloc(reinterpret_cast(&buff), + size * sizeof(float), + ACL_MEM_MALLOC_HUGE_FIRST)); + PADDLE_ENFORCE_NPU_SUCCESS(aclrtMemcpy( + reinterpret_cast(buff), size * sizeof(float), input.data(), + size * sizeof(float), ACL_MEMCPY_HOST_TO_DEVICE)); + VLOG(3) << "Build buff data successful."; + + aclrtStream stream = nullptr; + auto comm = paddle::platform::HCCLCommContext::Instance().Get(rid, place); + if (rank_id == 0) { + stream = comm->stream(); + } else { + auto dev_ctx = platform::DeviceContextPool::Instance().Get(place); + stream = static_cast(dev_ctx)->stream(); + } + PADDLE_ENFORCE_NPU_SUCCESS(platform::dynload::HcclBroadcast( + buff, size, HCCL_DATA_TYPE_FP32, 0, comm->comm(), stream)); + VLOG(3) << "Build connection successful."; #else PADDLE_THROW(platform::errors::PreconditionNotMet( "PaddlePaddle should compile with NPU.")); diff --git a/paddle/fluid/operators/compat/conv2d.pbtxt b/paddle/fluid/operators/compat/conv2d.pbtxt index d8a08b6b4101af..9e4c8b796a8980 100644 --- a/paddle/fluid/operators/compat/conv2d.pbtxt +++ b/paddle/fluid/operators/compat/conv2d.pbtxt @@ -41,6 +41,22 @@ def { } } extra { + attrs { + name: "Input_scale" + type: FLOAT + } + attrs { + name: "quantization_type" + type: STRING + } + attrs { + name: "bit_length" + type: INT + } + attrs { + name: "out_threshold" + type: FLOAT + } attrs { name: "@ENABLE_CACHE_RUNTIME_CONTEXT@" type: BOOLEAN diff --git a/paddle/fluid/operators/compat/cvm.pbtxt b/paddle/fluid/operators/compat/cvm.pbtxt new file mode 100644 index 00000000000000..ccbeabc1f1511c --- /dev/null +++ b/paddle/fluid/operators/compat/cvm.pbtxt @@ -0,0 +1,39 @@ +type: "cvm" +def { + inputs { + name: "X" + } + inputs { + name: "CVM" + } + outputs { + name: "Y" + } + attrs { + name: "use_cvm" + type: BOOLEAN + } +} +extra { + attrs { + name: "op_role" + type: INT + } + attrs { + name: "op_role_var" + type: STRINGS + } + attrs { + name: "op_namescope" + type: STRING + } + attrs { + name: "op_callstack" + type: STRINGS + } + attrs { + name: "op_device" + type: STRING + } +} + diff --git a/paddle/fluid/operators/compat/depthwise_conv2d.pbtxt b/paddle/fluid/operators/compat/depthwise_conv2d.pbtxt new file mode 100644 index 00000000000000..901ed164608071 --- /dev/null +++ b/paddle/fluid/operators/compat/depthwise_conv2d.pbtxt @@ -0,0 +1,177 @@ +type: "depthwise_conv2d" +def { + inputs { + name: "Input" + } + inputs { + name: "Filter" + } + inputs { + name: "Bias" + } + inputs { + name: "ResidualData" + } + outputs { + name: "Output" + } + attrs { + name: "strides" + type: INTS + } + attrs { + name: "paddings" + type: INTS + } + attrs { + name: "padding_algorithm" + type: STRING + } + attrs { + name: "groups" + type: INT + } + attrs { + name: "dilations" + type: INTS + } + attrs { + name: "data_format" + type: STRING + } +} +extra { + attrs { + name: "Input_scale" + type: FLOAT + } + attrs { + name: "quantization_type" + type: STRING + } + attrs { + name: "bit_length" + type: INT + } + attrs { + name: "out_threshold" + type: FLOAT + } + attrs { + name: "@ENABLE_CACHE_RUNTIME_CONTEXT@" + type: BOOLEAN + } + attrs { + name: "skip_quant" + type: BOOLEAN + } + attrs { + name: "is_test" + type: BOOLEAN + } + attrs { + name: "name" + type: STRING + } + attrs { + name: "use_cudnn" + type: BOOLEAN + } + attrs { + name: "fuse_relu_before_depthwise_conv" + type: BOOLEAN + } + attrs { + name: "use_mkldnn" + type: BOOLEAN + } + attrs { + name: "use_quantizer" + type: BOOLEAN + } + attrs { + name: "mkldnn_data_type" + type: STRING + } + attrs { + name: "fuse_relu" + type: BOOLEAN + } + attrs { + name: "fuse_brelu" + type: BOOLEAN + } + attrs { + name: "fuse_brelu_threshold" + type: FLOAT + } + attrs { + name: "fuse_activation" + type: STRING + } + attrs { + name: "fuse_alpha" + type: FLOAT + } + attrs { + name: "fuse_beta" + type: FLOAT + } + attrs { + name: "use_addto" + type: BOOLEAN + } + attrs { + name: "fuse_residual_connection" + type: BOOLEAN + } + attrs { + name: "Scale_in" + type: FLOAT + } + attrs { + name: "Scale_out" + type: FLOAT + } + attrs { + name: "Scale_in_eltwise" + type: FLOAT + } + attrs { + name: "Scale_weights" + type: FLOATS + } + attrs { + name: "force_fp32_output" + type: BOOLEAN + } + attrs { + name: "workspace_size_MB" + type: INT + } + attrs { + name: "exhaustive_search" + type: BOOLEAN + } + attrs { + name: "op_role" + type: INT + } + attrs { + name: "op_role_var" + type: STRINGS + } + attrs { + name: "op_namescope" + type: STRING + } + attrs { + name: "op_callstack" + type: STRINGS + } + attrs { + name: "op_device" + type: STRING + } +} + diff --git a/paddle/fluid/operators/compat/fake_channel_wise_dequantize_max_abs.pbtxt b/paddle/fluid/operators/compat/fake_channel_wise_dequantize_max_abs.pbtxt new file mode 100644 index 00000000000000..542a0ff649ff9c --- /dev/null +++ b/paddle/fluid/operators/compat/fake_channel_wise_dequantize_max_abs.pbtxt @@ -0,0 +1,47 @@ +type: "fake_channel_wise_dequantize_max_abs" +def { + inputs { + name: "X" + } + inputs { + name: "Scales" + } + outputs { + name: "Out" + } + attrs { + name: "quant_bits" + type: INTS + } + attrs { + name: "quant_axis" + type: INT + } +} +extra { + attrs { + name: "is_test" + type: BOOLEAN + } + attrs { + name: "op_role" + type: INT + } + attrs { + name: "op_role_var" + type: STRINGS + } + attrs { + name: "op_namescope" + type: STRING + } + attrs { + name: "op_callstack" + type: STRINGS + } + attrs { + name: "op_device" + type: STRING + } +} + diff --git a/paddle/fluid/operators/compat/fake_channel_wise_quantize_dequantize_abs_max.pbtxt b/paddle/fluid/operators/compat/fake_channel_wise_quantize_dequantize_abs_max.pbtxt new file mode 100644 index 00000000000000..7c49da93e71836 --- /dev/null +++ b/paddle/fluid/operators/compat/fake_channel_wise_quantize_dequantize_abs_max.pbtxt @@ -0,0 +1,46 @@ +type: "fake_channel_wise_quantize_dequantize_abs_max" +def { + inputs { + name: "X" + } + outputs { + name: "Out" + } + outputs { + name: "OutScale" + } + attrs { + name: "quant_axis" + type: INT + } + attrs { + name: "bit_length" + type: INT + } +} +extra { + attrs { + name: "is_test" + type: BOOLEAN + } + attrs { + name: "op_role" + type: INT + } + attrs { + name: "op_role_var" + type: STRINGS + } + attrs { + name: "op_namescope" + type: STRING + } + attrs { + name: "op_callstack" + type: STRINGS + } + attrs { + name: "op_device" + type: STRING + } +} diff --git a/paddle/fluid/operators/compat/fake_quantize_dequantize_abs_max.pbtxt b/paddle/fluid/operators/compat/fake_quantize_dequantize_abs_max.pbtxt new file mode 100644 index 00000000000000..bebb397e20bbe7 --- /dev/null +++ b/paddle/fluid/operators/compat/fake_quantize_dequantize_abs_max.pbtxt @@ -0,0 +1,38 @@ +type: "fake_quantize_dequantize_abs_max" +def { + inputs { + name: "X" + } + outputs { + name: "Out" + } + outputs { + name: "OutScale" + } + attrs { + name: "bit_length" + type: INT + } +} +extra { + attrs { + name: "op_role" + type: INT + } + attrs { + name: "op_role_var" + type: STRINGS + } + attrs { + name: "op_namescope" + type: STRING + } + attrs { + name: "op_callstack" + type: STRINGS + } + attrs { + name: "op_device" + type: STRING + } +} diff --git a/paddle/fluid/operators/compat/fill_constant.pbtxt b/paddle/fluid/operators/compat/fill_constant.pbtxt index 308348fd7e30de..26fecf623c19cd 100644 --- a/paddle/fluid/operators/compat/fill_constant.pbtxt +++ b/paddle/fluid/operators/compat/fill_constant.pbtxt @@ -24,7 +24,6 @@ def { name: "value" type: FLOAT } - } extra { attrs { diff --git a/paddle/fluid/operators/compat/hard_swish.pbtxt b/paddle/fluid/operators/compat/hard_swish.pbtxt index ccf387652ed325..9951513741a61a 100644 --- a/paddle/fluid/operators/compat/hard_swish.pbtxt +++ b/paddle/fluid/operators/compat/hard_swish.pbtxt @@ -24,6 +24,18 @@ extra { name: "op_role" type: INT } + attrs { + name: "use_mkldnn" + type: BOOLEAN + } + attrs { + name: "name" + type: STRING + } + attrs { + name: "@ENABLE_CACHE_RUNTIME_CONTEXT@" + type: BOOLEAN + } attrs { name: "op_role_var" type: STRINGS diff --git a/paddle/fluid/operators/compat/leaky_relu.pbtxt b/paddle/fluid/operators/compat/leaky_relu.pbtxt index 9df2e5916118c5..8618b72ca87485 100644 --- a/paddle/fluid/operators/compat/leaky_relu.pbtxt +++ b/paddle/fluid/operators/compat/leaky_relu.pbtxt @@ -16,6 +16,18 @@ extra { name: "use_mkldnn" type: BOOLEAN } + attrs { + name: "name" + type: STRING + } + attrs { + name: "@ENABLE_CACHE_RUNTIME_CONTEXT@" + type: BOOLEAN + } + attrs { + name: "is_test" + type: BOOLEAN + } attrs { name: "op_role" type: INT diff --git a/paddle/fluid/operators/compat/matmul.pbtxt b/paddle/fluid/operators/compat/matmul.pbtxt index e68a7f31b66340..8f29d936606089 100644 --- a/paddle/fluid/operators/compat/matmul.pbtxt +++ b/paddle/fluid/operators/compat/matmul.pbtxt @@ -23,6 +23,10 @@ def { } } extra { + attrs { + name: "head_number" + type: INT + } attrs { name: "Scale_out" type: FLOAT diff --git a/paddle/fluid/operators/compat/relu.pbtxt b/paddle/fluid/operators/compat/relu.pbtxt index 271ed91718cee4..9a184bf03d0a6d 100644 --- a/paddle/fluid/operators/compat/relu.pbtxt +++ b/paddle/fluid/operators/compat/relu.pbtxt @@ -52,4 +52,8 @@ extra { name: "is_test" type: BOOLEAN } + attrs { + name: "name" + type: STRINGS + } } diff --git a/paddle/fluid/operators/compat/relu6.pbtxt b/paddle/fluid/operators/compat/relu6.pbtxt index edd29037324430..340b13020144a8 100644 --- a/paddle/fluid/operators/compat/relu6.pbtxt +++ b/paddle/fluid/operators/compat/relu6.pbtxt @@ -6,16 +6,28 @@ def { outputs { name: "Out" } + attrs { + name: "threshold" + type: FLOAT + } } extra { attrs { - name: "threshold" + name: "name" + type: STRING + } + attrs { + name: "is_test" type: FLOAT } attrs { name: "use_mkldnn" type: BOOLEAN } + attrs { + name: "@ENABLE_CACHE_RUNTIME_CONTEXT@" + type: BOOLEAN + } attrs { name: "op_role" type: INT diff --git a/paddle/fluid/operators/compat/sequence_pool.pbtxt b/paddle/fluid/operators/compat/sequence_pool.pbtxt new file mode 100644 index 00000000000000..c45f457fe0d9ff --- /dev/null +++ b/paddle/fluid/operators/compat/sequence_pool.pbtxt @@ -0,0 +1,47 @@ +type: "sequence_pool" +def { + inputs { + name: "X" + } + outputs { + name: "Out" + } + outputs { + name: "MaxIndex" + } + attrs { + name: "pooltype" + type: STRING + } + attrs { + name: "pad_value" + type: FLOAT + } +} +extra { + attrs { + name: "is_test" + type: BOOLEAN + } + attrs { + name: "op_role" + type: INT + } + attrs { + name: "op_role_var" + type: STRINGS + } + attrs { + name: "op_namescope" + type: STRING + } + attrs { + name: "op_callstack" + type: STRINGS + } + attrs { + name: "op_device" + type: STRING + } +} + diff --git a/paddle/fluid/operators/compat/softmax.pbtxt b/paddle/fluid/operators/compat/softmax.pbtxt index 5cd155ed1c63a8..04f15ace15f449 100644 --- a/paddle/fluid/operators/compat/softmax.pbtxt +++ b/paddle/fluid/operators/compat/softmax.pbtxt @@ -10,12 +10,12 @@ def { name: "axis" type: INT } +} +extra { attrs { name: "data_format" type: STRING } -} -extra { attrs { name: "op_role" type: INT diff --git a/paddle/fluid/operators/compat/swish.pbtxt b/paddle/fluid/operators/compat/swish.pbtxt index 4f5ec127e48979..1dd8e577d9c738 100644 --- a/paddle/fluid/operators/compat/swish.pbtxt +++ b/paddle/fluid/operators/compat/swish.pbtxt @@ -12,6 +12,10 @@ extra { name: "beta" type: FLOAT } + attrs { + name: "name" + type: STRING + } attrs { name: "use_mkldnn" type: BOOLEAN diff --git a/paddle/fluid/operators/compat/transpose.pbtxt b/paddle/fluid/operators/compat/transpose.pbtxt index 97081e0afc29a8..1cd04a4da4a174 100644 --- a/paddle/fluid/operators/compat/transpose.pbtxt +++ b/paddle/fluid/operators/compat/transpose.pbtxt @@ -10,12 +10,12 @@ def { name: "axis" type: INTS } +} +extra { attrs { name: "data_format" type: STRING } -} -extra { attrs { name: "use_mkldnn" type: BOOLEAN diff --git a/paddle/fluid/operators/compat/transpose2.pbtxt b/paddle/fluid/operators/compat/transpose2.pbtxt index 19d991a6414d13..31aecd24bc911b 100644 --- a/paddle/fluid/operators/compat/transpose2.pbtxt +++ b/paddle/fluid/operators/compat/transpose2.pbtxt @@ -13,12 +13,12 @@ def { name: "axis" type: INTS } +} +extra { attrs { name: "data_format" type: STRING } -} -extra { attrs { name: "use_mkldnn" type: BOOLEAN diff --git a/paddle/fluid/operators/controlflow/compare_all_op.cc b/paddle/fluid/operators/controlflow/compare_all_op.cc index 9442c7583d98fe..ede349f737d899 100644 --- a/paddle/fluid/operators/controlflow/compare_all_op.cc +++ b/paddle/fluid/operators/controlflow/compare_all_op.cc @@ -30,29 +30,13 @@ class CompareReduceOpKernel auto* x = context.Input("X"); auto* y = context.Input("Y"); auto* z = context.Output("Out"); - bool shape_same = true; - Tensor tmp; - framework::DDim x_dims = x->dims(); - framework::DDim y_dims = y->dims(); - - // judge the two inputs shape is same, if not same, just return false - if (x_dims.size() != y_dims.size()) { - shape_same = false; - } else { - for (auto i = 0; i < x_dims.size(); i++) { - if (x_dims[i] != y_dims[i]) { - shape_same = false; - break; - } - } - } - bool* z_data = z->mutable_data(context.GetPlace()); - if (!shape_same) { + + if (x->dims() != y->dims()) { z_data[0] = false; } else { - tmp.mutable_data(x_dims, context.GetPlace()); + tmp.mutable_data(x->dims(), context.GetPlace()); if (x->numel() == 1 && y->numel() == 1) { bool* z_data = tmp.mutable_data(context.GetPlace()); z_data[0] = Functor()(x->data()[0], y->data()[0]); diff --git a/paddle/fluid/operators/controlflow/compare_all_op.cu b/paddle/fluid/operators/controlflow/compare_all_op.cu index 3753ed6b15f1e3..9e22d74d6e2aac 100644 --- a/paddle/fluid/operators/controlflow/compare_all_op.cu +++ b/paddle/fluid/operators/controlflow/compare_all_op.cu @@ -14,14 +14,18 @@ limitations under the License. */ #include #include "paddle/fluid/operators/controlflow/compare_all_op.h" +#include "paddle/fluid/operators/elementwise/elementwise_op_impl.cu.h" #include "paddle/fluid/operators/reduce_ops/cub_reduce.h" + +namespace ops = paddle::operators; +namespace plat = paddle::platform; + namespace paddle { namespace operators { template struct IdentityFunctor { HOSTDEVICE explicit inline IdentityFunctor() {} - HOSTDEVICE inline T operator()(const T& x) const { return x; } }; @@ -33,6 +37,24 @@ struct BitwiseAdd { return a & b; } }; + +template +struct CudaEqualReduceFunctor { + using ELEM_TYPE = T; + HOSTDEVICE bool operator()(const T args[]) const { + return (args[0] == args[1]); + } +}; + +template +struct CudaEqualReduceFunctor< + T, typename std::enable_if::value>::type> { + using ELEM_TYPE = T; + HOSTDEVICE bool operator()(const T args[]) const { + return fabs(static_cast(args[0] - args[1])) < 1e-8; + } +}; + template class CompareReduceOpKernel : public framework::OpKernel { @@ -44,32 +66,22 @@ class CompareReduceOpKernel auto* x = context.Input("X"); auto* y = context.Input("Y"); auto* z = context.Output("Out"); - bool shape_same = true; - + bool* z_data = z->mutable_data(context.GetPlace()); Tensor tmp; - framework::DDim x_dims = x->dims(); - framework::DDim y_dims = y->dims(); - if (x_dims.size() != y_dims.size()) { - shape_same = false; - } else { - for (auto i = 0; i < x_dims.size(); i++) { - if (x_dims[i] != y_dims[i]) { - shape_same = false; - break; - } - } - } - - bool* z_data = z->mutable_data(context.GetPlace()); - if (!shape_same) { + if (x->dims() != y->dims()) { thrust::device_ptr z_dev_ptr(z_data); thrust::fill(z_dev_ptr, z_dev_ptr + 1, false); return; } else { - tmp.mutable_data(x_dims, context.GetPlace()); - ElementwiseComputeEx(context, x, y, 0, - Functor(), &tmp); + tmp.mutable_data(x->dims(), context.GetPlace()); + const auto& cuda_ctx = + context.template device_context(); + std::vector ins = {x, y}; + std::vector outs = {&tmp}; + LaunchSameDimsElementwiseCudaKernel( + cuda_ctx, ins, &outs, Functor()); + // Reduce by 'bitwise and' operator std::vector reduce_dims; reduce_dims.resize(tmp.dims().size()); @@ -85,18 +97,17 @@ class CompareReduceOpKernel } // namespace operators } // namespace paddle -#define REGISTER_COMPARE_REDUCE_CUDA_KERNEL(op_type, functor) \ - REGISTER_OP_CUDA_KERNEL( \ - op_type, paddle::operators::CompareReduceOpKernel< \ - paddle::platform::CUDADeviceContext, functor>, \ - paddle::operators::CompareReduceOpKernel< \ - paddle::platform::CUDADeviceContext, functor>, \ - paddle::operators::CompareReduceOpKernel< \ - paddle::platform::CUDADeviceContext, functor>, \ - paddle::operators::CompareReduceOpKernel< \ - paddle::platform::CUDADeviceContext, functor>, \ - paddle::operators::CompareReduceOpKernel< \ - paddle::platform::CUDADeviceContext, functor>); - -REGISTER_COMPARE_REDUCE_CUDA_KERNEL(equal_all, - paddle::operators::EqualReduceFunctor); +#define REGISTER_COMPARE_REDUCE_CUDA_KERNEL(op_type, functor) \ + REGISTER_OP_CUDA_KERNEL( \ + op_type, \ + ops::CompareReduceOpKernel>, \ + ops::CompareReduceOpKernel>, \ + ops::CompareReduceOpKernel>, \ + ops::CompareReduceOpKernel>, \ + ops::CompareReduceOpKernel>); + +REGISTER_COMPARE_REDUCE_CUDA_KERNEL(equal_all, CudaEqualReduceFunctor) +#undef REGISTER_COMPARE_REDUCE_CUDA_KERNEL diff --git a/paddle/fluid/operators/controlflow/compare_op.cu b/paddle/fluid/operators/controlflow/compare_op.cu index 6f3a615edb44be..bf7861a03d8d4d 100644 --- a/paddle/fluid/operators/controlflow/compare_op.cu +++ b/paddle/fluid/operators/controlflow/compare_op.cu @@ -59,7 +59,6 @@ struct CudaNotEqualFunctor< template class CompareOpKernel : public framework::OpKernel { - public: public: using InT = typename Functor::ELEMENT_TYPE; using OutT = bool; diff --git a/paddle/fluid/operators/controlflow/fetch_op.cc b/paddle/fluid/operators/controlflow/fetch_op.cc index fdd1b776bd8fa3..d86b6b48422d94 100644 --- a/paddle/fluid/operators/controlflow/fetch_op.cc +++ b/paddle/fluid/operators/controlflow/fetch_op.cc @@ -44,11 +44,6 @@ static void DataCopy(const framework::LoDTensor &src_item, TensorCopySync(src_item, platform::CPUPlace(), dst_item); } #else -#ifdef PADDLE_WITH_ASCEND_CL - if (platform::is_npu_place(src_item.place())) { - platform::DeviceContextPool::Instance().Get(src_item.place())->Wait(); - } -#endif TensorCopySync(src_item, platform::CPUPlace(), dst_item); #endif } else { diff --git a/paddle/fluid/operators/correlation_op.cu b/paddle/fluid/operators/correlation_op.cu index 9b08f875bb6e6d..f488cc12e642b8 100644 --- a/paddle/fluid/operators/correlation_op.cu +++ b/paddle/fluid/operators/correlation_op.cu @@ -55,6 +55,7 @@ __forceinline__ __device__ T blockReduceSum(T val) { int wid = threadIdx.x / warpSize; val = warpReduceSum(val); + __syncthreads(); if (lane == 0) shared[wid] = val; __syncthreads(); diff --git a/paddle/fluid/operators/diagonal_op.cc b/paddle/fluid/operators/diagonal_op.cc new file mode 100644 index 00000000000000..dd5a84ade59ced --- /dev/null +++ b/paddle/fluid/operators/diagonal_op.cc @@ -0,0 +1,186 @@ +// Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "paddle/fluid/operators/diagonal_op.h" + +namespace paddle { +namespace operators { + +class DiagonalOp : public framework::OperatorWithKernel { + public: + using framework::OperatorWithKernel::OperatorWithKernel; + + void InferShape(framework::InferShapeContext *ctx) const override { + OP_INOUT_CHECK(ctx->HasInput("Input"), "Input", "Input", "diagonal"); + OP_INOUT_CHECK(ctx->HasOutput("Out"), "Output", "Out", "diagonal"); + + int offset_ = ctx->Attrs().Get("offset"); + int axis1 = ctx->Attrs().Get("axis1"); + int axis2 = ctx->Attrs().Get("axis2"); + + auto x_dims = ctx->GetInputDim("Input"); + int axis1_ = axis1 < 0 ? x_dims.size() + axis1 : axis1; + int axis2_ = axis2 < 0 ? x_dims.size() + axis2 : axis2; + + PADDLE_ENFORCE_GE( + x_dims.size(), 2, + platform::errors::OutOfRange("Input's dim is out of range (expected at " + "least 2 dimensions, but got %ld).", + x_dims.size())); + PADDLE_ENFORCE_LT( + axis1_, x_dims.size(), + platform::errors::OutOfRange( + "Attr(axis1) is out of range (expected to be in range of [%ld, " + "%ld], but got %ld).", + -(x_dims.size()), (x_dims.size() - 1), axis1)); + PADDLE_ENFORCE_LT( + axis2_, x_dims.size(), + platform::errors::OutOfRange( + "Attr(axis2) is out of range (expected to be in range of [%ld, " + "%ld], but got %ld).", + -(x_dims.size()), (x_dims.size() - 1), axis2)); + PADDLE_ENFORCE_NE(axis1_, axis2_, + platform::errors::InvalidArgument( + "The dimensions should not be identical " + "%d vs %d.", + axis1, axis2)); + + auto out_dims = vectorize(x_dims); + // from out_dims get the dim size of axis1_. + auto axis1_size = out_dims[axis1_]; + auto axis2_size = out_dims[axis2_]; + // delete two dims by attr axis1 and axis2 from out_dims. + /* example: + out_dim = [2, 3, 4]; + axis1 = 0; + axis2 = 1; + according to the attr of axis1 and axis2, we get: + out_dim = [4]. + */ + out_dims.erase(out_dims.begin() + std::max(axis1_, axis2_)); + out_dims.erase(out_dims.begin() + std::min(axis1_, axis2_)); + + if (offset_ == 0) { + out_dims.push_back(std::min(axis1_size, axis2_size)); + } else if (offset_ > 0) { + if ((axis2_size - offset_) > 0) { + out_dims.push_back(std::min(axis1_size, axis2_size - offset_)); + } else { + out_dims.push_back(0); + } + } else { + if ((axis1_size + offset_) > 0) { + out_dims.push_back(std::min(axis1_size + offset_, axis2_size)); + } else { + out_dims.push_back(0); + } + } + ctx->SetOutputDim("Out", framework::make_ddim(out_dims)); + } +}; + +class DiagonalOpMaker : public framework::OpProtoAndCheckerMaker { + public: + void Make() override { + AddInput("Input", + "(Tensor) The input tensor, from which the diagonals are taken."); + AddOutput( + "Out", + "(Tensor) The partial view of input with the its diagonal elements."); + AddAttr( + "offset", + R"DOC((int, default 0), offset of the diagonal from the main diagonal. Can be both positive and negative. Default: 0. + )DOC") + .SetDefault(0); + AddAttr( + "axis1", + R"DOC((int, default 0), the first axis of the 2-D planes from which the diagonals should be taken. + Can be either positive or negative. Default: 0. + )DOC") + .SetDefault(0); + AddAttr( + "axis2", + R"DOC((int, default 1), the second axis of the 2-D planes from which the diagonals should be taken. + Can be either positive or negative. Default: 1. + )DOC") + .SetDefault(1); + AddComment(R"DOC( +Diagonal Operator. +Return a partial view of input with the its diagonal elements of the input tensor. +The behavior of this operator is similar to how `numpy.diagonal` works. + +)DOC"); + } +}; + +class DiagonalGradOp : public framework::OperatorWithKernel { + public: + using framework::OperatorWithKernel::OperatorWithKernel; + + void InferShape(framework::InferShapeContext *ctx) const override { + OP_INOUT_CHECK(ctx->HasInput("Input"), "Input", "Input", "DiagonalGrad"); + OP_INOUT_CHECK(ctx->HasOutput(framework::GradVarName("Input")), "Output", + framework::GradVarName("Input"), "DiagonalGrad"); + + ctx->SetOutputDim(framework::GradVarName("Input"), + ctx->GetInputDim("Input")); + } + + protected: + framework::OpKernelType GetExpectedKernelType( + const framework::ExecutionContext &ctx) const override { + return framework::OpKernelType(OperatorWithKernel::IndicateVarDataType( + ctx, framework::GradVarName("Out")), + ctx.GetPlace()); + } +}; + +template +class DiagonalGradOpMaker : public framework::SingleGradOpMaker { + public: + using framework::SingleGradOpMaker::SingleGradOpMaker; + + protected: + void Apply(GradOpPtr grad_op) const override { + grad_op->SetType("diagonal_grad"); + grad_op->SetInput("Input", this->Input("Input")); + grad_op->SetInput(framework::GradVarName("Out"), this->OutputGrad("Out")); + grad_op->SetOutput(framework::GradVarName("Input"), + this->InputGrad("Input")); + grad_op->SetAttrMap(this->Attrs()); + } +}; + +DECLARE_NO_NEED_BUFFER_VARS_INFERER(DiagonalGradNoNeedBufferVarsInferer, + "Input"); + +} // namespace operators +} // namespace paddle + +namespace ops = paddle::operators; +REGISTER_OPERATOR(diagonal, ops::DiagonalOp, ops::DiagonalOpMaker, + ops::DiagonalGradOpMaker, + ops::DiagonalGradOpMaker); + +REGISTER_OPERATOR(diagonal_grad, ops::DiagonalGradOp, + ops::DiagonalGradNoNeedBufferVarsInferer) + +REGISTER_OP_CPU_KERNEL(diagonal, ops::DiagonalKernel, + ops::DiagonalKernel, ops::DiagonalKernel, + ops::DiagonalKernel, ops::DiagonalKernel); + +REGISTER_OP_CPU_KERNEL(diagonal_grad, ops::DiagonalGradKernel, + ops::DiagonalGradKernel, + ops::DiagonalGradKernel, + ops::DiagonalGradKernel); diff --git a/paddle/fluid/operators/diagonal_op.cu b/paddle/fluid/operators/diagonal_op.cu new file mode 100644 index 00000000000000..e2b5f24d6619e1 --- /dev/null +++ b/paddle/fluid/operators/diagonal_op.cu @@ -0,0 +1,273 @@ +/* Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. */ + +#include "paddle/fluid/framework/op_registry.h" +#include "paddle/fluid/operators/diagonal_op.h" +#include "paddle/fluid/platform/cuda_primitives.h" + +namespace paddle { +namespace operators { + +using platform::PADDLE_CUDA_NUM_THREADS; + +template +__global__ void Diagonal(const T* data1, T* data2, const int64_t offset_, + int64_t axis1_, int64_t axis2_, int64_t* x_stride, + int64_t* out_stride, int64_t numel, bool is_grad) { + CUDA_KERNEL_LOOP(idx, numel) { + int64_t idx_dim[X_DIM_SIZE] = {0}; + int64_t temp = 0; + for (size_t i = 0; i < X_DIM_SIZE - 1; i++) { + idx_dim[i] = (idx - temp) / x_stride[i]; + temp = temp + idx_dim[i] * x_stride[i]; + } + idx_dim[X_DIM_SIZE - 1] = idx - temp; + + int64_t axis1_dim = idx_dim[axis1_]; + int64_t axis2_dim = idx_dim[axis2_]; + + int64_t out_dim[OUT_DIM_SIZE] = {0}; + int temp_pos = 0; + for (int i = 0; i < X_DIM_SIZE; i++) { + if (i != axis1_ && i != axis2_) { + out_dim[temp_pos] = idx_dim[i]; + temp_pos++; + } + } + bool flag = false; + if (offset_ == 0 && axis1_dim == axis2_dim) { + out_dim[temp_pos] = axis1_dim; + flag = true; + } else if (offset_ > 0 && (axis1_dim + offset_) == axis2_dim) { + out_dim[temp_pos] = axis1_dim; + flag = true; + } else if (offset_ < 0 && (axis1_dim + offset_) == axis2_dim) { + out_dim[temp_pos] = axis2_dim; + flag = true; + } + if (!is_grad) { + if (flag) { + int64_t idx_output = 0; + for (size_t i = 0; i < OUT_DIM_SIZE - 1; i++) { + idx_output = idx_output + out_dim[i] * out_stride[i]; + } + idx_output = idx_output + out_dim[OUT_DIM_SIZE - 1]; + data2[idx_output] = data1[idx]; + } + } else { + if (flag) { + int64_t idx_output = 0; + for (size_t i = 0; i < OUT_DIM_SIZE - 1; i++) { + idx_output = idx_output + out_dim[i] * out_stride[i]; + } + idx_output = idx_output + out_dim[OUT_DIM_SIZE - 1]; + data2[idx] = data1[idx_output]; + } else { + data2[idx] = static_cast(0); + } + } + } +} + +template +class DiagonalCUDAKernel : public framework::OpKernel { + public: + void Compute(const framework::ExecutionContext& context) const override { + auto* input = context.Input("Input"); + const auto* input_data = input->data(); + auto input_dim = input->dims().Get(); + auto input_dim_size = input->dims().size(); + + std::vector res_in = vectorize(framework::stride(input->dims())); + paddle::framework::Tensor input_stride_tensor; + framework::TensorFromVector(res_in, context.device_context(), + &input_stride_tensor); + int64_t* input_stride = input_stride_tensor.data(); + + auto* output = context.Output("Out"); + auto* output_data = output->mutable_data(context.GetPlace()); + auto output_dim = output->dims().Get(); + auto output_dim_size = output->dims().size(); + + std::vector res_out = vectorize(framework::stride(output->dims())); + paddle::framework::Tensor output_stride_tensor; + framework::TensorFromVector(res_out, context.device_context(), + &output_stride_tensor); + int64_t* output_stride = output_stride_tensor.data(); + + const int64_t offset_ = context.Attr("offset"); + const int64_t axis1 = context.Attr("axis1"); + int64_t axis1_ = axis1 < 0 ? input_dim_size + axis1 : axis1; + const int64_t axis2 = context.Attr("axis2"); + int64_t axis2_ = axis2 < 0 ? input_dim_size + axis2 : axis2; + int64_t numel = input->numel(); + + int threads = PADDLE_CUDA_NUM_THREADS; + int blocks = (numel + threads - 1) / threads; + + switch (input_dim_size) { + case 2: + Diagonal<<>>(input_data, output_data, offset_, + axis1_, axis2_, input_stride, + output_stride, numel, false); + break; + case 3: + Diagonal<<>>(input_data, output_data, offset_, + axis1_, axis2_, input_stride, + output_stride, numel, false); + break; + case 4: + Diagonal<<>>(input_data, output_data, offset_, + axis1_, axis2_, input_stride, + output_stride, numel, false); + break; + case 5: + Diagonal<<>>(input_data, output_data, offset_, + axis1_, axis2_, input_stride, + output_stride, numel, false); + break; + case 6: + Diagonal<<>>(input_data, output_data, offset_, + axis1_, axis2_, input_stride, + output_stride, numel, false); + break; + case 7: + Diagonal<<>>(input_data, output_data, offset_, + axis1_, axis2_, input_stride, + output_stride, numel, false); + break; + case 8: + Diagonal<<>>(input_data, output_data, offset_, + axis1_, axis2_, input_stride, + output_stride, numel, false); + break; + case 9: + Diagonal<<>>(input_data, output_data, offset_, + axis1_, axis2_, input_stride, + output_stride, numel, false); + break; + default: + PADDLE_THROW(platform::errors::InvalidArgument( + "The rank of input should be less than 10, but received %d.", + input_dim_size)); + } + } +}; + +template +class DiagonalGradCUDAKernel : public framework::OpKernel { + public: + void Compute(const framework::ExecutionContext& context) const override { + const auto* dout = + context.Input(framework::GradVarName("Out")); + const auto* dout_data = dout->data(); + auto dout_dim = dout->dims().Get(); + auto dout_dim_size = dout->dims().size(); + + std::vector res_dout = vectorize(framework::stride(dout->dims())); + paddle::framework::Tensor dout_stride_tensor; + framework::TensorFromVector(res_dout, context.device_context(), + &dout_stride_tensor); + int64_t* dout_stride = dout_stride_tensor.data(); + + auto* dx = + context.Output(framework::GradVarName("Input")); + auto* dx_data = dx->mutable_data(context.GetPlace()); + auto dx_dim = dx->dims().Get(); + auto dx_dim_size = dx->dims().size(); + + std::vector res_dx = vectorize(framework::stride(dx->dims())); + paddle::framework::Tensor dx_stride_tensor; + framework::TensorFromVector(res_dx, context.device_context(), + &dx_stride_tensor); + int64_t* dx_stride = dx_stride_tensor.data(); + + const int64_t offset_ = context.Attr("offset"); + const int64_t axis1 = context.Attr("axis1"); + int64_t axis1_ = axis1 < 0 ? dx_dim_size + axis1 : axis1; + const int64_t axis2 = context.Attr("axis2"); + int64_t axis2_ = axis2 < 0 ? dx_dim_size + axis2 : axis2; + + int64_t numel = dx->numel(); + + int threads = PADDLE_CUDA_NUM_THREADS; + int blocks = (numel + threads - 1) / threads; + + switch (dx_dim_size) { + case 2: + Diagonal<<>>(dout_data, dx_data, offset_, + axis1_, axis2_, dx_stride, + dout_stride, numel, true); + break; + case 3: + Diagonal<<>>(dout_data, dx_data, offset_, + axis1_, axis2_, dx_stride, + dout_stride, numel, true); + break; + case 4: + Diagonal<<>>(dout_data, dx_data, offset_, + axis1_, axis2_, dx_stride, + dout_stride, numel, true); + break; + case 5: + Diagonal<<>>(dout_data, dx_data, offset_, + axis1_, axis2_, dx_stride, + dout_stride, numel, true); + break; + case 6: + Diagonal<<>>(dout_data, dx_data, offset_, + axis1_, axis2_, dx_stride, + dout_stride, numel, true); + break; + case 7: + Diagonal<<>>(dout_data, dx_data, offset_, + axis1_, axis2_, dx_stride, + dout_stride, numel, true); + break; + case 8: + Diagonal<<>>(dout_data, dx_data, offset_, + axis1_, axis2_, dx_stride, + dout_stride, numel, true); + break; + case 9: + Diagonal<<>>(dout_data, dx_data, offset_, + axis1_, axis2_, dx_stride, + dout_stride, numel, true); + break; + default: + PADDLE_THROW(platform::errors::InvalidArgument( + "The rank of output(input@Grad) should be less than 10, but " + "received %d.", + dx_dim_size)); + } + } +}; + +} // namespace operators +} // namespace paddle + +namespace ops = paddle::operators; +namespace plat = paddle::platform; +REGISTER_OP_CUDA_KERNEL(diagonal, ops::DiagonalCUDAKernel, + ops::DiagonalCUDAKernel, + ops::DiagonalCUDAKernel, + ops::DiagonalCUDAKernel, + ops::DiagonalCUDAKernel, + ops::DiagonalCUDAKernel); + +REGISTER_OP_CUDA_KERNEL(diagonal_grad, ops::DiagonalGradCUDAKernel, + ops::DiagonalGradCUDAKernel, + ops::DiagonalGradCUDAKernel, + ops::DiagonalGradCUDAKernel, + ops::DiagonalGradCUDAKernel); diff --git a/paddle/fluid/operators/diagonal_op.h b/paddle/fluid/operators/diagonal_op.h new file mode 100644 index 00000000000000..a0380e9e52cace --- /dev/null +++ b/paddle/fluid/operators/diagonal_op.h @@ -0,0 +1,163 @@ +// Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#pragma once +#include +#include +#include "paddle/fluid/framework/op_registry.h" + +namespace paddle { +namespace operators { +template + +std::vector ComputeDimStride(const std::vector dim) { + size_t dim_size = dim.size(); + std::vector dim_strides; + dim_strides.resize(dim_size); + for (size_t i = 0; i < dim_size - 1; i++) { + size_t temp_stride = 1; + for (size_t j = i + 1; j < dim_size; j++) { + temp_stride = temp_stride * dim[j]; + } + dim_strides[i] = temp_stride; + } + dim_strides[dim_size - 1] = 1; + return dim_strides; +} +template +class DiagonalKernel : public framework::OpKernel { + public: + void Compute(const framework::ExecutionContext& context) const override { + auto* input = context.Input("Input"); + const T* input_data = input->data(); + auto input_dim = vectorize(input->dims()); + auto input_dim_size = input_dim.size(); + + auto* output = context.Output("Out"); + T* output_data = output->mutable_data(context.GetPlace()); + auto output_dim = vectorize(output->dims()); + + const int64_t offset_ = context.Attr("offset"); + const int64_t axis1 = context.Attr("axis1"); + int64_t axis1_ = axis1 < 0 ? input_dim_size + axis1 : axis1; + const int64_t axis2 = context.Attr("axis2"); + int64_t axis2_ = axis2 < 0 ? input_dim_size + axis2 : axis2; + + std::vector input_stride = ComputeDimStride(input_dim); + std::vector output_stride = ComputeDimStride(output_dim); + + int64_t numel = input->numel(); + + for (int64_t idx = 0; idx < numel; idx++) { + std::vector idx_dim(input_dim_size); + int64_t temp = 0; + for (size_t i = 0; i < input_dim_size; i++) { + idx_dim[i] = (idx - temp) / input_stride[i]; + temp = temp + idx_dim[i] * input_stride[i]; + } + + int64_t axis1_dim = idx_dim[axis1_]; + int64_t axis2_dim = idx_dim[axis2_]; + + idx_dim.erase(idx_dim.begin() + std::max(axis1_, axis2_)); + idx_dim.erase(idx_dim.begin() + std::min(axis1_, axis2_)); + + bool flag = false; + if (offset_ == 0 && axis1_dim == axis2_dim) { + idx_dim.push_back(axis1_dim); + flag = true; + } else if (offset_ > 0 && (axis1_dim + offset_) == axis2_dim) { + idx_dim.push_back(axis1_dim); + flag = true; + } else if (offset_ < 0 && (axis1_dim + offset_) == axis2_dim) { + idx_dim.push_back(axis2_dim); + flag = true; + } + if (flag) { + int64_t idx_output = 0; + for (size_t i = 0; i < idx_dim.size(); i++) { + idx_output = idx_output + idx_dim[i] * output_stride[i]; + } + output_data[idx_output] = input_data[idx]; + } + } + } +}; + +template +class DiagonalGradKernel : public framework::OpKernel { + public: + void Compute(const framework::ExecutionContext& context) const override { + const auto* dout = + context.Input(framework::GradVarName("Out")); + const T* dout_data = dout->data(); + auto dout_dim = vectorize(dout->dims()); + + auto* dx = + context.Output(framework::GradVarName("Input")); + T* dx_data = dx->mutable_data(context.GetPlace()); + auto dx_dim = vectorize(dx->dims()); + auto dx_dim_size = dx_dim.size(); + + const int64_t offset_ = context.Attr("offset"); + const int64_t axis1 = context.Attr("axis1"); + int64_t axis1_ = axis1 < 0 ? dx_dim_size + axis1 : axis1; + const int64_t axis2 = context.Attr("axis2"); + int64_t axis2_ = axis2 < 0 ? dx_dim_size + axis2 : axis2; + + std::vector dout_stride = ComputeDimStride(dout_dim); + std::vector dx_stride = ComputeDimStride(dx_dim); + + int64_t numel = dx->numel(); + + for (int64_t idx = 0; idx < numel; idx++) { + std::vector idx_dim(dx_dim_size); + int64_t temp = 0; + for (size_t i = 0; i < dx_dim_size; i++) { + idx_dim[i] = (idx - temp) / dx_stride[i]; + temp = temp + idx_dim[i] * dx_stride[i]; + } + + int64_t axis1_dim = idx_dim[axis1_]; + int64_t axis2_dim = idx_dim[axis2_]; + + idx_dim.erase(idx_dim.begin() + std::max(axis1_, axis2_)); + idx_dim.erase(idx_dim.begin() + std::min(axis1_, axis2_)); + + bool flag = false; + if (offset_ == 0 && axis1_dim == axis2_dim) { + idx_dim.push_back(axis1_dim); + flag = true; + } else if (offset_ > 0 && (axis1_dim + offset_) == axis2_dim) { + idx_dim.push_back(axis1_dim); + flag = true; + } else if (offset_ < 0 && (axis1_dim + offset_) == axis2_dim) { + idx_dim.push_back(axis2_dim); + flag = true; + } + if (flag) { + int64_t idx_output = 0; + for (size_t i = 0; i < idx_dim.size(); i++) { + idx_output = idx_output + idx_dim[i] * dout_stride[i]; + } + dx_data[idx] = dout_data[idx_output]; + } else { + dx_data[idx] = static_cast(0); + } + } + } +}; + +} // namespace operators +} // namespace paddle diff --git a/paddle/fluid/operators/elementwise/elementwise_mod_op.cu b/paddle/fluid/operators/elementwise/elementwise_mod_op.cu index 92991ab3a0a24c..bb49fdbf12dfa3 100644 --- a/paddle/fluid/operators/elementwise/elementwise_mod_op.cu +++ b/paddle/fluid/operators/elementwise/elementwise_mod_op.cu @@ -12,13 +12,60 @@ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */ #include "paddle/fluid/operators/elementwise/elementwise_mod_op.h" +#include "paddle/fluid/operators/elementwise/elementwise_op_broadcast.cu.h" #include "paddle/fluid/platform/float16.h" namespace ops = paddle::operators; namespace plat = paddle::platform; +namespace paddle { +namespace operators { + +template +struct CudaModFunctor { + inline HOSTDEVICE T operator()(const T* args) const { + T res = args[0] % args[1]; + + // Accoding to #PR26732: in dividen % divsor + // remainder shall have the same sign as divsor. + if ((res != 0) && ((args[1] ^ res) < 0)) res += args[1]; + return res; + } +}; + +template +struct CudaModFunctor< + T, typename std::enable_if_t::value>> { + inline HOSTDEVICE T operator()(const T* args) const { + T res = fmod(args[0], args[1]); + + // Accoding to #PR26732: in dividen % divsor + // remainder shall have the same sign as divsor. + if ((res != 0) && ((res < 0) != (args[1] < 0))) res += args[1]; + return res; + } +}; + +template +class ElementwiseModKernel + : public framework::OpKernel { + public: + void Compute(const framework::ExecutionContext& ctx) const override { + std::vector ins; + std::vector outs; + const auto& cuda_ctx = + ctx.template device_context(); + int axis = PackTensorsIntoVector(ctx, &ins, &outs); + LaunchElementwiseCudaKernel( + cuda_ctx, ins, &outs, axis, CudaModFunctor()); + } +}; + +} // namespace operators +} // namespace paddle + REGISTER_OP_CUDA_KERNEL( elementwise_mod, ops::ElementwiseModKernel, ops::ElementwiseModKernel, - ops::ElementwiseModFPKernel, - ops::ElementwiseModFPKernel); + ops::ElementwiseModKernel, + ops::ElementwiseModKernel); diff --git a/paddle/fluid/operators/elementwise/elementwise_mod_op.h b/paddle/fluid/operators/elementwise/elementwise_mod_op.h index 87e940e2ed6319..03884f2a45883b 100644 --- a/paddle/fluid/operators/elementwise/elementwise_mod_op.h +++ b/paddle/fluid/operators/elementwise/elementwise_mod_op.h @@ -16,7 +16,6 @@ limitations under the License. */ #include "paddle/fluid/framework/eigen.h" #include "paddle/fluid/operators/elementwise/elementwise_op.h" -#include "paddle/fluid/operators/elementwise/elementwise_op_function.h" #include "paddle/fluid/operators/math/blas.h" namespace paddle { diff --git a/paddle/fluid/operators/elementwise/mkldnn/elementwise_mkldnn_op.h b/paddle/fluid/operators/elementwise/mkldnn/elementwise_mkldnn_op.h index e5d20893335f70..ddad70a6a5f31c 100644 --- a/paddle/fluid/operators/elementwise/mkldnn/elementwise_mkldnn_op.h +++ b/paddle/fluid/operators/elementwise/mkldnn/elementwise_mkldnn_op.h @@ -47,23 +47,13 @@ class EltwiseMKLDNNKernel : public framework::OpKernel { float scale_o = ctx.Attr("Scale_out"); int axis = ctx.Attr("axis"); - bool is_inplaced = x->IsSharedBufferWith(*z); - - std::string key = is_inplaced - ? platform::CreateKey(dev_ctx, ctx.OutputName("Out"), - x->format(), y->format()) - : ctx.OutputName("Out"); - platform::BinaryMKLDNNHandler handler( BINARY_OP, axis, dev_ctx, mkldnn_engine, ctx.GetPlace(), x, y, z, - scale_x, scale_y, scale_o, key); + scale_x, scale_y, scale_o, ctx.OutputName("Out")); const auto src_x_memory = handler.AcquireSrcMemory(x); const auto src_y_memory = handler.AcquireSecondSrcMemory(y); - - // For Inplace src and and dst are the same memory object - const auto dst_memory = - is_inplaced ? src_x_memory : handler.AcquireDstMemory(z); + const auto dst_memory = handler.AcquireDstMemory(z); const auto binary_prim = handler.AcquireForwardPrimitive(); diff --git a/paddle/fluid/operators/gelu_op_npu.cc b/paddle/fluid/operators/gelu_op_npu.cc index 6e60926cc7951a..4db82e96cfae7c 100644 --- a/paddle/fluid/operators/gelu_op_npu.cc +++ b/paddle/fluid/operators/gelu_op_npu.cc @@ -61,13 +61,14 @@ class GeluGradNPUKernel : public framework::OpKernel { ctx.template device_context() .stream(); - Tensor out(x->type()); - out.mutable_data(x->dims(), place); - const auto& runner_out = NpuOpRunner("Gelu", {*x}, {out}, {}); - runner_out.Run(stream); - + // NOTE(pangyoki): In the original implementation of GeluGrad op, the input + // is {*dout, *x, out}, where out = Gelu(x). However, we find that variable + // `out` was not actually used. In order to improve performance, the + // useless GELU operation was deleted. + // We directly use `*dout` as a placeholder to replace `out`, it will not + // be used in calculations. const auto& runner_dx = - NpuOpRunner("GeluGrad", {*dout, *x, out}, {*dx}, {}); + NpuOpRunner("GeluGrad", {*dout, *x, *dout}, {*dx}, {}); runner_dx.Run(stream); } }; diff --git a/paddle/fluid/operators/layer_norm_op.cu b/paddle/fluid/operators/layer_norm_op.cu old mode 100755 new mode 100644 index f955011675cf5d..6cd6a524e281db --- a/paddle/fluid/operators/layer_norm_op.cu +++ b/paddle/fluid/operators/layer_norm_op.cu @@ -64,17 +64,16 @@ static __forceinline__ __device__ U WarpReduceSum(U val) { } template -__forceinline__ __device__ U BlockReduceSum(U val) { - static __shared__ U shared[32]; +__forceinline__ __device__ U BlockReduceSum(U val, U *shared) { int lane = threadIdx.x % warpSize; int wid = threadIdx.x / warpSize; val = WarpReduceSum(val); // Each warp performs partial reduction + __syncthreads(); if (lane == 0) shared[wid] = val; // Write reduced value to shared memory __syncthreads(); // Wait for all partial reductions - // read from shared memory only if that warp existed val = (threadIdx.x < blockDim.x / warpSize) ? shared[lane] : static_cast(0); @@ -183,6 +182,9 @@ __global__ void LayerNormForward(const T *x, const U *scale, const U *bias, int64_t feature_size) { __shared__ U mean_share; __shared__ U var_share; + __shared__ U shared_mean[32]; // threadIdx.x / warpSize <= kMaxBlockDim / + // warpSize <= 1024/32 = 32; + __shared__ U shared_var[32]; int64_t beg_idx = blockIdx.x * feature_size + threadIdx.x; int64_t end_idx = (blockIdx.x + 1) * feature_size; @@ -196,8 +198,8 @@ __global__ void LayerNormForward(const T *x, const U *scale, const U *bias, var_val += (tmp * tmp); } - mean_val = BlockReduceSum(mean_val); - var_val = BlockReduceSum(var_val); + mean_val = BlockReduceSum(mean_val, shared_mean); + var_val = BlockReduceSum(var_val, shared_var); if (threadIdx.x == 0) { auto scale = static_cast(1.) / static_cast(feature_size); @@ -398,9 +400,9 @@ __global__ void LayerNormBackwardComputeGradInput( const U *__restrict__ mean, const U *__restrict__ var, const float epsilon, const U *gamma, T *grad_input) { #ifdef __HIPCC__ - for (auto i1 = hipBlockIdx_y; i1 < n1; i1 += hipGridDim_y) { + for (auto i1 = hipBlockIdx_x; i1 < n1; i1 += hipGridDim_x) { #else - for (auto i1 = blockIdx.y; i1 < n1; i1 += gridDim.y) { + for (auto i1 = blockIdx.x; i1 < n1; i1 += gridDim.x) { #endif U sum_loss1 = U(0); U sum_loss2 = U(0); @@ -541,8 +543,11 @@ __global__ void LayerNormBackwardGradientAll( } } - d_scale_partial = BlockReduceSum(d_scale_partial); - d_bias_partial = BlockReduceSum(d_bias_partial); + __shared__ U shared_scale[32]; // threadIdx.x / warpSize <= kMaxBlockDim / + // warpSize <= 1024/32 = 32; + __shared__ U shared_bias[32]; + d_scale_partial = BlockReduceSum(d_scale_partial, shared_scale); + d_bias_partial = BlockReduceSum(d_bias_partial, shared_bias); if (threadIdx.x == 0) { d_scale[blockIdx.x + col_offset] = d_scale_partial; @@ -864,9 +869,8 @@ static void LayerNormBackward(const T *x, const T *d_y, const U *scale, constexpr int BDIMX1 = 32; constexpr int BDIMY1 = 4; dim3 threads1(BDIMX1, BDIMY1, 1); - const dim3 blocks1(1, batch_size, 1); LayerNormBackwardComputeGradInput< - T, U, BDIMX1, BDIMY1><<>>( + T, U, BDIMX1, BDIMY1><<>>( d_y, x, batch_size, feature_size, mean, var, epsilon, scale, d_x); break; } diff --git a/paddle/fluid/operators/math/math_cuda_utils.h b/paddle/fluid/operators/math/math_cuda_utils.h index e97dbd20ca142a..8de4e8221c0e47 100644 --- a/paddle/fluid/operators/math/math_cuda_utils.h +++ b/paddle/fluid/operators/math/math_cuda_utils.h @@ -188,6 +188,7 @@ __inline__ __device__ T blockReduceSum(T val, unsigned mask) { val = warpReduceSum(val, mask); + __syncthreads(); if (lane == 0) shared[wid] = val; __syncthreads(); diff --git a/paddle/fluid/operators/matmul_op_xpu.cc b/paddle/fluid/operators/matmul_op_xpu.cc index 6fa96aca4be147..7097b5327d86fa 100644 --- a/paddle/fluid/operators/matmul_op_xpu.cc +++ b/paddle/fluid/operators/matmul_op_xpu.cc @@ -102,6 +102,7 @@ template static void MatMulXPUFunction(const Tensor *x, const Tensor *y, Tensor *out, bool trans_x, bool trans_y, const paddle::framework::ExecutionContext &ctx) { + using XPUType = typename XPUTypeTrait::Type; const auto &x_dims = x->dims(); const auto &y_dims = y->dims(); auto &dev_ctx = @@ -162,34 +163,36 @@ static void MatMulXPUFunction(const Tensor *x, const Tensor *y, Tensor *out, int ldout = n; if (batch_size <= 1) { int r = 0; - r = xpu::fc_fusion( - dev_ctx.x_context(), x->data(), y->data(), data_c, m, n, k, - mat_dim_a.trans_, mat_dim_b.trans_, nullptr, nullptr, nullptr, ldx, ldy, - ldout, alpha, 0, nullptr, xpu::Activation_t::LINEAR); + r = xpu::fc_fusion( + dev_ctx.x_context(), reinterpret_cast(x->data()), + reinterpret_cast(y->data()), + reinterpret_cast(data_c), m, n, k, mat_dim_a.trans_, + mat_dim_b.trans_, nullptr, nullptr, nullptr, ldx, ldy, ldout, alpha, 0, + nullptr, xpu::Activation_t::LINEAR); PADDLE_ENFORCE_EQ(r, XPU_SUCCESS, platform::errors::External( "XPU fc_fusion kernel return wrong value[%d %s]", r, XPUAPIErrorMsg[r])); } else { // batch matmul - int r = xpu::fc_batched( - dev_ctx.x_context(), // Context* ctx, - batch_size, // int batch_size, - mat_dim_a.trans_, // bool x_trans, - mat_dim_b.trans_, // bool w_trans, - m, // int m, - n, // int n, - k, // int k, - alpha, // float alpha, - reinterpret_cast(x->data()), // const TX* x, - mat_dim_a.stride_, // int stride_a, - reinterpret_cast(y->data()), // const TW* w, - mat_dim_b.stride_, // int stride_b, - 0.0, // float beta, - reinterpret_cast(data_c), // TY* y, - m * n, // int stride_c, - nullptr, // const float* x_maxptr, - nullptr); // const float* w_maxptr + int r = xpu::fc_batched( + dev_ctx.x_context(), // Context* ctx, + batch_size, // int batch_size, + mat_dim_a.trans_, // bool x_trans, + mat_dim_b.trans_, // bool w_trans, + m, // int m, + n, // int n, + k, // int k, + alpha, // float alpha, + reinterpret_cast(x->data()), // const TX* x, + mat_dim_a.stride_, // int stride_a, + reinterpret_cast(y->data()), // const TW* w, + mat_dim_b.stride_, // int stride_b, + 0.0, // float beta, + reinterpret_cast(data_c), // TY* y, + m * n, // int stride_c, + nullptr, // const float* x_maxptr, + nullptr); // const float* w_maxptr PADDLE_ENFORCE_EQ(r, XPU_SUCCESS, platform::errors::External( @@ -210,10 +213,14 @@ class MatMulXPUKernel : public framework::OpKernel { out->mutable_data(context.GetPlace()); bool trans_x = context.Attr("transpose_X"); bool trans_y = context.Attr("transpose_Y"); - if (std::getenv("XPU_PADDLE_MAT_MUL_FCINT32") != nullptr) { - MatMulXPUFunction(x, y, out, trans_x, trans_y, context); - } else { + if (std::is_same::value) { MatMulXPUFunction(x, y, out, trans_x, trans_y, context); + } else { + if (std::getenv("XPU_PADDLE_MAT_MUL_FCINT32") != nullptr) { + MatMulXPUFunction(x, y, out, trans_x, trans_y, context); + } else { + MatMulXPUFunction(x, y, out, trans_x, trans_y, context); + } } } }; @@ -224,6 +231,7 @@ class MatMulXPUKernel : public framework::OpKernel { template static framework::Tensor XPUFoldHeadAndLastDims( const DeviceContext &context, const framework::Tensor &input) { + using XPUType = typename XPUTypeTrait::Type; auto in_dims = input.dims(); if (in_dims.size() != 3) { return input; @@ -236,8 +244,9 @@ static framework::Tensor XPUFoldHeadAndLastDims( static_cast(in_dims[1]), static_cast(in_dims[2])}; std::vector axis_host = {1, 0, 2}; - int r = xpu::transpose(context.x_context(), input.data(), output.data(), - in_shape_host, axis_host); + int r = xpu::transpose( + context.x_context(), reinterpret_cast(input.data()), + reinterpret_cast(output.data()), in_shape_host, axis_host); PADDLE_ENFORCE_EQ(r, XPU_SUCCESS, platform::errors::External( "XPU transpose kernel return wrong value[%d %s]", r, @@ -280,10 +289,14 @@ class MatMulGradXPUKernel : public framework::OpKernel { const framework::Tensor &b, bool trans_b, framework::Tensor *out) const { out->mutable_data(context.GetPlace()); - if (std::getenv("XPU_PADDLE_MAT_MUL_GRAD_FCINT32") != nullptr) { - MatMulXPUFunction(&a, &b, out, trans_a, trans_b, context); - } else { + if (std::is_same::value) { MatMulXPUFunction(&a, &b, out, trans_a, trans_b, context); + } else { + if (std::getenv("XPU_PADDLE_MAT_MUL_GRAD_FCINT32") != nullptr) { + MatMulXPUFunction(&a, &b, out, trans_a, trans_b, context); + } else { + MatMulXPUFunction(&a, &b, out, trans_a, trans_b, context); + } } } @@ -370,10 +383,14 @@ class MatMulGradXPUKernel : public framework::OpKernel { } // namespace paddle namespace ops = paddle::operators; +namespace plat = paddle::platform; REGISTER_OP_XPU_KERNEL( - matmul, ops::MatMulXPUKernel); + matmul, ops::MatMulXPUKernel, + ops::MatMulXPUKernel); REGISTER_OP_XPU_KERNEL( matmul_grad, - ops::MatMulGradXPUKernel); + ops::MatMulGradXPUKernel, + ops::MatMulGradXPUKernel); #endif diff --git a/paddle/fluid/operators/matmul_v2_op.cc b/paddle/fluid/operators/matmul_v2_op.cc index 82706fd4875230..8ac81596a36d3f 100644 --- a/paddle/fluid/operators/matmul_v2_op.cc +++ b/paddle/fluid/operators/matmul_v2_op.cc @@ -85,9 +85,17 @@ class MatMulV2Op : public framework::OperatorWithKernel { protected: framework::OpKernelType GetExpectedKernelType( const framework::ExecutionContext& ctx) const override { - auto data_type = + auto input_data_type = OperatorWithKernel::IndicateOrPromoteVarDataTypes(ctx, "X", "Y"); - return framework::OpKernelType(data_type, ctx.device_context()); + +#ifdef PADDLE_WITH_MKLDNN + if (this->CanMKLDNNBeUsed(ctx, input_data_type)) { + return framework::OpKernelType(input_data_type, ctx.GetPlace(), + framework::DataLayout::kMKLDNN, + framework::LibraryType::kMKLDNN); + } +#endif + return framework::OpKernelType(input_data_type, ctx.GetPlace()); } framework::OpKernelType GetKernelTypeForVar( @@ -118,6 +126,14 @@ class MatMulV2OpMaker : public framework::OpProtoAndCheckerMaker { "Set true to transpose the last two dimensions of Y before " "doing multiplication") .SetDefault(false); + AddAttr("use_mkldnn", + "(bool, default false) Only used in mkldnn kernel") + .SetDefault(false); + AddAttr( + "mkldnn_data_type", + "(string, default \"float32\"). Data type of mkldnn kernel") + .SetDefault("float32") + .InEnum({"float32", "bfloat16"}); AddComment( R"DOC(Matrix multiplication Out = X * Y. A has shape (d0, d1 ... M, K), B has shape (d0, d1 ... K, N), Out has shape ((d0, d1 ... M, N)). diff --git a/paddle/fluid/operators/matmul_v2_op_npu.cc b/paddle/fluid/operators/matmul_v2_op_npu.cc index f499c24ea3206c..3d77c177500e38 100644 --- a/paddle/fluid/operators/matmul_v2_op_npu.cc +++ b/paddle/fluid/operators/matmul_v2_op_npu.cc @@ -138,10 +138,30 @@ class MatMulV2GradNPUKernel : public framework::OpKernel { } if (dy) { dy->mutable_data(ctx.GetPlace()); - const auto& runner_dy = - NpuOpRunner("BatchMatMul", {*x, *dout}, {*dy}, - {{"adj_x1", true}, {"adj_x2", false}}); - runner_dy.Run(stream); + if ((x->dims().size() == 3) && (dout->dims().size() == 3) && + (dy->dims().size() == 2)) { + framework::Tensor dout_; + dout_.ShareDataWith(*dout); + std::vector vec_dim = framework::vectorize(dout_.dims()); + std::vector vec_dim_v{vec_dim[0] * vec_dim[1], vec_dim[2]}; + dout_.Resize(framework::make_ddim(vec_dim_v)); + + framework::Tensor x_; + x_.ShareDataWith(*x); + std::vector vec_dim_x = framework::vectorize(x_.dims()); + std::vector vec_dim_x_v{vec_dim_x[0] * vec_dim_x[1], + vec_dim_x[2]}; + x_.Resize(framework::make_ddim(vec_dim_x_v)); + const auto& runner_dy = + NpuOpRunner("MatMul", {x_, dout_}, {*dy}, + {{"transpose_x1", true}, {"transpose_x2", false}}); + runner_dy.Run(stream); + } else { + const auto& runner_dy = + NpuOpRunner("BatchMatMul", {*x, *dout}, {*dy}, + {{"adj_x1", true}, {"adj_x2", false}}); + runner_dy.Run(stream); + } } } } diff --git a/paddle/fluid/operators/matmul_v2_op_xpu.cc b/paddle/fluid/operators/matmul_v2_op_xpu.cc index d992ef847db2ac..ae1e9358f68115 100644 --- a/paddle/fluid/operators/matmul_v2_op_xpu.cc +++ b/paddle/fluid/operators/matmul_v2_op_xpu.cc @@ -25,6 +25,7 @@ template static void MatMulXPUFunction(const Tensor* x, const Tensor* y, Tensor* out, bool trans_x, bool trans_y, const paddle::framework::ExecutionContext& ctx) { + using XPUType = typename XPUTypeTrait::Type; const auto& x_dims = x->dims(); const auto& y_dims = y->dims(); auto& dev_ctx = @@ -75,9 +76,11 @@ static void MatMulXPUFunction(const Tensor* x, const Tensor* y, Tensor* out, int batch_size = mat_dim_a.batch_size_; if (batch_size <= 1) { int r = 0; - r = xpu::fc(dev_ctx.x_context(), x->data(), y->data(), - data_c, m, n, k, mat_dim_a.trans_, - mat_dim_b.trans_, nullptr, nullptr, nullptr); + r = xpu::fc( + dev_ctx.x_context(), reinterpret_cast(x->data()), + reinterpret_cast(y->data()), + reinterpret_cast(data_c), m, n, k, mat_dim_a.trans_, + mat_dim_b.trans_, nullptr, nullptr, nullptr); PADDLE_ENFORCE_EQ( r, XPU_SUCCESS, platform::errors::External( @@ -87,24 +90,24 @@ static void MatMulXPUFunction(const Tensor* x, const Tensor* y, Tensor* out, r, XPUAPIErrorMsg[r], m, n, k, mat_dim_a.trans_, mat_dim_b.trans_)); } else { // batch matmul - int r = xpu::fc_batched( - dev_ctx.x_context(), // Context* ctx, - batch_size, // int batch_size, - mat_dim_a.trans_, // bool x_trans, - mat_dim_b.trans_, // bool w_trans, - m, // int m, - n, // int n, - k, // int k, - 1.0, // float alpha, - reinterpret_cast(x->data()), // const TX* x, - mat_dim_a.stride_, // int stride_a, - reinterpret_cast(y->data()), // const TW* w, - mat_dim_b.stride_, // int stride_b, - 0.0, // float beta, - reinterpret_cast(data_c), // TY* y, - m * n, // int stride_c, - nullptr, // const float* x_maxptr, - nullptr); // const float* w_maxptr + int r = xpu::fc_batched( + dev_ctx.x_context(), // Context* ctx, + batch_size, // int batch_size, + mat_dim_a.trans_, // bool x_trans, + mat_dim_b.trans_, // bool w_trans, + m, // int m, + n, // int n, + k, // int k, + 1.0, // float alpha, + reinterpret_cast(x->data()), // const TX* x, + mat_dim_a.stride_, // int stride_a, + reinterpret_cast(y->data()), // const TW* w, + mat_dim_b.stride_, // int stride_b, + 0.0, // float beta, + reinterpret_cast(data_c), // TY* y, + m * n, // int stride_c, + nullptr, // const float* x_maxptr, + nullptr); // const float* w_maxptr PADDLE_ENFORCE_EQ(r, XPU_SUCCESS, platform::errors::External( @@ -123,10 +126,14 @@ class MatMulV2XPUKernel : public framework::OpKernel { bool trans_x = ctx.Attr("trans_x"); bool trans_y = ctx.Attr("trans_y"); out->mutable_data(ctx.GetPlace()); - if (std::getenv("XPU_PADDLE_MAT_MUL_V2_FCINT32") != nullptr) { - MatMulXPUFunction(x, y, out, trans_x, trans_y, ctx); - } else { + if (std::is_same::value) { MatMulXPUFunction(x, y, out, trans_x, trans_y, ctx); + } else { + if (std::getenv("XPU_PADDLE_MAT_MUL_V2_FCINT32") != nullptr) { + MatMulXPUFunction(x, y, out, trans_x, trans_y, ctx); + } else { + MatMulXPUFunction(x, y, out, trans_x, trans_y, ctx); + } } } }; @@ -134,6 +141,7 @@ class MatMulV2XPUKernel : public framework::OpKernel { template static framework::Tensor XPUFoldHeadAndLastDims( const DeviceContext& context, const framework::Tensor& input) { + using XPUType = typename XPUTypeTrait::Type; auto in_dims = input.dims(); if (in_dims.size() != 3) { return input; @@ -147,8 +155,9 @@ static framework::Tensor XPUFoldHeadAndLastDims( static_cast(in_dims[2])}; std::vector axis_host = {1, 0, 2}; - int r = xpu::transpose(context.x_context(), input.data(), output.data(), - in_shape_host, axis_host); + int r = xpu::transpose( + context.x_context(), reinterpret_cast(input.data()), + reinterpret_cast(output.data()), in_shape_host, axis_host); PADDLE_ENFORCE_EQ(r, XPU_SUCCESS, platform::errors::External( "XPU transpose kernel return wrong value[%d %s]", r, @@ -166,10 +175,14 @@ class MatMulV2XPUGradKernel : public framework::OpKernel { const framework::Tensor& b, bool trans_b, framework::Tensor* out) const { out->mutable_data(ctx.GetPlace()); - if (std::getenv("XPU_PADDLE_MAT_MUL_GRAD_V2_FCINT32") != nullptr) { - MatMulXPUFunction(&a, &b, out, trans_a, trans_b, ctx); - } else { + if (std::is_same::value) { MatMulXPUFunction(&a, &b, out, trans_a, trans_b, ctx); + } else { + if (std::getenv("XPU_PADDLE_MAT_MUL_GRAD_V2_FCINT32") != nullptr) { + MatMulXPUFunction(&a, &b, out, trans_a, trans_b, ctx); + } else { + MatMulXPUFunction(&a, &b, out, trans_a, trans_b, ctx); + } } } @@ -261,8 +274,10 @@ class MatMulV2XPUGradKernel : public framework::OpKernel { } // namespace paddle namespace ops = paddle::operators; - -REGISTER_OP_XPU_KERNEL(matmul_v2, ops::MatMulV2XPUKernel); -REGISTER_OP_XPU_KERNEL(matmul_v2_grad, ops::MatMulV2XPUGradKernel); +namespace plat = paddle::platform; +REGISTER_OP_XPU_KERNEL(matmul_v2, ops::MatMulV2XPUKernel, + ops::MatMulV2XPUKernel); +REGISTER_OP_XPU_KERNEL(matmul_v2_grad, ops::MatMulV2XPUGradKernel, + ops::MatMulV2XPUGradKernel); #endif diff --git a/paddle/fluid/operators/mkldnn/concat_mkldnn_op.cc b/paddle/fluid/operators/mkldnn/concat_mkldnn_op.cc index df1b5af121da93..df4750321e3fce 100644 --- a/paddle/fluid/operators/mkldnn/concat_mkldnn_op.cc +++ b/paddle/fluid/operators/mkldnn/concat_mkldnn_op.cc @@ -14,6 +14,7 @@ limitations under the License. */ #include #include "paddle/fluid/operators/concat_op.h" +#include "paddle/fluid/operators/utils.h" #include "paddle/fluid/platform/mkldnn_helper.h" #include "paddle/fluid/platform/mkldnn_reuse.h" @@ -156,6 +157,17 @@ class ConcatMKLDNNOpKernel : public paddle::framework::OpKernel { "The axis is expected to be in range of [%d, %d), but got %d", -rank, rank, concat_axis)); platform::MKLDNNDeviceContext::tls().log_lib_version(); + + if (ctx.HasInput("AxisTensor")) { + auto* axis_tensor = ctx.Input("AxisTensor"); + concat_axis = GetDataFromTensor(axis_tensor)[0]; + auto out_dims = multi_input[0]->dims(); + for (size_t i = 1; i < multi_input.size(); ++i) { + out_dims[concat_axis] += multi_input[i]->dims()[concat_axis]; + } + output->Resize(out_dims); + } + if (concat_axis < 0) { concat_axis = concat_axis + rank; } diff --git a/paddle/fluid/operators/mkldnn/matmul_v2_mkldnn_op.cc b/paddle/fluid/operators/mkldnn/matmul_v2_mkldnn_op.cc new file mode 100644 index 00000000000000..50afd417170e0f --- /dev/null +++ b/paddle/fluid/operators/mkldnn/matmul_v2_mkldnn_op.cc @@ -0,0 +1,205 @@ +/* Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. */ + +#include "paddle/fluid/framework/op_registry.h" +#include "paddle/fluid/framework/tensor.h" +#include "paddle/fluid/operators/math/blas.h" +#include "paddle/fluid/platform/mkldnn_reuse.h" + +namespace paddle { +namespace operators { + +using dnnl::memory; +using dnnl::primitive; +using framework::DataLayout; +using framework::ExecutionContext; +using platform::GetMKLDNNFormat; +using platform::MKLDNNDeviceContext; +using platform::MKLDNNGetDataType; +using platform::to_void_cast; +using Tensor = framework::Tensor; + +template +class MatMulV2MKLDNNHandler : public platform::MKLDNNHandlerT { + public: + MatMulV2MKLDNNHandler(const MKLDNNDeviceContext& dev_ctx, + const mkldnn::engine engine, platform::Place cpu_place, + std::vector& x_dims, bool trans_x, + std::vector& y_dims, bool trans_y, + const std::string& uniq_name) + : platform::MKLDNNHandlerT( + dev_ctx, engine, cpu_place, + platform::CreateKey(dev_ctx, x_dims, uniq_name)) { + if (!this->isCached()) { + // M X K * K X N + const int MB_idx = x_dims.size() - 3; + const int H_idx = x_dims.size() - 2; + const int W_idx = x_dims.size() - 1; + + if (trans_x) std::swap(x_dims[H_idx], x_dims[W_idx]); + if (trans_y) std::swap(y_dims[H_idx], y_dims[W_idx]); + + const memory::dim M = x_dims[H_idx]; + const memory::dim K = x_dims[W_idx]; + const memory::dim N = y_dims[W_idx]; + + std::vector x_strides(x_dims.size() - 3, 1); + std::vector y_strides(x_dims.size() - 3, 1); + std::vector out_strides(x_dims.size() - 3, 1); + std::vector out_ddims(x_dims.size() - 3, 1); + + x_strides.reserve(x_dims.size()); + y_strides.reserve(x_dims.size()); + out_strides.reserve(x_dims.size()); + + if (!trans_x) { + x_strides.insert(x_strides.end(), {M * K, K, 1}); + } else { + x_strides.insert(x_strides.end(), {M * K, 1, M}); + } + + if (!trans_y) { + y_strides.insert(y_strides.end(), {N * K, N, 1}); + } else { + y_strides.insert(y_strides.end(), {N * K, 1, K}); + } + + out_strides.insert(out_strides.end(), {M * N, N, 1}); + out_ddims.insert(out_ddims.end(), + {std::max(x_dims[MB_idx], y_dims[MB_idx]), M, N}); + + for (int i = x_dims.size() - 4; i >= 0; --i) { + out_ddims[i] = std::max(x_dims[i], y_dims[i]); + x_strides[i] = x_dims[i + 1] * x_strides[i + 1]; + y_strides[i] = y_dims[i + 1] * y_strides[i + 1]; + out_strides[i] = out_ddims[i + 1] * out_strides[i + 1]; + } + + auto x_md = memory::desc(x_dims, MKLDNNGetDataType(), x_strides); + auto y_md = memory::desc(y_dims, MKLDNNGetDataType(), y_strides); + auto out_md = + memory::desc(out_ddims, MKLDNNGetDataType(), out_strides); + + this->AcquireForwardPrimitiveDescriptor(x_md, y_md, out_md); + } + } + + std::shared_ptr AcquireWeightsMemory(const Tensor* input) { + const T* input_data = input->data(); + return this->AcquireMemoryFromPrimitive(this->fwd_pd_->weights_desc(), + to_void_cast(input_data), + "@weights_mem_p"); + } +}; + +template +class MatMulV2MKLDNNKernel : public framework::OpKernel { + public: + void Compute(const ExecutionContext& ctx) const override { RunKernel(ctx); } + + private: + void CalculateMatrixDims(const ExecutionContext& ctx, + const std::vector& x_dims, + const std::vector& y_dims, + std::vector& x_bd_dims, + std::vector& y_bd_dims, + std::vector& out_dims, Tensor* out) const { + if (x_dims.size() == 1) { + x_bd_dims[x_bd_dims.size() - 1] = x_dims[0]; + } else { + for (size_t i = 0; i < x_dims.size(); ++i) { + x_bd_dims[i] = x_dims[i]; + } + } + if (y_dims.size() == 1) { + y_bd_dims[x_bd_dims.size() - 2] = y_dims[0]; + } else { + for (size_t i = 0; i < y_dims.size(); ++i) { + y_bd_dims[i] = y_dims[i]; + } + } + + if ((y_dims.size() == x_dims.size()) && y_dims.size() > 2) { + for (size_t i = 0; i < x_dims.size() - 2; ++i) { + PADDLE_ENFORCE_EQ( + x_dims[i] == y_dims[i] || x_dims[i] == 1 || y_dims[i] == 1, true, + platform::errors::InvalidArgument( + "Tensor dimensions are incorrect for broadcasting." + "Dimensions in X and Y must be same or equal to 1, but " + "received x_dim[%d]=%d and y_dims[%d]= %d", + i, x_dims[i], i, y_dims[i])); + out_dims[i] = std::max(x_dims[i], y_dims[i]); + } + out->Resize(framework::make_ddim(out_dims)); + } + } + + void RunKernel(const ExecutionContext& ctx) const { + const auto& dev_ctx = ctx.template device_context(); + const auto& onednn_engine = dev_ctx.GetEngine(); + + auto* x = ctx.Input("X"); + auto* y = ctx.Input("Y"); + auto* out = ctx.Output("Out"); + bool trans_x = ctx.Attr("trans_x"); + bool trans_y = ctx.Attr("trans_y"); + + auto x_dims = framework::vectorize(x->dims()); + auto y_dims = framework::vectorize(y->dims()); + auto out_dims = framework::vectorize(out->dims()); + + int ndims = std::max(x->dims().size(), y->dims().size()); + ndims = std::max(ndims, 3); + + std::vector x_bd_dims(ndims, 1); + std::vector y_bd_dims(ndims, 1); + + CalculateMatrixDims(ctx, x_dims, y_dims, x_bd_dims, y_bd_dims, out_dims, + out); + + MatMulV2MKLDNNHandler handler(dev_ctx, onednn_engine, ctx.GetPlace(), + x_bd_dims, trans_x, y_bd_dims, trans_y, + ctx.InputName("X")); + + const auto src_memory_p = handler.AcquireSrcMemory(x); + const auto weights_memory_p = handler.AcquireWeightsMemory(y); + const auto dst_memory_p = handler.AcquireDstMemory(out); + + auto matmul_p = handler.AcquireForwardPrimitive(); + + std::unordered_map matmul_args = { + {DNNL_ARG_SRC, *src_memory_p}, + {DNNL_ARG_WEIGHTS, *weights_memory_p}, + {DNNL_ARG_DST, *dst_memory_p}}; + + auto& astream = MKLDNNDeviceContext::tls().get_stream(); + matmul_p->execute(astream, matmul_args); + astream.wait(); + + out->set_layout(framework::DataLayout::kMKLDNN); + out->set_format( + GetMKLDNNFormat(dst_memory_p->get_desc().reshape(out_dims))); + } +}; +} // namespace operators +} // namespace paddle +namespace ops = paddle::operators; + +REGISTER_OP_KERNEL(matmul_v2, MKLDNN, ::paddle::platform::CPUPlace, + ops::MatMulV2MKLDNNKernel, + ops::MatMulV2MKLDNNKernel); + +// REGISTER_OP_KERNEL(matmul_grad_v2, MKLDNN, ::paddle::platform::CPUPlace, +// ops::MatMulV2GradMKLDNNKernel, +// ops::MatMulV2GradMKLDNNKernel); diff --git a/paddle/fluid/operators/mkldnn/split_mkldnn_op.cc b/paddle/fluid/operators/mkldnn/split_mkldnn_op.cc new file mode 100644 index 00000000000000..afbe330305b7e1 --- /dev/null +++ b/paddle/fluid/operators/mkldnn/split_mkldnn_op.cc @@ -0,0 +1,132 @@ +/* Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. */ + +#include "paddle/fluid/operators/utils.h" +#include "paddle/fluid/platform/mkldnn_reuse.h" + +namespace paddle { +namespace operators { + +using paddle::framework::Tensor; + +static inline std::vector> CalculateOutsDims( + const framework::DDim& in_dims, const size_t num, + const std::vector& sections, const size_t axis, + const int outs_number) { + std::vector> outs_dims(outs_number, + framework::vectorize(in_dims)); + + if (num > 0) { + PADDLE_ENFORCE_EQ(in_dims[axis] % num, 0, + platform::errors::InvalidArgument( + "The input's size along the split dimension " + "must be evenly divisible by Attr(num_or_sections). " + "But received Attr(num_or_sections) " + "= %d, input(X)'s shape = [%s], Attr(dim) = %d.", + num, in_dims, axis)); + + const size_t out_axis_dim = in_dims[axis] / num; + + for (auto& out_dim : outs_dims) out_dim[axis] = out_axis_dim; + } else { + for (size_t i = 0; i < outs_dims.size(); ++i) + outs_dims[i][axis] = sections[i]; + } + return outs_dims; +} + +template +class SplitMKLDNNKernel : public framework::OpKernel { + public: + void Compute(const framework::ExecutionContext& ctx) const override { + this->RunKernel(ctx); + } + + void RunKernel(const framework::ExecutionContext& ctx) const { + const auto& dev_ctx = + ctx.template device_context(); + const auto& onednn_engine = dev_ctx.GetEngine(); + + const auto* x = ctx.Input("X"); + auto outs = ctx.MultiOutput("Out"); + + int num = ctx.Attr("num"); + auto sections = ctx.Attr>("sections"); + int axis = ctx.Attr("axis"); + auto outs_number = outs.size(); + const auto x_dims = x->dims(); + + bool need_resize = false; + if (ctx.HasInput("AxisTensor")) { + auto* axis_tensor = ctx.Input("AxisTensor"); + axis = GetDataFromTensor(axis_tensor)[0]; + need_resize = true; + } + + auto sections_tensor_list = ctx.MultiInput("SectionsTensorList"); + if (sections_tensor_list.size() > 0) { + sections = GetDataFromTensorList(sections_tensor_list); + need_resize = true; + } + + if (need_resize) { + const auto outs_dims = + CalculateOutsDims(x->dims(), num, sections, axis, outs_number); + for (size_t i = 0; i < outs.size(); ++i) { + outs[i]->Resize(framework::make_ddim(outs_dims[i])); + } + } + + auto x_vec_dims = framework::vectorize(x_dims); + + mkldnn::memory::data_type x_type = framework::ToMKLDNNDataType(x->type()); + auto key = platform::CreateKey(dev_ctx, x_vec_dims, axis, num, sections, + x->format(), x_type); + + auto& astream = platform::MKLDNNDeviceContext::tls().get_stream(); + + std::vector offset(x_vec_dims.size(), 0); + + platform::ReorderMKLDNNHandler reorder_handler( + x_vec_dims, x->type(), x_type, dev_ctx, onednn_engine, key); + auto reorder_src_memory_p = reorder_handler.AcquireSrcMemory( + x->format(), platform::to_void_cast(x->data())); + + for (size_t i = 0; i < outs_number; ++i) { + auto out_vec_dims = framework::vectorize(outs[i]->dims()); + auto slice_mem_p = reorder_handler.AcquireSrcSubmemory( + out_vec_dims, offset, reorder_src_memory_p, i); + + auto reorder_dst_memory_p = reorder_handler.AcquireDstMemory( + outs[i], out_vec_dims, i, x->format(), ctx.GetPlace()); + auto reorder_p = + reorder_handler.AcquireReorder(reorder_dst_memory_p, slice_mem_p, i); + + reorder_p->execute(astream, *slice_mem_p, *reorder_dst_memory_p); + + offset[axis] += num > 0 ? x->dims()[axis] / num : sections[i]; + + outs[i]->set_layout(framework::DataLayout::kMKLDNN); + outs[i]->set_format(platform::GetMKLDNNFormat(*reorder_dst_memory_p)); + } + astream.wait(); + } +}; +} // namespace operators +} // namespace paddle + +namespace ops = paddle::operators; +REGISTER_OP_KERNEL(split, MKLDNN, paddle::platform::CPUPlace, + ops::SplitMKLDNNKernel, + ops::SplitMKLDNNKernel); diff --git a/paddle/fluid/operators/mkldnn/test_mkldnn_caching.cc b/paddle/fluid/operators/mkldnn/test_mkldnn_caching.cc index d6cd76b697f518..cad4f47ec14022 100644 --- a/paddle/fluid/operators/mkldnn/test_mkldnn_caching.cc +++ b/paddle/fluid/operators/mkldnn/test_mkldnn_caching.cc @@ -180,17 +180,5 @@ TEST(test_elementwise_add_reuse_cache, cpu_place) { "Wrong number of cached oneDNN objects")); } -TEST(test_elementwises_sequence_reuse_cache, cpu_place) { - framework::DDim dims({32, 64}); - platform::CPUPlace p; - CacheTester ct; - RunOperator(p, "elementwise_add", dims, "elementwise_add_out", true); - RunOperator(p, "elementwise_mul", dims, "elementwise_add_out", true); - RunOperator(p, "relu", dims, "elementwise_add_out", true); - PADDLE_ENFORCE_EQ(ct.Analyze(11), true, - platform::errors::InvalidArgument( - "Wrong number of cached oneDNN objects")); -} - } // namespace operators } // namespace paddle diff --git a/paddle/fluid/operators/mkldnn/test_mkldnn_op_inplace.cc b/paddle/fluid/operators/mkldnn/test_mkldnn_op_inplace.cc index 643de3fd5be70e..0612417c46ce30 100644 --- a/paddle/fluid/operators/mkldnn/test_mkldnn_op_inplace.cc +++ b/paddle/fluid/operators/mkldnn/test_mkldnn_op_inplace.cc @@ -128,12 +128,6 @@ TEST(test_softmax_inplace, cpu_place) { ASSERT_TRUE(TestMain(p, "softmax", dims, 1)); } -TEST(test_elementwise_add_inplace, cpu_place) { - framework::DDim dims({1, 12, 20, 20}); - platform::CPUPlace p; - ASSERT_TRUE(TestMain(p, "elementwise_add", dims, 2)); -} - TEST(test_relu_inplace, cpu_place) { framework::DDim dims({1, 12, 20, 20}); platform::CPUPlace p; diff --git a/paddle/fluid/operators/prelu_op.h b/paddle/fluid/operators/prelu_op.h index cfc0a2b6fb1128..60fd75ce3cffd3 100644 --- a/paddle/fluid/operators/prelu_op.h +++ b/paddle/fluid/operators/prelu_op.h @@ -39,13 +39,19 @@ class PReluKernel : public framework::OpKernel { int index = 0; int i = 0; if (mode == "channel") { - int temp = numel / (dim[0] * dim[1]); + int temp = 1; + for (int j = 2; j < dim.size(); j++) { + temp *= dim[j]; + } for (i = 0; i < numel; i++) { index = (i / temp) % dim[1]; o_ptr[i] = x_ptr[i] > 0 ? x_ptr[i] : alpha_ptr[index] * x_ptr[i]; } } else if (mode == "element") { - int temp = numel / dim[0]; + int temp = 1; + for (int j = 1; j < dim.size(); j++) { + temp *= dim[j]; + } for (i = 0; i < numel; i++) { index = i % temp; o_ptr[i] = x_ptr[i] > 0 ? x_ptr[i] : alpha_ptr[index] * x_ptr[i]; @@ -75,18 +81,23 @@ class PReluGradKernel : public framework::OpKernel { auto dim = x->dims(); int index = 0; int i = 0; - int temp = 0; if (dx) { T* dx_ptr = dx->mutable_data(context.GetPlace()); if (mode == "channel") { + int temp = 1; + for (int j = 2; j < dim.size(); j++) { + temp *= dim[j]; + } for (i = 0; i < numel; i++) { - temp = numel / (dim[0] * dim[1]); index = (i / temp) % dim[1]; dx_ptr[i] = x_ptr[i] > 0 ? dout_ptr[i] : alpha_ptr[index] * dout_ptr[i]; } } else if (mode == "element") { - temp = numel / dim[0]; + int temp = 1; + for (int j = 1; j < dim.size(); j++) { + temp *= dim[j]; + } for (i = 0; i < numel; i++) { index = i % temp; dx_ptr[i] = @@ -105,13 +116,19 @@ class PReluGradKernel : public framework::OpKernel { memset(dalpha_ptr, 0, sizeof(T) * dalpha->numel()); if (mode == "channel") { + int temp = 1; + for (int j = 2; j < dim.size(); j++) { + temp *= dim[j]; + } for (i = 0; i < numel; i++) { - temp = numel / (dim[0] * dim[1]); index = (i / temp) % dim[1]; dalpha_ptr[index] += x_ptr[i] > 0 ? 0 : x_ptr[i] * dout_ptr[i]; } } else if (mode == "element") { - temp = numel / dim[0]; + int temp = 1; + for (int j = 1; j < dim.size(); j++) { + temp *= dim[j]; + } for (i = 0; i < numel; i++) { index = i % temp; dalpha_ptr[index] += x_ptr[i] > 0 ? 0 : x_ptr[i] * dout_ptr[i]; diff --git a/paddle/fluid/operators/reduce_ops/reduce_all_op.cu b/paddle/fluid/operators/reduce_ops/reduce_all_op.cu index 89f3345fcbe42d..99a5caaad6ab80 100644 --- a/paddle/fluid/operators/reduce_ops/reduce_all_op.cu +++ b/paddle/fluid/operators/reduce_ops/reduce_all_op.cu @@ -13,7 +13,9 @@ // limitations under the License. #include "paddle/fluid/operators/reduce_ops/reduce_all_op.h" +#include "paddle/fluid/operators/reduce_ops/reduce_functor_op.h" +// reduce_prod REGISTER_OP_CUDA_KERNEL( - reduce_all, ops::BoolReduceKernel); + reduce_all, + ops::ReduceCudaKernel); diff --git a/paddle/fluid/operators/reduce_ops/reduce_any_op.cu b/paddle/fluid/operators/reduce_ops/reduce_any_op.cu index c0f94098a351ea..c7eafa2ac8760a 100644 --- a/paddle/fluid/operators/reduce_ops/reduce_any_op.cu +++ b/paddle/fluid/operators/reduce_ops/reduce_any_op.cu @@ -13,7 +13,10 @@ // limitations under the License. #include "paddle/fluid/operators/reduce_ops/reduce_any_op.h" +#include "paddle/fluid/operators/reduce_ops/reduce_functor_op.h" +#include "paddle/fluid/operators/reduce_ops/reduce_op.h" +// reduce_prod REGISTER_OP_CUDA_KERNEL( - reduce_any, ops::BoolReduceKernel); + reduce_any, + ops::ReduceCudaKernel); diff --git a/paddle/fluid/operators/reduce_ops/reduce_op.cu.h b/paddle/fluid/operators/reduce_ops/reduce_op.cu.h index 5fad6efdb34961..45279a224ac8dc 100644 --- a/paddle/fluid/operators/reduce_ops/reduce_op.cu.h +++ b/paddle/fluid/operators/reduce_ops/reduce_op.cu.h @@ -62,27 +62,6 @@ struct DivideFunctor { T n_inv; }; -static inline std::vector GetReduceDim(const std::vector& dims, - int dim_size, bool reduce_all) { - std::vector reduce_dims; - if (reduce_all) { - reduce_dims.resize(dim_size); - for (int i = 0; i < reduce_dims.size(); ++i) { - reduce_dims[i] = i; - } - } else { - for (auto e : dims) { - PADDLE_ENFORCE_LT(e, dim_size, - paddle::platform::errors::InvalidArgument( - "ReduceOp: invalid axis, when x_dims is %d, " - "axis[i] should less than x_dims, but got %d.", - dim_size, e)); - reduce_dims.push_back(e >= 0 ? e : e + dim_size); - } - } - return reduce_dims; -} - static inline int GetLastPow2(int n) { n |= (n >> 1); n |= (n >> 2); @@ -167,8 +146,9 @@ enum ReduceType { // reduce config template struct ReduceConfig { - ReduceConfig(std::vector origin_reduce_dims, std::vector x_dim) - : reduce_dims_origin(origin_reduce_dims), x_dim(x_dim) {} + ReduceConfig(const std::vector& origin_reduce_dims, + const std::vector& origin_x_dim) + : reduce_dims_origin(origin_reduce_dims), x_dim(origin_x_dim) {} // get the parameters of reduceKernel void Run() { @@ -530,22 +510,22 @@ __device__ __forceinline__ void ReduceAny( // module function designed for global function template + int BlockDim, int Rank, int ReduceRank> __device__ __forceinline__ void ReduceModule( const Tx* x, Ty* y, ReduceOp reducer, TransformOp transformer, Ty init, - int reduce_num, int left_num, int blocking_size, + int reduce_num, int left_num, int blocking_size, int reduce_type, paddle::framework::Array x_strides, paddle::framework::Array reduce_dim, paddle::framework::Array reduce_strides, paddle::framework::Array left_dim, paddle::framework::Array left_strides) { // reduce_rank == 1 && reduce_dim[0] == x_dim.size() - 1 - if (ReduceType == ReduceType::kReduceLastDim) { + if (reduce_type == ReduceType::kReduceLastDim) { ReduceLastDim( x, y, reducer, transformer, init, reduce_num); // reduce_rank == 1 && reduce_dim[0] != x_dim.size() - 1 - } else if (ReduceType == ReduceType::kReduceHigherDim) { + } else if (reduce_type == ReduceType::kReduceHigherDim) { ReduceHigherDim( x, y, reducer, transformer, init, reduce_num, left_num, blocking_size); @@ -558,57 +538,47 @@ __device__ __forceinline__ void ReduceModule( } template + int BlockDim, int Rank, int ReduceRank> __global__ void ReduceKernelFunction( const Tx* x, Ty* y, ReduceOp reducer, TransformOp transformer, Ty init, - int reduce_num, int left_num, int block_size, + int reduce_num, int left_num, int block_size, int reduce_type, paddle::framework::Array x_strides, paddle::framework::Array reduce_dim, paddle::framework::Array reduce_strides, paddle::framework::Array left_dim, paddle::framework::Array left_strides) { - ReduceModule(x, y, reducer, transformer, init, reduce_num, - left_num, block_size, x_strides, reduce_dim, - reduce_strides, left_dim, left_strides); + ReduceModule( + x, y, reducer, transformer, init, reduce_num, left_num, block_size, + reduce_type, x_strides, reduce_dim, reduce_strides, left_dim, + left_strides); } -template -static void LaunchKernel(const Tx* x_data, Ty* y_data, const ReduceOp& reducer, - const TransformOp& transformer, Ty init, - gpuStream_t stream, ReduceConfig config) { -#define CUB_REDUCE_TYPE_CASE(type) \ - case type: { \ - constexpr auto kReduceType = type; \ - ReduceKernelFunction< \ - Tx, Ty, ReduceOp, TransformOp, BlockDim, kRank, kReduceRank, \ - kReduceType><<>>( \ - x_data, config.output_data, reducer, transformer, init, \ - config.reduce_num, config.left_num, config.blocking_size, \ - detail::VectorToArray(config.x_strides), \ - detail::VectorToArray(config.reduce_dim), \ - detail::VectorToArray(config.reduce_strides), \ - detail::VectorToArray(config.left_dim), \ - detail::VectorToArray(config.left_strides)); \ - } break - - switch (config.reduce_type) { - CUB_REDUCE_TYPE_CASE(1); // reduceLastDim - CUB_REDUCE_TYPE_CASE(2); // ReduceHigherDim - CUB_REDUCE_TYPE_CASE(3); // reduceAny - } +template +static void LaunchReduceKernel(const Tx* x_data, Ty* y_data, + const ReduceOp& reducer, Ty init, + gpuStream_t stream, ReduceConfig config) { + using TransformOp = typename ReduceOp::Transformer; + + ReduceKernelFunction<<>>( + x_data, config.output_data, reducer, TransformOp(config.reduce_num), init, + config.reduce_num, config.left_num, config.blocking_size, + config.reduce_type, detail::VectorToArray(config.x_strides), + detail::VectorToArray(config.reduce_dim), + detail::VectorToArray(config.reduce_strides), + detail::VectorToArray(config.left_dim), + detail::VectorToArray(config.left_strides)); if (config.should_reduce_again) { dim3 block(config.block.x, 1, 1); dim3 grid(config.grid.x, 1, config.grid.z); - ReduceKernelFunction< - Ty, Ty, ReduceOp, detail::IdentityFunctor, 128, kRank, kReduceRank, - ReduceType::kReduceHigherDim><<>>( + ReduceKernelFunction, 128, + kRank, kReduceRank><<>>( config.output_data, y_data, reducer, detail::IdentityFunctor(config.grid.y), init, config.grid.y, - config.left_num, config.grid.y, + config.left_num, config.grid.y, ReduceType::kReduceHigherDim, detail::VectorToArray(config.x_strides), detail::VectorToArray(config.reduce_dim), detail::VectorToArray(config.reduce_strides), @@ -617,12 +587,10 @@ static void LaunchKernel(const Tx* x_data, Ty* y_data, const ReduceOp& reducer, } } -template -static void LaunchReduceKernel(const Tx* x_data, Ty* y_data, - const ReduceOp& reducer, - const TransformOp& transformer, Ty init, - gpuStream_t stream, ReduceConfig config) { +template +static void ReduceKernelImpl(const Tx* x_data, Ty* y_data, + const ReduceOp& reducer, Ty init, + gpuStream_t stream, ReduceConfig config) { int reduce_rank = config.reduce_strides.size(); int rank = config.x_strides.size(); @@ -632,11 +600,11 @@ static void LaunchReduceKernel(const Tx* x_data, Ty* y_data, switch (reduce_rank) { __VA_ARGS__; } \ } break -#define CUB_REDUCE_RANK_CASE(i, ...) \ - case i: { \ - constexpr auto kReduceRank = i; \ - LaunchKernel( \ - x_data, y_data, reducer, transformer, init, stream, config); \ +#define CUB_REDUCE_RANK_CASE(i, ...) \ + case i: { \ + constexpr auto kReduceRank = i; \ + LaunchReduceKernel( \ + x_data, y_data, reducer, init, stream, config); \ } break detail::CheckReduceRank(reduce_rank, rank); @@ -671,15 +639,13 @@ void TensorReduceFunctorImpl(const framework::Tensor& x, framework::Tensor* y, auto config = ReduceConfig(origin_reduce_dims, x_dim); config.Run(); // get the parameters of LaunchReduceKernel - auto x_data = x.data(); - auto y_data = y->mutable_data(x.place()); - // after config.run() // SetOutputData for ReduceHigherDim when should_reduce_again is true, // temp_output should be stored temp_data in output_data space or stored in // y_data; framework::Tensor tmp; - config.SetOutputData(y_data, x.place(), &tmp); + auto x_data = x.data(); + auto y_data = y->mutable_data(x.place()); if (config.reduce_num == 1) { auto out_dims = y->dims(); @@ -687,6 +653,9 @@ void TensorReduceFunctorImpl(const framework::Tensor& x, framework::Tensor* y, y->Resize(out_dims); return; } + + config.SetOutputData(y_data, x.place(), &tmp); + using TransformOp = typename ReduceOp::Transformer; auto reducer = ReduceOp(); // launch CUB::Reduce @@ -708,12 +677,11 @@ void TensorReduceFunctorImpl(const framework::Tensor& x, framework::Tensor* y, return; } -#define CUB_BLOCK_DIM_CASE(block_dim) \ - case block_dim: { \ - constexpr auto kBlockDim = block_dim; \ - LaunchReduceKernel, TransformOp>( \ - x_data, y_data, reducer, TransformOp(config.reduce_num), \ - reducer.initial(), stream, config); \ +#define CUB_BLOCK_DIM_CASE(block_dim) \ + case block_dim: { \ + constexpr auto kBlockDim = block_dim; \ + ReduceKernelImpl>( \ + x_data, y_data, reducer, reducer.initial(), stream, config); \ } break switch (detail::GetBlockDim(config.reduce_num)) { @@ -745,30 +713,5 @@ struct TensorReduceFunc { } }; -template class ReduceOp> -class ReduceCudaKernel : public framework::OpKernel { - public: - void Compute(const framework::ExecutionContext& context) const override { - bool reduce_all = context.Attr("reduce_all"); - const Tensor* input = context.Input("X"); - Tensor* output = context.Output("Out"); - auto out_dtype = context.Attr("out_dtype"); - std::vector dims = context.Attr>("dim"); - - std::vector reduce_dims = - detail::GetReduceDim(dims, input->dims().size(), reduce_all); - - gpuStream_t stream = context.cuda_device_context().stream(); - if (out_dtype >= 0) { - framework::VisitDataTypeSmall( - static_cast(out_dtype), - TensorReduceFunc(*input, output, reduce_dims, stream)); - } else { - TensorReduceFunctorImpl(*input, output, reduce_dims, - stream); - } - } -}; - } // namespace operators } // namespace paddle diff --git a/paddle/fluid/operators/reduce_ops/reduce_op.h b/paddle/fluid/operators/reduce_ops/reduce_op.h index 390c4d9709a60f..368fedececf533 100644 --- a/paddle/fluid/operators/reduce_ops/reduce_op.h +++ b/paddle/fluid/operators/reduce_ops/reduce_op.h @@ -23,6 +23,9 @@ limitations under the License. */ #include "paddle/fluid/operators/cast_op.h" #include "paddle/fluid/operators/math/math_function.h" #include "paddle/fluid/operators/reduce_ops/reduce_op_function.h" +#if defined(__HIPCC__) || defined(__NVCC__) +#include "paddle/fluid/operators/reduce_ops/reduce_op.cu.h" +#endif namespace paddle { namespace operators { @@ -60,6 +63,27 @@ inline void GetShuffledDim(const DDim& src_dims, DDim* dst_dims, } } +static inline std::vector GetReduceDim(const std::vector& dims, + int dim_size, bool reduce_all) { + std::vector reduce_dims; + if (reduce_all) { + reduce_dims.resize(dim_size); + int reduce_size = reduce_dims.size(); + for (int i = 0; i < reduce_size; ++i) { + reduce_dims[i] = i; + } + } else { + for (auto e : dims) { + PADDLE_ENFORCE_LT(e, dim_size, + paddle::platform::errors::InvalidArgument( + "ReduceOp: invalid axis, when x_dims is %d, " + "axis[i] should less than x_dims, but got %d.", + dim_size, e)); + reduce_dims.push_back(e >= 0 ? e : e + dim_size); + } + } + return reduce_dims; +} template void GetShuffledInput(const framework::ExecutionContext& context, const Tensor* input, Tensor* shuffled_input, @@ -308,6 +332,7 @@ class BoolReduceKernel : public framework::OpKernel { } } }; + template class ReduceGradKernel : public framework::OpKernel { @@ -636,6 +661,33 @@ If reduce_all is true, just reduce along all dimensions and output a scalar. virtual std::string GetOpType() const = 0; }; +#if defined(__HIPCC__) || defined(__NVCC__) +template class ReduceOp> +class ReduceCudaKernel : public framework::OpKernel { + public: + void Compute(const framework::ExecutionContext& context) const override { + bool reduce_all = context.Attr("reduce_all"); + const Tensor* input = context.Input("X"); + Tensor* output = context.Output("Out"); + auto out_dtype = context.Attr("out_dtype"); + std::vector dims = context.Attr>("dim"); + + std::vector reduce_dims = + GetReduceDim(dims, input->dims().size(), reduce_all); + + gpuStream_t stream = context.cuda_device_context().stream(); + if (out_dtype >= 0) { + framework::VisitDataTypeSmall( + static_cast(out_dtype), + TensorReduceFunc(*input, output, reduce_dims, stream)); + } else { + TensorReduceFunctorImpl(*input, output, reduce_dims, + stream); + } + } +}; +#endif + } // namespace operators } // namespace paddle diff --git a/paddle/fluid/operators/reduce_ops/reduce_prod_op.cu b/paddle/fluid/operators/reduce_ops/reduce_prod_op.cu index 4f259e415d2220..317a6e1d93c2e8 100644 --- a/paddle/fluid/operators/reduce_ops/reduce_prod_op.cu +++ b/paddle/fluid/operators/reduce_ops/reduce_prod_op.cu @@ -16,18 +16,8 @@ #include "paddle/fluid/operators/reduce_ops/reduce_op.cu.h" #include "paddle/fluid/operators/reduce_ops/reduce_prod_op.h" -// reduce_prod -#ifdef __HIPCC__ -// Eigen3/unsupported/Eigen/CXX11/src/Tensor/TensorReductionGpu.h:922 -// do not support double in HIPCC platform (Eigen3 to be fixed) -REGISTER_OP_CUDA_KERNEL( - reduce_prod, ops::ReduceCudaKernel, - ops::ReduceCudaKernel, - ops::ReduceCudaKernel); -#else REGISTER_OP_CUDA_KERNEL( reduce_prod, ops::ReduceCudaKernel, ops::ReduceCudaKernel, ops::ReduceCudaKernel, ops::ReduceCudaKernel); -#endif diff --git a/paddle/fluid/operators/rnn_op.cu.cc b/paddle/fluid/operators/rnn_op.cu.cc index 2be59c620441d6..07329a9175e525 100644 --- a/paddle/fluid/operators/rnn_op.cu.cc +++ b/paddle/fluid/operators/rnn_op.cu.cc @@ -29,15 +29,21 @@ namespace operators { using LoDTensor = framework::LoDTensor; using Tensor = framework::Tensor; +#ifdef PADDLE_WITH_HIP +using gpuRNNMode_t = miopenRNNMode_t; +using gpuDnnHandle_t = miopenHandle_t; +using gpuDnnDataType_t = miopenDataType_t; +#else +using gpuRNNMode_t = cudnnRNNMode_t; +using gpuDnnHandle_t = cudnnHandle_t; +using gpuDnnDataType_t = cudnnDataType_t; +#endif + class RNNDescriptors { public: RNNDescriptors(int seq_length, int batch_size, int input_size, int hidden_size, int num_layers, float dropout_prob, int seed, -#ifdef PADDLE_WITH_HIP - int weight_numel, miopenRNNMode_t mode, bool is_bidirec, -#else - int weight_numel, cudnnRNNMode_t mode, bool is_bidirec, -#endif + int weight_numel, gpuRNNMode_t mode, bool is_bidirec, bool is_test) : seq_length_(seq_length), batch_size_(batch_size), @@ -49,23 +55,14 @@ class RNNDescriptors { weight_numel_(weight_numel), mode_(mode), is_bidirec_(is_bidirec), - is_test_(is_test) { - } + is_test_(is_test) {} template -#ifdef PADDLE_WITH_HIP - void Create(const miopenHandle_t &handle, const platform::Place &place, -#else - void Create(const cudnnHandle_t &handle, const platform::Place &place, -#endif + void Create(const gpuDnnHandle_t &handle, const platform::Place &place, const std::vector &sequence_length, size_t *workspace_size, size_t *reserve_size, framework::Tensor *dropout_state) { int numDirections = is_bidirec_ ? 2 : 1; -#ifdef PADDLE_WITH_HIP - miopenDataType_t cudnn_type = platform::CudnnDataType::type; -#else - cudnnDataType_t cudnn_type = platform::CudnnDataType::type; -#endif + gpuDnnDataType_t cudnn_type = platform::CudnnDataType::type; // ------------------- cudnn x, y descriptors --------------------- std::vector dims_x = {batch_size_, input_size_, 1}; std::vector strides_x = {input_size_, 1, 1}; @@ -215,11 +212,7 @@ class RNNDescriptors { float dropout_prob_; int seed_; int weight_numel_; -#ifdef PADDLE_WITH_HIP - miopenRNNMode_t mode_; -#else - cudnnRNNMode_t mode_; -#endif + gpuRNNMode_t mode_; bool is_bidirec_; bool is_test_; #ifdef PADDLE_WITH_HIP @@ -296,6 +289,105 @@ void weight_to_tensor_list(const platform::Place &place, gpuStream_t stream, } } +#ifdef PADDLE_WITH_HIP +template +void weight_list_to_tensor(const platform::Place &place, gpuStream_t stream, + const std::vector &tensor_list, + Tensor *weight_whole, const size_t offset = 0UL) { + size_t weight_offset = offset; + auto weight_data = weight_whole->data(); + + for (size_t i = 0; i < tensor_list.size(); ++i) { + const T *in_data = tensor_list[i].data(); + auto in_size = tensor_list[i].numel(); + memory::Copy(BOOST_GET_CONST(platform::CUDAPlace, weight_whole->place()), + weight_data + weight_offset, + BOOST_GET_CONST(platform::CUDAPlace, tensor_list[i].place()), + in_data, in_size * sizeof(T), stream); + weight_offset += in_size; + } +} + +template +void weight_to_permuted_tensor(const platform::Place &place, gpuStream_t stream, + std::vector *weight_list, + Tensor *weight_whole, + const gpuRNNMode_t rnn_mode, + const bool is_bidirec) { + if (is_bidirec) { + for (size_t i = 0; i < weight_list->size(); i += 4) { + auto tmp = (*weight_list)[i + 1]; + (*weight_list)[i + 1] = (*weight_list)[i + 2]; + (*weight_list)[i + 2] = tmp; + } + } + size_t weight_offset = 0; + for (size_t i = 0; i < weight_list->size(); ++i) { + if (rnn_mode == miopenLSTM) { + std::vector split_tensor = (*weight_list)[i]->Chunk(4, 0); + weight_list_to_tensor( + place, stream, + {split_tensor[0], split_tensor[1], split_tensor[3], split_tensor[2]}, + weight_whole, weight_offset); + } else if (rnn_mode == miopenGRU) { + std::vector split_tensor = (*weight_list)[i]->Chunk(3, 0); + weight_list_to_tensor( + place, stream, {split_tensor[1], split_tensor[0], split_tensor[2]}, + weight_whole, weight_offset); + } else { + weight_list_to_tensor(place, stream, {*(*weight_list)[i]}, + weight_whole, weight_offset); + } + weight_offset += (*weight_list)[i]->numel(); + } +} + +template +void tensor_to_permuted_weight(const platform::Place &place, gpuStream_t stream, + const Tensor &tensor, + std::vector *weight_grad_list, + const gpuRNNMode_t rnn_mode, + const bool is_bidirec) { + if (is_bidirec) { + for (size_t i = 0; i < weight_grad_list->size(); i += 4) { + auto tmp = (*weight_grad_list)[i + 1]; + (*weight_grad_list)[i + 1] = (*weight_grad_list)[i + 2]; + (*weight_grad_list)[i + 2] = tmp; + } + } + size_t weight_offset = 0; + for (size_t i = 0; i < weight_grad_list->size(); ++i) { + auto numel_size = (*weight_grad_list)[i]->numel(); + Tensor temp; + temp.mutable_data({numel_size}, place); + temp.ShareDataWith(tensor.Slice(weight_offset, weight_offset + numel_size)); + + if (rnn_mode == miopenLSTM) { + std::vector split_tensor = temp.Chunk(4, 0); + weight_list_to_tensor( + place, stream, + {split_tensor[0], split_tensor[1], split_tensor[3], split_tensor[2]}, + (*weight_grad_list)[i]); + } else if (rnn_mode == miopenGRU) { + std::vector split_tensor = temp.Chunk(3, 0); + weight_list_to_tensor( + place, stream, {split_tensor[1], split_tensor[0], split_tensor[2]}, + (*weight_grad_list)[i]); + } else { + weight_list_to_tensor(place, stream, {temp}, (*weight_grad_list)[i]); + } + weight_offset += numel_size; + } + if (is_bidirec) { + for (size_t i = 0; i < weight_grad_list->size(); i += 4) { + auto tmp = (*weight_grad_list)[i + 1]; + (*weight_grad_list)[i + 1] = (*weight_grad_list)[i + 2]; + (*weight_grad_list)[i + 2] = tmp; + } + } +} +#endif + template class RNNCudnnKernel : public framework::OpKernel { public: @@ -314,7 +406,7 @@ class RNNCudnnKernel : public framework::OpKernel { int num_layers = ctx.Attr("num_layers"); auto mode = ctx.Attr("mode"); #ifdef PADDLE_WITH_HIP - miopenRNNMode_t rnn_mode = miopenLSTM; + gpuRNNMode_t rnn_mode = miopenLSTM; if (mode == "LSTM") rnn_mode = miopenLSTM; else if (mode == "GRU") @@ -324,7 +416,7 @@ class RNNCudnnKernel : public framework::OpKernel { else if (mode == "RNN_TANH") rnn_mode = miopenRNNTANH; #else - cudnnRNNMode_t rnn_mode = CUDNN_LSTM; + gpuRNNMode_t rnn_mode = CUDNN_LSTM; if (mode == "LSTM") rnn_mode = CUDNN_LSTM; else if (mode == "GRU") @@ -373,6 +465,11 @@ class RNNCudnnKernel : public framework::OpKernel { } bool has_seq_length = ctx.HasInput("SequenceLength"); +#ifdef PADDLE_WITH_HIP + PADDLE_ENFORCE_EQ(has_seq_length, false, + platform::errors::InvalidArgument( + "ROCm do not support SequenceLength yet.")); +#endif std::vector SequenceLength; if (has_seq_length) { auto *sequence_length = ctx.Input("SequenceLength"); @@ -400,14 +497,26 @@ class RNNCudnnKernel : public framework::OpKernel { [](int64_t num, const Tensor *t) { return num + t->numel(); }); bool continuous = is_continuous>(weight_list); +#ifdef PADDLE_WITH_HIP + // Need to permute weight, set continuous to false + continuous = false; +#endif if (!continuous) { LOG_FIRST_N(WARNING, 2) << "If the memory space of the Input WeightList is not continuous, " "less efficient calculation will be called. Please call " "flatten_parameters() to make the input memory continuous."; weight_whole.mutable_data({weight_numel}, place); +#ifdef PADDLE_WITH_HIP + // MIOPEN need to permute weight for miopenLSTM or miopenGRU + weight_to_permuted_tensor(place, stream, &weight_list, &weight_whole, + rnn_mode, is_bidirec); +#else weight_to_tensor(place, stream, weight_list, &weight_whole); +#endif w_data = weight_whole.data(); +#ifndef PADDLE_WITH_HIP + // MIOPEN need to permute weight, do not share with weight_grad if (is_test) { // maybe also reset small weights' ptr for training int offset = 0; for (size_t i = 0; i < weight_list.size(); ++i) { @@ -421,6 +530,7 @@ class RNNCudnnKernel : public framework::OpKernel { offset += len; } } +#endif } else { w_data = const_cast(weight_list[0]->data()); } @@ -486,11 +596,7 @@ class RNNCudnnKernel : public framework::OpKernel { } } -#ifdef PADDLE_WITH_HIP - void RNNInferece(const bool &has_seq_length, const miopenHandle_t &handle, -#else - void RNNInferece(const bool &has_seq_length, const cudnnHandle_t &handle, -#endif + void RNNInferece(const bool &has_seq_length, const gpuDnnHandle_t &handle, const int &seq_length, RNNDescriptors *rnn, const T *x_data, const T *init_h_data, const T *init_c_data, const T *w_data, T *out_data, T *last_h_data, T *last_c_data, @@ -607,9 +713,20 @@ class RNNGradCudnnKernel : public framework::OpKernel { Tensor weight_whole; T *weight_data = nullptr; +#ifdef PADDLE_WITH_HIP + // Need to permute weight, set continuous to false + continuous = false; +#endif + if (!continuous) { weight_whole.mutable_data({weight_numel}, place); +#ifdef PADDLE_WITH_HIP + // MIOPEN need to permute weight for miopenLSTM or miopenGRU + weight_to_permuted_tensor(place, stream, &weight_list, &weight_whole, + rnn_mode, is_bidirec); +#else weight_to_tensor(place, stream, weight_list, &weight_whole); +#endif weight_data = weight_whole.data(); } else { weight_data = const_cast(weight_list[0]->data()); @@ -621,6 +738,13 @@ class RNNGradCudnnKernel : public framework::OpKernel { zero(dev_ctx, &weight_grad, static_cast(0.0)); T *weight_grad_data = weight_grad.data(); +#ifdef PADDLE_WITH_HIP + // MIOPEN need to permute weight_grad_list, so do not share data with + // weight_grad + for (size_t i = 0; i < weight_grad_list.size(); ++i) { + weight_grad_list[i]->mutable_data(ctx.GetPlace()); + } +#else int offset = 0; for (size_t i = 0; i < weight_grad_list.size(); ++i) { size_t len = weight_grad_list[i]->numel(); @@ -631,6 +755,7 @@ class RNNGradCudnnKernel : public framework::OpKernel { .Resize(dim); offset += len; } +#endif Tensor input_grad_value; if (!in_grad) { @@ -672,6 +797,11 @@ class RNNGradCudnnKernel : public framework::OpKernel { } bool has_seq_length = ctx.HasInput("SequenceLength"); +#ifdef PADDLE_WITH_HIP + PADDLE_ENFORCE_EQ(has_seq_length, false, + platform::errors::InvalidArgument( + "ROCm do not support SequenceLength yet.")); +#endif std::vector SequenceLength; if (has_seq_length) { auto *sequence_length = ctx.Input("SequenceLength"); @@ -731,6 +861,9 @@ class RNNGradCudnnKernel : public framework::OpKernel { rnn.weight_desc(), weight_grad_data, workspace_data_.data(), workspace_size, const_cast(reserve_data), reserve_size)); + // permute weight grad list from weight grad tensor + tensor_to_permuted_weight(place, stream, weight_grad, + &weight_grad_list, rnn_mode, is_bidirec); #else PADDLE_ENFORCE_CUDA_SUCCESS(platform::dynload::cudnnRNNBackwardWeights( handle, rnn.rnn_desc(), seq_length, rnn.x_descs(), input->data(), diff --git a/paddle/fluid/operators/roll_op.cc b/paddle/fluid/operators/roll_op.cc index b1fe95203636fe..a0c28ae6cba16d 100644 --- a/paddle/fluid/operators/roll_op.cc +++ b/paddle/fluid/operators/roll_op.cc @@ -13,8 +13,10 @@ // limitations under the License. #include "paddle/fluid/operators/roll_op.h" + #include #include + #include "paddle/fluid/framework/op_version_registry.h" namespace paddle { @@ -37,12 +39,22 @@ class RollOp : public framework::OperatorWithKernel { auto dims = ctx->Attrs().Get>("axis"); auto shifts = ctx->Attrs().Get>("shifts"); - PADDLE_ENFORCE_EQ(dims.size(), shifts.size(), - platform::errors::InvalidArgument( - "Attr(dims).size() should be equl to " - "Attr(shifts).size(). But received " - "Attr(dims).size() = %d, Attr(shifts).size() = %d", - dims.size(), shifts.size())); + if (dims.size() != 0) { + PADDLE_ENFORCE_EQ(dims.size(), shifts.size(), + platform::errors::InvalidArgument( + "When dims.size() != 0, dims.size() " + "should be equal to " + "shifts.size(). But received " + "dims.size() = %d, shifts.size() = %d", + dims.size(), shifts.size())); + } else { + PADDLE_ENFORCE_EQ(shifts.size(), 1, + platform::errors::InvalidArgument( + "When dims.size() == 0, shifts.size() " + "should be equal to 1, But received " + "shifts.size() = %d", + shifts.size())); + } ctx->SetOutputDim("Out", ctx->GetInputDim("X")); auto type = ctx->GetInputsVarType("X")[0]; @@ -95,7 +107,7 @@ class RollOpMaker : public framework::OpProtoAndCheckerMaker { AddAttr>( "axis", "Axis along which to roll. It must have the same size " - "with shifts.") + "with shifts or size == 0") .SetDefault({}); AddComment(R"DOC( Roll the tensor along the given dimension(s). @@ -151,8 +163,9 @@ REGISTER_OP_VERSION(roll) paddle::framework::compatible::OpVersionDesc() .NewAttr("axis", "(std::vector) Axis along which to roll. " - "It must have the same size with shifts.", + "It must have the same size with shifts, or size = 0.", std::vector()) - .DeleteAttr("dims", - "(std::vector) Dims along which to roll. " - "It must have the same size with shifts.")); + .DeleteAttr( + "dims", + "(std::vector) Dims along which to roll. " + "It must have the same size with shifts, or size = 0.")); diff --git a/paddle/fluid/operators/roll_op.cu b/paddle/fluid/operators/roll_op.cu index 09309c492d2922..ce93c5f984e377 100644 --- a/paddle/fluid/operators/roll_op.cu +++ b/paddle/fluid/operators/roll_op.cu @@ -13,6 +13,7 @@ // limitations under the License. #pragma once +#include "paddle/fluid/framework/array.h" #include "paddle/fluid/framework/op_registry.h" #include "paddle/fluid/operators/roll_op.h" #include "paddle/fluid/platform/cuda_primitives.h" @@ -24,26 +25,31 @@ using platform::PADDLE_CUDA_NUM_THREADS; using Tensor = framework::Tensor; using LoDTensor = framework::LoDTensor; -template -__global__ void roll_cuda_kernel(const T* input, T* output, int64_t N, - int64_t* shifts, int64_t* strides, - int64_t* sizes, int64_t nums) { +template +__global__ void RollCudaKernel(const T* input, T* output, int64_t N, + paddle::framework::Array shifts, + paddle::framework::Array strides, + paddle::framework::Array sizes) { int64_t idx = blockIdx.x * blockDim.x + threadIdx.x; if (idx >= N) { return; } + int64_t output_idx = idx; int64_t dim_idx, dim_idx_shift; - for (int64_t i = 0; i < nums; i++) { - dim_idx = idx % (strides[i] * sizes[i]) / strides[i]; + +#pragma unroll Rank + for (size_t i = 0; i < Rank; i++) { + dim_idx = (idx / strides[i]) % sizes[i]; dim_idx_shift = (dim_idx + shifts[i]) % sizes[i]; output_idx = output_idx + (dim_idx_shift - dim_idx) * strides[i]; } output[output_idx] = input[idx]; } -template -class RollCUDAKernel : public framework::OpKernel { +template +class RollKernel + : public framework::OpKernel { public: void Compute(const framework::ExecutionContext& context) const override { auto* in = context.Input("X"); @@ -61,50 +67,62 @@ class RollCUDAKernel : public framework::OpKernel { auto input_dim = in->dims(); auto stride_dim = framework::stride(input_dim); - int64_t dim, size; - size_t gpu_memory_size_ = sizeof(int64_t) * nums; - std::vector strides, sizes; - strides.resize(nums); - sizes.resize(nums); - paddle::memory::AllocationPtr shifts_gpu = - memory::Alloc(context.GetPlace(), gpu_memory_size_); - paddle::memory::AllocationPtr strides_gpu = - memory::Alloc(context.GetPlace(), gpu_memory_size_); - paddle::memory::AllocationPtr sizes_gpu = - memory::Alloc(context.GetPlace(), gpu_memory_size_); - - for (size_t i = 0; i < nums; i++) { - dim = dims[i] >= 0 ? dims[i] : dims[i] + input_dim.size(); - size = input_dim[dim]; - shifts[i] = (shifts[i] % size + size) % size; - strides[i] = stride_dim[dim]; - sizes[i] = size; + std::vector strides(nums), sizes(nums); + if (dims.size() == 0) { + strides[0] = 1; + sizes[0] = numel; + shifts[0] = (shifts[0] % numel + numel) % numel; + } else { + for (size_t i = 0; i < nums; i++) { + int dim = dims[i] >= 0 ? dims[i] : dims[i] + input_dim.size(); + int64_t size = input_dim[dim]; + + shifts[i] = (shifts[i] % size + size) % size; + strides[i] = stride_dim[dim]; + sizes[i] = size; + } + } + +#define CALL_ROLL_CUDA_KERNEL(N) \ + case N: { \ + paddle::framework::Array _strides; \ + paddle::framework::Array _shifts; \ + paddle::framework::Array _sizes; \ + for (size_t idx = 0; idx < N; ++idx) { \ + _strides[idx] = strides[idx]; \ + _shifts[idx] = shifts[idx]; \ + _sizes[idx] = sizes[idx]; \ + } \ + RollCudaKernel< \ + T, \ + N><<<(numel + PADDLE_CUDA_NUM_THREADS - 1) / PADDLE_CUDA_NUM_THREADS, \ + PADDLE_CUDA_NUM_THREADS, 0, stream>>>(in_data, out_data, numel, \ + _shifts, _strides, _sizes); \ + break; \ + } + + switch (nums) { + CALL_ROLL_CUDA_KERNEL(1); + CALL_ROLL_CUDA_KERNEL(2); + CALL_ROLL_CUDA_KERNEL(3); + CALL_ROLL_CUDA_KERNEL(4); + CALL_ROLL_CUDA_KERNEL(5); + CALL_ROLL_CUDA_KERNEL(6); + CALL_ROLL_CUDA_KERNEL(7); + CALL_ROLL_CUDA_KERNEL(8); + CALL_ROLL_CUDA_KERNEL(9); + default: + PADDLE_THROW(platform::errors::InvalidArgument( + "shifts.size() should be less than 10, But received shifts.size() " + "= %d", + shifts.size())); } - paddle::memory::Copy( - BOOST_GET_CONST(platform::CUDAPlace, shifts_gpu->place()), - shifts_gpu->ptr(), platform::CPUPlace(), shifts.data(), - gpu_memory_size_, stream); - paddle::memory::Copy( - BOOST_GET_CONST(platform::CUDAPlace, strides_gpu->place()), - strides_gpu->ptr(), platform::CPUPlace(), strides.data(), - gpu_memory_size_, stream); - paddle::memory::Copy( - BOOST_GET_CONST(platform::CUDAPlace, sizes_gpu->place()), - sizes_gpu->ptr(), platform::CPUPlace(), sizes.data(), gpu_memory_size_, - stream); - int64_t* shifts_ptr = reinterpret_cast(shifts_gpu->ptr()); - int64_t* strides_ptr = reinterpret_cast(strides_gpu->ptr()); - int64_t* sizes_ptr = reinterpret_cast(sizes_gpu->ptr()); - - roll_cuda_kernel<<<(numel + PADDLE_CUDA_NUM_THREADS - 1) / - PADDLE_CUDA_NUM_THREADS, - PADDLE_CUDA_NUM_THREADS, 0, stream>>>( - in_data, out_data, numel, shifts_ptr, strides_ptr, sizes_ptr, nums); } }; -template -class RollGradCUDAKernel : public framework::OpKernel { +template +class RollGradKernel + : public framework::OpKernel { public: void Compute(const framework::ExecutionContext& context) const override { auto* in = context.Input(framework::GradVarName("Out")); @@ -121,46 +139,38 @@ class RollGradCUDAKernel : public framework::OpKernel { auto input_dim = in->dims(); auto stride_dim = framework::stride(input_dim); - int64_t dim, size; - size_t gpu_memory_size_ = sizeof(int64_t) * nums; - std::vector strides, sizes; - strides.resize(nums); - sizes.resize(nums); - paddle::memory::AllocationPtr shifts_gpu = - memory::Alloc(context.GetPlace(), gpu_memory_size_); - paddle::memory::AllocationPtr strides_gpu = - memory::Alloc(context.GetPlace(), gpu_memory_size_); - paddle::memory::AllocationPtr sizes_gpu = - memory::Alloc(context.GetPlace(), gpu_memory_size_); - - for (size_t i = 0; i < nums; i++) { - dim = dims[i] >= 0 ? dims[i] : dims[i] + input_dim.size(); - size = input_dim[dim]; - shifts[i] = ((0 - shifts[i]) % size + size) % size; - strides[i] = stride_dim[dim]; - sizes[i] = size; + std::vector strides(nums), sizes(nums); + if (dims.size() == 0) { + strides[0] = 1; + sizes[0] = numel; + shifts[0] = ((-shifts[0]) % numel + numel) % numel; + } else { + for (size_t i = 0; i < nums; i++) { + int dim = dims[i] >= 0 ? dims[i] : dims[i] + input_dim.size(); + int64_t size = input_dim[dim]; + + shifts[i] = ((-shifts[i]) % size + size) % size; + strides[i] = stride_dim[dim]; + sizes[i] = size; + } } - paddle::memory::Copy( - BOOST_GET_CONST(platform::CUDAPlace, shifts_gpu->place()), - shifts_gpu->ptr(), platform::CPUPlace(), shifts.data(), - gpu_memory_size_, stream); - paddle::memory::Copy( - BOOST_GET_CONST(platform::CUDAPlace, strides_gpu->place()), - strides_gpu->ptr(), platform::CPUPlace(), strides.data(), - gpu_memory_size_, stream); - paddle::memory::Copy( - BOOST_GET_CONST(platform::CUDAPlace, sizes_gpu->place()), - sizes_gpu->ptr(), platform::CPUPlace(), sizes.data(), gpu_memory_size_, - stream); - int64_t* shifts_ptr = reinterpret_cast(shifts_gpu->ptr()); - int64_t* strides_ptr = reinterpret_cast(strides_gpu->ptr()); - int64_t* sizes_ptr = reinterpret_cast(sizes_gpu->ptr()); - - roll_cuda_kernel<<<(numel + PADDLE_CUDA_NUM_THREADS - 1) / - PADDLE_CUDA_NUM_THREADS, - PADDLE_CUDA_NUM_THREADS, 0, stream>>>( - in_data, out_data, numel, shifts_ptr, strides_ptr, sizes_ptr, nums); + switch (nums) { + CALL_ROLL_CUDA_KERNEL(1); + CALL_ROLL_CUDA_KERNEL(2); + CALL_ROLL_CUDA_KERNEL(3); + CALL_ROLL_CUDA_KERNEL(4); + CALL_ROLL_CUDA_KERNEL(5); + CALL_ROLL_CUDA_KERNEL(6); + CALL_ROLL_CUDA_KERNEL(7); + CALL_ROLL_CUDA_KERNEL(8); + CALL_ROLL_CUDA_KERNEL(9); + default: + PADDLE_THROW(platform::errors::InvalidArgument( + "shifts.size() should be less than 10, But received shifts.size() " + "= %d", + shifts.size())); + } } }; @@ -169,13 +179,12 @@ class RollGradCUDAKernel : public framework::OpKernel { namespace ops = paddle::operators; REGISTER_OP_CUDA_KERNEL( - roll, ops::RollCUDAKernel, - ops::RollCUDAKernel, - ops::RollCUDAKernel, - ops::RollCUDAKernel); + roll, ops::RollKernel, + ops::RollKernel, + ops::RollKernel, + ops::RollKernel); REGISTER_OP_CUDA_KERNEL( - roll_grad, - ops::RollGradCUDAKernel, - ops::RollGradCUDAKernel, - ops::RollGradCUDAKernel, - ops::RollGradCUDAKernel); + roll_grad, ops::RollGradKernel, + ops::RollGradKernel, + ops::RollGradKernel, + ops::RollGradKernel); diff --git a/paddle/fluid/operators/roll_op.h b/paddle/fluid/operators/roll_op.h index 74dd37ed8388fe..da4f335ca7faa6 100644 --- a/paddle/fluid/operators/roll_op.h +++ b/paddle/fluid/operators/roll_op.h @@ -88,7 +88,13 @@ class RollKernel : public framework::OpKernel { TensorToVector(input, context.device_context(), &out_vec); size_t nums = shifts.size(); - const DDim input_dim = input.dims(); + DDim input_dim = input.dims(); + + // axis = none, reshape to 1-D tensor + if (dims.size() == 0) { + dims.push_back(0l); + input_dim = framework::Dim<1>(out_vec.size()); + } for (size_t i = 0; i < nums; i++) { PADDLE_ENFORCE_EQ( @@ -101,7 +107,7 @@ class RollKernel : public framework::OpKernel { } output->mutable_data(context.GetPlace()); framework::TensorFromVector(out_vec, context.device_context(), output); - output->Resize(input_dim); + output->Resize(input.dims()); } }; @@ -120,14 +126,20 @@ class RollGradKernel : public framework::OpKernel { TensorToVector(input, context.device_context(), &out_vec); size_t nums = shifts.size(); - const DDim input_dim = input.dims(); + DDim input_dim = input.dims(); + + // axis = none, reshape to 1-D tensor + if (dims.size() == 0) { + dims.push_back(0l); + input_dim = framework::Dim<1>(out_vec.size()); + } for (size_t i = 0; i < nums; i++) { shift_along_dim(out_vec.data(), input_dim, dims[i], 0 - shifts[i]); } output->mutable_data(context.GetPlace()); framework::TensorFromVector(out_vec, context.device_context(), output); - output->Resize(input_dim); + output->Resize(input.dims()); } }; diff --git a/paddle/fluid/operators/scatter.cu.h b/paddle/fluid/operators/scatter.cu.h index b116a78891a931..61e95c2b50eb72 100644 --- a/paddle/fluid/operators/scatter.cu.h +++ b/paddle/fluid/operators/scatter.cu.h @@ -33,6 +33,14 @@ __global__ void ScatterInitCUDAKernel(const IndexT* indices, T* output, int indices_i = i / slice_size; int slice_i = i - indices_i * slice_size; // offset inside the slice IndexT scatter_i = indices[indices_i]; + + PADDLE_ENFORCE(scatter_i >= 0, + "The index is out of bounds, " + "please check whether the dimensions of index and " + "input meet the requirements. It should " + "be greater than or equal to 0, but received [%d]", + scatter_i); + IndexT out_i = scatter_i * slice_size + slice_i; *(output + out_i) = static_cast(0); } @@ -46,6 +54,14 @@ __global__ void ScatterCUDAKernel(const T* params, const IndexT* indices, int indices_i = i / slice_size; int slice_i = i - indices_i * slice_size; // offset inside the slice IndexT scatter_i = indices[indices_i]; + + PADDLE_ENFORCE(scatter_i >= 0, + "The index is out of bounds, " + "please check whether the dimensions of index and " + "input meet the requirements. It should " + "be greater than or equal to 0, but received [%d]", + scatter_i); + IndexT out_i = scatter_i * slice_size + slice_i; if (overwrite) { *(output + out_i) = *(params + i); @@ -67,6 +83,15 @@ __global__ void ScatterNdCUDAKernel(const T* update, const IndexT* indices, int64_t temp = slice_size; for (int64_t j = end_size - 1; j >= 0; --j) { IndexT index_value = indices[indices_i * end_size + j]; + + PADDLE_ENFORCE( + index_value >= 0 && index_value < output_dims[j], + "The index is out of bounds, " + "please check whether the dimensions of index and " + "input meet the requirements. It should " + "be less than [%d] and greater or equal to 0, but received [%d]", + output_dims[j], index_value); + gather_i += (index_value * temp); temp *= output_dims[j]; } diff --git a/paddle/fluid/operators/scatter.h b/paddle/fluid/operators/scatter.h index 864a94a4235e65..2589033d2fef72 100644 --- a/paddle/fluid/operators/scatter.h +++ b/paddle/fluid/operators/scatter.h @@ -118,6 +118,15 @@ void ScatterAssign(const platform::DeviceContext& ctx, const Tensor& src, for (int i = 0; i < index_size; ++i) { IndexT index_ = p_index[i]; + + PADDLE_ENFORCE_GE(index_, 0, + platform::errors::OutOfRange( + "The index is out of bounds, " + "please check whether the dimensions of index and " + "input meet the requirements. It should " + "be greater than or equal to 0, but received [%d]", + index_)); + memcpy(p_output + index_ * slice_size, p_src + i * slice_size, slice_bytes); } } @@ -173,6 +182,15 @@ void ScatterAssignAdd(const framework::ExecutionContext& ctx, const Tensor& src, // if not in overwrite mode, need to init output data for (int i = 0; i < index_size; ++i) { const IndexT& index_ = p_index[i]; + + PADDLE_ENFORCE_GE(index_, 0, + platform::errors::OutOfRange( + "The index is out of bounds, " + "please check whether the dimensions of index and " + "input meet the requirements. It should " + "be greater than or equal to 0, but received [%d]", + index_)); + elementwise_inner_add(ctx, p_src, p_output, result_p_output, src, output, i, index_, slice_size, slice_bytes); @@ -233,6 +251,15 @@ void ScatterNdAdd(const framework::ExecutionContext& ctx, const Tensor& update, IndexT temp = 1; for (int64_t j = end_size - 1; j >= 0; --j) { IndexT index_value = p_index[i * end_size + j]; + PADDLE_ENFORCE_EQ( + (index_value >= 0 && index_value < output_dims[j]), true, + platform::errors::OutOfRange( + "The index is out of bounds, " + "please check whether the dimensions of index and " + "input meet the requirements. It should " + "be less than [%d] and greater or equal to 0, but received [%d]", + output_dims[j], index_value)); + index_ += (index_value * temp); temp *= output_dims[j]; } diff --git a/paddle/fluid/operators/share_data_op.cc b/paddle/fluid/operators/share_data_op.cc new file mode 100644 index 00000000000000..6fcc29e9002616 --- /dev/null +++ b/paddle/fluid/operators/share_data_op.cc @@ -0,0 +1,72 @@ +/* Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. */ + +#include "paddle/fluid/operators/share_data_op.h" +#include "paddle/fluid/framework/op_registry.h" + +namespace paddle { +namespace operators { + +class ShareDataOp : public framework::OperatorWithKernel { + public: + using framework::OperatorWithKernel::OperatorWithKernel; + + void InferShape(framework::InferShapeContext *ctx) const override { + OP_INOUT_CHECK(ctx->HasInput("X"), "Input", "X", "ShareData"); + OP_INOUT_CHECK(ctx->HasOutput("Out"), "Output", "Out", "ShareData"); + auto in_type = ctx->GetInputsVarType("X")[0]; + auto out_type = ctx->GetOutputsVarType("Out")[0]; + + PADDLE_ENFORCE_EQ( + in_type == framework::proto::VarType::LOD_TENSOR || + in_type == framework::proto::VarType::SELECTED_ROWS, + true, platform::errors::InvalidArgument( + "Type of Variable[X] must be LoDTensor or SelectedRows!")); + PADDLE_ENFORCE_EQ( + in_type, out_type, + platform::errors::InvalidArgument( + "The type of input (X) and output (Out) are inconsistent.")); + + ctx->ShareDim("X", "Out"); + } +}; + +class ShareDataOpMaker : public framework::OpProtoAndCheckerMaker { + public: + void Make() override { + AddInput("X", "(Tensor), The input tensor of share_data op"); + AddOutput("Out", "(Tensor), The output tensor of share_data op"); + AddComment(R"DOC( +ShareData Operator. + +Return a tensor $Out$ that shares data with the input tensor $X$ and without tensor copy. +)DOC"); + } +}; + +} // namespace operators +} // namespace paddle + +namespace ops = paddle::operators; +REGISTER_OPERATOR( + share_data, ops::ShareDataOp, ops::ShareDataOpMaker, + paddle::framework::EmptyGradOpMaker, + paddle::framework::EmptyGradOpMaker); +REGISTER_OP_CPU_KERNEL(share_data, ops::ShareDataKernel, + ops::ShareDataKernel, ops::ShareDataKernel, + ops::ShareDataKernel, + ops::ShareDataKernel, + ops::ShareDataKernel, + ops::ShareDataKernel, + ops::ShareDataKernel) diff --git a/paddle/fluid/operators/share_data_op.cu b/paddle/fluid/operators/share_data_op.cu new file mode 100644 index 00000000000000..20cdaafa43de72 --- /dev/null +++ b/paddle/fluid/operators/share_data_op.cu @@ -0,0 +1,25 @@ +/* Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. */ + +#include "paddle/fluid/operators/share_data_op.h" + +REGISTER_OP_CUDA_KERNEL( + share_data, paddle::operators::ShareDataKernel, + paddle::operators::ShareDataKernel, + paddle::operators::ShareDataKernel, + paddle::operators::ShareDataKernel, + paddle::operators::ShareDataKernel, + paddle::operators::ShareDataKernel, + paddle::operators::ShareDataKernel, + paddle::operators::ShareDataKernel); diff --git a/paddle/fluid/operators/share_data_op.h b/paddle/fluid/operators/share_data_op.h new file mode 100644 index 00000000000000..d876b4fabd5c09 --- /dev/null +++ b/paddle/fluid/operators/share_data_op.h @@ -0,0 +1,41 @@ +/* Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. + +Licensed under the Apache License, Version 2.0 (the "License"); +you may not use this file except in compliance with the License. +You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + +Unless required by applicable law or agreed to in writing, software +distributed under the License is distributed on an "AS IS" BASIS, +WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +See the License for the specific language governing permissions and +limitations under the License. */ + +#pragma once +#include "paddle/fluid/framework/op_registry.h" + +namespace paddle { +namespace operators { + +template +class ShareDataKernel : public framework::OpKernel { + public: + void Compute(const framework::ExecutionContext &ctx) const override { + auto *in_var = ctx.InputVar("X"); + auto *out_var = ctx.OutputVar("Out"); + if (in_var->IsType()) { + const auto &origin_tensor = in_var->Get(); + auto *detach_tensor = out_var->GetMutable(); + detach_tensor->ShareDataWith(origin_tensor); + } else { + const auto &origin_selected_rows = in_var->Get(); + auto *detach_selected_rows = + out_var->GetMutable(); + detach_selected_rows->mutable_value()->ShareDataWith( + origin_selected_rows.value()); + } + } +}; +} // namespace operators +} // namespace paddle diff --git a/paddle/fluid/operators/softmax_op.h b/paddle/fluid/operators/softmax_op.h index 08266318fb970b..68a1649d0a039d 100644 --- a/paddle/fluid/operators/softmax_op.h +++ b/paddle/fluid/operators/softmax_op.h @@ -65,6 +65,9 @@ class SoftmaxKernel : public framework::OpKernel { // allocate memory on device. Out->mutable_data(context.GetPlace()); + if (Out->numel() == 0) { + return; + } const int n = SizeToAxis(axis, X->dims()); const int d = SizeFromAxis(axis, X->dims()); @@ -97,6 +100,9 @@ class SoftmaxGradKernel : public framework::OpKernel { // allocate memory on device. dX->mutable_data(context.GetPlace()); + if (dX->numel() == 0) { + return; + } const int n = SizeToAxis(axis, dX->dims()); const int d = SizeFromAxis(axis, dX->dims()); diff --git a/paddle/fluid/operators/softmax_op_xpu.cc b/paddle/fluid/operators/softmax_op_xpu.cc index ed7034ef6ab416..3527478f766105 100644 --- a/paddle/fluid/operators/softmax_op_xpu.cc +++ b/paddle/fluid/operators/softmax_op_xpu.cc @@ -47,8 +47,8 @@ class SoftmaxXPUKernel : public framework::OpKernel { int len = x->numel(); T* clip_x_data = clip_x.mutable_data(context.GetPlace(), len * sizeof(T)); - r = xpu::clip(dev_ctx.x_context(), x->data(), clip_x_data, len, - -1e30, 1e30); + r = xpu::clip_v2(dev_ctx.x_context(), x->data(), clip_x_data, len, + static_cast(-1e20), static_cast(1e20)); PADDLE_ENFORCE_EQ(r, XPU_SUCCESS, platform::errors::External("XPU API(clip) return wrong " "value[%d %s]", diff --git a/paddle/fluid/operators/softmax_with_cross_entropy_op_xpu.cc b/paddle/fluid/operators/softmax_with_cross_entropy_op_xpu.cc index 8635def2ecf138..a79e31eb8d028d 100644 --- a/paddle/fluid/operators/softmax_with_cross_entropy_op_xpu.cc +++ b/paddle/fluid/operators/softmax_with_cross_entropy_op_xpu.cc @@ -54,8 +54,9 @@ class SoftmaxWithCrossEntropyXPUKernel : public framework::OpKernel { int len = logits->numel(); T* clip_logits_data = clip_logits.mutable_data(context.GetPlace(), len * sizeof(T)); - r = xpu::clip(dev_ctx.x_context(), logits->data(), clip_logits_data, - len, -1e30, 1e30); + r = xpu::clip_v2(dev_ctx.x_context(), logits->data(), + clip_logits_data, len, static_cast(-1e20), + static_cast(1e20)); PADDLE_ENFORCE_EQ( r, xpu::Error_t::SUCCESS, platform::errors::External("XPU kernel error. clip " diff --git a/paddle/fluid/operators/split_op.cc b/paddle/fluid/operators/split_op.cc index 0151778075de04..661e4ca727beec 100644 --- a/paddle/fluid/operators/split_op.cc +++ b/paddle/fluid/operators/split_op.cc @@ -73,8 +73,17 @@ class SplitOp : public framework::OperatorWithKernel { protected: framework::OpKernelType GetExpectedKernelType( const framework::ExecutionContext &ctx) const override { - return framework::OpKernelType(ctx.Input("X")->type(), - ctx.device_context()); + auto input_data_type = + framework::OperatorWithKernel::IndicateVarDataType(ctx, "X"); + +#ifdef PADDLE_WITH_MKLDNN + if (this->CanMKLDNNBeUsed(ctx, input_data_type)) { + return framework::OpKernelType(input_data_type, ctx.GetPlace(), + framework::DataLayout::kMKLDNN, + framework::LibraryType::kMKLDNN); + } +#endif + return framework::OpKernelType(input_data_type, ctx.GetPlace()); } framework::OpKernelType GetKernelTypeForVar( @@ -136,6 +145,14 @@ This operator splits the input tensor into multiple sub-tensors. "(int, default 0) " "The axis which the input will be split on.") .SetDefault(0); + AddAttr("use_mkldnn", + "(bool, default false) Only used in mkldnn kernel") + .SetDefault(false); + AddAttr( + "mkldnn_data_type", + "(string, default \"float32\"). Data type of mkldnn kernel") + .SetDefault("float32") + .InEnum({"float32", "bfloat16"}); } }; diff --git a/paddle/fluid/operators/test_common_infer_shape_functions.cc b/paddle/fluid/operators/test_common_infer_shape_functions.cc index ca8f6ce84fc571..60eeb66ae7d1ec 100644 --- a/paddle/fluid/operators/test_common_infer_shape_functions.cc +++ b/paddle/fluid/operators/test_common_infer_shape_functions.cc @@ -48,7 +48,7 @@ class DygraphInferShapeTest { void SetOpType(const std::string& op_type) { op_type_ = op_type; } void Run(std::function infer_shape) { imperative::DygraphInferShapeContext ctx( - &ins_, &outs_, &attrs_, op_type_); + &ins_, &outs_, &attrs_, {}, op_type_); infer_shape(&ctx); for (const auto& pair : expected_dims_) { auto out = outs_[pair.first][0]; diff --git a/paddle/fluid/operators/trace_op.cu b/paddle/fluid/operators/trace_op.cu index 336c1c40832b97..f3fe32e10a52b6 100644 --- a/paddle/fluid/operators/trace_op.cu +++ b/paddle/fluid/operators/trace_op.cu @@ -14,6 +14,7 @@ #include #include +#include "paddle/fluid/operators/math/math_function.h" #include "paddle/fluid/operators/reduce_ops/cub_reduce.h" #include "paddle/fluid/operators/trace_op.h" @@ -50,6 +51,9 @@ class TraceCUDAKernel : public framework::OpKernel { TensorReduce( diag, out, reduce_dims, static_cast(0), cub::Sum(), IdentityFunctor(), stream); + } else { + math::SetConstant functor; + functor(context.device_context(), out, static_cast(0)); } } }; diff --git a/paddle/fluid/operators/trace_op.h b/paddle/fluid/operators/trace_op.h index b7a6e559ed4ef6..ca9439cbed97dd 100644 --- a/paddle/fluid/operators/trace_op.h +++ b/paddle/fluid/operators/trace_op.h @@ -179,7 +179,7 @@ class TraceKernel : public framework::OpKernel { auto output_dims = out->dims(); - out->mutable_data(context.GetPlace()); + T* out_data = out->mutable_data(context.GetPlace()); const framework::Tensor diag = Diagonal(context, input, offset, dim1, dim2); @@ -191,6 +191,8 @@ class TraceKernel : public framework::OpKernel { auto reduce_dim = Eigen::array({1}); output.device(place) = x.sum(reduce_dim); out->Resize(output_dims); + } else { + std::fill(out_data, out_data + out->numel(), static_cast(0)); } } }; diff --git a/paddle/fluid/platform/dynload/dynamic_loader.cc b/paddle/fluid/platform/dynload/dynamic_loader.cc index be9cda4a2e9b6c..f0a46e0818af74 100644 --- a/paddle/fluid/platform/dynload/dynamic_loader.cc +++ b/paddle/fluid/platform/dynload/dynamic_loader.cc @@ -219,17 +219,17 @@ static inline void* GetDsoHandleFromSearchPath( for (auto dso : dso_names) { // 1. search in user config path by FLAGS dso_handle = GetDsoHandleFromSpecificPath(config_path, dso, dynload_flags); - // 2. search in extra paths + // 2. search in system default path + if (nullptr == dso_handle) { + dso_handle = GetDsoHandleFromDefaultPath(dso, dynload_flags); + } + // 3. search in extra paths if (nullptr == dso_handle) { for (auto path : extra_paths) { VLOG(3) << "extra_paths: " << path; dso_handle = GetDsoHandleFromSpecificPath(path, dso, dynload_flags); } } - // 3. search in system default path - if (nullptr == dso_handle) { - dso_handle = GetDsoHandleFromDefaultPath(dso, dynload_flags); - } if (nullptr != dso_handle) break; } diff --git a/paddle/fluid/platform/mkldnn_reuse.h b/paddle/fluid/platform/mkldnn_reuse.h index 514c0b3d3ce7f8..58622fb2529b83 100644 --- a/paddle/fluid/platform/mkldnn_reuse.h +++ b/paddle/fluid/platform/mkldnn_reuse.h @@ -599,17 +599,8 @@ class BinaryMKLDNNHandler : public platform::MKLDNNHandlerT { const std::string& uniq_name) : platform::MKLDNNHandlerT( dev_ctx, engine, cpu_place, - platform::CreateKey( - dev_ctx, framework::vectorize(x->dims()), uniq_name, - (algo == dnnl::algorithm::binary_mul ? "M" : ""))) { - // bradcasting combined with in-place may require - auto rankdiff = x->dims().size() - y->dims().size(); - if (rankdiff > 0) { - auto suffix = std::to_string(rankdiff); - this->key_ += suffix; - this->key_common_ += suffix; - } - + platform::CreateKey(dev_ctx, framework::vectorize(x->dims()), + uniq_name)) { if (!this->isCached()) { PADDLE_ENFORCE_EQ( x->layout(), DataLayout::kMKLDNN, @@ -629,18 +620,24 @@ class BinaryMKLDNNHandler : public platform::MKLDNNHandlerT { const auto src_y_tz = framework::vectorize(y->dims()); // if output tensor(z) is nullptr then we are computing into oneDNN // managed buffer - const auto dst_tz = - (z == nullptr) ? src_x_tz : framework::vectorize(z->dims()); + auto rankdiff = x->dims().size() - y->dims().size(); + const auto dst_tz = (z == nullptr) ? (rankdiff > 0 ? src_x_tz : src_y_tz) + : framework::vectorize(z->dims()); - const auto src0_md = dnnl::memory::desc( + auto src0_md = dnnl::memory::desc( src_x_tz, platform::MKLDNNGetDataType(), x->format()); auto src1_md = dnnl::memory::desc( src_y_tz, platform::MKLDNNGetDataType(), y->format()); - if (rankdiff > 0) { + if (rankdiff > 0) { // Second input is of smaller rank than first std::vector dims1_ex(rankdiff, 1); dims1_ex.insert(next(dims1_ex.begin(), (axis == -1 ? rankdiff : axis)), src_y_tz.begin(), src_y_tz.end()); src1_md = src1_md.reshape(dims1_ex); + } else if (rankdiff < 0) { // First input is of smaller than second + std::vector dims0_ex(-rankdiff, 1); + dims0_ex.insert(next(dims0_ex.begin(), (axis == -1 ? -rankdiff : axis)), + src_x_tz.begin(), src_x_tz.end()); + src0_md = src0_md.reshape(dims0_ex); } const auto dst_md = memory::desc(dst_tz, platform::MKLDNNGetDataType(), MKLDNNMemoryFormat::any); @@ -1023,6 +1020,27 @@ class ReorderMKLDNNHandler : public MKLDNNHandler { return this->AcquireMemory(dims_, dtype_, fmt, ptr, "@user_src_mem_p"); } + std::shared_ptr AcquireSrcSubmemory( + const std::vector& dims, const std::vector& offset, + const std::shared_ptr& mem_p, int submemory_number) { + std::string local_key = key_; + local_key.append("@submem") + .append(std::to_string(submemory_number)) + .append("_p"); + + auto sub_mem_p = + std::static_pointer_cast(dev_ctx_.GetBlob(local_key)); + if (sub_mem_p == nullptr) { + auto sub_md = mem_p->get_desc().submemory_desc(dims, {offset}); + sub_mem_p = std::make_shared(sub_md, engine_, + mem_p->get_data_handle()); + dev_ctx_.SetBlob(local_key, sub_mem_p); + } else { + sub_mem_p->set_data_handle(mem_p->get_data_handle()); + } + return sub_mem_p; + } + std::shared_ptr AcquireDstMemory( framework::Tensor* output, const MKLDNNMemoryFormat& fmt, platform::Place place) { @@ -1045,6 +1063,44 @@ class ReorderMKLDNNHandler : public MKLDNNHandler { return mem_p; } + std::shared_ptr AcquireDstMemory( + framework::Tensor* output, const std::vector& dims, + const int memory_number, const MKLDNNMemoryFormat& fmt, + platform::Place place) { + auto local_key = + key_ + "@user_dst_mem" + std::to_string(memory_number) + "_p"; + auto mem_p = + std::static_pointer_cast(dev_ctx_.GetBlob(local_key)); + if (mem_p == nullptr) { + auto dst_md = platform::MKLDNNMemDesc(dims, dtype_dst_, fmt); + auto dst_data = + output->mutable_data(place, vtype_dst_, dst_md.get_size()); + + mem_p = std::make_shared(dst_md, engine_, dst_data); + dev_ctx_.SetBlob(local_key, mem_p); + } else { + // Even if memory object exists , we may be using it for diffrent tensor + auto dst_data = + output->mutable_data(place, vtype_dst_, mem_p->get_desc().get_size()); + mem_p->set_data_handle(dst_data); + } + return mem_p; + } + + std::shared_ptr AcquireReorder( + std::shared_ptr dst_memory_p, + std::shared_ptr src_memory_p, int reorder_number) { + auto prim_key = key_ + "@reorder" + std::to_string(reorder_number) + "_p"; + auto reorder_p = + std::static_pointer_cast(dev_ctx_.GetBlob(prim_key)); + if (reorder_p == nullptr) { + reorder_p = + std::make_shared(*(src_memory_p), *(dst_memory_p)); + dev_ctx_.SetBlob(prim_key, reorder_p); + } + return reorder_p; + } + std::shared_ptr AcquireReorder( std::shared_ptr dst_memory_p, std::shared_ptr src_memory_p) { diff --git a/paddle/fluid/platform/xpu_header.h b/paddle/fluid/platform/xpu_header.h index 9f2befc123f224..99f4224b5d408a 100644 --- a/paddle/fluid/platform/xpu_header.h +++ b/paddle/fluid/platform/xpu_header.h @@ -1,4 +1,4 @@ -// Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. +// Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. @@ -20,6 +20,7 @@ #include #include "paddle/fluid/platform/errors.h" +#include "paddle/fluid/platform/float16.h" #include "xpu/api.h" #include "xpu/refactor/fusion.h" #include "xpu/refactor/math.h" @@ -58,4 +59,16 @@ static std::map XPUAPIErrorMsg = { {xpu::Error_t::RUNTIME_ERROR, "xpu api runtime error"}, {xpu::Error_t::NO_ENOUGH_WORKSPACE, "xpu api no enough workspace"}}; +template +class XPUTypeTrait { + public: + using Type = T; +}; + +template <> +class XPUTypeTrait { + public: + using Type = float16; +}; + #endif diff --git a/paddle/fluid/pybind/imperative.cc b/paddle/fluid/pybind/imperative.cc index 816281ce8a00d4..af7f03dc197166 100644 --- a/paddle/fluid/pybind/imperative.cc +++ b/paddle/fluid/pybind/imperative.cc @@ -135,12 +135,14 @@ static const platform::Place PyObjectToPlace(const py::object &place_obj) { return place_obj.cast(); } else if (py::isinstance(place_obj)) { return place_obj.cast(); + } else if (py::isinstance(place_obj)) { + return place_obj.cast(); } else if (py::isinstance(place_obj)) { return place_obj.cast(); } else { PADDLE_THROW(platform::errors::InvalidArgument( "Place should be one of " - "Place/CPUPlace/XPUPlace/CUDAPlace/CUDAPinnedPlace")); + "Place/CPUPlace/XPUPlace/CUDAPlace/CUDAPinnedPlace/NPUPlace")); } } @@ -172,9 +174,13 @@ static void InitTensorForVarBase(imperative::VarBase *self, SetTensorFromPyArray( tensor, array, BOOST_GET_CONST(platform::CUDAPinnedPlace, place), zero_copy); + } else if (platform::is_npu_place(place)) { + SetTensorFromPyArray( + tensor, array, BOOST_GET_CONST(platform::NPUPlace, place), zero_copy); } else { PADDLE_THROW(platform::errors::InvalidArgument( - "Place should be one of CPUPlace/XPUPlace/CUDAPlace/CUDAPinnedPlace")); + "Place should be one of " + "CPUPlace/XPUPlace/CUDAPlace/CUDAPinnedPlace/NPUPlace")); } if (stop_gradient != -1) { self->SetOverridedStopGradient(stop_gradient); @@ -718,6 +724,10 @@ void BindImperative(py::module *m_ptr) { py::arg("value"), py::arg("place"), py::arg("persistable") = false, py::arg("zero_copy") = false, py::arg("name") = "", py::arg("stop_gradient") = -1) + .def("__init__", &InitVarBaseFromNumpyWithArg, + py::arg("value"), py::arg("place"), py::arg("persistable") = false, + py::arg("zero_copy") = false, py::arg("name") = "", + py::arg("stop_gradient") = -1) .def("__init__", &InitVarBaseFromNumpyWithArgDefault, py::arg("value")) .def("__init__", &InitVarBaseFromTensorWithArgDefault, py::arg("tensor")) .def("__init__", &InitVarBaseFromNumpyWithKwargs) @@ -1452,6 +1462,16 @@ void BindImperative(py::module *m_ptr) { return new_var; }, py::return_value_policy::copy) + .def("_copy_to", + [](const std::shared_ptr &self, + const platform::NPUPlace &place, bool blocking) { + auto new_var = self->NewVarBase(place, blocking); + if (!blocking) { + IncreaseVarbaseReferenceCountUntilCopyComplete(self, place); + } + return new_var; + }, + py::return_value_policy::copy) .def("_copy_to", [](const std::shared_ptr &self, const platform::Place &place, bool blocking) { @@ -1578,6 +1598,11 @@ void BindImperative(py::module *m_ptr) { self.SetExpectedPlace(*p); VLOG(4) << "Tracer(" << &self << ")" << " set expected place " << *p; + } else if (py::isinstance(obj)) { + auto p = obj.cast(); + self.SetExpectedPlace(*p); + VLOG(4) << "Tracer(" << &self << ")" + << " set expected place " << *p; } else if (py::isinstance(obj)) { auto p = obj.cast(); self.SetExpectedPlace(*p); @@ -1586,7 +1611,7 @@ void BindImperative(py::module *m_ptr) { } else { PADDLE_THROW(platform::errors::InvalidArgument( "Incompatible Place Type: supports XPUPlace, CUDAPlace, " - "CPUPlace, " + "CPUPlace, NPUPlace" "and CUDAPinnedPlace, " "but got Unknown Type!")); } @@ -1647,6 +1672,19 @@ void BindImperative(py::module *m_ptr) { std::move(attrs), place, trace_backward); } }) + .def("trace", + [](imperative::Tracer &self, const std::string &type, + const PyNameVarBaseMap &ins, const PyNameVarBaseMap &outs, + framework::AttributeMap attrs, const platform::NPUPlace &place, + bool trace_backward) { + auto ins_map = ConvertToNameVarBaseMap(ins); + auto outs_map = ConvertToNameVarBaseMap(outs); + { + py::gil_scoped_release release; + self.TraceOp(type, std::move(ins_map), std::move(outs_map), + std::move(attrs), place, trace_backward); + } + }) .def("trace", [](imperative::Tracer &self, const std::string &type, const PyNameVarBaseMap &ins, const PyNameVarBaseMap &outs, @@ -1704,6 +1742,7 @@ void BindImperative(py::module *m_ptr) { m.def("varbase_copy", &VarBaseCopy); m.def("varbase_copy", &VarBaseCopy); m.def("varbase_copy", &VarBaseCopy); + m.def("varbase_copy", &VarBaseCopy); m.def( "dygraph_partial_grad", @@ -1804,6 +1843,12 @@ void BindImperative(py::module *m_ptr) { const py::args args, const py::kwargs kwargs) { return imperative::PyLayerApply(place, cls, args, kwargs); }); + + m.def("pylayer_apply", + [](const platform::NPUPlace &place, const py::object &cls, + const py::args args, const py::kwargs kwargs) { + return imperative::PyLayerApply(place, cls, args, kwargs); + }); } } // namespace pybind diff --git a/paddle/fluid/pybind/op_function.h b/paddle/fluid/pybind/op_function.h index e0886ac144ab13..eaa70adcc89fe4 100644 --- a/paddle/fluid/pybind/op_function.h +++ b/paddle/fluid/pybind/op_function.h @@ -209,11 +209,16 @@ inline bool PyObject_CheckLongOrToLong(PyObject** obj) { PyObject_IsInstance(*obj, (PyObject*)g_varbase_pytype)) { // NOLINT return true; } - auto to = PyNumber_Long(*obj); - if (to) { - *obj = to; - return true; + + if (std::string(((PyTypeObject*)(*obj)->ob_type)->tp_name) // NOLINT + .find("numpy") != std::string::npos) { + auto to = PyNumber_Long(*obj); + if (to) { + *obj = to; + return true; + } } + return false; } @@ -223,10 +228,13 @@ inline bool PyObject_CheckFloatOrToFloat(PyObject** obj) { PyObject_IsInstance(*obj, (PyObject*)g_varbase_pytype)) { // NOLINT return true; } - auto to = PyNumber_Float(*obj); - if (to) { - *obj = to; - return true; + if (std::string(((PyTypeObject*)(*obj)->ob_type)->tp_name) // NOLINT + .find("numpy") != std::string::npos) { + auto to = PyNumber_Float(*obj); + if (to) { + *obj = to; + return true; + } } return false; } diff --git a/paddle/fluid/pybind/op_function_generator.cc b/paddle/fluid/pybind/op_function_generator.cc index 619f14c30f1b71..b2205391a253c3 100644 --- a/paddle/fluid/pybind/op_function_generator.cc +++ b/paddle/fluid/pybind/op_function_generator.cc @@ -268,7 +268,7 @@ static PyObject * %s(PyObject *self, PyObject *args, PyObject *kwargs) imperative::GetCurrentTracer()->TraceOp("%s", ins, outs, attrs, {%s}); PyEval_RestoreThread(tstate); tstate = nullptr; - return %s; + %s } catch(...) { if (tstate) { @@ -488,13 +488,13 @@ std::string GenerateOpFunctionsBody( viwe_input_name, viwe_output_name); } if (outs_num == 0) { - return_str = "Py_None"; + return_str = "Py_INCREF(Py_None);\n return Py_None;"; } else if (outs_num == 1) { - return_str = "MakeReturnPyObject(" + return_str + ")"; + return_str = "return MakeReturnPyObject(" + return_str + ");"; } else { - return_str = "MakeReturnPyObject(" + + return_str = "return MakeReturnPyObject(" + paddle::string::Sprintf(RETURN_TUPLE_TEMPLATE, return_str) + - ")"; + ");"; } std::string function_args = ""; if (input_args == "") { diff --git a/paddle/fluid/pybind/ps_gpu_wrapper_py.cc b/paddle/fluid/pybind/ps_gpu_wrapper_py.cc index 0c239f8157e5df..bdd7abe1d8332a 100644 --- a/paddle/fluid/pybind/ps_gpu_wrapper_py.cc +++ b/paddle/fluid/pybind/ps_gpu_wrapper_py.cc @@ -48,6 +48,8 @@ void BindPSGPUWrapper(py::module* m) { .def("end_pass", &framework::PSGPUWrapper::EndPass, py::call_guard()) .def("build_gpu_ps", &framework::PSGPUWrapper::BuildGPUPS, + py::call_guard()) + .def("finalize", &framework::PSGPUWrapper::Finalize, py::call_guard()); } // end PSGPUWrapper #endif diff --git a/paddle/fluid/pybind/pybind.cc b/paddle/fluid/pybind/pybind.cc index 86084297c4ae65..a93ce4ecd48260 100644 --- a/paddle/fluid/pybind/pybind.cc +++ b/paddle/fluid/pybind/pybind.cc @@ -225,7 +225,9 @@ OpSupportedInfos(const std::string &place, [](unsigned char c) { return std::toupper(c); }); using fn_type = std::add_pointer::type; std::unordered_map is_target_place{ - {"GPU", &platform::is_gpu_place}, {"CPU", &platform::is_cpu_place}, + {"GPU", &platform::is_gpu_place}, + {"CPU", &platform::is_cpu_place}, + {"XPU", &platform::is_xpu_place}, }; PADDLE_ENFORCE_NE( is_target_place.count(query_place), 0, @@ -1308,7 +1310,7 @@ All parameter, weight, gradient are variables in Paddle. if (info != nullptr) { if (info->HasOpProtoAndChecker()) { auto op_checker = info->Checker(); - res = op_checker->GetAttrsDefaultValuesMap(); + res = op_checker->GetDefaultAttrsMap(); } } return res; @@ -1716,6 +1718,8 @@ All parameter, weight, gradient are variables in Paddle. .def("_equals", &IsSamePlace) .def("_equals", &IsSamePlace) + .def("get_device_id", + [](const platform::NPUPlace &self) { return self.GetDeviceId(); }) .def("__str__", string::to_string); py::class_(m, "Place") diff --git a/paddle/scripts/paddle_build.bat b/paddle/scripts/paddle_build.bat index ede4003bd86d25..b1cdfbaf6f21b4 100644 --- a/paddle/scripts/paddle_build.bat +++ b/paddle/scripts/paddle_build.bat @@ -26,22 +26,22 @@ if not defined cache_dir set cache_dir=%work_dir:Paddle=cache% if not exist %cache_dir%\tools ( git clone https://github.com/zhouwei25/tools.git %cache_dir%\tools ) -taskkill /f /im cmake.exe 2>NUL -taskkill /f /im ninja.exe 2>NUL -taskkill /f /im MSBuild.exe 2>NUL -taskkill /f /im cl.exe 2>NUL -taskkill /f /im lib.exe 2>NUL -taskkill /f /im link.exe 2>NUL -taskkill /f /im vctip.exe 2>NUL -taskkill /f /im cvtres.exe 2>NUL -taskkill /f /im rc.exe 2>NUL -taskkill /f /im mspdbsrv.exe 2>NUL -taskkill /f /im csc.exe 2>NUL -taskkill /f /im python.exe 2>NUL -taskkill /f /im nvcc.exe 2>NUL -taskkill /f /im cicc.exe 2>NUL -taskkill /f /im ptxas.exe 2>NUL -taskkill /f /im op_function_generator.exe 2>NUL +taskkill /f /im cmake.exe /t 2>NUL +taskkill /f /im ninja.exe /t 2>NUL +taskkill /f /im MSBuild.exe /t 2>NUL +taskkill /f /im cl.exe /t 2>NUL +taskkill /f /im lib.exe /t 2>NUL +taskkill /f /im link.exe /t 2>NUL +taskkill /f /im vctip.exe /t 2>NUL +taskkill /f /im cvtres.exe /t 2>NUL +taskkill /f /im rc.exe /t 2>NUL +taskkill /f /im mspdbsrv.exe /t 2>NUL +taskkill /f /im csc.exe /t 2>NUL +taskkill /f /im python.exe /t 2>NUL +taskkill /f /im nvcc.exe /t 2>NUL +taskkill /f /im cicc.exe /t 2>NUL +taskkill /f /im ptxas.exe /t 2>NUL +taskkill /f /im op_function_generator.exe /t 2>NUL wmic process where name="op_function_generator.exe" call terminate 2>NUL wmic process where name="cvtres.exe" call terminate 2>NUL wmic process where name="rc.exe" call terminate 2>NUL @@ -72,7 +72,7 @@ if not defined INFERENCE_DEMO_INSTALL_DIR set INFERENCE_DEMO_INSTALL_DIR=%cache_ if not defined LOG_LEVEL set LOG_LEVEL=normal if not defined PRECISION_TEST set PRECISION_TEST=OFF if not defined NIGHTLY_MODE set PRECISION_TEST=OFF -if not defined retry_times set retry_times=2 +if not defined retry_times set retry_times=3 if not defined PYTHON_ROOT set PYTHON_ROOT=C:\Python37 rem -------set cache build directory----------- @@ -193,7 +193,7 @@ rem ------Build windows avx whl package------ set WITH_AVX=ON set ON_INFER=OFF set CUDA_ARCH_NAME=All -set retry_times=3 +set retry_times=4 call :cmake || goto cmake_error call :build || goto build_error @@ -205,7 +205,7 @@ rem ------Build windows no-avx whl package------ set WITH_AVX=OFF set ON_INFER=OFF set CUDA_ARCH_NAME=All -set retry_times=3 +set retry_times=4 call :cmake || goto cmake_error call :build || goto build_error @@ -366,18 +366,26 @@ echo ======================================== for /F %%# in ('wmic cpu get NumberOfLogicalProcessors^|findstr [0-9]') do set /a PARALLEL_PROJECT_COUNT=%%#*4/5 echo "PARALLEL PROJECT COUNT is %PARALLEL_PROJECT_COUNT%" + set build_times=1 +rem MSbuild will build third_party first to improve compiler stability. +if NOT %GENERATOR% == "Ninja" ( + goto :build_tp +) else ( + goto :build_paddle +) + :build_tp echo Build third_party the %build_times% time: - if %GENERATOR% == "Ninja" ( ninja third_party ) else ( MSBuild /m /p:PreferredToolArchitecture=x64 /p:Configuration=Release /verbosity:%LOG_LEVEL% third_party.vcxproj ) + if %ERRORLEVEL% NEQ 0 ( set /a build_times=%build_times%+1 - if %build_times% GTR %retry_times% ( + if %build_times% GEQ %retry_times% ( exit /b 7 ) else ( echo Build third_party failed, will retry! @@ -392,20 +400,20 @@ set build_times=1 rem clcache.exe -z rem -------clean up environment again----------- -taskkill /f /im cmake.exe 2>NUL -taskkill /f /im MSBuild.exe 2>NUL -taskkill /f /im cl.exe 2>NUL -taskkill /f /im lib.exe 2>NUL -taskkill /f /im link.exe 2>NUL -taskkill /f /im vctip.exe 2>NUL -taskkill /f /im cvtres.exe 2>NUL -taskkill /f /im rc.exe 2>NUL -taskkill /f /im mspdbsrv.exe 2>NUL -taskkill /f /im csc.exe 2>NUL -taskkill /f /im nvcc.exe 2>NUL -taskkill /f /im cicc.exe 2>NUL -taskkill /f /im ptxas.exe 2>NUL -taskkill /f /im op_function_generator.exe 2>NUL +taskkill /f /im cmake.exe /t 2>NUL +taskkill /f /im MSBuild.exe /t 2>NUL +taskkill /f /im cl.exe /t 2>NUL +taskkill /f /im lib.exe /t 2>NUL +taskkill /f /im link.exe /t 2>NUL +taskkill /f /im vctip.exe /t 2>NUL +taskkill /f /im cvtres.exe /t 2>NUL +taskkill /f /im rc.exe /t 2>NUL +taskkill /f /im mspdbsrv.exe /t 2>NUL +taskkill /f /im csc.exe /t 2>NUL +taskkill /f /im nvcc.exe /t 2>NUL +taskkill /f /im cicc.exe /t 2>NUL +taskkill /f /im ptxas.exe /t 2>NUL +taskkill /f /im op_function_generator.exe /t 2>NUL wmic process where name="cmake.exe" call terminate 2>NUL wmic process where name="op_function_generator.exe" call terminate 2>NUL wmic process where name="cvtres.exe" call terminate 2>NUL @@ -414,7 +422,7 @@ wmic process where name="cl.exe" call terminate 2>NUL wmic process where name="lib.exe" call terminate 2>NUL if "%WITH_TESTING%"=="ON" ( - for /F "tokens=1 delims= " %%# in ('tasklist ^| findstr /i test') do taskkill /f /im %%# + for /F "tokens=1 delims= " %%# in ('tasklist ^| findstr /i test') do taskkill /f /im %%# /t ) echo Build Paddle the %build_times% time: @@ -430,7 +438,7 @@ if %GENERATOR% == "Ninja" ( if %ERRORLEVEL% NEQ 0 ( set /a build_times=%build_times%+1 - if %build_times% GTR %retry_times% ( + if %build_times% GEQ %retry_times% ( exit /b 7 ) else ( echo Build Paddle failed, will retry! @@ -783,24 +791,24 @@ rem ---------------------------------------------------------------------------- echo ======================================== echo Clean up environment at the end ... echo ======================================== -taskkill /f /im cmake.exe 2>NUL -taskkill /f /im ninja.exe 2>NUL -taskkill /f /im MSBuild.exe 2>NUL -taskkill /f /im git.exe 2>NUL -taskkill /f /im cl.exe 2>NUL -taskkill /f /im lib.exe 2>NUL -taskkill /f /im link.exe 2>NUL -taskkill /f /im git-remote-https.exe 2>NUL -taskkill /f /im vctip.exe 2>NUL -taskkill /f /im cvtres.exe 2>NUL -taskkill /f /im rc.exe 2>NUL -taskkill /f /im mspdbsrv.exe 2>NUL -taskkill /f /im csc.exe 2>NUL -taskkill /f /im python.exe 2>NUL -taskkill /f /im nvcc.exe 2>NUL -taskkill /f /im cicc.exe 2>NUL -taskkill /f /im ptxas.exe 2>NUL -taskkill /f /im op_function_generator.exe 2>NUL +taskkill /f /im cmake.exe /t 2>NUL +taskkill /f /im ninja.exe /t 2>NUL +taskkill /f /im MSBuild.exe /t 2>NUL +taskkill /f /im git.exe /t 2>NUL +taskkill /f /im cl.exe /t 2>NUL +taskkill /f /im lib.exe /t 2>NUL +taskkill /f /im link.exe /t 2>NUL +taskkill /f /im git-remote-https.exe /t 2>NUL +taskkill /f /im vctip.exe /t 2>NUL +taskkill /f /im cvtres.exe /t 2>NUL +taskkill /f /im rc.exe /t 2>NUL +taskkill /f /im mspdbsrv.exe /t 2>NUL +taskkill /f /im csc.exe /t 2>NUL +taskkill /f /im python.exe /t 2>NUL +taskkill /f /im nvcc.exe /t 2>NUL +taskkill /f /im cicc.exe /t 2>NUL +taskkill /f /im ptxas.exe /t 2>NUL +taskkill /f /im op_function_generator.exe /t 2>NUL wmic process where name="op_function_generator.exe" call terminate 2>NUL wmic process where name="cvtres.exe" call terminate 2>NUL wmic process where name="rc.exe" call terminate 2>NUL @@ -808,7 +816,7 @@ wmic process where name="cl.exe" call terminate 2>NUL wmic process where name="lib.exe" call terminate 2>NUL wmic process where name="python.exe" call terminate 2>NUL if "%WITH_TESTING%"=="ON" ( - for /F "tokens=1 delims= " %%# in ('tasklist ^| findstr /i test') do taskkill /f /im %%# + for /F "tokens=1 delims= " %%# in ('tasklist ^| findstr /i test') do taskkill /f /im %%# /t ) echo Windows CI run successfully! exit /b 0 diff --git a/paddle/scripts/paddle_build.sh b/paddle/scripts/paddle_build.sh index a0e630818d853f..309db1c6ee8c55 100755 --- a/paddle/scripts/paddle_build.sh +++ b/paddle/scripts/paddle_build.sh @@ -79,37 +79,12 @@ function cmake_base() { # Delete previous built whl packages rm -rf python/dist 2>/dev/null || true - # Support build for all python versions, currently - # including cp27-cp27m and cp27-cp27mu. + # Support build for all python3 versions PYTHON_FLAGS="" SYSTEM=`uname -s` if [ "$SYSTEM" == "Darwin" ]; then echo "Using python abi: $1" - if [[ "$1" == "cp27-cp27m" ]] || [[ "$1" == "" ]]; then - if [ -d "/Library/Frameworks/Python.framework/Versions/2.7" ]; then - export LD_LIBRARY_PATH=/Library/Frameworks/Python.framework/Versions/2.7 - export DYLD_LIBRARY_PATH=/Library/Frameworks/Python.framework/Versions/2.7 - export PATH=/Library/Frameworks/Python.framework/Versions/2.7/bin/:${PATH} - PYTHON_FLAGS="-DPYTHON_EXECUTABLE:FILEPATH=/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 - -DPYTHON_INCLUDE_DIR:PATH=/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 - -DPYTHON_LIBRARY:FILEPATH=/Library/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib" - pip install --user -r ${PADDLE_ROOT}/python/requirements.txt - else - exit 1 - fi - elif [ "$1" == "cp35-cp35m" ]; then - if [ -d "/Library/Frameworks/Python.framework/Versions/3.5" ]; then - export LD_LIBRARY_PATH=/Library/Frameworks/Python.framework/Versions/3.5/lib/ - export DYLD_LIBRARY_PATH=/Library/Frameworks/Python.framework/Versions/3.5/lib/ - export PATH=/Library/Frameworks/Python.framework/Versions/3.5/bin/:${PATH} - PYTHON_FLAGS="-DPYTHON_EXECUTABLE:FILEPATH=/Library/Frameworks/Python.framework/Versions/3.5/bin/python3 - -DPYTHON_INCLUDE_DIR:PATH=/Library/Frameworks/Python.framework/Versions/3.5/include/python3.5m/ - -DPYTHON_LIBRARY:FILEPATH=/Library/Frameworks/Python.framework/Versions/3.5/lib/libpython3.5m.dylib" - pip3.5 install --user -r ${PADDLE_ROOT}/python/requirements.txt - else - exit 1 - fi - elif [ "$1" == "cp36-cp36m" ]; then + if [ "$1" == "cp36-cp36m" ] || [ "$1" == "" ]; then if [ -d "/Library/Frameworks/Python.framework/Versions/3.6" ]; then export LD_LIBRARY_PATH=/Library/Frameworks/Python.framework/Versions/3.6/lib/ export DYLD_LIBRARY_PATH=/Library/Frameworks/Python.framework/Versions/3.6/lib/ @@ -161,42 +136,7 @@ function cmake_base() { else if [ "$1" != "" ]; then echo "using python abi: $1" - if [ "$1" == "cp27-cp27m" ]; then - export LD_LIBRARY_PATH=/opt/_internal/cpython-2.7.15-ucs2/lib:${LD_LIBRARY_PATH#/opt/_internal/cpython-2.7.15-ucs4/lib:} - export PATH=/opt/python/cp27-cp27m/bin/:${PATH} - PYTHON_FLAGS="-DPYTHON_EXECUTABLE:FILEPATH=/opt/python/cp27-cp27m/bin/python - -DPYTHON_INCLUDE_DIR:PATH=/opt/python/cp27-cp27m/include/python2.7 - -DPYTHON_LIBRARIES:FILEPATH=/opt/_internal/cpython-2.7.15-ucs2/lib/libpython2.7.so" - pip install -r ${PADDLE_ROOT}/python/requirements.txt - elif [ "$1" == "cp27-cp27mu" ]; then - export LD_LIBRARY_PATH=/opt/_internal/cpython-2.7.15-ucs4/lib:${LD_LIBRARY_PATH#/opt/_internal/cpython-2.7.15-ucs2/lib:} - export PATH=/opt/python/cp27-cp27mu/bin/:${PATH} - PYTHON_FLAGS="-DPYTHON_EXECUTABLE:FILEPATH=/opt/python/cp27-cp27mu/bin/python - -DPYTHON_INCLUDE_DIR:PATH=/opt/python/cp27-cp27mu/include/python2.7 - -DPYTHON_LIBRARIES:FILEPATH=/opt/_internal/cpython-2.7.15-ucs4/lib/libpython2.7.so" - pip install -r ${PADDLE_ROOT}/python/requirements.txt - elif [ "$1" == "cp27-cp27m-gcc82" ]; then - export LD_LIBRARY_PATH=/opt/_internal/cpython-2.7.15-ucs2/lib:${LD_LIBRARY_PATH#/opt/_internal/cpython-2.7.15-ucs4/lib:} - export PATH=/opt/python/cp27-cp27m/bin/:${PATH} - PYTHON_FLAGS="-DPYTHON_EXECUTABLE:FILEPATH=/opt/python/cp27-cp27m/bin/python - -DPYTHON_INCLUDE_DIR:PATH=/opt/python/cp27-cp27m/include/python2.7 - -DPYTHON_LIBRARIES:FILEPATH=/opt/_internal/cpython-2.7.15-ucs2/lib/libpython2.7.so" - pip install -r ${PADDLE_ROOT}/python/requirements.txt - elif [ "$1" == "cp27-cp27mu-gcc82" ]; then - export LD_LIBRARY_PATH=/opt/_internal/cpython-2.7.15-ucs4/lib:${LD_LIBRARY_PATH#/opt/_internal/cpython-2.7.15-ucs2/lib:} - export PATH=/opt/python/cp27-cp27mu/bin/:${PATH} - PYTHON_FLAGS="-DPYTHON_EXECUTABLE:FILEPATH=/opt/python/cp27-cp27mu/bin/python - -DPYTHON_INCLUDE_DIR:PATH=/opt/python/cp27-cp27mu/include/python2.7 - -DPYTHON_LIBRARIES:FILEPATH=/opt/_internal/cpython-2.7.15-ucs4/lib/libpython2.7.so" - pip install -r ${PADDLE_ROOT}/python/requirements.txt - elif [ "$1" == "cp35-cp35m" ]; then - export LD_LIBRARY_PATH=/opt/_internal/cpython-3.5.1/lib/:${LD_LIBRARY_PATH} - export PATH=/opt/_internal/cpython-3.5.1/bin/:${PATH} - export PYTHON_FLAGS="-DPYTHON_EXECUTABLE:FILEPATH=/opt/_internal/cpython-3.5.1/bin/python3 - -DPYTHON_INCLUDE_DIR:PATH=/opt/_internal/cpython-3.5.1/include/python3.5m - -DPYTHON_LIBRARIES:FILEPATH=/opt/_internal/cpython-3.5.1/lib/libpython3.so" - pip3.5 install -r ${PADDLE_ROOT}/python/requirements.txt - elif [ "$1" == "cp36-cp36m" ]; then + if [ "$1" == "cp36-cp36m" ]; then export LD_LIBRARY_PATH=/opt/_internal/cpython-3.6.0/lib/:${LD_LIBRARY_PATH} export PATH=/opt/_internal/cpython-3.6.0/bin/:${PATH} export PYTHON_FLAGS="-DPYTHON_EXECUTABLE:FILEPATH=/opt/_internal/cpython-3.6.0/bin/python3 @@ -604,11 +544,7 @@ EOF set -x set +ex - if [ "$1" == "cp27-cp27m" ]; then - pip uninstall -y paddlepaddle - elif [ "$1" == "cp35-cp35m" ]; then - pip3.5 uninstall -y paddlepaddle - elif [ "$1" == "cp36-cp36m" ]; then + if [ "$1" == "cp36-cp36m" ]; then pip3.6 uninstall -y paddlepaddle elif [ "$1" == "cp37-cp37m" ]; then pip3.7 uninstall -y paddlepaddle @@ -619,13 +555,7 @@ EOF fi set -ex - if [ "$1" == "cp27-cp27m" ]; then - set -e - pip install --user ${INSTALL_PREFIX:-/paddle/build}/opt/paddle/share/wheels/*.whl - python ${PADDLE_ROOT}/paddle/scripts/installation_validate.py - elif [ "$1" == "cp35-cp35m" ]; then - pip3.5 install --user ${INSTALL_PREFIX:-/paddle/build}/opt/paddle/share/wheels/*.whl - elif [ "$1" == "cp36-cp36m" ]; then + if [ "$1" == "cp36-cp36m" ]; then pip3.6 install --user ${INSTALL_PREFIX:-/paddle/build}/opt/paddle/share/wheels/*.whl elif [ "$1" == "cp37-cp37m" ]; then pip3.7 install --user ${INSTALL_PREFIX:-/paddle/build}/opt/paddle/share/wheels/*.whl @@ -675,8 +605,10 @@ EOF if [ $need_retry_ut_count -lt $exec_retry_threshold ];then while ( [ $exec_times -lt $retry_time ] ) do + set +e retry_unittests_record="$retry_unittests_record$failed_test_lists" failed_test_lists_ult=`echo "${failed_test_lists}"` + set -e if [[ "${exec_times}" == "1" ]];then if [[ "${failed_test_lists}" == "" ]];then break @@ -831,11 +763,6 @@ function generate_api_spec() { awk -F '(' '{print $NF}' $spec_path >${spec_path}.doc awk -F '(' '{$NF="";print $0}' $spec_path >${spec_path}.api - if [ "$1" == "cp35-cp35m" ] || [ "$1" == "cp36-cp36m" ] || [ "$1" == "cp37-cp37m" ] || [ "$1" == "cp38-cp38" ] || [ "$1" == "cp39-cp39" ]; then - # Use sed to make python2 and python3 sepc keeps the same - sed -i 's/arg0: str/arg0: unicode/g' $spec_path - sed -i "s/\(.*Transpiler.*\).__init__ (ArgSpec(args=\['self'].*/\1.__init__ /g" $spec_path - fi python ${PADDLE_ROOT}/tools/diff_use_default_grad_op_maker.py \ ${PADDLE_ROOT}/paddle/fluid/op_use_default_grad_maker_${spec_kind}.spec @@ -1297,8 +1224,10 @@ set +x if [ $need_retry_ut_count -lt $exec_retry_threshold ];then while ( [ $exec_times -lt $retry_time ] ) do + set +e retry_unittests_record="$retry_unittests_record$failed_test_lists" failed_test_lists_ult=`echo "${failed_test_lists}" |grep -Po '[^ ].*$'` + set -e if [[ "${exec_times}" == "1" ]];then if [[ "${failed_test_lists}" == "" ]];then break @@ -1450,7 +1379,6 @@ function precise_card_test_single { mkdir ${PADDLE_ROOT}/build/ut_map/$case fi set -x - mkdir ${PADDLE_ROOT}/build/ut_map/$case find paddle/fluid -name '*.gcda'|xargs -I {} cp --path {} ut_map/$case find paddle/fluid -name '*.gcno'|xargs -I {} cp --path {} ut_map/$case python ${PADDLE_ROOT}/tools/get_single_test_cov.py ${PADDLE_ROOT} $case & @@ -1747,70 +1675,38 @@ EOF ref_web=https://paddle-wheel.bj.bcebos.com/${PADDLE_BRANCH}-${ref_gpu}-${ref_mkl} - ref_paddle2=paddlepaddle${install_gpu}-${PADDLE_BRANCH}-cp27-cp27mu-linux_x86_64.whl - ref_paddle35=paddlepaddle${install_gpu}-${PADDLE_BRANCH}-cp35-cp35m-linux_x86_64.whl ref_paddle36=paddlepaddle${install_gpu}-${PADDLE_BRANCH}-cp36-cp36m-linux_x86_64.whl ref_paddle37=paddlepaddle${install_gpu}-${PADDLE_BRANCH}-cp37-cp37m-linux_x86_64.whl ref_paddle38=paddlepaddle${install_gpu}-${PADDLE_BRANCH}-cp38-cp38-linux_x86_64.whl ref_paddle39=paddlepaddle${install_gpu}-${PADDLE_BRANCH}-cp39-cp39-linux_x86_64.whl - ref_paddle2_whl=paddlepaddle${install_gpu}-${PADDLE_BRANCH}-cp27-cp27mu-linux_x86_64.whl - ref_paddle35_whl=paddlepaddle${install_gpu}-${PADDLE_BRANCH}-cp35-cp35m-linux_x86_64.whl ref_paddle36_whl=paddlepaddle${install_gpu}-${PADDLE_BRANCH}-cp36-cp36m-linux_x86_64.whl ref_paddle37_whl=paddlepaddle${install_gpu}-${PADDLE_BRANCH}-cp37-cp37m-linux_x86_64.whl ref_paddle38_whl=paddlepaddle${install_gpu}-${PADDLE_BRANCH}-cp38-cp38-linux_x86_64.whl ref_paddle39_whl=paddlepaddle${install_gpu}-${PADDLE_BRANCH}-cp39-cp39-linux_x86_64.whl if [[ ${PADDLE_BRANCH} != "0.0.0" && ${WITH_MKL} == "ON" && ${WITH_GPU} == "ON" ]]; then - ref_paddle2=paddlepaddle${install_gpu}-${PADDLE_BRANCH}.post${ref_CUDA_MAJOR}${CUDNN_MAJOR}-cp27-cp27mu-linux_x86_64.whl - ref_paddle35=paddlepaddle${install_gpu}-${PADDLE_BRANCH}.post${ref_CUDA_MAJOR}${CUDNN_MAJOR}-cp35-cp35m-linux_x86_64.whl ref_paddle36=paddlepaddle${install_gpu}-${PADDLE_BRANCH}.post${ref_CUDA_MAJOR}${CUDNN_MAJOR}-cp36-cp36m-linux_x86_64.whl ref_paddle37=paddlepaddle${install_gpu}-${PADDLE_BRANCH}.post${ref_CUDA_MAJOR}${CUDNN_MAJOR}-cp37-cp37m-linux_x86_64.whl ref_paddle38=paddlepaddle${install_gpu}-${PADDLE_BRANCH}.post${ref_CUDA_MAJOR}${CUDNN_MAJOR}-cp38-cp38-linux_x86_64.whl ref_paddle39=paddlepaddle${install_gpu}-${PADDLE_BRANCH}.post${ref_CUDA_MAJOR}${CUDNN_MAJOR}-cp39-cp39-linux_x86_64.whl - ref_paddle2_whl=paddlepaddle${install_gpu}-${PADDLE_BRANCH}.post${ref_CUDA_MAJOR}${CUDNN_MAJOR}-cp27-cp27mu-linux_x86_64.whl - ref_paddle35_whl=paddlepaddle${install_gpu}-${PADDLE_BRANCH}.post${ref_CUDA_MAJOR}${CUDNN_MAJOR}-cp35-cp35m-linux_x86_64.whl ref_paddle36_whl=paddlepaddle${install_gpu}-${PADDLE_BRANCH}.post${ref_CUDA_MAJOR}${CUDNN_MAJOR}-cp36-cp36m-linux_x86_64.whl ref_paddle37_whl=paddlepaddle${install_gpu}-${PADDLE_BRANCH}.post${ref_CUDA_MAJOR}${CUDNN_MAJOR}-cp37-cp37m-linux_x86_64.whl ref_paddle38_whl=paddlepaddle${install_gpu}-${PADDLE_BRANCH}.post${ref_CUDA_MAJOR}${CUDNN_MAJOR}-cp38-cp38-linux_x86_64.whl ref_paddle39_whl=paddlepaddle${install_gpu}-${PADDLE_BRANCH}.post${ref_CUDA_MAJOR}${CUDNN_MAJOR}-cp39-cp39-linux_x86_64.whl fi - #ref_paddle2_mv1="" - #ref_paddle2_mv2="" - ref_paddle35_mv1="" - ref_paddle35_mv2="" ref_paddle36_mv1="" ref_paddle36_mv2="" - #ref_paddle37_mv1="" - #ref_paddle37_mv2="" if [[ ${PADDLE_BRANCH} == "0.0.0" && ${WITH_GPU} == "ON" ]]; then - #ref_paddle2_whl=paddlepaddle_gpu-1.5.1-cp27-cp27mu-linux_x86_64.whl - ref_paddle35_whl=paddlepaddle_gpu-1.5.1-cp35-cp35m-linux_x86_64.whl ref_paddle36_whl=paddlepaddle_gpu-1.5.1-cp36-cp36m-linux_x86_64.whl - #ref_paddle37_whl=paddlepaddle_gpu-1.5.1-cp37-cp37m-linux_x86_64.whl - #ref_paddle2_mv1="mv ref_paddle2 paddlepaddle_gpu-1.5.1-cp27-cp27mu-linux_x86_64.whl &&" - #ref_paddle2_mv2="&& mv paddlepaddle_gpu-1.5.1-cp27-cp27mu-linux_x86_64.whl ref_paddle2" - ref_paddle35_mv1="mv ${ref_paddle35} ${ref_paddle35_whl} &&" - ref_paddle35_mv2="&& mv ${ref_paddle35_whl} ${ref_paddle35}" ref_paddle36_mv1="mv ${ref_paddle36} ${ref_paddle36_whl} &&" ref_paddle36_mv2="&& mv ${ref_paddle36_whl} ${ref_paddle36}" - #ref_paddle37_mv1="mv ref_paddle37 paddlepaddle_gpu-1.5.1-cp37-cp37m-linux_x86_64.whl &&" - #ref_paddle37_mv2="&& mv paddlepaddle_gpu-1.5.1-cp37-cp37m-linux_x86_64.whl ref_paddle37" fi if [[ ${PADDLE_BRANCH} == "0.0.0" && ${WITH_GPU} != "ON" ]]; then - #ref_paddle2_whl=paddlepaddle_gpu-1.5.1-cp27-cp27mu-linux_x86_64.whl - ref_paddle35_whl=paddlepaddle-1.5.1-cp35-cp35m-linux_x86_64.whl ref_paddle36_whl=paddlepaddle-1.5.1-cp36-cp36m-linux_x86_64.whl - #ref_paddle37_whl=paddlepaddle_gpu-1.5.1-cp37-cp37m-linux_x86_64.whl - #ref_paddle2_mv1="mv ref_paddle2 paddlepaddle_gpu-1.5.1-cp27-cp27mu-linux_x86_64.whl &&" - #ref_paddle2_mv2="&& mv paddlepaddle_gpu-1.5.1-cp27-cp27mu-linux_x86_64.whl ref_paddle2" - ref_paddle35_mv1="mv ${ref_paddle35} ${ref_paddle35_whl} &&" - ref_paddle35_mv2="&& mv ${ref_paddle35_whl} ${ref_paddle35}" ref_paddle36_mv1="mv ${ref_paddle36} ${ref_paddle36_whl} &&" ref_paddle36_mv2="&& mv ${ref_paddle36_whl} ${ref_paddle36}" - #ref_paddle37_mv1="mv ref_paddle37 paddlepaddle_gpu-1.5.1-cp37-cp37m-linux_x86_64.whl &&" - #ref_paddle37_mv2="&& mv paddlepaddle_gpu-1.5.1-cp37-cp37m-linux_x86_64.whl ref_paddle37" fi cat > ${PADDLE_ROOT}/build/Dockerfile <> ${PADDLE_ROOT}/build/Dockerfile <&2 @@ -2147,6 +2042,23 @@ function reuse_so_cache() { fi } +function find_temporary_files() { + set +x + jsonData=`curl \ + -H "Authorization: token ${GITHUB_API_TOKEN}"\ + -H "Accept: application/vnd.github.v3+json" \ + https://api.github.com/repos/PaddlePaddle/Paddle/pulls/${GIT_PR_ID}/files` + + result=`echo ${jsonData}|python ${PADDLE_ROOT}/tools/check_file_suffix.py` + + if [ ${#result} -gt 0 ] + then + echo ${result} + exit 65 + fi +} + + function main() { local CMD=$1 local parallel_number=$2 @@ -2159,14 +2071,21 @@ function main() { set +e check_style_info=$(check_style) check_style_code=$? + find_temporary_files generate_upstream_develop_api_spec ${PYTHON_ABI:-""} ${parallel_number} cmake_gen_and_build ${PYTHON_ABI:-""} ${parallel_number} check_sequence_op_unittest generate_api_spec ${PYTHON_ABI:-""} "PR" set +e + example_info_gpu="" + example_code_gpu=0 + if [ "${WITH_GPU}" == "ON" ] ; then + example_info_gpu=$(exec_samplecode_test gpu) + example_code_gpu=$? + fi example_info=$(exec_samplecode_test cpu) example_code=$? - summary_check_problems $check_style_code $example_code "$check_style_info" "$example_info" + summary_check_problems $check_style_code $[${example_code_gpu} + ${example_code}] "$check_style_info" "${example_info_gpu}\n${example_info}" assert_api_spec_approvals ;; build) @@ -2206,6 +2125,7 @@ function main() { test_fluid_lib ;; build_inference_lib) + python ${PADDLE_ROOT}/tools/remove_grad_op_and_kernel.py cmake_gen ${PYTHON_ABI:-""} gen_fluid_lib ${parallel_number} ;; @@ -2234,6 +2154,7 @@ function main() { enable_unused_var_check ;; gpu_cicheck_coverage) + check_approvals_of_unittest 1 parallel_test check_coverage check_change_of_unittest ${PYTHON_ABI:-""} diff --git a/python/paddle/__init__.py b/python/paddle/__init__.py index c81ee72d7f2ba3..773ae61a691c5a 100755 --- a/python/paddle/__init__.py +++ b/python/paddle/__init__.py @@ -118,6 +118,7 @@ from .tensor.logic import is_tensor # noqa: F401 from .tensor.manipulation import cast # noqa: F401 from .tensor.manipulation import concat # noqa: F401 +from .tensor.manipulation import broadcast_tensors # noqa: F401 from .tensor.manipulation import expand # noqa: F401 from .tensor.manipulation import broadcast_to # noqa: F401 from .tensor.manipulation import expand_as # noqa: F401 @@ -203,6 +204,7 @@ from .tensor.math import addmm # noqa: F401 from .tensor.math import clip # noqa: F401 from .tensor.math import trace # noqa: F401 +from .tensor.math import diagonal # noqa: F401 from .tensor.math import kron # noqa: F401 from .tensor.math import isfinite # noqa: F401 from .tensor.math import isinf # noqa: F401 @@ -503,5 +505,7 @@ 'check_shape', 'trunc', 'digamma', - 'standard_normal' + 'standard_normal', + 'diagonal', + 'broadcast_tensors', ] diff --git a/python/paddle/amp/grad_scaler.py b/python/paddle/amp/grad_scaler.py index 770b660a9e11ff..827a320b2cc9c4 100644 --- a/python/paddle/amp/grad_scaler.py +++ b/python/paddle/amp/grad_scaler.py @@ -145,3 +145,290 @@ def minimize(self, optimizer, *args, **kwargs): optimizer.clear_grad() """ return super(GradScaler, self).minimize(optimizer, *args, **kwargs) + + def is_enable(self): + """ + Enable loss scaling or not. + + Returns: + bool: enable loss scaling return True else return False. + + Examples: + .. code-block:: python + + import paddle + scaler = paddle.amp.GradScaler(enable=True, + init_loss_scaling=1024, + incr_ratio=2.0, + decr_ratio=0.5, + incr_every_n_steps=1000, + decr_every_n_nan_or_inf=2, + use_dynamic_loss_scaling=True) + enable = scaler.is_enable() + print(enable) # True + """ + return super(GradScaler, self).is_enable() + + def is_use_dynamic_loss_scaling(self): + """ + Whether to use dynamic loss scaling. + + Returns: + bool: if fixed loss_scaling is used return False, if the loss scaling is updated dynamicly return true. + + Examples: + .. code-block:: python + + import paddle + scaler = paddle.amp.GradScaler(enable=True, + init_loss_scaling=1024, + incr_ratio=2.0, + decr_ratio=0.5, + incr_every_n_steps=1000, + decr_every_n_nan_or_inf=2, + use_dynamic_loss_scaling=True) + use_dynamic_loss_scaling = scaler.is_use_dynamic_loss_scaling() + print(use_dynamic_loss_scaling) # True + """ + return super(GradScaler, self).is_use_dynamic_loss_scaling() + + def get_init_loss_scaling(self): + """ + Return the initial loss scaling factor. + + Reurns: + float: the initial loss scaling factor. + + Examples: + .. code-block:: python + + import paddle + scaler = paddle.amp.GradScaler(enable=True, + init_loss_scaling=1024, + incr_ratio=2.0, + decr_ratio=0.5, + incr_every_n_steps=1000, + decr_every_n_nan_or_inf=2, + use_dynamic_loss_scaling=True) + init_loss_scaling = scaler.get_init_loss_scaling() + print(init_loss_scaling) # 1024 + """ + return super(GradScaler, self).get_init_loss_scaling() + + def set_init_loss_scaling(self, new_init_loss_scaling): + """ + Set the initial loss scaling factor by `new_init_loss_scaling`. + + Args: + new_init_loss_scaling(int): The new_init_loss_scaling used to update initial loss scaling factor. + + Examples: + .. code-block:: python + + import paddle + scaler = paddle.amp.GradScaler(enable=True, + init_loss_scaling=1024, + incr_ratio=2.0, + decr_ratio=0.5, + incr_every_n_steps=1000, + decr_every_n_nan_or_inf=2, + use_dynamic_loss_scaling=True) + print(scaler.get_init_loss_scaling()) # 1024 + new_init_loss_scaling = 1000 + scaler.set_init_loss_scaling(new_init_loss_scaling) + print(scaler.get_init_loss_scaling()) # 1000 + """ + super(GradScaler, self).set_init_loss_scaling(new_init_loss_scaling) + + def get_incr_ratio(self): + """ + Return the multiplier to use when increasing the loss scaling. + + Reurns: + float: the multiplier to use when increasing the loss scaling. + + Examples: + .. code-block:: python + + import paddle + scaler = paddle.amp.GradScaler(enable=True, + init_loss_scaling=1024, + incr_ratio=2.0, + decr_ratio=0.5, + incr_every_n_steps=1000, + decr_every_n_nan_or_inf=2, + use_dynamic_loss_scaling=True) + incr_ratio = scaler.get_incr_ratio() + print(incr_ratio) # 2.0 + """ + return super(GradScaler, self).get_incr_ratio() + + def set_incr_ratio(self, new_incr_ratio): + """ + Set the multiplier to use when increasing the loss scaling by `new_incr_ratio`, `new_incr_ratio` should > 1.0. + + Args: + new_incr_ratio(float): The new_incr_ratio used to update the multiplier to use when increasing the loss scaling. + + Examples: + .. code-block:: python + + import paddle + scaler = paddle.amp.GradScaler(enable=True, + init_loss_scaling=1024, + incr_ratio=2.0, + decr_ratio=0.5, + incr_every_n_steps=1000, + decr_every_n_nan_or_inf=2, + use_dynamic_loss_scaling=True) + print(scaler.get_incr_ratio()) # 2.0 + new_incr_ratio = 3.0 + scaler.set_incr_ratio(new_incr_ratio) + print(scaler.get_incr_ratio()) # 3.0 + """ + super(GradScaler, self).set_incr_ratio(new_incr_ratio) + + def get_decr_ratio(self): + """ + Get the less-than-one-multiplier to use when decreasing the loss scaling. + + Reurns: + float: the less-than-one-multiplier to use when decreasing the loss scaling. + + Examples: + .. code-block:: python + + import paddle + scaler = paddle.amp.GradScaler(enable=True, + init_loss_scaling=1024, + incr_ratio=2.0, + decr_ratio=0.5, + incr_every_n_steps=1000, + decr_every_n_nan_or_inf=2, + use_dynamic_loss_scaling=True) + decr_ratio = scaler.get_decr_ratio() + print(decr_ratio) # 0.5 + """ + return super(GradScaler, self).get_decr_ratio() + + def set_decr_ratio(self, new_decr_ratio): + """ + Set the less-than-one-multiplier to use when decreasing the loss scaling by `new_incr_ratio`, `new_decr_ratio` should < 1.0. + + Args: + new_decr_ratio(float): The new_decr_ratio used to update the less-than-one-multiplier to use when decreasing the loss scaling. + + Examples: + .. code-block:: python + + import paddle + scaler = paddle.amp.GradScaler(enable=True, + init_loss_scaling=1024, + incr_ratio=2.0, + decr_ratio=0.5, + incr_every_n_steps=1000, + decr_every_n_nan_or_inf=2, + use_dynamic_loss_scaling=True) + print(scaler.get_decr_ratio()) # 0.5 + new_decr_ratio = 0.1 + scaler.set_decr_ratio(new_decr_ratio) + print(scaler.get_decr_ratio()) # 0.1 + """ + super(GradScaler, self).set_decr_ratio(new_decr_ratio) + + def get_incr_every_n_steps(self): + """ + Return the num `n`, `n` represent increases loss scaling every `n` consecutive steps with finite gradients. + + Reurns: + int: the num `n`, `n` represent increases loss scaling every `n` consecutive steps with finite gradients. + + Examples: + .. code-block:: python + + import paddle + scaler = paddle.amp.GradScaler(enable=True, + init_loss_scaling=1024, + incr_ratio=2.0, + decr_ratio=0.5, + incr_every_n_steps=1000, + decr_every_n_nan_or_inf=2, + use_dynamic_loss_scaling=True) + incr_every_n_steps = scaler.get_incr_every_n_steps() + print(incr_every_n_steps) # 1000 + """ + return super(GradScaler, self).get_incr_every_n_steps() + + def set_incr_every_n_steps(self, new_incr_every_n_steps): + """ + Set the num `n` by `new_incr_every_n_steps`, `n` represent increases loss scaling every `n` consecutive steps with finite gradients. + + Args: + new_incr_every_n_steps(int): The new_incr_every_n_steps used to update the num `n`, `n` represent increases loss scaling every `n` consecutive steps with finite gradients. + + Examples: + .. code-block:: python + + import paddle + scaler = paddle.amp.GradScaler(enable=True, + init_loss_scaling=1024, + incr_ratio=2.0, + decr_ratio=0.5, + incr_every_n_steps=1000, + decr_every_n_nan_or_inf=2, + use_dynamic_loss_scaling=True) + print(scaler.get_incr_every_n_steps()) # 1000 + new_incr_every_n_steps = 2000 + scaler.set_incr_every_n_steps(new_incr_every_n_steps) + print(scaler.get_incr_every_n_steps()) # 2000 + """ + super(GradScaler, self).set_incr_every_n_steps(new_incr_every_n_steps) + + def get_decr_every_n_nan_or_inf(self): + """ + Return the num `n`, `n` represent decreases loss scaling every `n` accumulated steps with nan or inf gradients. + + Reurns: + int: the num `n`, `n` represent decreases loss scaling every `n` accumulated steps with nan or inf gradients. + + Examples: + .. code-block:: python + + import paddle + scaler = paddle.amp.GradScaler(enable=True, + init_loss_scaling=1024, + incr_ratio=2.0, + decr_ratio=0.5, + incr_every_n_steps=1000, + decr_every_n_nan_or_inf=2, + use_dynamic_loss_scaling=True) + decr_every_n_nan_or_inf = scaler.get_decr_every_n_nan_or_inf() + print(decr_every_n_nan_or_inf) # 2 + """ + return super(GradScaler, self).get_decr_every_n_nan_or_inf() + + def set_decr_every_n_nan_or_inf(self, new_decr_every_n_nan_or_inf): + """ + Set the num `n` by `new_decr_every_n_nan_or_inf`, `n` represent decreases loss scaling every `n` accumulated steps with nan or inf gradients. + + Args: + new_decr_every_n_nan_or_inf(int): The new_decr_every_n_nan_or_inf used to update the num `n`, `n` represent decreases loss scaling every `n` accumulated steps with nan or inf gradients. + + Examples: + .. code-block:: python + + import paddle + scaler = paddle.amp.GradScaler(enable=True, + init_loss_scaling=1024, + incr_ratio=2.0, + decr_ratio=0.5, + incr_every_n_steps=1000, + decr_every_n_nan_or_inf=2, + use_dynamic_loss_scaling=True) + print(scaler.get_decr_every_n_nan_or_inf()) # 2 + new_decr_every_n_nan_or_inf = 3 + scaler.set_decr_every_n_nan_or_inf(new_decr_every_n_nan_or_inf) + print(scaler.get_decr_every_n_nan_or_inf()) # 3 + """ + super(GradScaler, + self).set_decr_every_n_nan_or_inf(new_decr_every_n_nan_or_inf) diff --git a/python/paddle/dataset/flowers.py b/python/paddle/dataset/flowers.py index 45a4c36f42ecd5..8ca948b49bc4a7 100644 --- a/python/paddle/dataset/flowers.py +++ b/python/paddle/dataset/flowers.py @@ -114,35 +114,26 @@ def reader_creator(data_file, :return: data reader :rtype: callable ''' - scio = try_import('scipy.io') - - labels = scio.loadmat(label_file)['labels'][0] - indexes = scio.loadmat(setid_file)[dataset_name][0] - - img2label = {} - for i in indexes: - img = "jpg/image_%05d.jpg" % i - img2label[img] = labels[i - 1] - file_list = batch_images_from_tar(data_file, dataset_name, img2label) def reader(): - while True: - with open(file_list, 'r') as f_list: - for file in f_list: - file = file.strip() - batch = None - with open(file, 'rb') as f: - batch = pickle.load(f, encoding='bytes') - - if six.PY3: - batch = cpt.to_text(batch) - data_batch = batch['data'] - labels_batch = batch['label'] - for sample, label in six.moves.zip(data_batch, - labels_batch): - yield sample, int(label) - 1 - if not cycle: - break + scio = try_import('scipy.io') + + labels = scio.loadmat(label_file)['labels'][0] + indexes = scio.loadmat(setid_file)[dataset_name][0] + + img2label = {} + for i in indexes: + img = "jpg/image_%05d.jpg" % i + img2label[img] = labels[i - 1] + + tf = tarfile.open(data_file) + mems = tf.getmembers() + file_id = 0 + for mem in mems: + if mem.name in img2label: + image = tf.extractfile(mem).read() + label = img2label[mem.name] + yield image, int(label) - 1 if use_xmap: return xmap_readers(mapper, reader, min(4, cpu_count()), buffered_size) diff --git a/python/paddle/device.py b/python/paddle/device.py index 93e439ecf0aa42..cf445917dd5b7f 100644 --- a/python/paddle/device.py +++ b/python/paddle/device.py @@ -133,12 +133,20 @@ def _convert_to_place(device): selected_xpus = os.getenv("FLAGS_selected_xpus", "0").split(",") device_id = int(selected_xpus[0]) place = core.XPUPlace(device_id) + elif lower_device == 'npu': + if not core.is_compiled_with_npu(): + raise ValueError("The device should not be 'npu', " + "since PaddlePaddle is not compiled with NPU") + selected_npus = os.getenv("FLAGS_selected_npus", "0").split(",") + device_id = int(selected_npus[0]) + place = core.NPUPlace(device_id) else: avaliable_gpu_device = re.match(r'gpu:\d+', lower_device) avaliable_xpu_device = re.match(r'xpu:\d+', lower_device) - if not avaliable_gpu_device and not avaliable_xpu_device: + avaliable_npu_device = re.match(r'npu:\d+', lower_device) + if not avaliable_gpu_device and not avaliable_xpu_device and not avaliable_npu_device: raise ValueError( - "The device must be a string which is like 'cpu', 'gpu', 'gpu:x', 'xpu' or 'xpu:x'" + "The device must be a string which is like 'cpu', 'gpu', 'gpu:x', 'xpu', 'xpu:x', 'npu' or 'npu:x'" ) if avaliable_gpu_device: if not core.is_compiled_with_cuda(): @@ -158,19 +166,28 @@ def _convert_to_place(device): device_id = device_info_list[1] device_id = int(device_id) place = core.XPUPlace(device_id) + if avaliable_npu_device: + if not core.is_compiled_with_npu(): + raise ValueError( + "The device should not be {}, since PaddlePaddle is " + "not compiled with NPU".format(avaliable_npu_device)) + device_info_list = device.split(':', 1) + device_id = device_info_list[1] + device_id = int(device_id) + place = core.NPUPlace(device_id) return place def set_device(device): """ - Paddle supports running calculations on various types of devices, including CPU, GPU and XPU. + Paddle supports running calculations on various types of devices, including CPU, GPU, XPU and NPU. They are represented by string identifiers. This function can specify the global device which the OP will run. Parameters: device(str): This parameter determines the specific running device. - It can be ``cpu``, ``gpu:x`` and ``xpu:x``, where ``x`` is the - index of the GPUs or XPUs. + It can be ``cpu``, ``gpu``, ``xpu``, ``npu``, ``gpu:x``, ``xpu:x`` and ``npu:x``, + where ``x`` is the index of the GPUs, XPUs or NPUs. Examples: @@ -191,7 +208,7 @@ def set_device(device): def get_device(): """ This funciton can get the current global device of the program is running. - It's a string which is like 'cpu', 'gpu:x' and 'xpu:x'. if the global device is not + It's a string which is like 'cpu', 'gpu:x', 'xpu:x' and 'npu:x'. if the global device is not set, it will return a string which is 'gpu:x' when cuda is avaliable or it will return a string which is 'cpu' when cuda is not avaliable. @@ -213,5 +230,8 @@ def get_device(): elif isinstance(place, core.XPUPlace): device_id = place.get_device_id() device = 'xpu:' + str(device_id) + elif isinstance(place, core.NPUPlace): + device_id = place.get_device_id() + device = 'npu:' + str(device_id) return device diff --git a/python/paddle/distributed/collective.py b/python/paddle/distributed/collective.py index 1a09cf5394fba8..5256749c9405ee 100644 --- a/python/paddle/distributed/collective.py +++ b/python/paddle/distributed/collective.py @@ -19,6 +19,7 @@ from ..fluid.framework import OpProtoHolder from ..fluid.framework import in_dygraph_mode from ..fluid.framework import convert_np_dtype_to_dtype_ +from ..fluid.framework import _varbase_creator from ..fluid.data_feeder import convert_dtype from ..fluid.data_feeder import check_variable_and_dtype from ..fluid.data_feeder import check_type @@ -31,6 +32,7 @@ from .fleet import fleet import paddle.fluid as fluid import paddle.fluid.core as core +import paddle.fluid.dygraph_utils as dygraph_utils __all__ = [] @@ -92,8 +94,6 @@ def is_member(self): return True def get_group_rank(self, rank): - if self.id == 0: - return rank if self.is_member() and rank in self.ranks: return self.ranks.index(rank) else: @@ -126,7 +126,8 @@ def _get_group_map(): global _group_map if not _group_map: genv = _get_global_env() - _group_map[0] = Group(genv.rank, genv.world_size, 0) + _group_map[0] = Group(genv.rank, genv.world_size, + list(range(genv.world_size))) return _group_map @@ -159,7 +160,7 @@ def get_group(id=0): """ gm = _get_group_map() - return gm[group] if group in gm else None + return gm[id] if id in gm else None def barrier(group=None): @@ -463,7 +464,6 @@ def all_reduce(tensor, op=ReduceOp.SUM, group=None, use_calc_stream=True): tensor, 'use_calc_stream', use_calc_stream, 'ring_id', ring_id) else: raise ValueError("Unknown parameter: {}.".format(op)) - return out check_variable_and_dtype( tensor, 'tensor', ['float16', 'float32', 'float64', 'int32', 'int64'], @@ -1014,6 +1014,27 @@ def _c_softmax_with_cross_entropy(logits, else: return loss, softmax + attrs = { + 'ring_id': ring_id, + 'rank': rank, + 'nranks': nranks, + } + helper = LayerHelper('c_softmax_with_cross_entropy', **locals()) + softmax = helper.create_variable_for_type_inference(dtype=logits.dtype) + loss = helper.create_variable_for_type_inference(dtype=logits.dtype) + helper.append_op( + type='c_softmax_with_cross_entropy', + inputs={'Logits': logits, + 'Label': label}, + outputs={'Softmax': softmax, + 'Loss': loss}, + attrs=attrs) + + if return_softmax: + return loss, softmax + + return loss + def _linear(x, weight, bias=None, name=None): """ @@ -1199,6 +1220,65 @@ def _parallel_embedding(x, return out +def _parallel_embedding_npu(x, + per_part_embeddings, + origin_size, + param_attr, + inner_rank, + num_partitions, + name, + group=None): + """ + NPU Parallel Embedding + """ + if group is not None and not group.is_member(): + return + ring_id = 0 if group is None else group.id + + origin_num_embeddings = origin_size[0] + embedding = paddle.nn.Embedding( + per_part_embeddings, + origin_size[1], + padding_idx=per_part_embeddings - 1, + sparse=False, + weight_attr=param_attr, + name=name) + + origin_input_shape = x.shape + if len(origin_input_shape) == 2: + x = paddle.unsqueeze(x, axis=-1) + else: + assert origin_input_shape[-1] == 1, ( + "The last dimension size of x must be 1.") + x_shard = paddle.shard_index(x, origin_num_embeddings, num_partitions, + inner_rank, per_part_embeddings - 1) + if len(origin_input_shape) == 2: + x_shard = paddle.squeeze(x_shard, axis=-1) + emb_out = embedding(x_shard) + startup_block = paddle.static.default_startup_program().global_block() + main_block = paddle.static.default_main_program().global_block() + startup_block.vars[embedding.weight.name].is_distributed = True + main_block.vars[embedding.weight.name].is_distributed = True + out = main_block.create_var( + shape=emb_out.shape, + dtype=emb_out.dtype, + type=emb_out.type, + lod_level=emb_out.lod_level, + persistable=False, + is_data=False, + need_check_feed=emb_out.desc.need_check_feed()) + main_block.append_op( + type='c_allreduce_sum', + inputs={'X': emb_out}, + outputs={'Out': out}, + attrs={ + 'ring_id': ring_id, + 'use_calc_stream': True, + 'use_model_parallel': True + }) + return out + + def split(x, size, operation, @@ -1312,16 +1392,28 @@ def split(x, "but received vocabulary={} num_partitions={}".format(size[0], num_partitions) per_part_size = size[0] // num_partitions - emb_out = _parallel_embedding( - x, - per_part_size, - size, - weight_attr, - inner_rank, - num_partitions, - name, - group=None) - return emb_out + if core.is_compiled_with_npu(): + emb_out = _parallel_embedding_npu( + x, + per_part_size, + size, + weight_attr, + inner_rank, + num_partitions, + name, + group=None) + return emb_out + else: + emb_out = _parallel_embedding( + x, + per_part_size, + size, + weight_attr, + inner_rank, + num_partitions, + name, + group=None) + return emb_out else: should_split = False if axis == 0: diff --git a/python/paddle/distributed/fleet/base/distributed_strategy.py b/python/paddle/distributed/fleet/base/distributed_strategy.py index c4aa9213469738..5308964b1c162d 100644 --- a/python/paddle/distributed/fleet/base/distributed_strategy.py +++ b/python/paddle/distributed/fleet/base/distributed_strategy.py @@ -254,6 +254,28 @@ def build_strategy(self, strategy): getattr(self.strategy.build_strategy, f.name).extend(getattr(strategy, f.name)) + @property + def gradient_scale_configs(self): + """ + Set the strategy of gradient scale + Examples: + + .. code-block:: python + import paddle.distributed.fleet as fleet + strategy = fleet.DistributedStrategy() + strategy.gradient_scale_configs = {'scale_strategy': 'avg'} + + Note that, strategy must be in 'avg', 'sum' or 'customized' + """ + return get_msg_dict(self.strategy.gradient_scale_configs) + + @gradient_scale_configs.setter + @is_strict_auto + def gradient_scale_configs(self, config): + check_configs_key(self.strategy.gradient_scale_configs, config, + 'gradient_scale_configs') + assign_configs_value(self.strategy.gradient_scale_configs, config) + @property def a_sync(self): """ diff --git a/python/paddle/distributed/fleet/base/fleet_base.py b/python/paddle/distributed/fleet/base/fleet_base.py index 9e5a31d6899e07..2a9b15c732541a 100644 --- a/python/paddle/distributed/fleet/base/fleet_base.py +++ b/python/paddle/distributed/fleet/base/fleet_base.py @@ -30,7 +30,7 @@ from . import topology as tp from .topology import ParallelMode from ..meta_parallel import TensorParallel, model_parallel_random_seed -from ..meta_parallel import PipelineParallel +from ..meta_parallel import PipelineParallel, ShardingParallel from ..meta_optimizers import HybridParallelOptimizer from ..meta_optimizers import HybridParallelGradScaler @@ -253,6 +253,40 @@ def init(self, role_maker=None, is_collective=False, strategy=None): warnings.warn( "The dygraph hybrid parallel environment has been initialized." ) + elif self._is_collective: + use_sharding = self._user_defined_strategy.sharding + + # global group + global_rank = self.worker_index() + global_world_size = self.worker_num() + # NOTE(wangxi): see sharding_optimizer + global_ring_id = 3 if use_sharding else 0 + global_ranks = list(range(global_world_size)) + + if tp._HYBRID_PARALLEL_GROUP is None: tp._CommunicateGroup() + cg = tp._HYBRID_PARALLEL_GROUP + self._hcg = cg + cg.set_comm_group('global', global_rank, global_world_size, + global_ring_id, global_ranks) + + # hybrid group + if use_sharding is False: return + + sharding_configs = self._user_defined_strategy.sharding_configs + mp_degree = int(sharding_configs['mp_degree']) + + if mp_degree > 1: + assert global_world_size % mp_degree == 0 + # NOTE(wangxi): mp_ring_id sync with sharding_optimizer.py _build_groups + mp_ring_id = 0 + mp_rank = global_rank % mp_degree + mp_group_id = global_rank // mp_degree + mp_group_ranks = [ + idx for idx in global_ranks + if idx // mp_degree == mp_group_id + ] + cg.set_comm_group('model', mp_rank, mp_degree, mp_ring_id, + mp_group_ranks) def _init_hybrid_parallel_env(self): """initialize the hybrid environment @@ -261,9 +295,11 @@ def _init_hybrid_parallel_env(self): self.dp_degree = self.hybrid_configs["dp_degree"] self.mp_degree = self.hybrid_configs["mp_degree"] self.pp_degree = self.hybrid_configs["pp_degree"] + self.sharding_degree = self.hybrid_configs["sharding_degree"] assert self.mp_degree >= 0, "mp_degree should be greater or equal to 0" assert self.pp_degree >= 0, "pp_degree should be greater or equal to 0" + assert self.sharding_degree >= 0, "sharding_degree should be greater or equal to 0" self.mp_degree = max(self.mp_degree, 1) self.pp_degree = max(self.pp_degree, 1) @@ -275,8 +311,11 @@ def _init_hybrid_parallel_env(self): self.dp_degree = max(self.dp_degree, 1) self._topology = tp.CommunicateTopology( - hybrid_group_names=["data", "pipe", "model"], - dims=[self.dp_degree, self.pp_degree, self.mp_degree]) + hybrid_group_names=["data", "pipe", "sharding", "model"], + dims=[ + self.dp_degree, self.pp_degree, self.sharding_degree, + self.mp_degree + ]) self._hcg = tp.HybridCommunicateGroup(self._topology) @@ -852,7 +891,11 @@ def forward(self, x): assert model is not None, "model should not be None" if self.worker_num() <= 1: return model - if self._hcg.get_parallel_mode() == ParallelMode.DATA_PARALLEL: + + if self._hcg.get_parallel_mode() == ParallelMode.SHARDING_PARALLEL: + distributed_model = ShardingParallel( + model, self._hcg, strategy=self._user_defined_strategy) + elif self._hcg.get_parallel_mode() == ParallelMode.DATA_PARALLEL: distributed_model = paddle.DataParallel( model, comm_buffer_size=self._user_defined_strategy. @@ -867,6 +910,7 @@ def forward(self, x): elif self._hcg.get_parallel_mode() == ParallelMode.PIPELINE_PARALLEL: distributed_model = PipelineParallel( model, self._hcg, strategy=self._user_defined_strategy) + return distributed_model @dygraph_only diff --git a/python/paddle/distributed/fleet/base/topology.py b/python/paddle/distributed/fleet/base/topology.py index 850f3581421705..004b3fb0f666bc 100644 --- a/python/paddle/distributed/fleet/base/topology.py +++ b/python/paddle/distributed/fleet/base/topology.py @@ -30,12 +30,13 @@ class ParallelMode(object): DATA_PARALLEL = 0 TENSOR_PARALLEL = 1 PIPELINE_PARALLEL = 2 + SHARDING_PARALLEL = 3 class CommunicateTopology(object): def __init__(self, - hybrid_group_names=["data", "pipe", "model"], - dims=[1, 1, 1]): + hybrid_group_names=["data", "pipe", "sharding", "model"], + dims=[1, 1, 1, 1]): self._parallel_names = hybrid_group_names self._dims = dims self.coordinate = collections.namedtuple('Coordinate', @@ -122,15 +123,17 @@ def __init__(self, topology): self._dp_degree = self._topo.get_dim('data') self._mp_degree = self._topo.get_dim('model') self._pp_degree = self._topo.get_dim('pipe') + self._sharding_degree = self._topo.get_dim('sharding') self._data_parallel_id = self._get_data_parallel_id() self._model_parallel_id = self._get_model_parallel_id() + self._sharding_parallel_id = self._get_sharding_parallel_id() self.stage_id = self._get_pipe_parallel_id() assert self._check_vaild_topo( ), "Here is an unreasonable topogy setting. world_size: {}, but" \ - "dp_num: {}, mp_num: {}, pp_num: {}".format(self.nranks, self._dp_degree, - self._mp_degree, self._pp_degree) + "mp_num: {}, sharding_num: {}, pp_num: {}, dp_num: {}".format(self.nranks, + self._mp_degree, self._sharding_degree, self._pp_degree, self._dp_degree) # create comm group for data parallel self._dp_group, self._dp_comm_group = self._set_comm_group("data") @@ -141,6 +144,10 @@ def __init__(self, topology): # create comm group for pipe parallel self._pp_group, self._pp_comm_group = self._set_comm_group("pipe") + # create comm group for sharding parallel + self._sharding_group, self._sharding_comm_group = self._set_comm_group( + "sharding") + # create global group for check inf_nan / clip global norm self._check_group, self._check_comm_group = self._set_check_group( "data") @@ -149,19 +156,44 @@ def __init__(self, topology): self.is_first_stage = (self.stage_id == 0) self.is_last_stage = (self.stage_id == (self._pp_degree - 1)) - debug_str = "HybridParallelInfo: rank_id: %d, dp_degree: %d, " \ - "mp_degree: %d, pp_degree: %d" % (self.global_rank, self._dp_degree, - self._mp_degree,self._pp_degree) - debug_str += ", dp_group: %s, mp_group: %s, pp_group: %s, check/clip group: %s" % ( - self._dp_group, self._mp_group, self._pp_group, self._check_group) + debug_str = "HybridParallelInfo: rank_id: %d, mp_degree: %d, " \ + "sharding_degree: %d, pp_degree: %d, dp_degree: %d" % (self.global_rank, self._mp_degree, + self._sharding_degree, self._pp_degree, self._dp_degree) + debug_str += ", mp_group: %s, sharding_group: %s, pp_group: %s, dp_group: %s, check/clip group: %s" % ( + self._mp_group, self._sharding_group, self._pp_group, + self._dp_group, self._check_group) logger.info(debug_str) + # create p2p_groups and no new group + self._p2p_groups = self._build_p2p_lists() + global _HYBRID_PARALLEL_GROUP _HYBRID_PARALLEL_GROUP = self + def _build_p2p_lists(self): + comm_lists = self._topo.get_comm_list('pipe') + p2p_lists = [] + for rank in range(self.nranks): + for comm_ranks in comm_lists: + assert len(comm_ranks) == self._pp_degree + if rank in comm_ranks: + idx = comm_ranks.index(rank) + next_rank = comm_ranks[(idx + 1) % self._pp_degree] + p2p_lists.append([rank, next_rank]) + break + assert len( + p2p_lists) == self.nranks, "len(p2p_lists) should be equal nranks" + return p2p_lists + def get_parallel_mode(self): - # there are three modes : DataParallel / TensorParallel / PipelineParallel - if self._mp_degree == 1 and self._pp_degree == 1: + # there are four modes : DataParallel / TensorParallel / PipelineParallel / ShardingParallel + # NOTE when sharding conjugates with other parallel, sharding should act like a optimizer and + # adding its parallel logic within that parallelism + # when use sharding alone, it should have its own parallelism for its parallel logic + # TODO modify 3 others parallel to support sharding + if self._mp_degree == 1 and self._pp_degree == 1 and self._dp_degree == 1 and self._sharding_degree > 1: + return ParallelMode.SHARDING_PARALLEL + elif self._mp_degree == 1 and self._pp_degree == 1: return ParallelMode.DATA_PARALLEL elif self._mp_degree > 1 and self._pp_degree == 1: # initialize the seed @@ -170,7 +202,7 @@ def get_parallel_mode(self): return ParallelMode.PIPELINE_PARALLEL def _check_vaild_topo(self): - return self._dp_degree * self._mp_degree * self._pp_degree == self.nranks + return self._dp_degree * self._mp_degree * self._pp_degree * self._sharding_degree == self.nranks def _set_comm_group(self, parallel_method="data"): parallel_group = [] @@ -255,6 +287,26 @@ def get_pipe_parallel_world_size(self): def get_pipe_parallel_group(self): return self._pp_comm_group + # sharding parallel message: + def _get_sharding_parallel_id(self): + return self._topo.get_coord(self.global_rank).sharding + + def get_sharding_parallel_rank(self): + return self._sharding_parallel_id + + def get_sharding_parallel_world_size(self): + return self._sharding_degree + + def get_sharding_parallel_group(self): + return self._sharding_comm_group + + def get_sharding_parallel_group_src_rank(self): + # TODO should the src rank related to the shard rank for each parameter ? + return self._sharding_comm_group.ranks[0] + + def get_p2p_groups(self): + return self._p2p_groups + # check parallel group def get_check_parallel_group(self): return self._check_comm_group @@ -262,3 +314,31 @@ def get_check_parallel_group(self): def get_rank_from_stage(self, stage_id, **kwargs): return self._topo.get_rank_from_stage( self.global_rank, pipe=stage_id, **kwargs) + + +class _CommunicateGroup(object): + """ tmp for static """ + + def __init__(self): + global _HYBRID_PARALLEL_GROUP + _HYBRID_PARALLEL_GROUP = self + self.groups = dict() + + def set_comm_group(self, group_name, group_rank, group_size, ring_id, + group_ranks): + group = paddle.distributed.collective.Group(group_rank, group_size, + ring_id, group_ranks) + self.groups[group_name] = group + + def get_group(self, group_name): + assert group_name in self.groups + return self.groups[group_name] + + def get_model_parallel_group(self): + return self.get_group('model') + + def get_model_parallel_world_size(self): + return self.get_group('model').nranks + + def get_model_parallel_rank(self): + return self.get_group('model').rank diff --git a/python/paddle/distributed/fleet/elastic.py b/python/paddle/distributed/fleet/elastic.py index b919c4737576d5..aa950fc26f6595 100644 --- a/python/paddle/distributed/fleet/elastic.py +++ b/python/paddle/distributed/fleet/elastic.py @@ -18,6 +18,7 @@ import six import logging import signal +import random logging.basicConfig(level=os.environ.get('LOGLEVEL', 'INFO').upper()) logger = logging.getLogger("ELASTIC") @@ -129,10 +130,14 @@ def __init__(self, args): # etcd data self.prefix = "/paddle/" + name - self.node_prefix = self.prefix + '/nodes/' + self.node_prefix = self.prefix + '/nodes' self.np_path = self.prefix + '/np' self.endpoints_path = self.prefix + '/endpoints' - self.host_path = '{}{}'.format(self.node_prefix, time.time()) + + node_tag = ''.join( + random.choice('abcdefghijklmnopqrstuvwxyz') for _ in range(6)) + self.host_path = '{}/{}{}'.format(self.node_prefix, node_tag, + time.time()) self.np = np + scale ''' @@ -195,9 +200,14 @@ def endpoints_call_back(event): self.watches = [host_watch, np_watch, endpoints_watch] + self.launcher = None + def exit(self, completed=False): logger.info('manager exist completed {}'.format(completed)) + if self.launcher: + self.launcher.stop() + if not self.enable: return @@ -262,6 +272,7 @@ def wait(self): if not self.enable: return + idx = 1 while not self.stopped: if self._match(): logger.info('ready with hosts {}'.format(self.hosts)) @@ -269,6 +280,14 @@ def wait(self): return logger.info('not ready for np {} with hosts {}'.format(self.np, self.hosts)) + + # reset hosts every 30s to prevent fake deadlock + if idx % 10 == 0: + self.etcd.delete_prefix(self.node_prefix) + logger.info('reset np {} with hosts {}'.format(self.np, + self.hosts)) + + idx += 1 time.sleep(3) return @@ -288,7 +307,6 @@ def watch(self): logger.info('job exit with code {}'.format(ret)) # process is completed if ret >= 0 or error else completed = True if ret == 0 else False - self.launcher.stop() self.exit(completed=completed) if completed: return ElasticStatus.COMPLETED @@ -303,6 +321,8 @@ def watch(self): time.sleep(3) + if self.launcher: + self.launcher.stop() return ElasticStatus.EXIT def signal_handler(self, sigint, frame): diff --git a/python/paddle/distributed/fleet/launch.py b/python/paddle/distributed/fleet/launch.py index 07862a07c92c41..f407892e79acf6 100644 --- a/python/paddle/distributed/fleet/launch.py +++ b/python/paddle/distributed/fleet/launch.py @@ -293,7 +293,8 @@ def launch(self): def stop(self): logger.info("collective lauchner stop ...") - self._terminate_procs() + if not self._terminate_procs(): + logger.error("kill process failed") if os.path.exists(self.gloo_rendezvous_dir): shutil.rmtree(self.gloo_rendezvous_dir) diff --git a/python/paddle/distributed/fleet/launch_utils.py b/python/paddle/distributed/fleet/launch_utils.py index ee5eb807fad701..4b1eef72ee9177 100644 --- a/python/paddle/distributed/fleet/launch_utils.py +++ b/python/paddle/distributed/fleet/launch_utils.py @@ -29,6 +29,7 @@ import paddle import paddle.fluid as fluid +from distutils.util import strtobool logger = logging.getLogger("root") logger.propagate = False @@ -349,7 +350,7 @@ def add_arguments(argname, type, default, help, argparser, **kwargs): add_argument("name", str, "Jonh", "User name.", parser) args = parser.parse_args() """ - type = distutils.util.strtobool if type == bool else type + type = strtobool if type == bool else type argparser.add_argument( "--" + argname, default=default, @@ -685,7 +686,7 @@ def get_device_proc_info(args): gpus = get_gpus(args.gpus) if args.nproc_per_node is not None: assert (len(gpus) % int(args.nproc_per_node)) ==0, \ - "gpus' number:{} mod args.nproc_per_node:{} must == 0".format(len(gpus), arg.nproc_per_node) + "gpus' number:{} mod args.nproc_per_node:{} must == 0".format(len(gpus), args.nproc_per_node) n = int(len(gpus) / int(args.nproc_per_node)) devices_per_proc = [ @@ -699,7 +700,7 @@ def get_device_proc_info(args): xpus = get_xpus(args.xpus) if args.nproc_per_node is not None: assert (len(xpus) % int(args.nproc_per_node)) == 0, \ - "xpus' number:{} mod args.nproc_per_node:{} must == 0".format(len(xpus), arg.nproc_per_node) + "xpus' number:{} mod args.nproc_per_node:{} must == 0".format(len(xpus), args.nproc_per_node) n = int(len(xpus) / int(args.nproc_per_node)) devices_per_proc = [ diff --git a/python/paddle/distributed/fleet/meta_optimizers/amp_optimizer.py b/python/paddle/distributed/fleet/meta_optimizers/amp_optimizer.py index 9ffb47789ee987..e3a781424e6d5f 100755 --- a/python/paddle/distributed/fleet/meta_optimizers/amp_optimizer.py +++ b/python/paddle/distributed/fleet/meta_optimizers/amp_optimizer.py @@ -124,3 +124,6 @@ def amp_init(self, use_fp16_test=False): return self.wrapped_opt.amp_init(place, scope, test_program, use_fp16_test) + + def get_loss_scaling(self): + return self.wrapped_opt.get_loss_scaling() diff --git a/python/paddle/distributed/fleet/meta_optimizers/ascend/ascend_parser.py b/python/paddle/distributed/fleet/meta_optimizers/ascend/ascend_parser.py index 3331a45b3d9479..8f1a4de86de0d9 100644 --- a/python/paddle/distributed/fleet/meta_optimizers/ascend/ascend_parser.py +++ b/python/paddle/distributed/fleet/meta_optimizers/ascend/ascend_parser.py @@ -136,7 +136,7 @@ def dtype2ge(self, dtype): def dtype2np(self, index): assert index in self.dtype2np_map, "index[%d] is not supported %d" % ( - dtype) + index) return self.dtype2np_map[index] @@ -342,7 +342,7 @@ def _apply(self): y = self._get_ge_input(self.op.input_arg_names[1]) pow = core.GEOperatorFactory.create_operator( "dotpow" + self._accumulated_op_id(), - "Pow").set_input("x1", x1).set_input("x2", y) + "Pow").set_input("x1", x).set_input("x2", y) return [pow], [[0]] @@ -918,15 +918,15 @@ def _apply(self): scatter_value = core.GEOperatorFactory.create_operator( "scatter" + self._accumulated_op_id(), "TensorScatterAdd").set_input( - "x", x_var).set_input("indices", index_var).set_input( - "updates", updatesi_var) + "x", x).set_input("indices", index).set_input("updates", + updates) else: scatter_value = core.GEOperatorFactory.create_operator( "scatter" + self._accumulated_op_id(), "TensorScatterUpdate").set_input( - "x", x_var).set_input("indices", index_var).set_input( - "updates", updates_var) - return [x_var, index_var, updates_var, scatter_value], [[-1]] + "x", x).set_input("indices", index).set_input("updates", + updates) + return [x, index, updates, scatter_value], [[-1]] class CastParser(AscendParserBase): diff --git a/python/paddle/distributed/fleet/meta_optimizers/dygraph_optimizer/dygraph_sharding_optimizer.py b/python/paddle/distributed/fleet/meta_optimizers/dygraph_optimizer/dygraph_sharding_optimizer.py new file mode 100755 index 00000000000000..4bddde6b5b62e6 --- /dev/null +++ b/python/paddle/distributed/fleet/meta_optimizers/dygraph_optimizer/dygraph_sharding_optimizer.py @@ -0,0 +1,198 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +###### +from functools import reduce + +import paddle +from paddle import framework +from ...utils.log_util import logger + + +def _is_trainable(param: paddle.Tensor) -> bool: + return not param.stop_gradient + + +class DygraphShardingOptimizer(object): + """ + A wrapper for Sharding Optimizer in Dygraph. + + .. warning: DygraphShardingOptimizer is experimental and subject to change. + + .. ZeRO: https://arxiv.org/abs/1910.02054 + + """ + + # TODO (JZ-LIANG) + # TO support following featrues in future: + # 1. fused update parameter sync + # 2. parameters_groups + # 3. dynamic trainable params, which is the case bewteen pretraining and finetuning + # 4. option to choose fuse comm (more GPU MEM need) or un-fuse comm + + def __init__( + self, + hcg, + user_defined_strategy, + params, + inner_optimizer_class, + **inner_optimizer_kargs, ): + + if not isinstance(params, list): + raise TypeError( + "`parameters` argument given to the DygraphShardingOptimizer should be " + "an iterable of paddle Tensors, but got argument type is `{}`.". + format(type(params))) + self._parameter_list = params + self._reference_is_trainable_params = list( + map(_is_trainable, self._parameter_list)) + + self._inner_optimizer_class = inner_optimizer_class + self._inner_optimizer_kargs = inner_optimizer_kargs + + # sharding parallel information + # TODO better way to get the hcg & user_defined_strategy + self._hcg = hcg + self._user_defined_strategy = user_defined_strategy + self._sharding_world_size = self._hcg.get_sharding_parallel_world_size() + self._sharding_rank = self._hcg.get_sharding_parallel_rank() + + # logic partitioning + self._build_sharding_mapping() + + # actually create opt ops + self._buid_inner_optimizer() + + def clear_grad(self): + """ + should clear grad for all parameters in model + """ + for p in self._parameter_list: + if not p.stop_gradient: + p.clear_gradient() + + def _build_sharding_mapping(self): + + self._rank2params = self._partition_parameters() + self._param2rank = self._map_param_to_rank() + + def _partition_parameters(self): + """ + Partitions parameters among sharding ranks. + + Return: + Dict[int, List] + """ + # TODO(JZ-LIANG) support multiple partition methods + # method1: greedy even but unorder + # method2: roughly even with oreder + + mapping = {} + for rank_ in range(self._sharding_world_size): + mapping[rank_] = [] + sizes = [0] * self._sharding_world_size + for param in self._parameter_list: + rank = sizes.index(min(sizes)) + mapping[rank].append(param) + numel = reduce(lambda x, y: x * y, param.shape) + assert numel > 0, "param [{}] should larger than 0, but it is [{}]".format( + param.name, numel) + sizes[rank] += numel + + return mapping + + def _map_param_to_rank(self): + """ + mapping parameters to the shard which holds it. + + Return: + Dict[str, int] + """ + mapping = {} + for rank, params in self._rank2params.items(): + for param in params: + mapping[param.name] = rank + return mapping + + def _buid_inner_optimizer(self): + # we rely on the inner opt to determine whether a parameter is stop_gradient or not: + # create moment + # update related ops: clip, regular, opt + self._inner_optimizer = self._inner_optimizer_class( + parameters=self._rank2params[self._sharding_rank], + **self._inner_optimizer_kargs) + + def _sharding_sync_parameters(self): + """ + sync parameter across sharding group + """ + # TODO speed up this functional + + logger.debug("sharding start sync parameters") + with framework.no_grad(): + # TODO detach not need (?) + for rank, params in self._rank2params.items(): + for param in params: + paddle.distributed.broadcast( + param, + # the collective API need src rank to be the global rank id + # instead of the relative logic rank id within group + src=self._hcg.get_sharding_parallel_group().ranks[rank], + group=self._hcg.get_sharding_parallel_group(), + use_calc_stream=True) + + def _update_trainable(self): + """ + allow user to update trainable parameters list during training + """ + raise NotImplementedError + + def minimize(self, + loss, + startup_program=None, + parameters=None, + no_grad_set=None): + + # NOTE in dygraph mode, the only different between step and minimize is that minimize + # allow user to customize the parameters for updating on each step + + input_param_names = set([param.name for param in parameters]) + parameters = list( + filter(lambda x: x.name in input_param_names, self._rank2params[ + self._sharding_rank])) + result = self._inner_optimizer.minimize(loss, startup_program, + parameters, no_grad_set) + + # sync parameters accross sharding ranks + self._sharding_sync_parameters() + + return result + + def step(self): + # TODO Check whether the model trainable param changed and update state accordingly + + # actually updating + self._inner_optimizer.step() + + # sync parameters accross sharding ranks + self._sharding_sync_parameters() + + # TODO is it a good way to make _grad_clip a property + @property + def _grad_clip(self): + assert self._inner_optimizer is not None, "inner opt of sharding is not initiliazed." + return self._inner_optimizer._grad_clip + + def __getattr__(self, item): + return getattr(self._inner_optimizer, item) diff --git a/python/paddle/distributed/fleet/meta_optimizers/dygraph_optimizer/hybrid_parallel_optimizer.py b/python/paddle/distributed/fleet/meta_optimizers/dygraph_optimizer/hybrid_parallel_optimizer.py index bceabeee3c3dce..e3a5947bf60fc1 100644 --- a/python/paddle/distributed/fleet/meta_optimizers/dygraph_optimizer/hybrid_parallel_optimizer.py +++ b/python/paddle/distributed/fleet/meta_optimizers/dygraph_optimizer/hybrid_parallel_optimizer.py @@ -17,7 +17,7 @@ import paddle from paddle.optimizer import Optimizer from paddle.fluid.clip import ClipGradByGlobalNorm -from ...utils.hybrid_parallel_util import fused_allreduce_gradients +from ...utils.hybrid_parallel_util import fused_allreduce_gradients, sharding_reduce_gradients from ...base.topology import ParallelMode from paddle.fluid.dygraph import base as imperative_base from paddle.fluid import framework @@ -98,6 +98,9 @@ def __init__(self, optimizer, hcg, strategy): self._need_dp = (self._hcg.get_data_parallel_world_size() > 1) + self._sharding_enable = ( + self._hcg.get_sharding_parallel_world_size() > 1) + if isinstance(self._inner_opt._grad_clip, ClipGradByGlobalNorm) and not self._use_dp_mode: logger.warning("using ClipGradByGlobalNorm in TensorParallel, the origin " \ @@ -108,6 +111,11 @@ def __init__(self, optimizer, hcg, strategy): @imperative_base.no_grad @framework.dygraph_only def step(self): + # Here should use global parameter list + if self._sharding_enable: + sharding_reduce_gradients( + list(self._inner_opt._parameter_list), self._hcg) + if not self._use_dp_mode and self._need_dp: fused_allreduce_gradients( list(self._inner_opt._parameter_list), self._hcg) @@ -119,15 +127,19 @@ def minimize(self, startup_program=None, parameters=None, no_grad_set=None): - assert isinstance(loss, Variable), "The loss should be an Tensor." parameter_list = parameters if parameters \ - else self._parameter_list + else self._inner_opt._parameter_list + + # Here should use global parameter list + if self._sharding_enable: + sharding_reduce_gradients( + list(self._inner_opt._parameter_list), self._hcg) if not self._use_dp_mode and self._need_dp: fused_allreduce_gradients(list(parameter_list), self._hcg) - return self._inner_opt.minimize(loss, startup_program, parameters, + return self._inner_opt.minimize(loss, startup_program, parameter_list, no_grad_set) def __getattr__(self, item): diff --git a/python/paddle/distributed/fleet/meta_optimizers/graph_execution_optimizer.py b/python/paddle/distributed/fleet/meta_optimizers/graph_execution_optimizer.py index 22ed3f2ac41603..5827f6bb3a183c 100644 --- a/python/paddle/distributed/fleet/meta_optimizers/graph_execution_optimizer.py +++ b/python/paddle/distributed/fleet/meta_optimizers/graph_execution_optimizer.py @@ -18,6 +18,7 @@ from .meta_optimizer_base import MetaOptimizerBase from ..base.private_helper_function import wait_server_ready import logging +from paddle.static import BuildStrategy __all__ = [] @@ -147,6 +148,17 @@ def _try_to_compile(self, startup_program, main_program, loss): local_build_strategy.nccl_comm_num = \ dist_strategy.nccl_comm_num + gradient_scale_configs = self.user_defined_strategy.gradient_scale_configs + scale_strategys = { + 'avg': BuildStrategy.GradientScaleStrategy.CoeffNumDevice, + 'sum': BuildStrategy.GradientScaleStrategy.One, + 'customized': BuildStrategy.GradientScaleStrategy.Customized, + } + assert gradient_scale_configs['scale_strategy'] in scale_strategys, \ + "gradient_scale_configs.scale_strategy must be 'avg', 'sum' or 'customized'" + local_build_strategy.gradient_scale_strategy = \ + scale_strategys[gradient_scale_configs['scale_strategy']] + if self.user_defined_strategy.recompute == True: logging.warn( "set enable_sequential_execution=True since you have enable the recompute strategy" diff --git a/python/paddle/distributed/fleet/meta_optimizers/localsgd_optimizer.py b/python/paddle/distributed/fleet/meta_optimizers/localsgd_optimizer.py index 3340672e0f925b..9052111d22c2eb 100644 --- a/python/paddle/distributed/fleet/meta_optimizers/localsgd_optimizer.py +++ b/python/paddle/distributed/fleet/meta_optimizers/localsgd_optimizer.py @@ -16,6 +16,7 @@ import paddle from paddle.fluid import program_guard, layers, default_main_program +from paddle.fluid import default_startup_program from .meta_optimizer_base import MetaOptimizerBase from .common import OpRole, OP_ROLE_KEY, CollectiveHelper, is_update_op diff --git a/python/paddle/distributed/fleet/meta_parallel/__init__.py b/python/paddle/distributed/fleet/meta_parallel/__init__.py index 4e32ff5723c418..fe7f23f3d8cc33 100644 --- a/python/paddle/distributed/fleet/meta_parallel/__init__.py +++ b/python/paddle/distributed/fleet/meta_parallel/__init__.py @@ -24,5 +24,6 @@ from .parallel_layers import get_rng_state_tracker # noqa: F401 from .tensor_parallel import TensorParallel # noqa: F401 from .pipeline_parallel import PipelineParallel # noqa: F401 +from .sharding_parallel import ShardingParallel # noqa: F401 __all__ = [] diff --git a/python/paddle/distributed/fleet/meta_parallel/parallel_layers/mp_layers.py b/python/paddle/distributed/fleet/meta_parallel/parallel_layers/mp_layers.py index f091c890f68542..2555d73462b780 100644 --- a/python/paddle/distributed/fleet/meta_parallel/parallel_layers/mp_layers.py +++ b/python/paddle/distributed/fleet/meta_parallel/parallel_layers/mp_layers.py @@ -56,7 +56,7 @@ def __init__(self, self._weight_attr = weight_attr self._name = name - if self.is_mp: + if self.is_mp and paddle.in_dynamic_mode(): with get_rng_state_tracker().rng_state(): self.weight = self.create_parameter( attr=self._weight_attr, @@ -121,7 +121,7 @@ def __init__(self, self._weight_attr = weight_attr self._dtype = self._helper.get_default_dtype() - if self.is_mp: + if self.is_mp and paddle.in_dynamic_mode(): with get_rng_state_tracker().rng_state(): self.weight = self.create_parameter( shape=[in_features, self.output_size_per_partition], @@ -198,7 +198,7 @@ def __init__(self, self.input_size_per_partition = in_features // self.world_size - if self.is_mp: + if self.is_mp and paddle.in_dynamic_mode(): with get_rng_state_tracker().rng_state(): self.weight = self.create_parameter( shape=[self.input_size_per_partition, self.out_features], diff --git a/python/paddle/distributed/fleet/meta_parallel/pipeline_parallel.py b/python/paddle/distributed/fleet/meta_parallel/pipeline_parallel.py index 0bb6315290ed72..343e6db04c24c4 100644 --- a/python/paddle/distributed/fleet/meta_parallel/pipeline_parallel.py +++ b/python/paddle/distributed/fleet/meta_parallel/pipeline_parallel.py @@ -24,6 +24,7 @@ from ..utils.hybrid_parallel_util import broadcast_dp_parameters from ..utils.log_util import logger from ..meta_optimizers.dygraph_optimizer import HybridParallelOptimizer +from .pp_utils import p2p_communication as p2p __all__ = [] @@ -63,6 +64,7 @@ def __init__(self, layers, hcg, strategy): self.prev_stage_id = self.stage_id - 1 self.next_stage_id = self.stage_id + 1 self.pp_group = self._hcg.get_pipe_parallel_group() + p2p.initialize_p2p_groups(hcg) self.is_first_stage = self.stage_id == 0 self.is_last_stage = (self.stage_id == (self.num_stages - 1)) @@ -275,97 +277,86 @@ def _send_meta(self, data, peer): if isinstance(data, paddle.Tensor): tensor_type = paddle.to_tensor([0]) # send tensor type - paddle.distributed.send( - tensor_type, peer, use_calc_stream=True, group=self.pp_group) + p2p.send(tensor_type, self.next_stage_id) # send len(shape) dims = paddle.to_tensor(len(data.shape)) - paddle.distributed.send( - dims, peer, use_calc_stream=True, group=self.pp_group) + p2p.send(dims, self.next_stage_id) # send shape shape = paddle.to_tensor(data.shape) - paddle.distributed.send( - shape, peer, use_calc_stream=True, group=self.pp_group) + p2p.send(shape, self.next_stage_id) # send dtype dtype = paddle.to_tensor(paddle_2_number(data.dtype)) - paddle.distributed.send( - dtype, peer, use_calc_stream=True, group=self.pp_group) + p2p.send(dtype, self.next_stage_id) elif isinstance(data, tuple): tensor_type = paddle.to_tensor([1]) - paddle.distributed.send( - tensor_type, peer, use_calc_stream=True, group=self.pp_group) + p2p.send(tensor_type, self.next_stage_id) + nums = paddle.to_tensor(len(data)) - paddle.distributed.send( - nums, peer, use_calc_stream=True, group=self.pp_group) + p2p.send(nums, self.next_stage_id) + for idx, d in enumerate(data): assert isinstance(d, paddle.Tensor) # send len(shape) dims = paddle.to_tensor(len(d.shape)) - paddle.distributed.send( - dims, peer, use_calc_stream=True, group=self.pp_group) + p2p.send(dims, self.next_stage_id) # send shape shape = paddle.to_tensor(d.shape) - paddle.distributed.send( - shape, peer, use_calc_stream=True, group=self.pp_group) + p2p.send(shape, self.next_stage_id) # send dtype dtype = paddle.to_tensor(paddle_2_number(d.dtype)) - paddle.distributed.send( - dtype, peer, use_calc_stream=True, group=self.pp_group) + p2p.send(dtype, self.next_stage_id) def _recv_meta(self, peer): tensor_type = paddle.to_tensor([0]) - paddle.distributed.recv( - tensor_type, peer, use_calc_stream=True, group=self.pp_group) + p2p.recv(tensor_type, self.prev_stage_id) + tensor_type = tensor_type.item() if tensor_type == 0: # recv len(shape) dims = paddle.to_tensor([0]) - paddle.distributed.recv( - dims, peer, use_calc_stream=True, group=self.pp_group) + p2p.recv(dims, self.prev_stage_id) + dims = dims.item() # recv shape shape = paddle.to_tensor([0] * dims) - paddle.distributed.recv( - shape, peer, use_calc_stream=True, group=self.pp_group) + p2p.recv(shape, self.prev_stage_id) + shape = shape.numpy().tolist() # recv dtype dtype = paddle.to_tensor([0]) - paddle.distributed.recv( - dtype, peer, use_calc_stream=True, group=self.pp_group) + p2p.recv(dtype, self.prev_stage_id) + return self._allocate_cache( shape, dtype=number_2_dtype(dtype.item()), num_caches=1)[0] elif tensor_type == 1: num = paddle.to_tensor([0]) - paddle.distributed.recv( - num, peer, use_calc_stream=True, group=self.pp_group) + p2p.recv(num, self.prev_stage_id) num = num.item() shapes = [] dtypes = [] for i in range(num): # recv len(shape) dims = paddle.to_tensor([0]) - paddle.distributed.recv( - dims, peer, use_calc_stream=True, group=self.pp_group) + p2p.recv(dims, self.prev_stage_id) # recv shape dims = dims.item() shape = paddle.to_tensor([0] * dims) - paddle.distributed.recv( - shape, peer, use_calc_stream=True, group=self.pp_group) + p2p.recv(shape, self.prev_stage_id) shapes.append(shape.numpy().tolist()) # recv dtype dtype = paddle.to_tensor([0]) - paddle.distributed.recv( - dtype, peer, use_calc_stream=True, group=self.pp_group) + p2p.recv(dtype, self.prev_stage_id) dtypes.append(number_2_dtype(dtype.item())) caches = self._allocate_caches(shapes, dtypes, num_caches=1)[0] @@ -380,39 +371,25 @@ def _send_activations(self, cache_id): self._send_meta(outputs, self.next_stage_id) if isinstance(outputs, paddle.Tensor): - paddle.distributed.send( - outputs, - self.next_stage_id, - use_calc_stream=True, - group=self.pp_group) + p2p.send(outputs, self.next_stage_id) + elif isinstance(outputs, tuple): for output in outputs: - paddle.distributed.send( - output, - self.next_stage_id, - use_calc_stream=True, - group=self.pp_group) + p2p.send(output, self.next_stage_id) def _send_gradients(self, cache_id): inputs = self.caches['inputs'][cache_id] if isinstance(inputs, paddle.Tensor): assert inputs.grad is not None - paddle.distributed.send( - paddle.to_tensor(inputs.grad), - self.prev_stage_id, - use_calc_stream=True, - group=self.pp_group) + p2p.send(inputs.grad, self.prev_stage_id) else: for idx, d in enumerate(inputs): # Skip tensors that will not produce a grad if not is_float_tensor(d): assert d.grad is None continue - paddle.distributed.send( - d.grad, - self.prev_stage_id, - use_calc_stream=True, - group=self.pp_group) + p2p.send(d.grad, self.prev_stage_id) + self.caches['inputs'][cache_id] = None def _recv_activations(self, cache_id): @@ -421,11 +398,7 @@ def _recv_activations(self, cache_id): self.recv_cache = self._recv_meta(self.prev_stage_id) if isinstance(self.recv_cache, paddle.Tensor): - paddle.distributed.recv( - self.recv_cache, - self.prev_stage_id, - use_calc_stream=True, - group=self.pp_group) + p2p.recv(self.recv_cache, self.prev_stage_id) inputs = self.recv_cache.clone().detach() inputs.stop_gradient = not is_float_tensor(inputs) else: @@ -433,12 +406,7 @@ def _recv_activations(self, cache_id): inputs = [None] * len(self.recv_cache) for idx, d in enumerate(self.recv_cache): assert isinstance(d, paddle.Tensor) - - paddle.distributed.recv( - d, - self.prev_stage_id, - use_calc_stream=True, - group=self.pp_group) + p2p.recv(d, self.prev_stage_id) inputs[idx] = d.clone().detach() inputs = tuple(inputs) @@ -466,19 +434,11 @@ def _recv_gradients(self, cache_id): sizes, dtypes, num_caches=1)[0] if isinstance(self.grad_tensors, paddle.Tensor): - paddle.distributed.recv( - self.grad_tensors, - self.next_stage_id, - use_calc_stream=True, - group=self.pp_group) + p2p.recv(self.grad_tensors, self.next_stage_id) else: assert isinstance(outputs, tuple) for d in self.grad_tensors: - paddle.distributed.recv( - d, - self.next_stage_id, - use_calc_stream=True, - group=self.pp_group) + p2p.recv(d, self.next_stage_id) def _step(self): self.optimizer.step() diff --git a/python/paddle/distributed/fleet/meta_parallel/pp_utils/p2p_communication.py b/python/paddle/distributed/fleet/meta_parallel/pp_utils/p2p_communication.py new file mode 100644 index 00000000000000..f81164b778cc27 --- /dev/null +++ b/python/paddle/distributed/fleet/meta_parallel/pp_utils/p2p_communication.py @@ -0,0 +1,72 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import paddle + +_groups = None +_hcg = None + + +def initialize_p2p_groups(hcg): + global _groups, _hcg + _groups = [ + paddle.distributed.new_group(ranks=group) + for group in hcg.get_p2p_groups() + ] + _hcg = hcg + + +def send(tensor, dest_stage): + global _groups, _hcg + src_stage = _hcg.get_stage_id() + src_rank = _hcg.get_rank_from_stage(stage_id=src_stage) + + _is_valid_communciate(src_stage, dest_stage) + group = _get_send_recv_group(src_stage, dest_stage) + dst_rank = _hcg.get_rank_from_stage(stage_id=dest_stage) + return paddle.distributed.broadcast(tensor, src_rank, group=group) + + +def recv(tensor, src_stage): + global _groups, _hcg + dest_stage = _hcg.get_stage_id() + + _is_valid_communciate(src_stage, dest_stage) + group = _get_send_recv_group(src_stage, dest_stage) + src_rank = _hcg.get_rank_from_stage(stage_id=src_stage) + return paddle.distributed.broadcast(tensor, src_rank, group=group) + + +def _is_valid_communciate(src_stage, dest_stage): + first_stage = 0 + last_stage = _hcg.get_pipe_parallel_world_size() - 1 + assert abs(src_stage-dest_stage) == 1 or \ + (src_stage == first_stage and dest_stage == last_stage) or \ + (src_stage == last_stage and dest_stage == first_stage) + + +def _get_send_recv_group(src_stage, dest_stage): + global _groups, _hcg + stage_id = None + first_stage = 0 + last_stage = _hcg.get_pipe_parallel_world_size() - 1 + if (src_stage == first_stage and dest_stage == last_stage) or \ + (dest_stage == first_stage and src_stage == last_stage): + stage_id = last_stage + elif src_stage > dest_stage: + stage_id = dest_stage + else: + stage_id = src_stage + group_id = _hcg.get_rank_from_stage(stage_id=stage_id) + return _groups[group_id] diff --git a/python/paddle/distributed/fleet/meta_parallel/sharding_parallel.py b/python/paddle/distributed/fleet/meta_parallel/sharding_parallel.py new file mode 100644 index 00000000000000..953a76d874e558 --- /dev/null +++ b/python/paddle/distributed/fleet/meta_parallel/sharding_parallel.py @@ -0,0 +1,33 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from paddle.fluid.dygraph.layers import Layer +from .meta_parallel_base import MetaParallelBase +from ..utils.hybrid_parallel_util import broadcast_sharding_parameters +from ..utils.log_util import logger + +__all__ = [] + + +class ShardingParallel(MetaParallelBase): + def __init__(self, layers, hcg, **kwargs): + super(ShardingParallel, self).__init__(layers, hcg, **kwargs) + + def _prepare_for_model(self): + logger.info("start broadcast sharding parameters") + broadcast_sharding_parameters(self._layers, self._hcg) + + # TODO (JZ-LIANG) to support Sharding-DP + + logger.info("sharding's parameters is ready") diff --git a/python/paddle/distributed/fleet/utils/hybrid_parallel_util.py b/python/paddle/distributed/fleet/utils/hybrid_parallel_util.py index ddbd6111b46099..81bed60050de29 100644 --- a/python/paddle/distributed/fleet/utils/hybrid_parallel_util.py +++ b/python/paddle/distributed/fleet/utils/hybrid_parallel_util.py @@ -119,3 +119,46 @@ def fused_allreduce_gradients(parameter_list, hcg): logger.debug("dp start fuse allreduce gradients") with framework.no_grad(): _apply_collective_grads(parameter_list, data_parallel_group) + + +def sharding_reduce_gradients(parameter_list, hcg): + # TODO allreduce --> reduce + # TODO merge grad / nrank with dp + logger.debug("sharding start gradients sync") + with framework.no_grad(): + + sharding_nrank = hcg.get_sharding_parallel_group().nranks + for param in parameter_list: + if param.trainable and (param._grad_ivar() is not None): + + g_var = param._grad_ivar() + + # need use trace_op to allreduce + # paddle.distributed.all_reduce( + # g_var, group=hcg.get_sharding_parallel_group(), use_calc_stream=True) + paddle.fluid.framework._dygraph_tracer().trace_op( + type="c_allreduce_sum", + inputs={'X': g_var}, + outputs={'Out': g_var}, + attrs={ + 'ring_id': hcg.get_sharding_parallel_group().id, + 'use_calc_stream': True + }) + + # grad / sharding_rank + div_factor = paddle.to_tensor(sharding_nrank, dtype=g_var.dtype) + paddle.fluid.framework._dygraph_tracer().trace_op( + type="elementwise_div", + inputs={'X': g_var, + 'Y': div_factor}, + outputs={'Out': g_var}, + attrs={'axis': -1}) + + +def broadcast_sharding_parameters(model, hcg): + # TODO TO save memory, use un-fused broadcast to avoid potentional OOM + logger.debug("sharding start init parameters sync") + sharding_parallel_group = hcg.get_sharding_parallel_group() + src_rank = hcg.get_sharding_parallel_group_src_rank() + sync_params_buffers( + model, sharding_parallel_group, src_rank, is_model_parallel=False) diff --git a/python/paddle/fluid/__init__.py b/python/paddle/fluid/__init__.py index f9e0e0ae047a25..fb1be483083a8c 100644 --- a/python/paddle/fluid/__init__.py +++ b/python/paddle/fluid/__init__.py @@ -93,7 +93,7 @@ from . import generator from .core import _cuda_synchronize from .generator import Generator -from .trainer_desc import TrainerDesc, DistMultiTrainer, PipelineTrainer, MultiTrainer, HeterXpuTrainer, HeterBoxTrainer +from .trainer_desc import TrainerDesc, DistMultiTrainer, PipelineTrainer, MultiTrainer, HeterXpuTrainer from .transpiler import HashName, RoundRobin from .backward import append_backward diff --git a/python/paddle/fluid/backward.py b/python/paddle/fluid/backward.py index 708167a0273996..9ce5f851846e8f 100755 --- a/python/paddle/fluid/backward.py +++ b/python/paddle/fluid/backward.py @@ -462,6 +462,7 @@ def _addup_repetitive_outputs_(op_descs, block_idx): var_rename_count = collections.defaultdict(int) renamed_vars = collections.defaultdict(list) renamed_var_start_idx = collections.defaultdict(list) + var_device = collections.defaultdict(str) for idx, op_desc in enumerate(op_descs): op_device_attr_name = core.op_proto_and_checker_maker.kOpDeviceAttrName( ) @@ -528,16 +529,19 @@ def _addup_repetitive_outputs_(op_descs, block_idx): arg_names[arg_idx] = new_name op_desc.set_output(param_name, arg_names) renamed_vars[var_name].append(new_name) + # record the latest device, for shared param + var_device[var_name] = op_device for var_name, inputs in six.iteritems(renamed_vars): if len(renamed_vars[var_name]) > 1: if len(renamed_vars[var_name]) > _MAX_ADD_NUM_: - _accumulate_gradients_by_sum_op_(var_name, renamed_vars, - pending_sum_ops, len(op_descs)) + _accumulate_gradients_by_sum_op_( + var_name, renamed_vars, pending_sum_ops, + len(op_descs), var_device[var_name]) else: - _accumulate_gradients_by_add_ops_(var_name, renamed_vars, - pending_sum_ops, - len(op_descs)) + _accumulate_gradients_by_add_ops_( + var_name, renamed_vars, pending_sum_ops, + len(op_descs), var_device[var_name]) # sum_op descs are sorted according to their insert position for key, value in collections.OrderedDict( diff --git a/python/paddle/fluid/contrib/mixed_precision/fp16_lists.py b/python/paddle/fluid/contrib/mixed_precision/fp16_lists.py index 5cfa77b3d9a4f8..18f635ee8064cb 100644 --- a/python/paddle/fluid/contrib/mixed_precision/fp16_lists.py +++ b/python/paddle/fluid/contrib/mixed_precision/fp16_lists.py @@ -152,8 +152,14 @@ def _update_list(self): # The set of ops that don't support fp16 calculation # lookup_table fp16 is slower than fp32, though fp16 is supported. -_, _, _sys_unsupported_fp16_list = core.op_supported_infos( - 'GPU', core.VarDesc.VarType.FP16) +_sys_unsupported_fp16_list = [] +if core.is_compiled_with_xpu(): + _, _, _sys_unsupported_fp16_list = core.op_supported_infos( + 'XPU', core.VarDesc.VarType.FP16) +else: + _, _, _sys_unsupported_fp16_list = core.op_supported_infos( + 'GPU', core.VarDesc.VarType.FP16) + unsupported_fp16_list = {'lookup_table', 'lookup_table_v2'} | _sys_unsupported_fp16_list diff --git a/python/paddle/fluid/contrib/slim/quantization/imperative/__init__.py b/python/paddle/fluid/contrib/slim/quantization/imperative/__init__.py index 77872e88a0733b..7210da93f7bf57 100644 --- a/python/paddle/fluid/contrib/slim/quantization/imperative/__init__.py +++ b/python/paddle/fluid/contrib/slim/quantization/imperative/__init__.py @@ -14,9 +14,6 @@ from __future__ import print_function -from . import quant_nn -from .quant_nn import * - from . import qat from .qat import * @@ -33,7 +30,6 @@ from .ptq_registry import * __all__ = [] -__all__ += quant_nn.__all__ __all__ += qat.__all__ __all__ += ptq.__all__ __all__ += ptq_config.__all__ diff --git a/python/paddle/fluid/contrib/slim/quantization/imperative/qat.py b/python/paddle/fluid/contrib/slim/quantization/imperative/qat.py index 600ce6397e1af3..3b4f9a757437aa 100644 --- a/python/paddle/fluid/contrib/slim/quantization/imperative/qat.py +++ b/python/paddle/fluid/contrib/slim/quantization/imperative/qat.py @@ -20,6 +20,7 @@ import warnings import paddle +import paddle.nn.quant.quant_layers as quant_layers from paddle.fluid import dygraph, core, framework, unique_name from paddle.fluid.executor import Executor, global_scope from paddle.fluid.param_attr import ParamAttr @@ -28,7 +29,6 @@ from paddle.fluid.io import load_inference_model, save_inference_model from paddle.fluid.log_helper import get_logger from .. import quantization_pass -from . import quant_nn from . import utils __all__ = ['ImperativeQuantAware'] @@ -39,7 +39,7 @@ class ImperativeQuantAware(object): """ - Applying quantization aware training (QAT) to dgraph model. + Applying quantization aware training (QAT) to the dgraph model. """ def __init__(self, @@ -329,12 +329,12 @@ def _get_input_quantized_layer(self, layer): "The layer %s is unsupported to be quantized." \ % layer.full_name() - return quant_nn.__dict__[quant_layer_name](layer, **self._kwargs) + return quant_layers.__dict__[quant_layer_name](layer, **self._kwargs) class ImperativeQuantizeOutputs(object): """ - Calculate the output scales for some layers. + Calculate the output scales for target layers. """ def __init__(self, moving_rate=0.9): @@ -371,11 +371,11 @@ def apply(self, model): utils.find_parent_layer_and_sub_name(model, cur_name) if isinstance(cur_layer, tuple(utils.fake_quant_output_layers)): - cur_quant_layer = quant_nn.FakeQuantMAOutputScaleLayer( + cur_quant_layer = quant_layers.FakeQuantMAOutputScaleLayer( cur_layer, self._moving_rate) else: - cur_quant_layer = quant_nn.MAOutputScaleLayer(cur_layer, - self._moving_rate) + cur_quant_layer = quant_layers.MAOutputScaleLayer( + cur_layer, self._moving_rate) setattr(parent_layer, sub_name, cur_quant_layer) @@ -433,7 +433,7 @@ def save_quantized_model(self, layer, path, input_spec=None, **config): model_filename=model_filename, params_filename=params_filename)) - self._save_output_scale(infer_program, scope) + self._gather_scales(infer_program, scope) self._set_skip_quant_attr(infer_program) @@ -455,36 +455,79 @@ def _is_target_layer(self, layer): """ flag = False if isinstance(layer, dygraph.Layer): - # exclude fake_quant ops in quant_nn file + # exclude fake_quant ops in quant_layers file if utils.is_leaf_layer(layer) and \ not isinstance(layer, tuple(utils.fake_quant_leaf_layers)): flag = True - # consider QuantizedConv2D and QuantizedLinear ops + if isinstance(layer, tuple(utils.fake_quant_wrap_layers)): flag = True - if isinstance(layer, paddle.nn.quant.FloatFunctionalLayer): - flag = True + + if isinstance(layer, paddle.nn.quant.FloatFunctionalLayer): + flag = True + return flag - def _save_output_scale(self, program, scope): + def _gather_scales(self, program, scope): """ - Save all output scales to the corresponding ops in static - inference program and delete 'moving_average_abs_max_scale' ops. + Get all scales from fake ops, save them into the corresponding ops + and delete all moving_average_abs_max_scale ops. """ - for block in program.blocks: - for op in block.ops: - if op.type == "moving_average_abs_max_scale": - in_var_name = op.input('X')[0] - out_var_name = op.output('Out')[0] - out_scale_name = op.output('OutScale')[0] - - out_scale = utils.load_variable_data(scope, out_scale_name) - previous_op = utils.find_previous_op(block, in_var_name) - previous_op._set_attr("out_threshold", float(out_scale)) - - next_ops = utils.find_next_ops(block, out_var_name) - for next_op in next_ops: - next_op._rename_input(out_var_name, in_var_name) + + def _gather_input_scale(): + target_ops = [] + skip_ops = utils.fake_quantize_dequantize_op_types + \ + ["moving_average_abs_max_scale"] + for block in program.blocks: + for op in block.ops: + if op.type not in skip_ops: + target_ops.append(op) + + for op in target_ops: + for in_var_name in utils._get_op_input_var_names(op): + previous_op = utils.find_previous_op(op.block, in_var_name) + + if previous_op is not None and \ + ("quantize_dequantize" in previous_op.type or \ + previous_op.type == "moving_average_abs_max_scale"): + scale_name = previous_op.output('OutScale')[0] + in_scale = utils.load_variable_data(scope, scale_name) + in_scale = utils.fp_numpy_to_naive(in_scale) + argname, index = utils._get_input_name_index( + op, in_var_name) + op._set_attr(argname + str(index) + "_threshold", + in_scale) + + def _gather_output_scale(): + target_ops = [] + for block in program.blocks: + for op in block.ops: + if op.type == "moving_average_abs_max_scale": + target_ops.append(op) + + for op in target_ops: + in_var_name = op.input('X')[0] + out_var_name = op.output('Out')[0] + block = op.block + previous_op = utils.find_previous_op(block, in_var_name) + next_ops = utils.find_next_ops(block, out_var_name) + + out_scale_name = op.output('OutScale')[0] + out_scale = utils.load_variable_data(scope, out_scale_name) + out_scale = utils.fp_numpy_to_naive(out_scale) + + if previous_op.type != "feed": + argname, index = utils._get_output_name_index(previous_op, + in_var_name) + previous_op._set_attr(argname + str(index) + "_threshold", + out_scale) + previous_op._set_attr("out_threshold", out_scale) + + for next_op in next_ops: + next_op._rename_input(out_var_name, in_var_name) + + _gather_input_scale() + _gather_output_scale() def _set_skip_quant_attr(self, program): """ diff --git a/python/paddle/fluid/contrib/slim/quantization/imperative/utils.py b/python/paddle/fluid/contrib/slim/quantization/imperative/utils.py index 98eefc73608122..4158c52d5ae257 100644 --- a/python/paddle/fluid/contrib/slim/quantization/imperative/utils.py +++ b/python/paddle/fluid/contrib/slim/quantization/imperative/utils.py @@ -16,8 +16,12 @@ import numpy as np import paddle +import paddle.nn.quant.quant_layers as quant_layers -from . import quant_nn +from ..quantization_pass import _get_op_input_var_names +from ..quantization_pass import _get_op_output_var_names +from ..quantization_pass import _get_output_name_index +from ..quantization_pass import _get_input_name_index layer_name_map = { 'Conv2D': paddle.nn.Conv2D, @@ -54,13 +58,15 @@ ] fake_quant_leaf_layers = [ - quant_nn.FakeQuantAbsMax, - quant_nn.FakeQuantChannelWiseAbsMax, - quant_nn.FakeQuantMovingAverageAbsMax, - quant_nn.MovingAverageAbsMaxScale, + quant_layers.FakeQuantAbsMax, + quant_layers.FakeQuantChannelWiseAbsMax, + quant_layers.FakeQuantMovingAverageAbsMax, + quant_layers.MovingAverageAbsMaxScale, ] -fake_quant_wrap_layers = [quant_nn.QuantizedConv2D, quant_nn.QuantizedLinear] +fake_quant_wrap_layers = [ + quant_layers.QuantizedConv2D, quant_layers.QuantizedLinear +] # The weight format of these layers is Cin * Cout * H * W spec_channel_axis_layers = [paddle.nn.Conv2D, paddle.nn.Conv2DTranspose] @@ -94,6 +100,7 @@ def find_previous_op(block, var_name): for op in block.ops: if var_name in op.output_arg_names: return op + return None def find_next_ops(block, var_name): @@ -244,3 +251,10 @@ def cal_kl_scaling_factor(hist, abs_max, bits): break min_kl_index = starting_iter return (min_kl_index + 0.5) * bin_width + + +def fp_numpy_to_naive(x_np): + if x_np.size == 1: + return float(x_np) + else: + return x_np.tolist() diff --git a/python/paddle/fluid/contrib/slim/quantization/quantization_pass.py b/python/paddle/fluid/contrib/slim/quantization/quantization_pass.py index 320c14d4e9ca41..b3b12a477e2a0a 100644 --- a/python/paddle/fluid/contrib/slim/quantization/quantization_pass.py +++ b/python/paddle/fluid/contrib/slim/quantization/quantization_pass.py @@ -141,12 +141,21 @@ def _get_op_input_var_names(op): - """ """ + """ + Get the input var names of the op. + Args: + op(IrNode, Operator): the input op. + Returns: + input_var_names or None. + """ assert isinstance(op, (IrNode, Operator)), \ "The input op should be IrNode or Operator." var_names = [] op_name = op.name() if isinstance(op, IrNode) \ else op.type + if op_name not in _op_real_in_out_name: + return [] + name_list = _op_real_in_out_name[op_name][0] for name in name_list: var_name = op.input(name) @@ -163,6 +172,9 @@ def _get_input_name_index(op, input_var_name): "The input op should be IrNode or Operator." op_name = op.name() if isinstance(op, IrNode) \ else op.type + if op_name not in _op_real_in_out_name: + return None + res = None for argname in _op_real_in_out_name[op_name][0]: var_names = op.input(argname) @@ -179,6 +191,9 @@ def _get_op_output_var_names(op): var_names = [] op_name = op.name() if isinstance(op, IrNode) \ else op.type + if op_name not in _op_real_in_out_name: + return [] + name_list = _op_real_in_out_name[op_name][1] for name in name_list: var_name = op.output(name) @@ -195,6 +210,9 @@ def _get_output_name_index(op, output_var_name): "The input op should be IrNode or Operator." op_name = op.name() if isinstance(op, IrNode) \ else op.type + if op_name not in _op_real_in_out_name: + return None + name_list = _op_real_in_out_name[op_name][1] res = None for name in name_list: @@ -1148,7 +1166,7 @@ def apply(self, graph): ], "the dim of scale_v should be 1 or 2" if scale_v.ndim == 2: scale_v = scale_v[0] - if scale_v.size == 1: + if scale_v.size == 1 and self._weight_quantize_type == 'abs_max': scale_v = scale_v[0] else: scale_v = scale_v.tolist() @@ -1183,7 +1201,8 @@ def apply(self, graph): if op_node_desc.has_attr("quantization_type") and \ op_node_desc.attr("quantization_type") == "qat_with_weight": if self._weight_quantize_type == 'channel_wise_abs_max': - self._insert_post_channel_dequant_op(graph, op_node) + self._insert_post_channel_dequant_op(graph, op_node, + quant_axis) else: self._insert_post_dequant_op(graph, op_node) @@ -1210,7 +1229,7 @@ def _remove_fake_quant_and_dequant_op(self, graph, op_node): v.node] graph.safe_remove_nodes(op_node) - def _insert_post_channel_dequant_op(self, graph, op_node): + def _insert_post_channel_dequant_op(self, graph, op_node, quant_axis): persistable_vars = [p.name() for p in graph.all_persistable_nodes()] for var_node in op_node.inputs: name = var_node.name() @@ -1258,6 +1277,7 @@ def _insert_post_channel_dequant_op(self, graph, op_node): op_type='fake_channel_wise_dequantize_max_abs', attrs={ 'quant_bits': [self._weight_bits, self._activation_bits], + 'quant_axis': quant_axis, 'op_role': core.op_proto_and_checker_maker.OpRole.Forward }, inputs={ diff --git a/python/paddle/fluid/contrib/slim/quantization/quantize_transpiler_v2.py b/python/paddle/fluid/contrib/slim/quantization/quantize_transpiler_v2.py index cde3d991a7f2fd..753d68f7970327 100644 --- a/python/paddle/fluid/contrib/slim/quantization/quantize_transpiler_v2.py +++ b/python/paddle/fluid/contrib/slim/quantization/quantize_transpiler_v2.py @@ -17,6 +17,7 @@ import numpy as np from .... import core from ....framework import Program, Operator, Variable, program_guard +from ....executor import global_scope from .... import unique_name from ....layer_helper import LayerHelper from ....param_attr import ParamAttr @@ -27,26 +28,49 @@ __name__, logging.INFO, fmt='%(asctime)s-%(levelname)s: %(message)s') +def find_next_ops(block, var_name): + """ + Find all followed ops for the input variable. + """ + res_ops = [] + for op in block.ops: + if var_name in op.input_arg_names: + res_ops.append(op) + return res_ops + + +def load_variable_data(scope, var_name): + ''' + Load variable value from scope + ''' + var_node = scope.find_var(var_name) + assert var_node is not None, \ + "Cannot find " + var_name + " in scope." + return np.array(var_node.get_tensor()) + + class QuantizeTranspilerV2(object): def __init__(self, weight_bits=8, activation_bits=8, weight_quantize_type='abs_max', - activation_quantize_type='abs_max', - quantizable_op_type=['conv2d', 'depthwise_conv2d', 'mul'], + activation_quantize_type='moving_average_abs_max', + quantizable_op_type=[ + 'conv2d', + 'depthwise_conv2d', + 'mul', + ], skip_pattern=['skip_quant']): """ - Add quant_dequant op before the quantized op to quantize the fluid Program. - It is a patch for distributed quantization, we will support others module for - distributed quantization. + Apply fake quant for the quantized ops. Args: weight_bits(int): the bit of quantized weight. activation_bits(int): the bit of quantized activation. weight_quantize_type(str): the quantization type for weight. - Only support to be 'abs_max' for now. + Only support to be 'abs_max' and 'channel_wise_abs_max'. activation_quantize_type(str): the quantization type for activation. - Only support to be 'abs_max' for now. + Only support to be 'abs_max' and 'moving_average_abs_max'. quantizable_op_type(str): set the op type for quantization. skip_pattern(str|list): The user-defined quantization skip pattern, which will be presented in the name scope of an op. When the skip pattern is @@ -55,28 +79,37 @@ def __init__(self, self._weight_bits = weight_bits self._activation_bits = activation_bits - assert activation_quantize_type == "abs_max", \ - "activation_quantize_type should be abs_max for now." - assert weight_quantize_type == "abs_max", \ - "weight_quantize_type should be abs_max for now." + assert activation_quantize_type in \ + ["abs_max", "moving_average_abs_max"], \ + "activation_quantize_type should be abs_max " \ + "or moving_average_abs_max for now." + assert weight_quantize_type in ["abs_max", "channel_wise_abs_max"], \ + "weight_quantize_type should be abs_max or channel_wise_abs_max." self._activation_quantize_type = activation_quantize_type self._weight_quantize_type = weight_quantize_type + for op_type in quantizable_op_type: + assert op_type in ['conv2d', 'depthwise_conv2d', 'mul'], \ + "Quantize op should be ['conv2d', 'depthwise_conv2d', 'mul']" self._quantizable_ops = quantizable_op_type self._quantizable_grad_ops = [ '%s_grad' % (op) for op in self._quantizable_ops ] self._skip_pattern = skip_pattern - self.helper = LayerHelper(self.__class__.__name__) + self._helper = LayerHelper(self.__class__.__name__) - def apply(self, program, startup_program): + self._moving_rate = 0.9 + self._out_ch_axis1_ops = ['conv2d_transpose', 'mul', 'matmul'] + + def apply(self, program, startup_program, is_test=False): """ Apply quantization to fluid Program. Args: program(Program): the train or test program to be quantized. startup_program(Program): the corresponding startup_program. + is_test(bool): Whethe the program is used for test. Returns: None """ @@ -85,7 +118,7 @@ def apply(self, program, startup_program): assert isinstance(startup_program, Program), \ "startup_program must be the instance of Program" - quant_dequant_vars = [ + var_rename_map = [ collections.OrderedDict() for _ in range(len(program.blocks)) ] with program_guard(program, startup_program): @@ -94,13 +127,104 @@ def apply(self, program, startup_program): for op in ops: if op.type in self._quantizable_ops and \ (not self._is_skip_quant(op)): - self._transform_forward(block, op, quant_dequant_vars) + self._transform_forward(block, op, var_rename_map, + is_test) + for block in program.blocks: ops = list(block.ops) for op in ops: if op.type in self._quantizable_grad_ops and \ (not self._is_skip_quant(op)): - self._transform_backward(block, op, quant_dequant_vars) + self._transform_backward(block, op, var_rename_map) + + def convert(self, test_program, scope=None): + """ + Convert the test program. + Get the out scale from the moving_average_abs_max_scale op and save the + out scale into the quantized op. + Args: + test_program(Program): the test program to be converted. + scope(fluid.Scope, optional): The scope of the program, use it to load + and save variables. If scope=None, get scope by global_scope(). + """ + scope = global_scope() if scope == None else scope + + for block in test_program.blocks: + for op in block.ops: + if op.has_attr("quantization_type") \ + and op.attr("quantization_type") == "qat_with_weight": + # quant op -> var1 -> fake op -> var2 + assert len(op.output_arg_names) == 1 + var1_name = op.output_arg_names[0] + + fake_ops = find_next_ops(block, var1_name) + assert len(fake_ops) == 1 + fake_op = fake_ops[0] + assert fake_op.type == "moving_average_abs_max_scale" + + out_scale_name = fake_op.output("OutScale") + out_threshold = load_variable_data(scope, out_scale_name[0]) + op._set_attr("out_threshold", float(out_threshold)) + + var2_name = fake_op.output("Out")[0] + op._rename_output(var1_name, var2_name) + fake_op._rename_output(var2_name, var1_name) + + def _transform_forward(self, block, op, var_rename_map, is_test): + """ + Insert fake quant op before the target ops. + """ + op._set_attr("quantization_type", "qat_with_weight") + + # insert fake quant op before the quantized op + for in_name in op.input_arg_names: + block_id = block.idx + idx = block.ops.index(op) + + if in_name in var_rename_map[block_id]: + new_in_name = var_rename_map[block_id][in_name] + else: + in_var = block.var(in_name) + if in_var.dtype != core.VarDesc.VarType.FP32: + continue + + quant_bits = self._weight_bits if in_var.persistable \ + else self._activation_bits + quant_type = self._weight_quantize_type if in_var.persistable \ + else self._activation_quantize_type + + if quant_type == "abs_max": + new_var = self._insert_abs_max_fq_op(block, idx, in_var, + quant_bits) + elif quant_type == "moving_average_abs_max": + new_var = self._insert_ma_abs_max_fq_op(block, idx, in_var, + quant_bits, is_test) + elif quant_type == "channel_wise_abs_max": + ch_axis = 1 if op.type in self._out_ch_axis1_ops else 0 + new_var = self._insert_pc_abs_max_fq_op(block, idx, in_var, + quant_bits, ch_axis) + else: + _logger.error("Don't support the quant_type: %s" % + quant_type) + continue + + new_in_name = new_var.name + var_rename_map[block_id][in_name] = new_in_name + + op._rename_input(in_name, new_in_name) + + # insert out scale op followed the quantized op + for out_name in op.output_arg_names: + next_ops = find_next_ops(block, out_name) + + idx = block.ops.index(op) + out_var = block.var(out_name) + new_out_var = self._insert_ma_abs_max_scale_op( + block, idx + 1, out_var, is_test, True) + + for next_op in next_ops: + if "_grad" not in next_op.type: + next_op._rename_input(out_name, new_out_var.name) def _is_skip_quant(self, op): """ @@ -117,49 +241,35 @@ def _is_skip_quant(self, op): self._skip_pattern) != -1 return user_skipped - def _transform_forward(self, block, op, quant_dequant_vars): - op._set_attr("quantization_type", "qat_with_weight") - idx = block.ops.index(op) - block_id = block.idx - for in_name in op.input_arg_names: - if in_name in quant_dequant_vars[block_id]: - quant_dequant_var = quant_dequant_vars[block_id][in_name] - else: - in_var = block.var(in_name) - quant_bits = self._weight_bits if in_var.persistable \ - else self._activation_bits - quant_type = self._weight_quantize_type if in_var.persistable \ - else self._activation_quantize_type - if quant_type == "abs_max": - quant_dequant_var = self._insert_quant_dequant_abs_max_op( - block, idx, in_var, quant_bits) - else: - _logger.error("Quant_type only supported to be abs_max") - quant_dequant_vars[block_id][in_name] = quant_dequant_var - op._rename_input(in_name, quant_dequant_var.name) - - def _transform_backward(self, block, op, quant_dequant_vars): + def _transform_backward(self, block, op, var_rename_map): + """ + Update the backword of the target ops. + Note: for the grad ops, only rename the input, skip rename the output. + """ block_id = block.idx no_dequanted_input_vars = True for name in op.input_arg_names: - if name in quant_dequant_vars[block_id]: - dequant_var = quant_dequant_vars[block_id][name] - op._rename_input(name, dequant_var.name) + if name in var_rename_map[block_id]: + new_var_name = var_rename_map[block_id][name] + op._rename_input(name, new_var_name) no_dequanted_input_vars = False if no_dequanted_input_vars: raise ValueError("There is no dequanted inputs for op %s." % (op.type)) - def _insert_quant_dequant_abs_max_op(self, block, idx, in_var, quant_bits): + def _insert_abs_max_fq_op(self, block, idx, in_var, quant_bits): + """ + Inset abs max fake quant op. + """ quant_dequant_var = block.create_var( type=in_var.type, name="{}.quant_dequant".format(in_var.name), shape=in_var.shape, dtype=in_var.dtype) - scale_var = self.helper.create_parameter( + scale_var = self._helper.create_parameter( attr=ParamAttr( name="{}.quant_dequant.scale".format(in_var.name), - initializer=Constant(0.001), + initializer=Constant(0.), trainable=False), shape=[1], dtype=in_var.dtype) @@ -175,3 +285,157 @@ def _insert_quant_dequant_abs_max_op(self, block, idx, in_var, quant_bits): inputs=inputs, outputs=outputs) return quant_dequant_var + + def _insert_ma_abs_max_fq_op(self, block, idx, in_var, quant_bits, is_test): + """ + Insert moving average abs max fake quant op. + """ + quant_dequant_var = block.create_var( + type=in_var.type, + name="{}.quant_dequant".format(in_var.name), + shape=in_var.shape, + dtype=in_var.dtype) + + scale_var = self._helper.create_parameter( + attr=ParamAttr( + name="{}.quant_dequant.scale".format(in_var.name), + initializer=Constant(0.), + trainable=False), + shape=[1], + dtype=in_var.dtype) + scale_var.stop_gradient = True + + if not is_test: + state_var = self._helper.create_parameter( + attr=ParamAttr( + name="{}.quant_dequant.state".format(in_var.name), + initializer=Constant(0), + trainable=False), + shape=[1], + dtype=in_var.dtype) + state_var.stop_gradient = True + + accum_var = self._helper.create_parameter( + attr=ParamAttr( + name="{}.quant_dequant.accum".format(in_var.name), + initializer=Constant(0), + trainable=False), + shape=[1], + dtype=in_var.dtype) + accum_var.stop_gradient = True + + attrs = { + 'moving_rate': self._moving_rate, + 'bit_length': quant_bits, + 'is_test': is_test + } + inputs = {'X': in_var, 'InScale': scale_var} + outputs = {'Out': quant_dequant_var, 'OutScale': scale_var} + if not is_test: + inputs['InState'] = state_var + inputs['InAccum'] = accum_var + outputs['OutState'] = state_var + outputs['OutAccum'] = accum_var + + block._insert_op( + idx, + type='fake_quantize_dequantize_moving_average_abs_max', + attrs=attrs, + inputs=inputs, + outputs=outputs) + return quant_dequant_var + + def _insert_pc_abs_max_fq_op(self, block, idx, in_var, quant_bits, ch_axis): + """ + Insert per channel abs max fake quant op. + """ + quant_dequant_var = block.create_var( + type=in_var.type, + name="{}.quant_dequant".format(in_var.name), + shape=in_var.shape, + dtype=in_var.dtype) + + scale_var = self._helper.create_parameter( + attr=ParamAttr( + name="{}.quant_dequant.scale".format(in_var.name), + initializer=Constant(0.), + trainable=False), + shape=[in_var.shape[ch_axis]], + dtype=in_var.dtype) + scale_var.stop_gradient = True + + inputs = {'X': in_var} + outputs = {'Out': quant_dequant_var, 'OutScale': scale_var} + attrs = {'bit_length': quant_bits, 'quant_axis': ch_axis} + block._insert_op( + idx, + type='fake_channel_wise_quantize_dequantize_abs_max', + attrs=attrs, + inputs=inputs, + outputs=outputs) + return quant_dequant_var + + def _insert_ma_abs_max_scale_op(self, + block, + idx, + in_var, + is_test, + has_out_var=False): + """ + Insert moving average abs max scale op. + """ + scale_var = self._helper.create_parameter( + attr=ParamAttr( + name="{}.outscale.scale".format(in_var.name), + initializer=Constant(0.), + trainable=False), + shape=[1], + dtype=in_var.dtype) + scale_var.stop_gradient = True + + attrs = {'moving_rate': self._moving_rate, 'is_test': is_test} + inputs = {'X': in_var} + outputs = {'OutScale': scale_var} + + if not is_test: + state_var = self._helper.create_parameter( + attr=ParamAttr( + name="{}.outscale.state".format(in_var.name), + initializer=Constant(0), + trainable=False), + shape=[1], + dtype=in_var.dtype) + state_var.stop_gradient = True + + accum_var = self._helper.create_parameter( + attr=ParamAttr( + name="{}.outscale.accum".format(in_var.name), + initializer=Constant(0), + trainable=False), + shape=[1], + dtype=in_var.dtype) + accum_var.stop_gradient = True + + inputs['InState'] = state_var + inputs['InAccum'] = accum_var + outputs['OutState'] = state_var + outputs['OutAccum'] = accum_var + + if has_out_var: + out_var = block.create_var( + type=in_var.type, + name="{}.tmp".format(in_var.name), + shape=in_var.shape, + dtype=in_var.dtype) + + outputs['Out'] = out_var + + block._insert_op( + idx, + type='moving_average_abs_max_scale', + attrs=attrs, + inputs=inputs, + outputs=outputs) + + if has_out_var: + return out_var diff --git a/python/paddle/fluid/contrib/slim/tests/test_imperative_qat.py b/python/paddle/fluid/contrib/slim/tests/test_imperative_qat.py index 3cc61ce8c58088..39d44060abfb38 100644 --- a/python/paddle/fluid/contrib/slim/tests/test_imperative_qat.py +++ b/python/paddle/fluid/contrib/slim/tests/test_imperative_qat.py @@ -31,7 +31,7 @@ from paddle.nn import Linear, Conv2D, Softmax from paddle.fluid.log_helper import get_logger from paddle.fluid.dygraph.io import INFER_MODEL_SUFFIX, INFER_PARAMS_SUFFIX -from paddle.fluid.contrib.slim.quantization.imperative.quant_nn import QuantizedConv2D +from paddle.nn.quant.quant_layers import QuantizedConv2D from imperative_test_utils import fix_model_dict, ImperativeLenet diff --git a/python/paddle/fluid/contrib/slim/tests/test_moving_average_abs_max_scale_op.py b/python/paddle/fluid/contrib/slim/tests/test_moving_average_abs_max_scale_op.py index 10c01566d05ee2..656fb1dda3bd11 100644 --- a/python/paddle/fluid/contrib/slim/tests/test_moving_average_abs_max_scale_op.py +++ b/python/paddle/fluid/contrib/slim/tests/test_moving_average_abs_max_scale_op.py @@ -20,7 +20,7 @@ import paddle import paddle.fluid as fluid from paddle.fluid import core -from paddle.fluid.contrib.slim.quantization.imperative import quant_nn +import paddle.nn.quant.quant_layers as quant_layers paddle.enable_static() @@ -45,7 +45,7 @@ def check_backward(self, use_cuda): name='image', shape=[784], dtype='float32') label = fluid.layers.data(name='label', shape=[1], dtype='int64') fc_tmp = fluid.layers.fc(image, size=10, act='softmax') - out_scale = quant_nn.MovingAverageAbsMaxScale( + out_scale = quant_layers.MovingAverageAbsMaxScale( name=fc_tmp.name, dtype=fc_tmp.dtype) fc_tmp_1 = out_scale(fc_tmp) cross_entropy = fluid.layers.softmax_with_cross_entropy(fc_tmp, diff --git a/python/paddle/fluid/contrib/slim/tests/test_quantize_transpiler_v2.py b/python/paddle/fluid/contrib/slim/tests/test_quantize_transpiler_v2.py index 00f2b597d934ba..aa9f6a1801cbf6 100644 --- a/python/paddle/fluid/contrib/slim/tests/test_quantize_transpiler_v2.py +++ b/python/paddle/fluid/contrib/slim/tests/test_quantize_transpiler_v2.py @@ -79,6 +79,7 @@ def build_program(main, startup, is_test): random.seed(0) np.random.seed(0) + # 1 Define program train_program = fluid.Program() startup_program = fluid.Program() test_program = fluid.Program() @@ -93,15 +94,14 @@ def build_program(main, startup, is_test): test_graph = IrGraph(core.Graph(test_program.desc), for_test=True) test_graph.draw('.', 'test_program_1') + # 2 Apply quantization qt = QuantizeTranspilerV2( activation_quantize_type=activation_quant_type, - weight_quantize_type=weight_quant_type, - quantizable_op_type=[ - 'conv2d', 'depthwise_conv2d', 'mul', 'pool2d' - ]) - qt.apply(train_program, startup_program) - qt.apply(test_program, startup_program) + weight_quantize_type=weight_quant_type) + qt.apply(train_program, startup_program, is_test=False) + qt.apply(test_program, startup_program, is_test=True) + # 3 Train place = fluid.CUDAPlace(0) if use_cuda else fluid.CPUPlace() exe = fluid.Executor(place) scope = fluid.Scope() @@ -120,28 +120,32 @@ def build_program(main, startup, is_test): build_strategy.fuse_all_reduce_ops = False binary = fluid.CompiledProgram(train_program).with_data_parallel( loss_name=loss.name, build_strategy=build_strategy) - iters = 2 + iters = 5 batch_size = 8 train_reader = paddle.batch( paddle.dataset.mnist.train(), batch_size=batch_size) feeder = fluid.DataFeeder(feed_list=feeds, place=place) with fluid.scope_guard(scope): - for _ in range(iters): + for idx in range(iters): data = next(train_reader()) loss_v = exe.run(binary, feed=feeder.feed(data), fetch_list=[loss]) - if not for_ci: - print('{}: {}'.format('loss', loss_v)) + if not for_ci and idx % 20 == 0: + print('{}: {}'.format('loss', np.mean(loss_v))) + print('{}: {}'.format('loss', np.mean(loss_v))) + + # 4 Convert + qt.convert(test_program, scope) if not for_ci: with fluid.scope_guard(scope): fluid.io.save_inference_model('./infer_model', ['image', 'label'], [loss], exe, test_program) - def test_quantize_program_gpu(self): + def test_gpu_1(self): if fluid.core.is_compiled_with_cuda(): self.quantize_program( use_cuda=True, @@ -150,7 +154,16 @@ def test_quantize_program_gpu(self): weight_quant_type='abs_max', for_ci=True) - def test_quantize_program_cpu(self): + def test_gpu_2(self): + if fluid.core.is_compiled_with_cuda(): + self.quantize_program( + use_cuda=True, + seed=1, + activation_quant_type='moving_average_abs_max', + weight_quant_type='channel_wise_abs_max', + for_ci=True) + + def test_cpu_1(self): self.quantize_program( use_cuda=False, seed=2, @@ -158,6 +171,14 @@ def test_quantize_program_cpu(self): weight_quant_type='abs_max', for_ci=True) + def test_cpu_2(self): + self.quantize_program( + use_cuda=False, + seed=2, + activation_quant_type='moving_average_abs_max', + weight_quant_type='channel_wise_abs_max', + for_ci=True) + if __name__ == '__main__': unittest.main() diff --git a/python/paddle/fluid/core.py b/python/paddle/fluid/core.py index 7886b6b3f7ad7c..ce9511a3766728 100644 --- a/python/paddle/fluid/core.py +++ b/python/paddle/fluid/core.py @@ -290,17 +290,13 @@ def to_list(s): else: from .. import compat as cpt sys.stderr.write( - "WARNING: AVX is supported on local machine, but you have installed " - "paddlepaddle without avx core. Hence, no_avx core which has worse " - "preformance will be imported.\nYou could reinstall paddlepaddle by " - "'python -m pip install --force-reinstall paddlepaddle-gpu[==version]' or rebuild " - "paddlepaddle WITH_AVX=ON to get better performance.\n" - "The original error is: %s\n" % cpt.get_exception_message(e)) + "Hint: Your machine support AVX, but the installed paddlepaddle doesn't have avx core. " + "Hence, no-avx core with worse preformance will be imported.\nIf you like, you could " + "reinstall paddlepaddle by 'python -m pip install --force-reinstall paddlepaddle-gpu[==version]' " + "to get better performance.\nThe original error is: %s\n" % + cpt.get_exception_message(e)) load_noavx = True else: - sys.stderr.write( - "WARNING: AVX is not support on your machine. Hence, no_avx core will be imported, " - "It has much worse preformance than avx core.\n") load_noavx = True if load_noavx: @@ -339,17 +335,14 @@ def to_list(s): current_path + os.sep + 'core_noavx.' + core_suffix + '\n') elif avx_supported(): sys.stderr.write( - "Error: AVX is support on your machine, but you have installed " - "paddlepaddle without avx core, you should reinstall paddlepaddle by " - "'python -m pip install --force-reinstall paddlepaddle-gpu[==version]\n" + "Error: The installed PaddlePaddle is incorrect. You should reinstall it by " + "'python -m pip install --force-reinstall paddlepaddle-gpu[==version]'\n" ) else: sys.stderr.write( - "Error: AVX is not support on your machine, but you have installed " - "paddlepaddle without no_avx core, you should reinstall paddlepaddle by " - "'python -m pip install --force-reinstall paddlepaddle-gpu[==version] -f " - "https://paddlepaddle.org.cn/whl/mkl/stable/noavx.html or " - "https://paddlepaddle.org.cn/whl/openblas/stable/noavx.html\n") + "Error: Your machine doesn't support AVX, but the installed PaddlePaddle is avx core, " + "you should reinstall paddlepaddle with no-avx core.\n") + raise e diff --git a/python/paddle/fluid/dataloader/flat.py b/python/paddle/fluid/dataloader/flat.py index db3a725ece01c2..32c8ef02dd915b 100644 --- a/python/paddle/fluid/dataloader/flat.py +++ b/python/paddle/fluid/dataloader/flat.py @@ -120,7 +120,7 @@ def _restore(structure, field_idx): elif isinstance(field, (Sequence, Mapping)): field_idx = _restore(structure[k], field_idx) else: - raise TypeError("wrong flat data type: {}".format(type(batch))) + raise TypeError("wrong flat data type: {}".format(type(structure))) return field_idx diff --git a/python/paddle/fluid/distributed/fleet.py b/python/paddle/fluid/distributed/fleet.py index 8f3d2defb9f063..cd2611956850ff 100644 --- a/python/paddle/fluid/distributed/fleet.py +++ b/python/paddle/fluid/distributed/fleet.py @@ -13,6 +13,7 @@ import sys from .. import core from . import ps_instance +from google.protobuf import text_format __all__ = ['Fleet'] diff --git a/python/paddle/fluid/distributed/node.py b/python/paddle/fluid/distributed/node.py index a15f94f4d17fca..5a1e9362c2fbcb 100644 --- a/python/paddle/fluid/distributed/node.py +++ b/python/paddle/fluid/distributed/node.py @@ -12,6 +12,8 @@ # See the License for the specific language governing permissions and import ps_pb2 as pslib +# NOTE: reduce removed in fuctools in python3 +from functools import reduce class Server(object): diff --git a/python/paddle/fluid/dygraph/amp/auto_cast.py b/python/paddle/fluid/dygraph/amp/auto_cast.py index b14b2be7394a1c..7af8c18e33f8f7 100644 --- a/python/paddle/fluid/dygraph/amp/auto_cast.py +++ b/python/paddle/fluid/dygraph/amp/auto_cast.py @@ -130,9 +130,10 @@ def amp_guard(enable=True, custom_white_list=None, custom_black_list=None): raise ValueError( "current_tracer is None, maybe it is not in imperative mode.") - if enable and not tracer._expected_place.is_gpu_place(): + if enable and not (tracer._expected_place.is_gpu_place() or + tracer._expected_place.is_xpu_place()): warnings.warn( - 'amp_guard can only be enabled on CUDAPlace, current place is %s, so it makes no effect.' + 'amp_guard can only be enabled on CUDAPlace and XPUPlace, current place is %s, so it makes no effect.' % tracer._expected_place) enable = False diff --git a/python/paddle/fluid/dygraph/amp/loss_scaler.py b/python/paddle/fluid/dygraph/amp/loss_scaler.py index ff57f30dcd2ec7..1817b78b60b907 100644 --- a/python/paddle/fluid/dygraph/amp/loss_scaler.py +++ b/python/paddle/fluid/dygraph/amp/loss_scaler.py @@ -90,9 +90,10 @@ def __init__(self, raise ValueError( "current_tracer is None, maybe it is not in imperative mode.") - if enable and not tracer._expected_place.is_gpu_place(): + if enable and not (tracer._expected_place.is_gpu_place() or + tracer._expected_place.is_xpu_place()): warnings.warn( - 'AmpScaler can only be enabled on CUDAPlace, current place is %s, so it makes no effect.' + 'AmpScaler can only be enabled on CUDAPlace and XPUPlace, current place is %s, so it makes no effect.' % tracer._expected_place) enable = False @@ -243,3 +244,115 @@ def _update(self): self._incr_count = 0 return + + def is_enable(self): + """ + Enable loss scaling or not. + + Returns: + bool: enable loss scaling return True else return False. + """ + return self._enable + + def is_use_dynamic_loss_scaling(self): + """ + Whether to use dynamic loss scaling. + + Returns: + bool: if fixed loss_scaling is used return False, if the loss scaling is updated dynamicly return true. + """ + return self._use_dynamic_loss_scaling + + def get_init_loss_scaling(self): + """ + Return the initial loss scaling factor. + + Reurns: + float: the initial loss scaling factor. + """ + return self._init_loss_scaling + + def set_init_loss_scaling(self, new_init_loss_scaling): + """ + Set the initial loss scaling factor by `new_init_loss_scaling`. + + Args: + new_init_loss_scaling(int): The new_init_loss_scaling used to update initial loss scaling factor.s + """ + self._init_loss_scaling = new_init_loss_scaling + self._scale = to_variable( + np.array([self._init_loss_scaling]).astype(np.float32)) + + def get_incr_ratio(self): + """ + Return the multiplier to use when increasing the loss scaling. + + Reurns: + float: the multiplier to use when increasing the loss scaling. + """ + return self._incr_ratio + + def set_incr_ratio(self, new_incr_ratio): + """ + Set the multiplier to use when increasing the loss scaling by `new_incr_ratio`, `new_incr_ratio` should > 1.0. + + Args: + new_incr_ratio(float): The new_incr_ratio used to update the multiplier to use when increasing the loss scaling. + """ + assert new_incr_ratio > 1.0, "The new_incr_ratio must be > 1.0." + self._incr_ratio = new_incr_ratio + + def get_decr_ratio(self): + """ + Get the less-than-one-multiplier to use when decreasing the loss scaling. + + Reurns: + float: the less-than-one-multiplier to use when decreasing the loss scaling. + """ + return self._decr_ratio + + def set_decr_ratio(self, new_decr_ratio): + """ + Set the less-than-one-multiplier to use when decreasing the loss scaling by `new_incr_ratio`, `new_decr_ratio` should < 1.0. + + Args: + new_decr_ratio(float): The new_decr_ratio used to update the less-than-one-multiplier to use when decreasing the loss scaling. + """ + assert new_decr_ratio < 1.0, "The new_decr_ratio must be < 1.0." + self._decr_ratio = new_decr_ratio + + def get_incr_every_n_steps(self): + """ + Return the num `n`, `n` represent increases loss scaling every `n` consecutive steps with finite gradients. + + Reurns: + int: the num `n`, `n` represent increases loss scaling every `n` consecutive steps with finite gradients. + """ + return self._incr_every_n_steps + + def set_incr_every_n_steps(self, new_incr_every_n_steps): + """ + Set the num `n` by `new_incr_every_n_steps`, `n` represent increases loss scaling every `n` consecutive steps with finite gradients. + + Args: + new_incr_every_n_steps(int): The new_incr_every_n_steps used to update the num `n`, `n` represent increases loss scaling every `n` consecutive steps with finite gradients. + """ + self._incr_every_n_steps = new_incr_every_n_steps + + def get_decr_every_n_nan_or_inf(self): + """ + Return the num `n`, `n` represent decreases loss scaling every `n` accumulated steps with nan or inf gradients. + + Reurns: + int: the num `n`, `n` represent decreases loss scaling every `n` accumulated steps with nan or inf gradients. + """ + return self._decr_every_n_nan_or_inf + + def set_decr_every_n_nan_or_inf(self, new_decr_every_n_nan_or_inf): + """ + Set the num `n` by `new_decr_every_n_nan_or_inf`, `n` represent decreases loss scaling every `n` accumulated steps with nan or inf gradients. + + Args: + new_decr_every_n_nan_or_inf(int): The new_decr_every_n_nan_or_inf used to update the num `n`, `n` represent decreases loss scaling every `n` accumulated steps with nan or inf gradients. + """ + self._decr_every_n_nan_or_inf = new_decr_every_n_nan_or_inf diff --git a/python/paddle/fluid/dygraph/dygraph_to_static/function_spec.py b/python/paddle/fluid/dygraph/dygraph_to_static/function_spec.py index 031351ca118ef5..c25574c39dafe0 100644 --- a/python/paddle/fluid/dygraph/dygraph_to_static/function_spec.py +++ b/python/paddle/fluid/dygraph/dygraph_to_static/function_spec.py @@ -103,8 +103,11 @@ def _replace_value_with_input_spec(self, args): for idx, input_var in enumerate(flatten(args)): if isinstance(input_var, np.ndarray): input_var = paddle.static.InputSpec.from_numpy(input_var) + _set_spec_stop_gradient(input_var, True) elif isinstance(input_var, core.VarBase): + stop_gradient = input_var.stop_gradient input_var = paddle.static.InputSpec.from_tensor(input_var) + _set_spec_stop_gradient(input_var, stop_gradient) args_with_spec.append(input_var) @@ -172,13 +175,15 @@ def to_static_inputs_with_spec(self, input_with_spec, main_program): block = main_program.global_block() for i, var_spec in enumerate(flat_input_spec): if isinstance(var_spec, paddle.static.InputSpec): + stop_gradient = getattr(var_spec, 'stop_gradient', False) feed_layer = block.create_var( # TODO(Aurelius84): consider a more elegant way to name this name=var_spec.name or "feed_%s" % i, shape=var_spec.shape, dtype=var_spec.dtype, is_data=True, - need_check_feed=False) + need_check_feed=False, + stop_gradient=stop_gradient) else: feed_layer = var_spec inputs.append(feed_layer) @@ -302,7 +307,7 @@ def check_type_and_len(input, spec, check_length=False): if isinstance(rest_input, (core.VarBase, np.ndarray)): logging_utils.warn( "The inputs constain `{}` without specificing InputSpec, its shape and dtype will be treated immutable. " - "Please specific InputSpec information in `@declarative` if you expect them as mutable inputs.". + "Please specific InputSpec information in `@to_static` if you expect them as mutable inputs.". format(type_name(rest_input))) input_with_spec.extend(inputs[len(input_spec):]) @@ -380,3 +385,12 @@ def _replace_spec_name(name, input_spec): return processed_specs else: return input_spec + + +def _set_spec_stop_gradient(spec, stop_gradient): + """ + Set new attribute ``stop_gradient`` for InputSpec to avoid generating redundant grad_op + while append_backward. + """ + assert isinstance(spec, paddle.static.InputSpec) + spec.stop_gradient = stop_gradient diff --git a/python/paddle/fluid/dygraph/dygraph_to_static/partial_program.py b/python/paddle/fluid/dygraph/dygraph_to_static/partial_program.py index 84bac98013adea..4d12c3c2b99803 100644 --- a/python/paddle/fluid/dygraph/dygraph_to_static/partial_program.py +++ b/python/paddle/fluid/dygraph/dygraph_to_static/partial_program.py @@ -35,6 +35,7 @@ class NestSequence(object): def __init__(self, raw_input, need_check=False): self.__raw_input = raw_input + self.__input_list = self.tolist() self.__var_ids = self._get_var_ids() self._check_non_variable(need_check) @@ -48,12 +49,12 @@ def restore(self, value_list): """ Restores the nested sequence from value list. """ - assert len(self.tolist()) == len(value_list) + assert len(self.__input_list) == len(value_list) return pack_sequence_as(self.__raw_input, value_list) def _get_var_ids(self): var_ids = [] - for idx, var in enumerate(self.tolist()): + for idx, var in enumerate(self.__input_list): if isinstance(var, (framework.Variable, core.VarBase)): var_ids.append(idx) @@ -65,7 +66,7 @@ def _check_non_variable(self, need_check): """ if need_check: warning_types = set() - for var in self.tolist(): + for var in self.__input_list: if not isinstance(var, (framework.Variable, core.VarBase)): warning_types.add(type(var)) if warning_types: @@ -80,7 +81,7 @@ def var_ids(self): return self.__var_ids def __getitem__(self, item): - return self.tolist()[item] + return self.__input_list[item] class LazyInitialized(object): @@ -106,7 +107,7 @@ def _change_is_test_status(program, is_test): return program -class PartialProgramLayer(layers.Layer): +class PartialProgramLayer: """ PartialProgramLayer wraps all the ops from layers decorated by `@declarative` and execute them as a static subgraph. @@ -134,7 +135,9 @@ def __init__(self, main_program, inputs, outputs, parameters=None): self._params = parameters if parameters is not None else [] self._origin_main_program = self._verify_program(main_program) - self._inner_scope = core.Scope() + self._tmp_scope_vec = self._create_scope_vec() + # A fake_var to handle empty input or output + self.__fake_vars = _create_fake_var() # Set default mode to train self._double_grads = self._get_double_grads(self._origin_main_program) self.training = True @@ -217,19 +220,19 @@ def _get_double_grads(self, program): var_desc.name(), var_desc.type(), False) double_grads.append(var_base) - return double_grads + return self._valid_vars(double_grads) - def forward(self, inputs): - in_vars, out_vars, tmp_scope_vec = self._prepare(inputs) + def __call__(self, inputs): + in_vars, out_vars = self._prepare(inputs) attrs = ('global_block', self.program.desc.block(0), 'start_op_index', 0, 'end_op_index', self._infer_program.desc.block(0).op_size(), 'is_test', not self.training) core.ops.run_program( - valid_vars(in_vars), - valid_vars(self._params), - valid_vars(out_vars), tmp_scope_vec, - valid_vars(self._double_grads), *attrs) + self._valid_vars(in_vars), + self._valid_vars(self._params), + self._valid_vars(out_vars), self._tmp_scope_vec, self._double_grads, + *attrs) restored_nest_out = self._restore_out(out_vars) return self._remove_no_value(restored_nest_out) @@ -264,7 +267,6 @@ def _prepare(self, inputs): expected_place): var = value._copy_to(expected_place, False) var.stop_gradient = True - var.name = value.name else: var = value var.name = self._inputs[i].desc.name() @@ -272,25 +274,29 @@ def _prepare(self, inputs): continue input_vars.append(var) - # Create VarBase to receive output data. - out_vars = [] - for idx in self._outputs.var_ids: - var = self._outputs[idx] + def create_out(var_id): + var = self._outputs[var_id] assert isinstance(var, framework.Variable) var_desc = var.desc var_base = core.VarBase(var_desc.dtype(), var_desc.shape(), var_desc.name(), var_desc.type(), False) - out_vars.append(var_base) + return var_base + + # Create VarBase to receive output data. + out_vars = list(map(create_out, self._outputs.var_ids)) + + return input_vars, out_vars + def _create_scope_vec(self): # Hold forward variables tmp_scope_vec = core.VarBase(core.VarDesc.VarType.FP32, [], "program_out_scope", core.VarDesc.VarType.STEP_SCOPES, True) - tmp_scope_vec.value().set_scope(self._inner_scope) - - return input_vars, out_vars, tmp_scope_vec + inner_scope = core.Scope() + tmp_scope_vec.value().set_scope(inner_scope) + return tmp_scope_vec def _restore_out(self, out_vars): """ @@ -311,8 +317,9 @@ def _clone_for_test(self, main_program): return main_program.clone(for_test=True) def _is_no_value(self, var): - if isinstance(var, core.VarBase): - if var.shape == [1] and var.numpy()[0] == RETURN_NO_VALUE_MAGIC_NUM: + if isinstance(var, core.VarBase) and var.shape == [1]: + # NOTE: .numpy() will insert MemcpySync operation, it hits performance. + if var.numpy()[0] == RETURN_NO_VALUE_MAGIC_NUM: return True return False @@ -405,20 +412,22 @@ def _check_params_all_inited(self, main_program): "Please define the layer with parameters in `__init__` function." % name) + def _valid_vars(self, vars): + """ + Note: run_program_op.InferShape requires `X`/'Out' not be null. + But it's common in dy2static, fake varBase is created to handle the + problem. + """ + return vars if vars else self.__fake_vars + -def valid_vars(vars): +def _create_fake_var(): """ - Note: run_program_op.InferShape requires `X`/'Out' not be null. - But it's common in dy2static, fake varBase is created to handle the - problem. + Create a fake_var (force on CPU) to handle empty input or output """ - if vars: - return vars return [ - core.VarBase( - value=[1], - name='Fake_var', - place=framework._current_expected_place()) + core.VarBase(core.VarDesc.VarType.FP32, [], "Fake_var", + core.VarDesc.VarType.RAW, False) ] diff --git a/python/paddle/fluid/dygraph/dygraph_to_static/utils.py b/python/paddle/fluid/dygraph/dygraph_to_static/utils.py index 9a59111b32113f..351a9dcfa3aa2a 100644 --- a/python/paddle/fluid/dygraph/dygraph_to_static/utils.py +++ b/python/paddle/fluid/dygraph/dygraph_to_static/utils.py @@ -485,8 +485,7 @@ def remove_if_exit(filepath): os.remove(filepath) source = ast_to_source_code(ast_root) - import_fluid = "import paddle\nimport paddle.fluid as fluid\n" - source = import_fluid + source + source = _inject_import_statements() + source f = tempfile.NamedTemporaryFile( mode='w', suffix='.py', delete=False, encoding='utf-8') @@ -519,6 +518,14 @@ def remove_if_exit(filepath): return callable_func, f.name +def _inject_import_statements(): + import_statements = [ + "import paddle", "import paddle.fluid as fluid", "from typing import *", + "import numpy as np" + ] + return '\n'.join(import_statements) + '\n' + + def recover_globals_attribute(src_obj, dst_obj): attr_name = '__globals__' diff --git a/python/paddle/fluid/dygraph/math_op_patch.py b/python/paddle/fluid/dygraph/math_op_patch.py index f6986265e2fbb3..dee11da4ac9ac1 100644 --- a/python/paddle/fluid/dygraph/math_op_patch.py +++ b/python/paddle/fluid/dygraph/math_op_patch.py @@ -202,12 +202,17 @@ def __impl__(self, other_var): # 2. create varbase for scalar lhs_dtype = self.dtype if not isinstance(other_var, core.VarBase): - if reverse: - other_var = create_tensor( - other_var, dtype=lhs_dtype, shape=self.shape) + if isinstance(other_var, complex): + import paddle + other_var = paddle.to_tensor(other_var, dtype='complex64') else: - # add fill_op - other_var = create_scalar(value=other_var, dtype=lhs_dtype) + if reverse: + other_var = create_tensor( + other_var, dtype=lhs_dtype, shape=self.shape) + else: + # add fill_op + other_var = create_scalar( + value=other_var, dtype=lhs_dtype) # 3. promote types or unify right var type to left var rhs_dtype = other_var.dtype diff --git a/python/paddle/fluid/framework.py b/python/paddle/fluid/framework.py index 80c27c585d8104..9e06f107a37585 100644 --- a/python/paddle/fluid/framework.py +++ b/python/paddle/fluid/framework.py @@ -947,35 +947,43 @@ def __init__(self, self._stop_gradient = stop_gradient self.is_data = is_data - @fake_interface_only def detach(self): """ - **Notes**: - **This API is ONLY available in Dygraph mode** - Returns a new Variable, detached from the current graph. + It will share data with origin Variable and without tensor copy. + In addition, the detached Variable doesn't provide gradient propagation. Returns: ( :ref:`api_guide_Variable_en` | dtype is same as current Variable): The detached Variable. - Examples: .. code-block:: python - import paddle.fluid as fluid - from paddle.fluid.dygraph.base import to_variable - from paddle.fluid.dygraph import Linear - import numpy as np + import paddle - data = np.random.uniform(-1, 1, [30, 10, 32]).astype('float32') - with fluid.dygraph.guard(): - linear = Linear(32, 64) - data = to_variable(data) - x = linear(data) - y = x.detach() + paddle.enable_static() + + # create a static Variable + x = paddle.static.data(name='x', shape=[3, 2, 1]) + # create a detached Variable + y = x.detach() """ - pass + + assert self.type == core.VarDesc.VarType.SELECTED_ROWS or \ + self.type == core.VarDesc.VarType.LOD_TENSOR, \ + "only support a variable with SELECTED_ROWS or LOD_TENSOR to be detached" + + output = self.block.create_var( + name=unique_name.generate_with_ignorable_key("detach_" + self.name), + dtype=self.dtype, + type=self.type, + persistable=self.persistable, + stop_gradient=True) + + self.block.append_op( + type='share_data', inputs={'X': [self]}, outputs={'Out': [output]}) + return output @fake_interface_only def numpy(self): @@ -1810,6 +1818,35 @@ def set_value(self, value, scope=None): t.set(value, place) + def size(self): + """ + Returns the number of elements for current Variable, which is a int64 Variable with shape [1] + + Returns: + Variable: the number of elements for current Variable + + Examples: + .. code-block:: python + + import paddle + + paddle.enable_static() + + # create a static Variable + x = paddle.static.data(name='x', shape=[3, 2, 1]) + + # get the number of elements of the Variable + y = x.size() + """ + + output = self.block.create_var( + name=unique_name.generate_with_ignorable_key(self.name + "_size"), + dtype=core.VarDesc.VarType.INT64) + + self.block.append_op( + type='size', inputs={'Input': [self]}, outputs={'Out': [output]}) + return output + def get_all_op_protos(): """ diff --git a/python/paddle/fluid/incubate/fleet/base/fleet_base.py b/python/paddle/fluid/incubate/fleet/base/fleet_base.py index 77a202317912f2..105180030ace82 100644 --- a/python/paddle/fluid/incubate/fleet/base/fleet_base.py +++ b/python/paddle/fluid/incubate/fleet/base/fleet_base.py @@ -181,7 +181,7 @@ def split_files(self, files): trainers = self.worker_num() remainder = len(files) % trainers - blocksize = len(files) / trainers + blocksize = len(files) // trainers blocks = [blocksize] * trainers for i in range(remainder): diff --git a/python/paddle/fluid/incubate/fleet/utils/fleet_util.py b/python/paddle/fluid/incubate/fleet/utils/fleet_util.py index 979334ed2eaa4c..d02be8af4b144f 100644 --- a/python/paddle/fluid/incubate/fleet/utils/fleet_util.py +++ b/python/paddle/fluid/incubate/fleet/utils/fleet_util.py @@ -244,7 +244,7 @@ def get_global_auc(self, new_pos = 0.0 new_neg = 0.0 total_ins_num = 0 - for i in xrange(num_bucket): + for i in range(num_bucket): index = num_bucket - 1 - i new_pos = pos + global_pos[0][index] total_ins_num += global_pos[0][index] @@ -1240,8 +1240,8 @@ def get_online_pass_interval(self, days, hours, split_interval, hours = os.popen("echo -n " + hours).read().split(" ") split_interval = int(split_interval) split_per_pass = int(split_per_pass) - splits_per_day = 24 * 60 / split_interval - pass_per_day = splits_per_day / split_per_pass + splits_per_day = 24 * 60 // split_interval + pass_per_day = splits_per_day // split_per_pass left_train_hour = int(hours[0]) right_train_hour = int(hours[-1]) @@ -1425,7 +1425,7 @@ def get_metric(name): relative_ctr_error = 0.0 k_max_span = 0.01 k_relative_error_bound = 0.05 - for i in xrange(num_bucket): + for i in range(num_bucket): click = global_pos[0][i] show = global_pos[0][i] + global_neg[0][i] ctr = float(i) / num_bucket diff --git a/python/paddle/fluid/incubate/fleet/utils/http_server.py b/python/paddle/fluid/incubate/fleet/utils/http_server.py index 50933ce5d1bd35..b4ee29a065a7c0 100644 --- a/python/paddle/fluid/incubate/fleet/utils/http_server.py +++ b/python/paddle/fluid/incubate/fleet/utils/http_server.py @@ -14,8 +14,9 @@ """Http Server.""" import logging -import BaseHTTPServer -import SimpleHTTPServer +# NOTE: HTTPServer has a different name in python2 and python3 +from http.server import HTTPServer +import http.server as SimpleHTTPServer import time import threading import socket @@ -123,7 +124,7 @@ def send_status_code(self, code): self.end_headers() -class KVHTTPServer(BaseHTTPServer.HTTPServer, object): +class KVHTTPServer(HTTPServer, object): """ it is a http server storing kv pairs. """ diff --git a/python/paddle/fluid/layers/math_op_patch.py b/python/paddle/fluid/layers/math_op_patch.py index 9433e0e5ee0e5f..feb723d9c8b43a 100644 --- a/python/paddle/fluid/layers/math_op_patch.py +++ b/python/paddle/fluid/layers/math_op_patch.py @@ -45,6 +45,7 @@ "__rpow__": "A **= B", "__floordiv__": "A //B", "__mod__": "A % B", + "__matmul__": "A @ B", "__eq__": "A == B", "__ne__": "A != B", "__lt__": "A < B", @@ -195,6 +196,28 @@ def _scalar_op_(var, scale, bias): def _neg_(var): return _scalar_op_(var, -1.0, 0.0) + @property + def _ndim_(self): + """ + Returns the dimension of current Variable + + Returns: + the dimension + + Examples: + .. code-block:: python + + import paddle + + paddle.enable_static() + + # create a static Variable + x = paddle.static.data(name='x', shape=[3, 2, 1]) + # print the dimension of the Variable + print(x.ndim) + """ + return len(self.shape) + def _scalar_add_(var, value): return _scalar_op_(var, 1.0, value) @@ -228,9 +251,9 @@ def __impl__(self, other_var): other_var = float(other_var) # division is a special case # NOTE(chenweihang): because we cast tensor to float32 instead float64, - # the division result can only guarantee the numerical accuracy of 6 digits - # after the decimal point. The result of numpy calculation is of float64 type, - # so the calculation result here and the calculation result of numpy are + # the division result can only guarantee the numerical accuracy of 6 digits + # after the decimal point. The result of numpy calculation is of float64 type, + # so the calculation result here and the calculation result of numpy are # different after 6 decimal point. If necessary, we can also use float64 here. # torch's behavior here is consistent with ours if op_type == 'elementwise_div' and self.dtype in _supported_int_dtype_: @@ -238,7 +261,7 @@ def __impl__(self, other_var): # here use `scale` replace `elementwise` to get better performance # but only +, -, * can use this method # NOTE(chentianyu03): / can not use `scale` method，because the result of - # `scale` method (self*(1/other_var)) do not exactly equal with the result + # `scale` method (self*(1/other_var)) do not exactly equal with the result # of `elementwise_div` method. if scalar_method is not None: return scalar_method(self, other_var) @@ -321,6 +344,9 @@ def __impl__(self, other_var): # b=-a ('__neg__', _neg_), ('astype', astype), + ('dim', lambda x: len(x.shape)), + ('ndimension', lambda x: len(x.shape)), + ('ndim', _ndim_), ('__add__', _binary_creator_('__add__', 'elementwise_add', False, _scalar_add_)), # a+b == b+a. Do not need to reverse explicitly @@ -347,6 +373,8 @@ def __impl__(self, other_var): 'elementwise_floordiv', False, None)), ('__mod__', _binary_creator_('__mod__', 'elementwise_mod', False, None)), + ('__matmul__', _binary_creator_('__matmul__', "matmul_v2", False, + None)), # for logical compare ('__eq__', _binary_creator_('__eq__', 'equal', False, None)), ('__ne__', _binary_creator_('__ne__', 'not_equal', False, None)), diff --git a/python/paddle/fluid/layers/ops.py b/python/paddle/fluid/layers/ops.py index a6ab50df08cc15..cc5c327b974f74 100755 --- a/python/paddle/fluid/layers/ops.py +++ b/python/paddle/fluid/layers/ops.py @@ -87,26 +87,26 @@ _new_OP = _OP if _OP in __deprecated_func_name__: _new_OP = __deprecated_func_name__[_OP] - func = generate_activation_fn(_OP) - func = deprecated( - since="2.0.0", update_to="paddle.nn.functional.%s" % (_new_OP))(func) - globals()[_OP] = func + _func = generate_activation_fn(_OP) + _func = deprecated( + since="2.0.0", update_to="paddle.nn.functional.%s" % (_new_OP))(_func) + globals()[_OP] = _func for _OP in set(__unary_func__): _new_OP = _OP if _OP in __deprecated_func_name__: _new_OP = __deprecated_func_name__[_OP] - func = generate_activation_fn(_OP) - func = deprecated(since="2.0.0", update_to="paddle.%s" % (_new_OP))(func) - globals()[_OP] = func + _func = generate_activation_fn(_OP) + _func = deprecated(since="2.0.0", update_to="paddle.%s" % (_new_OP))(_func) + globals()[_OP] = _func for _OP in set(__inplace_unary_func__): _new_OP = _OP if _OP in __deprecated_func_name__: _new_OP = __deprecated_func_name__[_OP] - func = generate_inplace_fn(_OP) - func = deprecated(since="2.0.0", update_to="paddle.%s" % (_new_OP))(func) - globals()[_OP] = func + _func = generate_inplace_fn(_OP) + _func = deprecated(since="2.0.0", update_to="paddle.%s" % (_new_OP))(_func) + globals()[_OP] = _func add_sample_code(globals()["sigmoid"], r""" Examples: diff --git a/python/paddle/fluid/layers/rnn.py b/python/paddle/fluid/layers/rnn.py index 7f815e1c74dfa6..c0ad3e3bea7d71 100644 --- a/python/paddle/fluid/layers/rnn.py +++ b/python/paddle/fluid/layers/rnn.py @@ -880,6 +880,9 @@ class BeamSearchDecoder(Decoder): :code:`BeamSearchDecoder.tile_beam_merge_with_batch` . The most common case for this is the encoder output in attention mechanism. + Returns: + BeamSearchDecoder: An instance of decoder which can be used in \ + `paddle.nn.dynamic_decode` to implement decoding. Examples: diff --git a/python/paddle/fluid/tests/unittests/CMakeLists.txt b/python/paddle/fluid/tests/unittests/CMakeLists.txt index 023b092b774a84..0356aead2e0bf6 100644 --- a/python/paddle/fluid/tests/unittests/CMakeLists.txt +++ b/python/paddle/fluid/tests/unittests/CMakeLists.txt @@ -25,6 +25,7 @@ list(APPEND DIST_TEST_OPS test_parallel_dygraph_control_flow) list(APPEND DIST_TEST_OPS test_parallel_dygraph_dataparallel) list(APPEND DIST_TEST_OPS test_parallel_dygraph_pipeline_parallel) list(APPEND DIST_TEST_OPS test_parallel_dygraph_tensor_parallel) +list(APPEND DIST_TEST_OPS test_parallel_dygraph_sharding_parallel) list(APPEND DIST_TEST_OPS test_parallel_dygraph_mp_layers) set(MIXED_DIST_TEST_OPS ${DIST_TEST_OPS}) #remove distribute unittests. @@ -70,6 +71,7 @@ list(APPEND MIXED_DIST_TEST_OPS test_fleet_graph_executor) list(APPEND MIXED_DIST_TEST_OPS test_fleet_meta_optimizer_base) list(APPEND MIXED_DIST_TEST_OPS test_fleet_distributed_strategy) list(APPEND MIXED_DIST_TEST_OPS test_fleet_auto) +list(APPEND MIXED_DIST_TEST_OPS test_fleet_static_mp_layers) foreach(TEST_OP ${MIXED_DIST_TEST_OPS}) list(REMOVE_ITEM TEST_OPS ${TEST_OP}) endforeach() @@ -105,6 +107,7 @@ if(((NOT WITH_ROCM) AND (NOT WITH_GPU)) OR WIN32) LIST(REMOVE_ITEM TEST_OPS test_collective_wait) LIST(REMOVE_ITEM TEST_OPS test_memcpy_op) LIST(REMOVE_ITEM TEST_OPS test_raw_program_optimizer) + LIST(REMOVE_ITEM TEST_OPS test_fleet_gradient_scale) endif() if(WIN32) @@ -184,6 +187,7 @@ if ((NOT WITH_GPU) AND (NOT WITH_ROCM)) list(REMOVE_ITEM TEST_OPS test_parallel_dygraph_dataparallel) list(REMOVE_ITEM TEST_OPS test_parallel_dygraph_pipeline_parallel) list(REMOVE_ITEM TEST_OPS test_parallel_dygraph_tensor_parallel) + list(REMOVE_ITEM TEST_OPS test_parallel_dygraph_sharding_parallel) list(REMOVE_ITEM TEST_OPS test_parallel_dygraph_mp_layers) LIST(REMOVE_ITEM TEST_OPS test_imperative_auto_mixed_precision) LIST(REMOVE_ITEM TEST_OPS test_fleet_base_single) @@ -525,6 +529,7 @@ if(WITH_DISTRIBUTE) py_test_modules(test_fleet_private_function MODULES test_fleet_private_function ENVS ${dist_ENVS}) py_test_modules(test_fleet_meta_optimizer_base MODULES test_fleet_meta_optimizer_base ENVS ${dist_ENVS}) py_test_modules(test_fleet_distributed_strategy MODULES test_fleet_distributed_strategy) + py_test_modules(test_fleet_static_mp_layers MODULES test_fleet_static_mp_layers) #py_test_modules(test_fleet_auto MODULES test_fleet_auto ENVS ${dist_ENVS}) if(NOT WIN32) py_test_modules(test_fleet_localsgd_meta_optimizer MODULES test_fleet_localsgd_meta_optimizer ENVS ${dist_ENVS}) @@ -880,6 +885,7 @@ if(WITH_DISTRIBUTE AND WITH_GPU AND WITH_NCCL) set_tests_properties(test_parallel_dygraph_control_flow PROPERTIES TIMEOUT 120) set_tests_properties(test_parallel_dygraph_pipeline_parallel PROPERTIES TIMEOUT 120) set_tests_properties(test_parallel_dygraph_tensor_parallel PROPERTIES TIMEOUT 200) + set_tests_properties(test_parallel_dygraph_sharding_parallel PROPERTIES TIMEOUT 120) set_tests_properties(test_parallel_dygraph_mp_layers PROPERTIES TIMEOUT 120) if(${NCCL_VERSION} VERSION_GREATER_EQUAL 2212) set_tests_properties(test_parallel_dygraph_sparse_embedding PROPERTIES TIMEOUT 120) diff --git a/python/paddle/fluid/tests/unittests/asp/test_asp_pruning_1d.py b/python/paddle/fluid/tests/unittests/asp/test_asp_pruning_1d.py index ee4b2c002f5afa..6ebc89b18738c7 100644 --- a/python/paddle/fluid/tests/unittests/asp/test_asp_pruning_1d.py +++ b/python/paddle/fluid/tests/unittests/asp/test_asp_pruning_1d.py @@ -15,6 +15,7 @@ from __future__ import print_function +import unittest import paddle from paddle.fluid.contrib import sparsity from paddle.fluid.tests.unittests.asp.asp_pruning_base import TestASPHelperPruningBase diff --git a/python/paddle/fluid/tests/unittests/asp/test_asp_pruning_2d_best.py b/python/paddle/fluid/tests/unittests/asp/test_asp_pruning_2d_best.py index 1b8b1e4a06ae4c..b21f8edf4f4772 100644 --- a/python/paddle/fluid/tests/unittests/asp/test_asp_pruning_2d_best.py +++ b/python/paddle/fluid/tests/unittests/asp/test_asp_pruning_2d_best.py @@ -16,6 +16,7 @@ from __future__ import print_function import paddle +import unittest from paddle.fluid.contrib import sparsity from paddle.fluid.tests.unittests.asp.asp_pruning_base import TestASPHelperPruningBase diff --git a/python/paddle/fluid/tests/unittests/asp/test_asp_pruning_2d_greedy.py b/python/paddle/fluid/tests/unittests/asp/test_asp_pruning_2d_greedy.py index 4bdd310f0209a9..8ec8ab485250e0 100644 --- a/python/paddle/fluid/tests/unittests/asp/test_asp_pruning_2d_greedy.py +++ b/python/paddle/fluid/tests/unittests/asp/test_asp_pruning_2d_greedy.py @@ -15,6 +15,7 @@ from __future__ import print_function +import unittest import paddle from paddle.fluid.contrib import sparsity from paddle.fluid.tests.unittests.asp.asp_pruning_base import TestASPHelperPruningBase diff --git a/python/paddle/fluid/tests/unittests/dygraph_to_static/test_origin_info.py b/python/paddle/fluid/tests/unittests/dygraph_to_static/test_origin_info.py index 144b16873aa9bc..016a1b3b588ab0 100644 --- a/python/paddle/fluid/tests/unittests/dygraph_to_static/test_origin_info.py +++ b/python/paddle/fluid/tests/unittests/dygraph_to_static/test_origin_info.py @@ -65,7 +65,7 @@ def set_test_func(self): self.func = simple_func def set_static_lineno(self): - self.static_abs_lineno_list = [3, 4, 5] + self.static_abs_lineno_list = [5, 6, 7] def set_dygraph_info(self): self.line_num = 3 @@ -149,7 +149,7 @@ def set_test_func(self): self.func = nested_func def set_static_lineno(self): - self.static_abs_lineno_list = [3, 5, 6, 7, 8] + self.static_abs_lineno_list = [5, 7, 8, 9, 10] def set_dygraph_info(self): self.line_num = 5 @@ -174,7 +174,7 @@ def set_test_func(self): self.func = decorated_func def set_static_lineno(self): - self.static_abs_lineno_list = [3, 4] + self.static_abs_lineno_list = [5, 6] def set_dygraph_info(self): self.line_num = 2 @@ -208,7 +208,7 @@ def set_test_func(self): self.func = decorated_func2 def set_static_lineno(self): - self.static_abs_lineno_list = [3, 4] + self.static_abs_lineno_list = [5, 6] def set_dygraph_info(self): self.line_num = 2 diff --git a/python/paddle/fluid/tests/unittests/dygraph_to_static/test_typing.py b/python/paddle/fluid/tests/unittests/dygraph_to_static/test_typing.py new file mode 100644 index 00000000000000..c3c0453bde3f40 --- /dev/null +++ b/python/paddle/fluid/tests/unittests/dygraph_to_static/test_typing.py @@ -0,0 +1,124 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import paddle +import unittest +import numpy as np +from typing import Tuple, List, Dict, TypeVar + + +class BaseLayer(paddle.nn.Layer): + def __init__(self, in_size, out_size): + super(BaseLayer, self).__init__() + self._linear = paddle.nn.Linear(in_size, out_size) + + def build(self, x): + out1 = self._linear(x) + out2 = paddle.mean(out1) + return out1, out2 + + +class LinearNetWithTuple(BaseLayer): + def __init__(self, in_size, out_size): + super(LinearNetWithTuple, self).__init__(in_size, out_size) + + def forward(self, x) -> Tuple[paddle.Tensor, str]: + out1, out2 = self.build(x) + return (out2, 'str') + + +class LinearNetWithTuple2(BaseLayer): + def __init__(self, in_size, out_size): + super(LinearNetWithTuple2, self).__init__(in_size, out_size) + + def forward(self, x) -> Tuple[paddle.Tensor, np.array]: + out1, out2 = self.build(x) + return (out2, np.ones([4, 16])) + + +class LinearNetWithList(BaseLayer): + def __init__(self, in_size, out_size): + super(LinearNetWithList, self).__init__(in_size, out_size) + + def forward(self, x) -> List[paddle.Tensor]: + out1, out2 = self.build(x) + return [out2] + + +class LinearNetWithDict(BaseLayer): + def __init__(self, in_size, out_size): + super(LinearNetWithDict, self).__init__(in_size, out_size) + + def forward(self, x) -> Dict[str, paddle.Tensor]: + out1, out2 = self.build(x) + return {'out': out2} + + +class TestTyping(unittest.TestCase): + def setUp(self): + self.in_num = 16 + self.out_num = 16 + self.x = paddle.randn([4, 16]) + self.spec = [paddle.static.InputSpec(shape=[None, 16], dtype='float32')] + + def build_net(self): + return LinearNetWithTuple(self.in_num, self.out_num) + + def save_and_load(self, suffix=''): + path = './layer_typing_' + suffix + paddle.jit.save(self.net, path, input_spec=self.spec) + return paddle.jit.load(path) + + def run_dy(self): + out, _ = self.net(self.x) + return out + + def test_type(self): + self.net = self.build_net() + out = self.run_dy() + load_net = self.save_and_load('tuple') + load_out = load_net(self.x) + self.assertTrue(np.allclose(out, load_out)) + + +class TestTypingTuple(TestTyping): + def build_net(self): + return LinearNetWithTuple2(self.in_num, self.out_num) + + def run_dy(self): + out, np_data = self.net(self.x) + self.assertTrue(np.equal(np_data, np.ones_like(np_data)).all()) + return out + + +class TestTypingList(TestTyping): + def build_net(self): + return LinearNetWithList(self.in_num, self.out_num) + + def run_dy(self): + out = self.net(self.x)[0] + return out + + +class TestTypingDict(TestTyping): + def build_net(self): + return LinearNetWithDict(self.in_num, self.out_num) + + def run_dy(self): + out = self.net(self.x)['out'] + return out + + +if __name__ == '__main__': + unittest.main() diff --git a/python/paddle/fluid/tests/unittests/elastic_demo.py b/python/paddle/fluid/tests/unittests/elastic_demo.py new file mode 100644 index 00000000000000..c5177c0f52950c --- /dev/null +++ b/python/paddle/fluid/tests/unittests/elastic_demo.py @@ -0,0 +1,23 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import os, sys +import time + +sys.stderr.write("{}-DISTRIBUTED_TRAINER_ENDPOINTS={}\n".format(os.environ[ + 'PADDLE_TRAINER_ID'], os.environ['DISTRIBUTED_TRAINER_ENDPOINTS'])) +sys.stderr.write("{}-PADDLE_TRAINERS={}\n".format(os.environ[ + 'PADDLE_TRAINER_ID'], os.environ['PADDLE_TRAINERS'])) + +time.sleep(600) diff --git a/python/paddle/fluid/tests/unittests/hybrid_parallel_communicate_group.py b/python/paddle/fluid/tests/unittests/hybrid_parallel_communicate_group.py index 0a9785475b561a..53d0f95a236672 100644 --- a/python/paddle/fluid/tests/unittests/hybrid_parallel_communicate_group.py +++ b/python/paddle/fluid/tests/unittests/hybrid_parallel_communicate_group.py @@ -21,7 +21,8 @@ class TestNewGroupAPI(object): def __init__(self): paddle.distributed.init_parallel_env() - topo = fleet.CommunicateTopology(["data", "model", "pipe"], [2, 1, 1]) + topo = fleet.CommunicateTopology(["data", "model", "sharding", "pipe"], + [2, 1, 1, 1]) self.hcg = fleet.HybridCommunicateGroup(topo) d1 = np.array([1, 2, 3]) diff --git a/python/paddle/fluid/tests/unittests/hybrid_parallel_sharding_model.py b/python/paddle/fluid/tests/unittests/hybrid_parallel_sharding_model.py new file mode 100644 index 00000000000000..2995e4dbf84018 --- /dev/null +++ b/python/paddle/fluid/tests/unittests/hybrid_parallel_sharding_model.py @@ -0,0 +1,297 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import division +from __future__ import print_function + +import paddle +import numpy as np +import random +import paddle.distributed as dist +import paddle.fluid as fluid +import paddle.distributed.fleet as fleet +from paddle.io import DataLoader, Dataset +from paddle.distributed.fleet.meta_optimizers.dygraph_optimizer.dygraph_sharding_optimizer import DygraphShardingOptimizer +import unittest + +vocab_size = 20 +hidden_size = 10 +inner_size = 8 +output_size = 10 +seq_length = 2 +batch_size = 4 +STEPS = 10 + + +def parallel_matmul(lm_output, logit_weights, parallel_output): + hcg = fleet.get_hybrid_communicate_group() + model_parallel_group = hcg.get_model_parallel_group() + world_size = hcg.get_model_parallel_world_size() + rank = hcg.get_model_parallel_rank() + + if world_size > 1: + input_parallel = paddle.distributed.collective._c_identity( + lm_output, group=model_parallel_group) + + logits = paddle.matmul(input_parallel, logit_weights, transpose_y=True) + + if parallel_output: + return logits + + return paddle.distributed.collective._c_concat( + logits, group=model_parallel_group) + else: + logits = paddle.matmul(lm_output, logit_weights, transpose_y=True) + return logits + + +class SimpleMPNet(fluid.dygraph.Layer): + def __init__(self, vocab_size, hidden_size, inner_size, output_size, np_fc1, + np_fc2, mp_id): + super(SimpleMPNet, self).__init__() + + if mp_id == 0: + init_fc1_data = np_fc1[:, :(inner_size // 2)] + init_fc2_data = np_fc2[:(inner_size // 2), :] + else: + init_fc1_data = np_fc1[:, (inner_size // 2):] + init_fc2_data = np_fc2[(inner_size // 2):, :] + + self.linear1 = fleet.meta_parallel.ColumnParallelLinear( + hidden_size, + inner_size, + weight_attr=paddle.framework.ParamAttr( + initializer=paddle.nn.initializer.Assign(init_fc1_data)), + gather_output=False, + has_bias=True) + + self.linear2 = fleet.meta_parallel.RowParallelLinear( + inner_size, + hidden_size, + weight_attr=paddle.framework.ParamAttr( + initializer=paddle.nn.initializer.Assign(init_fc2_data)), + input_is_parallel=True, + has_bias=True) + + self.linear3 = paddle.nn.Linear( + hidden_size, + output_size, + weight_attr=paddle.framework.ParamAttr( + initializer=paddle.nn.initializer.Constant(0.0)), + bias_attr=paddle.framework.ParamAttr( + initializer=paddle.nn.initializer.Constant(0.0))) + + self.embedding = fleet.meta_parallel.VocabParallelEmbedding( + vocab_size, + hidden_size, + weight_attr=paddle.nn.initializer.Constant(value=0.5)) + + def forward(self, x): + x = self.embedding(x) + x = self.linear1(x) + x = self.linear2(x) + x = self.linear3(x) + x = parallel_matmul(x, self.embedding.weight, False) + return x + + +class SimpleDPNet(fluid.dygraph.Layer): + def __init__(self, vocab_size, hidden_size, inner_size, output_size, np_fc1, + np_fc2): + + super(SimpleDPNet, self).__init__() + self.linear1 = paddle.nn.Linear( + hidden_size, + inner_size, + weight_attr=paddle.framework.ParamAttr( + initializer=paddle.nn.initializer.Assign(np_fc1)), + bias_attr=paddle.framework.ParamAttr( + initializer=paddle.nn.initializer.Constant(0.0))) + + self.linear2 = paddle.nn.Linear( + inner_size, + hidden_size, + weight_attr=paddle.framework.ParamAttr( + initializer=paddle.nn.initializer.Assign(np_fc2)), + bias_attr=paddle.framework.ParamAttr( + initializer=paddle.nn.initializer.Constant(0.0))) + + self.linear3 = paddle.nn.Linear( + hidden_size, + output_size, + weight_attr=paddle.framework.ParamAttr( + initializer=paddle.nn.initializer.Constant(0.0)), + bias_attr=paddle.framework.ParamAttr( + initializer=paddle.nn.initializer.Constant(0.0))) + + self.embedding = paddle.nn.Embedding( + vocab_size, + hidden_size, + weight_attr=paddle.nn.initializer.Constant(value=0.5)) + + def forward(self, x): + x = self.embedding(x) + x = self.linear1(x) + x = self.linear2(x) + x = self.linear3(x) + x = paddle.matmul(x, self.embedding.weight, transpose_y=True) + return x + + +class TestDistMPTraning(unittest.TestCase): + def setUp(self): + random.seed(2021) + np.random.seed(2021) + paddle.seed(2021) + + self.strategy = fleet.DistributedStrategy() + self.strategy.hybrid_configs = { + "sharding_degree": 2, + "dp_degree": 1, + "mp_degree": 1, + "pp_degree": 1, + } + fleet.init(is_collective=True, strategy=self.strategy) + self.data = [ + np.random.randint(0, vocab_size, ( + batch_size, + seq_length, )) for _ in range(STEPS) + ] + + def train_batch(self, batch, model, optimizer): + + output = model(batch) + loss = output.mean() + loss.backward() # do backward + optimizer.step() # update parameters + optimizer.clear_grad() + return loss + + def build_optimizer(self, + model, + strategy=None, + is_sharding=True, + Optimizer="adam"): + + if Optimizer == "adam": + if is_sharding: + optimizer = DygraphShardingOptimizer( + hcg=fleet.get_hybrid_communicate_group(), + user_defined_strategy=strategy, + params=model.parameters(), + inner_optimizer_class=paddle.optimizer.Adam, + learning_rate=0.001, + weight_decay=0.00001, ) + else: + optimizer = paddle.optimizer.Adam( + parameters=model.parameters(), + learning_rate=0.001, + weight_decay=0.00001, ) + else: + if is_sharding: + optimizer = DygraphShardingOptimizer( + hcg=fleet.get_hybrid_communicate_group(), + user_defined_strategy=strategy, + params=model.parameters(), + inner_optimizer_class=paddle.optimizer.Momentum, + learning_rate=0.001, ) + else: + optimizer = paddle.optimizer.Momentum( + learning_rate=0.001, parameters=model.parameters()) + return optimizer + + def build_model_optimizer(self, Optimizer="adam"): + hcg = fleet.get_hybrid_communicate_group() + word_size = hcg.get_model_parallel_world_size() + sharding_id = hcg.get_sharding_parallel_rank() + dp_id = hcg.get_data_parallel_rank() + rank_id = dist.get_rank() + + np_fc1 = np.random.random_sample((hidden_size, inner_size)) + np_fc2 = np.random.random_sample((inner_size, hidden_size)) + + model_a = SimpleDPNet(vocab_size, hidden_size, inner_size, output_size, + np_fc1, np_fc2) + optimizer_a = self.build_optimizer( + model_a, + strategy=self.strategy, + is_sharding=True, + Optimizer=Optimizer) + model_a = fleet.distributed_model(model_a) + optimizer_a = fleet.distributed_optimizer(optimizer_a) + + model_b = SimpleDPNet(vocab_size, hidden_size, inner_size, output_size, + np_fc1, np_fc2) + optimizer_b = self.build_optimizer( + model_b, + strategy=self.strategy, + is_sharding=False, + Optimizer=Optimizer) + + return model_a, optimizer_a, model_b, optimizer_b + + def sharding_model(self, Optimizer, sharded_accumulators): + model_a, optimizer_a, model_b, optimizer_b = self.build_model_optimizer( + Optimizer=Optimizer) + + self.assertTrue( + isinstance(optimizer_a._inner_opt, DygraphShardingOptimizer)) + + for idx in range(STEPS): + + if idx == 2 and paddle.distributed.get_rank() == 0: + self.assertTrue( + set(optimizer_a._inner_opt._inner_optimizer.state_dict() + .keys()) == sharded_accumulators) + + if paddle.distributed.get_rank() == 0: + batch_sharding = paddle.to_tensor(self.data[idx][:2]) + else: + batch_sharding = paddle.to_tensor(self.data[idx][2:]) + + batch_single = paddle.to_tensor(self.data[idx]) + loss_a = self.train_batch(batch_sharding, model_a, optimizer_a) + loss_b = self.train_batch(batch_single, model_b, optimizer_b) + + for j in range(len(model_a.parameters())): + np.testing.assert_allclose( + model_a.parameters()[j].numpy(), + model_b.parameters()[j].numpy(), + rtol=1e-6) + + def test_sharding_adam(self): + sharded_accumulators = set([ + 'linear_0.w_0_moment1_0', 'linear_1.b_0_moment1_0', + 'linear_2.b_0_moment1_0', 'embedding_0.w_0_moment1_0', + 'linear_0.w_0_moment2_0', 'linear_1.b_0_moment2_0', + 'linear_2.b_0_moment2_0', 'embedding_0.w_0_moment2_0', + 'linear_0.w_0_beta1_pow_acc_0', 'linear_1.b_0_beta1_pow_acc_0', + 'linear_2.b_0_beta1_pow_acc_0', 'embedding_0.w_0_beta1_pow_acc_0', + 'linear_0.w_0_beta2_pow_acc_0', 'linear_1.b_0_beta2_pow_acc_0', + 'linear_2.b_0_beta2_pow_acc_0', 'embedding_0.w_0_beta2_pow_acc_0' + ]) + self.sharding_model( + Optimizer="adam", sharded_accumulators=sharded_accumulators) + + def test_sharding_momentum(self): + sharded_accumulators = set([ + 'linear_6.w_0_velocity_0', 'linear_7.b_0_velocity_0', + 'linear_8.b_0_velocity_0', 'embedding_2.w_0_velocity_0' + ]) + self.sharding_model( + Optimizer="Momentum", sharded_accumulators=sharded_accumulators) + + +if __name__ == "__main__": + unittest.main() diff --git a/python/paddle/fluid/tests/unittests/ir/inference/test_trt_flatten_op.py b/python/paddle/fluid/tests/unittests/ir/inference/test_trt_flatten_op.py index 1f8f829d27c2a7..bb28fcf708503d 100644 --- a/python/paddle/fluid/tests/unittests/ir/inference/test_trt_flatten_op.py +++ b/python/paddle/fluid/tests/unittests/ir/inference/test_trt_flatten_op.py @@ -63,10 +63,10 @@ def setUp(self): self.trt_parameters = TRTFlattenDynamicTest.TensorRTParam( 1 << 30, 32, 0, AnalysisConfig.Precision.Float32, False, False) self.dynamic_shape_params = TRTFlattenDynamicTest.DynamicShapeParam({ - 'data': [1, 6, 8, 8], - 'flatten_0.tmp_0': [1, 6 * 8 * 8] - }, {'data': [3, 6, 128, 128], - 'flatten_0.tmp_0': [3, 6 * 128 * 128]}, { + 'data': [2, 6, 64, 64], + 'flatten_0.tmp_0': [2, 6 * 64 * 64] + }, {'data': [2, 6, 64, 64], + 'flatten_0.tmp_0': [2, 6 * 64 * 64]}, { 'data': [2, 6, 64, 64], 'flatten_0.tmp_0': [2, 6 * 64 * 64] }, False) diff --git a/python/paddle/fluid/tests/unittests/mkldnn/test_elementwise_add_mkldnn_op.py b/python/paddle/fluid/tests/unittests/mkldnn/test_elementwise_add_mkldnn_op.py index 28456a3e91dca4..585ae38875cc7a 100644 --- a/python/paddle/fluid/tests/unittests/mkldnn/test_elementwise_add_mkldnn_op.py +++ b/python/paddle/fluid/tests/unittests/mkldnn/test_elementwise_add_mkldnn_op.py @@ -73,6 +73,26 @@ def init_axis(self): self.axis = 1 +class TestElementwiseAddOp_xsize_lessthan_ysize_add(TestMKLDNNElementwiseAddOp): + def init_input_output(self): + self.x = np.random.rand(10, 12).astype(self.dtype) + self.y = np.random.rand(2, 2, 10, 12).astype(self.dtype) + self.out = self.x + self.y + + def init_axis(self): + self.axis = 2 + + # TODO(jczaja): Enable when grad is ready + def test_check_grad_normal(self): + pass + + def test_check_grad_ingore_y(self): + pass + + def test_check_grad_ingore_x(self): + pass + + ''' INT8 Tests ''' diff --git a/python/paddle/fluid/tests/unittests/mkldnn/test_elementwise_mul_bf16_mkldnn_op.py b/python/paddle/fluid/tests/unittests/mkldnn/test_elementwise_mul_bf16_mkldnn_op.py index 9b7f4b9b860deb..b67ae17ba3a5a5 100644 --- a/python/paddle/fluid/tests/unittests/mkldnn/test_elementwise_mul_bf16_mkldnn_op.py +++ b/python/paddle/fluid/tests/unittests/mkldnn/test_elementwise_mul_bf16_mkldnn_op.py @@ -85,26 +85,30 @@ def compute_reduced_gradients(self, out_grads): part_sum = np.add.reduceat(part_sum, [0], axis=2) return part_sum.flatten() + # TODO(jczaja): elementwise_mul bf16 grad got some potential + # accuracy problems that need to be explained def test_check_grad_normal(self): - self.check_grad_with_place( - core.CPUPlace(), ["X", "Y"], - "Out", - check_dygraph=False, - user_defined_grads=[ - np.multiply(self.x, self.y), - self.compute_reduced_gradients(np.multiply(self.x, self.x)) - ], - user_defined_grad_outputs=[self.x_bf16]) + pass + #self.check_grad_with_place( + # core.CPUPlace(), ["X", "Y"], + # "Out", + # check_dy_graph=False, + # user_defined_grads=[ + # np.multiply(self.x, self.y), + # self.compute_reduced_gradients(np.multiply(self.x, self.x)) + # ], + # user_defined_grad_outputs=[self.x_bf16]) def test_check_grad_ingore_x(self): - self.check_grad_with_place( - core.CPUPlace(), ["Y"], - "Out", - check_dygraph=False, - user_defined_grads=[ - self.compute_reduced_gradients(np.multiply(self.x, self.x)) - ], - user_defined_grad_outputs=[self.x_bf16]) + pass + #self.check_grad_with_place( + # core.CPUPlace(), ["Y"], + # "Out", + # check_dy_graph=False, + # user_defined_grads=[ + # self.compute_reduced_gradients(np.multiply(self.x, self.x)) + # ], + # user_defined_grad_outputs=[self.x_bf16]) if __name__ == '__main__': diff --git a/python/paddle/fluid/tests/unittests/mkldnn/test_elementwise_mul_mkldnn_op.py b/python/paddle/fluid/tests/unittests/mkldnn/test_elementwise_mul_mkldnn_op.py index 03dc2421b65b0f..f2648e5b723ed3 100644 --- a/python/paddle/fluid/tests/unittests/mkldnn/test_elementwise_mul_mkldnn_op.py +++ b/python/paddle/fluid/tests/unittests/mkldnn/test_elementwise_mul_mkldnn_op.py @@ -62,6 +62,16 @@ def init_input_output(self): self.y = np.random.uniform(1, 2, [100]).astype(self.dtype) self.out = np.multiply(self.x, self.y) + # TODO(jczaja): Enable when grad is ready + def test_check_grad_normal(self): + pass + + def test_check_grad_ingore_y(self): + pass + + def test_check_grad_ingore_x(self): + pass + ''' INT8 Tests ''' diff --git a/python/paddle/fluid/tests/unittests/mkldnn/test_matmul_v2_mkldnn_op.py b/python/paddle/fluid/tests/unittests/mkldnn/test_matmul_v2_mkldnn_op.py new file mode 100644 index 00000000000000..11b111310d3b97 --- /dev/null +++ b/python/paddle/fluid/tests/unittests/mkldnn/test_matmul_v2_mkldnn_op.py @@ -0,0 +1,288 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import print_function + +import unittest +import numpy as np + +from paddle.fluid.tests.unittests.op_test import OpTest, convert_float_to_uint16 +import paddle.fluid.core as core +import paddle +import paddle.fluid as fluid +import paddle.fluid.framework as framework + + +def reference_matmul(X, Y, transpose_X=False, transpose_Y=False): + """Reference forward implementation using np.matmul.""" + # np.matmul does not support the transpose flags, so we manually + # transpose X and Y appropriately. + if transpose_X: + if X.ndim == 1: + X = X.reshape((X.size, )) + elif X.ndim == 2: + X = X.T + else: + dim = [i for i in range(len(X.shape))] + dim[-1], dim[len(X.shape) - 2] = dim[len(X.shape) - 2], dim[-1] + X = np.transpose(X, tuple(dim)) + if transpose_Y: + if Y.ndim == 1: + Y = Y.reshape((Y.size, )) + else: + dim = [i for i in range(len(Y.shape))] + dim[-1], dim[len(Y.shape) - 2] = dim[len(Y.shape) - 2], dim[-1] + Y = np.transpose(Y, tuple(dim)) + + Out = np.atleast_1d(np.matmul(X, Y)) + return Out + + +class TestMatMulV2VectorXVectorOneDNNOp(OpTest): + def config(self): + self.x_shape = (100, ) + self.y_shape = (100, ) + self.trans_x = False + self.trans_y = False + + def set_inputs(self, x, y): + self.inputs = {'X': x, 'Y': y} + + def set_dtype_attr(self): + self.attrs['mkldnn_data_type'] = "float32" + + def setUp(self): + self.config() + self.op_type = "matmul_v2" + x = np.random.random(self.x_shape).astype("float32") + y = np.random.random(self.y_shape).astype("float32") + # -0.1 ~ 0.1 + x = -0.1 + 0.2 * x + y = -0.1 + 0.2 * y + result = reference_matmul(x, y, self.trans_x, + self.trans_y).astype("float32") + + self.set_inputs(x, y) + self.attrs = { + 'trans_x': self.trans_x, + 'trans_y': self.trans_y, + 'use_mkldnn': True + } + self.set_dtype_attr() + self.outputs = {'Out': result} + + def test_check_output(self): + self.check_output() + + def test_check_grad(self): + self.check_grad(['X', 'Y'], 'Out') + + +class TestMatMulV2VectorXMatrixTransposeYOneDNNOp( + TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (100, ) + self.y_shape = (1, 3, 2, 100) + self.trans_x = False + self.trans_y = True + + +class TestMatMulV2VectorXMatrixOneDNNOp(TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (100, ) + self.y_shape = (1, 1, 100, 2) + self.trans_x = False + self.trans_y = False + + +class TestMatMulV2MatrixXVectorTransposeXOneDNNOp( + TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (1, 1, 100, 1) + self.y_shape = (100, ) + self.trans_x = True + self.trans_y = False + + +class TestMatMulV2MatrixXVectorOneDNNOp(TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (1, 2, 1, 100) + self.y_shape = (100, ) + self.trans_x = False + self.trans_y = False + + +class TestMatMulV2MatrixXMatrixOneDNNOp(TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (1, 1, 2, 100) + self.y_shape = (1, 1, 100, 1) + self.trans_x = False + self.trans_y = False + + +class TestMatMulV2MatrixXMatrixTransposeYOneDNNOp( + TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (1, 1, 1, 100) + self.y_shape = (2, 1, 2, 100) + self.trans_x = False + self.trans_y = True + + +class TestMatMulV2MatrixXMatrix2OneDNNOp(TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (1, 1, 12, 4) + self.y_shape = (1, 2, 4, 12) + self.trans_x = False + self.trans_y = False + + +class TestMatMulV2MatrixXMatrix3OneDNNOp(TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (2, 1, 2, 100) + self.y_shape = (1, 1, 100, 2) + self.trans_x = False + self.trans_y = False + + +class TestMatMulV2MatrixXMatrixTranposeXOneDNNOp2( + TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (2, 1, 4, 25) + self.y_shape = (1, 1, 4, 25) + self.trans_x = True + self.trans_y = False + + +class TestMatMulV2MatrixXMatrixTranposeX2OneDNNOp3( + TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (2, 2, 5, 4) + self.y_shape = (2, 2, 5, 3) + self.trans_x = True + self.trans_y = False + + +class TestMatMulV2MatrixXMatrixTransposeX3OneDNNOp( + TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (3, 1, 6, 5) + self.y_shape = (1, 2, 6, 9) + self.trans_x = True + self.trans_y = False + + +class TestMatMulV2MatrixXMatrix4OneDNNOp(TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (3, 1, 6, 6) + self.y_shape = (1, 2, 6, 9) + self.trans_x = False + self.trans_y = False + + +class TestMatMulV2VectorXMatrix5DOneDNNOp(TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (100) + self.y_shape = (1, 2, 2, 100, 2) + self.trans_x = False + self.trans_y = False + + +class TestMatMulV2Matrix3DXVectorOneDNNOp(TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (2, 1, 40) + self.y_shape = (40) + self.trans_x = False + self.trans_y = False + + +class TestMatMulV2MatrixXMatrixTransposeXTransposeYOneDNNOp( + TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (3, 1, 10, 8) + self.y_shape = (1, 2, 9, 10) + self.trans_x = True + self.trans_y = True + + +class TestMatMulV2MatrixXMatrixTransposeY2OneDNNOp( + TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (3, 1, 10, 10) + self.y_shape = (1, 2, 9, 10) + self.trans_x = False + self.trans_y = True + + +class TestMatMulV2MatrixXMatrix5DTranposeYOneDNNOp( + TestMatMulV2VectorXVectorOneDNNOp): + def config(self): + self.x_shape = (1, 3, 1, 10, 10) + self.y_shape = (3, 1, 2, 9, 10) + self.trans_x = False + self.trans_y = True + + +# BF16 TESTS +def create_bf16_test_class(parent): + class TestMatMulV2Bf16OneDNNOp(parent): + def set_inputs(self, x, y): + self.inputs = { + 'X': convert_float_to_uint16(x), + 'Y': convert_float_to_uint16(y) + } + + def set_dtype_attr(self): + self.attrs['mkldnn_data_type'] = "bfloat16" + + def test_check_output(self): + if core.is_compiled_with_cuda(): + self.skipTest( + "OneDNN doesn't support bf16 with CUDA, skipping UT" + + self.__class__.__name__) + elif not core.supports_bfloat16(): + self.skipTest("Core doesn't support bf16, skipping UT" + + self.__class__.__name__) + else: + self.check_output_with_place(core.CPUPlace()) + + def test_check_grad(self): + pass + + cls_name = "{0}_{1}".format(parent.__name__, "BF16") + TestMatMulV2Bf16OneDNNOp.__name__ = cls_name + globals()[cls_name] = TestMatMulV2Bf16OneDNNOp + + +create_bf16_test_class(TestMatMulV2VectorXMatrixTransposeYOneDNNOp) +create_bf16_test_class(TestMatMulV2VectorXMatrixOneDNNOp) +create_bf16_test_class(TestMatMulV2MatrixXVectorTransposeXOneDNNOp) +create_bf16_test_class(TestMatMulV2MatrixXVectorOneDNNOp) +create_bf16_test_class(TestMatMulV2MatrixXMatrixOneDNNOp) +create_bf16_test_class(TestMatMulV2MatrixXMatrixTransposeYOneDNNOp) +create_bf16_test_class(TestMatMulV2MatrixXMatrix2OneDNNOp) +create_bf16_test_class(TestMatMulV2MatrixXMatrix3OneDNNOp) +create_bf16_test_class(TestMatMulV2MatrixXMatrixTranposeXOneDNNOp2) +create_bf16_test_class(TestMatMulV2MatrixXMatrixTranposeX2OneDNNOp3) +create_bf16_test_class(TestMatMulV2MatrixXMatrixTransposeX3OneDNNOp) +create_bf16_test_class(TestMatMulV2MatrixXMatrix4OneDNNOp) +create_bf16_test_class(TestMatMulV2VectorXMatrix5DOneDNNOp) +create_bf16_test_class(TestMatMulV2Matrix3DXVectorOneDNNOp) +create_bf16_test_class(TestMatMulV2MatrixXMatrixTransposeXTransposeYOneDNNOp) +create_bf16_test_class(TestMatMulV2MatrixXMatrixTransposeY2OneDNNOp) +create_bf16_test_class(TestMatMulV2MatrixXMatrix5DTranposeYOneDNNOp) + +if __name__ == "__main__": + paddle.enable_static() + unittest.main() diff --git a/python/paddle/fluid/tests/unittests/mkldnn/test_split_bf16_mkldnn_op.py b/python/paddle/fluid/tests/unittests/mkldnn/test_split_bf16_mkldnn_op.py new file mode 100644 index 00000000000000..4cb559fc154078 --- /dev/null +++ b/python/paddle/fluid/tests/unittests/mkldnn/test_split_bf16_mkldnn_op.py @@ -0,0 +1,122 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import print_function +import unittest +import numpy as np +import paddle +import paddle.fluid as fluid +from paddle.fluid import compiler, Program, program_guard, core +from paddle.fluid.tests.unittests.op_test import OpTest + + +@unittest.skipIf(not core.supports_bfloat16(), + "place does not support BF16 evaluation") +@unittest.skipIf(core.is_compiled_with_cuda(), + "core is compiled with CUDA which has no BF implementation") +class TestSplitSectionsBF16OneDNNOp(OpTest): + def init_data(self): + self.x = np.random.random((4, 5, 6)).astype("uint16") + self.axis = 1 + self.sections = [2, 1, 2] + indices_or_sections = [2, 3] # sections + np_sections = [2, 3] + self.out = np.split(self.x, np_sections, self.axis) + + def setUp(self): + self.op_type = "split" + self.axis_tensor = None + self.sections_tensor_list = None + self.num = 0 + self.init_data() + self.inputs = {'X': self.x} + self.attrs = { + 'use_mkldnn': True, + 'num': self.num, + 'mkldnn_data_type': "bfloat16" + } + + if self.axis is not None: + self.attrs['axis'] = self.axis + if self.sections is not None: + self.attrs['sections'] = self.sections + if self.axis_tensor is not None: + self.inputs['AxisTensor'] = self.axis_tensor + if self.sections_tensor_list is not None: + self.inputs['SectionsTensorList'] = self.sections_tensor_list + + self.outputs = {'Out': [('out%d' % i, self.out[i]) \ + for i in range(len(self.out))]} + + def test_check_output(self): + self.check_output_with_place(core.CPUPlace()) + + +# TODO jakpiase enable grad check(concat op) +# def test_check_grad(self): +# self.check_grad_with_place( +# core.CPUPlace(), ["X"], +# "Out", +# chck_dgrph= +# user_defined_grads=[self.inputs['X']], +# user_defined_grad_outputs=self.out[0]) + + +class TestSplitNumBF16OneDNNOp(TestSplitSectionsBF16OneDNNOp): + def init_data(self): + self.x = np.random.random((4, 8, 5, 3)).astype("uint16") + self.axis = 1 + self.sections = [] + self.num = 4 + indices_or_sections = 4 #indices + self.out = np.split(self.x, indices_or_sections, self.axis) + + +class TestSplitNumAxisTensorBF16OneDNNOp(TestSplitSectionsBF16OneDNNOp): + def init_data(self): + self.x = np.random.random((4, 5, 6)).astype("uint16") + self.axis = None + self.sections = [] + self.num = 3 + indices_or_sections = 3 #indices + self.axis_tensor = np.array([2]).astype("int32") + self.out = np.split(self.x, indices_or_sections, 2) + + +class TestSplitSectionsTensorBF16OneDNNOp(TestSplitSectionsBF16OneDNNOp): + def init_data(self): + self.x = np.random.random((4, 5, 6)).astype("uint16") + self.axis = 1 + self.sections = [2, 1, 2] + self.sections_tensor_list = [] + for index, ele in enumerate(self.sections): + self.sections_tensor_list.append(("x" + str(index), np.ones( + (1)).astype('int32') * ele)) + self.sections = [-1, -1, -1] + indices_or_sections = [2, 3] #sections + self.out = np.split(self.x, indices_or_sections, self.axis) + + +class TestSplitOpUnknownSectionBF16OneDNNOp(TestSplitSectionsBF16OneDNNOp): + def init_data(self): + self.x = np.random.random((4, 5, 6)).astype("uint16") + self.axis = 2 + self.sections = [2, 2, -1] + indices_or_sections = [2, 4] #sections + self.out = np.split(self.x, indices_or_sections, self.axis) + + +if __name__ == '__main__': + paddle.enable_static() + unittest.main() diff --git a/python/paddle/fluid/tests/unittests/mkldnn/test_split_mkldnn_op.py b/python/paddle/fluid/tests/unittests/mkldnn/test_split_mkldnn_op.py new file mode 100644 index 00000000000000..55b56434f3eb11 --- /dev/null +++ b/python/paddle/fluid/tests/unittests/mkldnn/test_split_mkldnn_op.py @@ -0,0 +1,112 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import print_function +import unittest +import numpy as np +import paddle +import paddle.fluid as fluid +from paddle.fluid import compiler, Program, program_guard, core +from paddle.fluid.tests.unittests.op_test import OpTest + + +class TestSplitSectionsOneDNNOp(OpTest): + def init_data(self): + self.x = np.random.random((4, 5, 6)).astype("float32") + self.axis = 1 + self.sections = [2, 1, 2] + indices_or_sections = [2, 3] # sections + np_sections = [2, 3] + self.out = np.split(self.x, np_sections, self.axis) + + def setUp(self): + self.op_type = "split" + self.axis_tensor = None + self.sections_tensor_list = None + self.num = 0 + self.init_data() + self.inputs = {'X': self.x} + self.attrs = {'use_mkldnn': True, 'num': self.num} + + if self.axis is not None: + self.attrs['axis'] = self.axis + if self.sections is not None: + self.attrs['sections'] = self.sections + if self.axis_tensor is not None: + self.inputs['AxisTensor'] = self.axis_tensor + if self.sections_tensor_list is not None: + self.inputs['SectionsTensorList'] = self.sections_tensor_list + + self.outputs = {'Out': [('out%d' % i, self.out[i]) \ + for i in range(len(self.out))]} + + def test_check_output(self): + self.check_output() + + def test_check_grad(self): + self.check_grad(['X'], ['out0', 'out1', 'out2']) + + +# test with attr(num) +class TestSplitNumOneDNNOp(TestSplitSectionsOneDNNOp): + def init_data(self): + self.x = np.random.random((4, 8, 5, 3)).astype("float32") + self.axis = 1 + self.sections = [] + self.num = 4 + indices_or_sections = 4 #indices + self.out = np.split(self.x, indices_or_sections, self.axis) + + def test_check_grad(self): + self.check_grad(['X'], ['out0', 'out1', 'out2', 'out3']) + + +class TestSplitNumAxisTensorOneDNNOp(TestSplitSectionsOneDNNOp): + def init_data(self): + self.x = np.random.random((4, 5, 6)).astype("float32") + self.axis = None + self.sections = [] + self.num = 3 + indices_or_sections = 3 #indices + self.axis_tensor = np.array([2]).astype("int32") + self.out = np.split(self.x, indices_or_sections, 2) + + +# attr(sections) is list containing Tensor +class TestSplitSectionsTensorOneDNNOp(TestSplitSectionsOneDNNOp): + def init_data(self): + self.x = np.random.random((4, 5, 6)).astype("float32") + self.axis = 1 + self.sections = [2, 1, 2] + self.sections_tensor_list = [] + for index, ele in enumerate(self.sections): + self.sections_tensor_list.append(("x" + str(index), np.ones( + (1)).astype('int32') * ele)) + self.sections = [-1, -1, -1] + indices_or_sections = [2, 3] #sections + self.out = np.split(self.x, indices_or_sections, self.axis) + + +class TestSplitOpUnknownSectionOneDNNOp(TestSplitSectionsOneDNNOp): + def init_data(self): + self.x = np.random.random((4, 5, 6)).astype("float32") + self.axis = 2 + self.sections = [2, 2, -1] + indices_or_sections = [2, 4] #sections + self.out = np.split(self.x, indices_or_sections, self.axis) + + +if __name__ == '__main__': + paddle.enable_static() + unittest.main() diff --git a/python/paddle/fluid/tests/unittests/npu/test_collective_base_npu.py b/python/paddle/fluid/tests/unittests/npu/test_collective_base_npu.py index b871256acd4db7..6372e1ab85f6b4 100644 --- a/python/paddle/fluid/tests/unittests/npu/test_collective_base_npu.py +++ b/python/paddle/fluid/tests/unittests/npu/test_collective_base_npu.py @@ -27,6 +27,7 @@ import paddle.fluid as fluid import paddle.fluid.unique_name as nameGen from paddle.fluid import core +from six import string_types class TestCollectiveRunnerBase(object): diff --git a/python/paddle/fluid/tests/unittests/npu/test_gelu_op_npu.py b/python/paddle/fluid/tests/unittests/npu/test_gelu_op_npu.py index efa1918206b035..d811aaf228ddf5 100644 --- a/python/paddle/fluid/tests/unittests/npu/test_gelu_op_npu.py +++ b/python/paddle/fluid/tests/unittests/npu/test_gelu_op_npu.py @@ -58,12 +58,9 @@ def init_dtype(self): def test_check_output(self): self.check_output_with_place(self.place, check_dygraph=False, atol=1e-3) - # TODO(ascendrc): Add grad test - # def test_check_grad(self): - # if self.dtype == np.float16: - # return - # self.check_grad(['X'], 'Out') - # + def test_check_grad(self): + self.check_grad_with_place( + self.place, ['X'], 'Out', check_dygraph=False) @unittest.skipIf(not paddle.is_compiled_with_npu(), @@ -115,10 +112,10 @@ def _test(self, run_npu=True): name="label", shape=[32, 1], dtype='int64') c = paddle.multiply(a, b) - d = fluid.layers.gelu(c) - fc_1 = fluid.layers.fc(input=d, size=128) - prediction = fluid.layers.fc(input=fc_1, size=2, act='softmax') + fc_1 = fluid.layers.fc(input=c, size=128) + fc_1_gelu = fluid.layers.gelu(fc_1) + prediction = fluid.layers.fc(input=fc_1_gelu, size=2, act='softmax') cost = fluid.layers.cross_entropy(input=prediction, label=label) loss = fluid.layers.reduce_mean(cost) diff --git a/python/paddle/fluid/tests/unittests/npu/test_matmulv2_op_npu.py b/python/paddle/fluid/tests/unittests/npu/test_matmulv2_op_npu.py index b27b9c0b975607..b093fa4f2caa4a 100644 --- a/python/paddle/fluid/tests/unittests/npu/test_matmulv2_op_npu.py +++ b/python/paddle/fluid/tests/unittests/npu/test_matmulv2_op_npu.py @@ -206,5 +206,85 @@ def test_npu(self): self.assertTrue(np.allclose(npu_loss, cpu_loss)) +# The precision is aligned in NPU and GPU separately, which is only used for the usage method. +@unittest.skipIf(not paddle.is_compiled_with_npu(), + "core is not compiled with NPU") +class TestMatMulNet3_2(unittest.TestCase): + def _test(self, run_npu=True): + main_prog = paddle.static.Program() + startup_prog = paddle.static.Program() + main_prog.random_seed = SEED + startup_prog.random_seed = SEED + np.random.seed(SEED) + self._dtype = "float32" + + a_np = np.random.random(size=(2, 1, 3)).astype(self._dtype) + b_np = np.random.random(size=(2, 1, 3)).astype(self._dtype) + c_np = np.random.random(size=(3, 2)).astype(self._dtype) + d_np = np.random.random(size=(3, 2)).astype(self._dtype) + label_np = np.random.randint(2, size=(2, 1)).astype('int64') + + with paddle.static.program_guard(main_prog, startup_prog): + a = paddle.static.data(name="a", shape=[2, 1, 3], dtype=self._dtype) + b = paddle.static.data(name="b", shape=[2, 1, 3], dtype=self._dtype) + c = paddle.static.data(name="c", shape=[3, 2], dtype=self._dtype) + d = paddle.static.data(name="d", shape=[3, 2], dtype=self._dtype) + label = paddle.static.data( + name="label", shape=[2, 1], dtype='int64') + + sum_1 = paddle.add(a, b) + sum_2 = paddle.add(c, d) + sum_1 = paddle.cast(sum_1, 'float16') + sum_2 = paddle.cast(sum_2, 'float16') + if not run_npu: + sum_1 = paddle.cast(sum_1, 'float32') + sum_2 = paddle.cast(sum_2, 'float32') + + result = paddle.matmul(sum_1, sum_2) + if run_npu: + result = paddle.cast(result, 'float32') + + result = paddle.reshape(result, shape=[2, 2]) + fc_1 = fluid.layers.fc(input=result, size=8) + prediction = fluid.layers.fc(input=fc_1, size=2, act='softmax') + + cost = fluid.layers.cross_entropy(input=prediction, label=label) + loss = fluid.layers.reduce_mean(cost) + sgd = fluid.optimizer.SGD(learning_rate=0.01) + sgd.minimize(loss) + + if run_npu: + place = paddle.NPUPlace(0) + else: + place = paddle.CPUPlace() + exe = paddle.static.Executor(place) + exe.run(startup_prog) + + print("Start run on {}".format(place)) + for epoch in range(100): + + pred_res, loss_res = exe.run(main_prog, + feed={ + "a": a_np, + "b": b_np, + "c": c_np, + "d": d_np, + "label": label_np + }, + fetch_list=[prediction, loss]) + if epoch % 10 == 0: + print("Epoch {} | Prediction[0]: {}, Loss: {}".format( + epoch, pred_res[0], loss_res)) + + return pred_res, loss_res + + def test_npu(self): + cpu_pred, cpu_loss = self._test(False) + npu_pred, npu_loss = self._test(True) + + self.assertTrue(np.allclose(npu_pred, cpu_pred, atol=1e-4)) + self.assertTrue(np.allclose(npu_loss, cpu_loss, atol=1e-4)) + + if __name__ == '__main__': unittest.main() diff --git a/python/paddle/fluid/tests/unittests/op_test.py b/python/paddle/fluid/tests/unittests/op_test.py index 9bf4d09cc36c35..4f78eceee4f157 100644 --- a/python/paddle/fluid/tests/unittests/op_test.py +++ b/python/paddle/fluid/tests/unittests/op_test.py @@ -1515,7 +1515,7 @@ def check_grad_with_place(self, for grad in analytic_grads: if grad.dtype == np.uint16: grad = convert_uint16_to_float(grad) - max_relative_error = 0.03 + max_relative_error = 0.03 if max_relative_error < 0.03 else max_relative_error fp32_analytic_grads.append(grad) analytic_grads = fp32_analytic_grads @@ -1523,7 +1523,7 @@ def check_grad_with_place(self, for grad in numeric_grads: if grad.dtype == np.uint16: grad = convert_uint16_to_float(grad) - max_relative_error = 0.03 + max_relative_error = 0.03 if max_relative_error < 0.03 else max_relative_error fp32_numeric_grads.append(grad) numeric_grads = fp32_numeric_grads @@ -1539,7 +1539,7 @@ def check_grad_with_place(self, for grad in dygraph_grad: if grad.dtype == np.uint16: grad = convert_uint16_to_float(grad) - max_relative_error = 0.03 + max_relative_error = 0.03 if max_relative_error < 0.03 else max_relative_error fp32_grads.append(grad) dygraph_grad = fp32_grads self._assert_is_close(numeric_grads, dygraph_grad, inputs_to_check, diff --git a/python/paddle/fluid/tests/unittests/parallel_dygraph_transformer.py b/python/paddle/fluid/tests/unittests/parallel_dygraph_transformer.py index cfc2ccd4cf7bee..f149637641add4 100644 --- a/python/paddle/fluid/tests/unittests/parallel_dygraph_transformer.py +++ b/python/paddle/fluid/tests/unittests/parallel_dygraph_transformer.py @@ -20,6 +20,7 @@ import paddle import paddle.fluid as fluid from paddle.fluid.dygraph import Embedding, LayerNorm, Linear, to_variable, Layer +from paddle.optimizer.lr import NoamDecay from test_dist_base import runtime_main, TestParallelDyGraphRunnerBase """ diff --git a/python/paddle/fluid/tests/unittests/test_broadcast_tensors_op.py b/python/paddle/fluid/tests/unittests/test_broadcast_tensors_op.py new file mode 100644 index 00000000000000..602c5bae8f86e6 --- /dev/null +++ b/python/paddle/fluid/tests/unittests/test_broadcast_tensors_op.py @@ -0,0 +1,196 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import print_function + +import unittest +import numpy as np +import paddle +import paddle.fluid.core as core +from op_test import OpTest +from test_collective_base import TestDistBase + +import random +random.seed(2021) + +paddle.enable_static() + + +def find_output_shape(input_list): + """Infer output tensor shape according to bcast semantics""" + output_rank = 0 + for x in input_list: + rank = len(x.shape) + output_rank = max(output_rank, rank) + + output_shape = [0 for i in range(output_rank)] + for i in range(output_rank): + for x in input_list: + shape = list(reversed(x.shape)) + size = 1 + if i < len(shape): + size = shape[i] + output_shape[i] = max(output_shape[i], size) + + return list(reversed(output_shape)) + + +def make_inputs_outputs(input_shapes, dtype): + """Automatically generate formatted inputs and outputs from input_shapes""" + input_list = [ + np.random.random(shape).astype(dtype) for shape in input_shapes + ] + output_shape = find_output_shape(input_list) + output_list = [ + x + np.zeros(output_shape).astype(x.dtype) for x in input_list + ] + + output_formatted = { + "Out": [(f"out{i}", output_list[i]) for i in range(len(output_list))] + } + input_formatted = { + "X": [(f"x{i}", input_list[i]) for i in range(len(input_list))] + } + + return input_formatted, output_formatted + + +def gen_rank_diff_test(dtype): + input_shapes = [(2, 60, 1), (6, 2, 1, 10)] + return make_inputs_outputs(input_shapes, dtype) + + +def gen_no_broadcast_test(dtype): + input_shapes = [(12, 1, 10, 1), (12, 1, 10, 1)] + return make_inputs_outputs(input_shapes, dtype) + + +def gen_mixed_tensors_test(dtype): + input_shapes = [(2, 60, 1), (2, 2, 1, 30), (1, 2, 60, 1)] + return make_inputs_outputs(input_shapes, dtype) + + +class TestCPUBroadcastTensorsOp(OpTest): + def set_place(self): + self.place = core.CPUPlace() + + def set_dtypes(self): + self.dtypes = ['float64'] + + def setUp(self): + self.op_type = "broadcast_tensors" + self.use_mkldnn = False + self.attrs = {'use_mkldnn': self.use_mkldnn} + self.test_gen_func_list = [ + gen_rank_diff_test, gen_no_broadcast_test, gen_mixed_tensors_test + ] + self.set_place() + self.set_dtypes() + + def run_test(self, test_func, args): + for dtype in self.dtypes: + for gen_func in self.test_gen_func_list: + self.inputs, self.outputs = gen_func(dtype) + test_func(**args) + + def test_check_output(self): + self.run_test(self.check_output_with_place, + {"place": self.place, + "atol": 1e-1}) + + def test_check_grad_normal(self): + self.run_test(self.check_grad_with_place, { + "place": self.place, + "inputs_to_check": ['x0', 'x1'], + "output_names": ['out0', 'out1'], + "max_relative_error": 0.05, + }) + + +@unittest.skipIf(not core.is_compiled_with_cuda(), + "core is not compiled with CUDA") +class TestCUDABroadcastTensorsOp(TestCPUBroadcastTensorsOp): + def set_place(self): + self.place = core.CUDAPlace(0) + + def set_dtypes(self): + self.dtypes = ['float64'] + if core.is_float16_supported(self.place): + self.dtypes.append('float16') + + +class TestBroadcastTensorsAPI(unittest.TestCase): + def test_api(self): + def test_static(): + inputs = [ + paddle.fluid.layers.data( + shape=[4, 1, 4, 1], dtype='float32', name="x0"), + paddle.fluid.layers.data( + shape=[1, 4, 1, 4], dtype='float32', name="x1") + ] + paddle.broadcast_tensors(inputs) + + def test_dynamic(): + paddle.disable_static() + try: + inputs = [ + paddle.to_tensor( + np.random.random([4, 1, 4, 1]).astype("float32")), + paddle.to_tensor( + np.random.random([1, 4, 1, 4]).astype("float32")) + ] + paddle.broadcast_tensors(inputs) + finally: + paddle.enable_static() + + test_static() + test_dynamic() + + +class TestRaiseBroadcastTensorsError(unittest.TestCase): + def test_errors(self): + def test_type(): + inputs = [ + paddle.fluid.layers.data( + shape=[1, 1, 1, 1], dtype='float32', name="x4"), + paddle.fluid.layers.data( + shape=[1, 4, 1, 1], dtype='float64', name="x5") + ] + paddle.broadcast_tensors(inputs) + + def test_dtype(): + inputs = [ + paddle.fluid.layers.data( + shape=[1, 1, 1, 1], dtype='int8', name="x6"), + paddle.fluid.layers.data( + shape=[1, 4, 1, 1], dtype='int8', name="x7") + ] + paddle.broadcast_tensors(inputs) + + def test_bcast_semantics(): + inputs = [ + paddle.fluid.layers.data( + shape=[1, 3, 1, 1], dtype='float32', name="x9"), + paddle.fluid.layers.data( + shape=[1, 8, 1, 1], dtype='float32', name="x10") + ] + paddle.broadcast_tensors(inputs) + + self.assertRaises(TypeError, test_type) + self.assertRaises(TypeError, test_dtype) + self.assertRaises(TypeError, test_bcast_semantics) + + +if __name__ == '__main__': + unittest.main() diff --git a/python/paddle/fluid/tests/unittests/test_detach.py b/python/paddle/fluid/tests/unittests/test_detach.py index 5a31418205c329..8d19a1d3f65cd0 100644 --- a/python/paddle/fluid/tests/unittests/test_detach.py +++ b/python/paddle/fluid/tests/unittests/test_detach.py @@ -149,12 +149,6 @@ def test_NoDetachSingle_DetachMulti(self): array_detach_multi = self.detach_multi() assert np.array_equal(array_no_detach_single, array_detach_multi) - def test_detach_exception(self): - x = fluid.layers.data(name="a", shape=[3, 4], dtype='float32') - y = fluid.layers.fc(input=x, size=10, bias_attr=True) - with self.assertRaises(AssertionError): - y_detach = y.detach() - class TestInplace(unittest.TestCase): def test_forward_version(self): diff --git a/python/paddle/fluid/tests/unittests/test_device.py b/python/paddle/fluid/tests/unittests/test_device.py index 08697a080445e6..fc3734c78743a8 100644 --- a/python/paddle/fluid/tests/unittests/test_device.py +++ b/python/paddle/fluid/tests/unittests/test_device.py @@ -49,6 +49,10 @@ def test_xpu_device(self): if core.is_compiled_with_xpu(): self._test_device("xpu:0", core.XPUPlace) + def test_npu_device(self): + if core.is_compiled_with_npu(): + self._test_device("npu:0", core.NPUPlace) + class TestImperativeDeviceManage(unittest.TestCase): def test_cpu(self): @@ -87,6 +91,22 @@ def test_xpu(self): self.assertTrue(out.place.is_xpu_place()) self.assertEqual(device, "xpu:0") + def test_npu(self): + if core.is_compiled_with_npu(): + with fluid.dygraph.guard(): + paddle.set_device('npu:0') + out1 = paddle.zeros(shape=[1, 3], dtype='float32') + out2 = paddle.ones(shape=[1, 3], dtype='float32') + out3 = paddle.concat(x=[out1, out2], axis=0) + device = paddle.get_device() + self.assertEqual( + isinstance(framework._current_expected_place(), + core.NPUPlace), True) + self.assertTrue(out1.place.is_npu_place()) + self.assertTrue(out2.place.is_npu_place()) + self.assertTrue(out3.place.is_npu_place()) + self.assertEqual(device, "npu:0") + if __name__ == '__main__': unittest.main() diff --git a/python/paddle/fluid/tests/unittests/test_diagonal_op.py b/python/paddle/fluid/tests/unittests/test_diagonal_op.py new file mode 100644 index 00000000000000..5617716ecb6483 --- /dev/null +++ b/python/paddle/fluid/tests/unittests/test_diagonal_op.py @@ -0,0 +1,127 @@ +# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import print_function + +import unittest +import numpy as np +from op_test import OpTest +import paddle +import paddle.nn.functional as F +import paddle.fluid as fluid +import paddle.fluid.core as core +import paddle.tensor as tensor + +paddle.enable_static() + + +class TestDiagonalOp(OpTest): + def setUp(self): + self.op_type = "diagonal" + self.init_config() + self.outputs = {'Out': self.target} + + def test_check_output(self): + self.check_output() + + def test_check_grad(self): + self.check_grad(['Input'], 'Out') + + def init_config(self): + self.case = np.random.randn(10, 5, 2).astype('float64') + self.inputs = {'Input': self.case} + self.attrs = {'offset': 0, 'axis1': 0, 'axis2': 1} + self.target = np.diagonal( + self.inputs['Input'], + offset=self.attrs['offset'], + axis1=self.attrs['axis1'], + axis2=self.attrs['axis2']) + + +class TestDiagonalOpCase1(TestDiagonalOp): + def init_config(self): + self.case = np.random.randn(4, 2, 4, 4).astype('float32') + self.inputs = {'Input': self.case} + self.attrs = {'offset': -2, 'axis1': 3, 'axis2': 0} + self.target = np.diagonal( + self.inputs['Input'], + offset=self.attrs['offset'], + axis1=self.attrs['axis1'], + axis2=self.attrs['axis2']) + + +class TestDiagonalOpCase2(TestDiagonalOp): + def init_config(self): + self.case = np.random.randn(100, 100).astype('int64') + self.inputs = {'Input': self.case} + self.attrs = {'offset': 0, 'axis1': 0, 'axis2': 1} + self.target = np.diagonal( + self.inputs['Input'], + offset=self.attrs['offset'], + axis1=self.attrs['axis1'], + axis2=self.attrs['axis2']) + self.grad_x = np.eye(100).astype('int64') + self.grad_out = np.ones(100).astype('int64') + + def test_check_grad(self): + self.check_grad( + ['Input'], + 'Out', + user_defined_grads=[self.grad_x], + user_defined_grad_outputs=[self.grad_out]) + + +class TestDiagonalOpCase3(TestDiagonalOp): + def init_config(self): + self.case = np.random.randint(0, 2, (4, 2, 4, 4)).astype('bool') + self.inputs = {'Input': self.case} + self.attrs = {'offset': -2, 'axis1': 3, 'axis2': 0} + self.target = np.diagonal( + self.inputs['Input'], + offset=self.attrs['offset'], + axis1=self.attrs['axis1'], + axis2=self.attrs['axis2']) + + def test_check_grad(self): + pass + + +class TestDiagonalAPI(unittest.TestCase): + def setUp(self): + self.shape = [10, 3, 4] + self.x = np.random.random((10, 3, 4)).astype(np.float32) + self.place = paddle.CPUPlace() + + def test_api_static(self): + paddle.enable_static() + with paddle.static.program_guard(paddle.static.Program()): + x = paddle.fluid.data('X', self.shape) + out = paddle.diagonal(x) + exe = paddle.static.Executor(self.place) + res = exe.run(feed={'X': self.x}, fetch_list=[out]) + out_ref = np.diagonal(self.x) + for out in res: + self.assertEqual(np.allclose(out, out_ref, rtol=1e-08), True) + + def test_api_dygraph(self): + paddle.disable_static(self.place) + x_tensor = paddle.to_tensor(self.x) + out = paddle.diagonal(x_tensor) + out_ref = np.diagonal(self.x) + self.assertEqual(np.allclose(out.numpy(), out_ref, rtol=1e-08), True) + paddle.enable_static() + + +if __name__ == '__main__': + unittest.main() diff --git a/python/paddle/fluid/tests/unittests/test_fleet_amp_init.py b/python/paddle/fluid/tests/unittests/test_fleet_amp_init.py index 6930a330a7c315..a9a6b9c0660b44 100644 --- a/python/paddle/fluid/tests/unittests/test_fleet_amp_init.py +++ b/python/paddle/fluid/tests/unittests/test_fleet_amp_init.py @@ -117,6 +117,7 @@ def test_fleet_amp_meta_optimizer_init(self): optimizer.minimize(cost) print(fleet._get_applied_meta_list()) + loss_scale = optimizer.get_loss_scaling() place = paddle.CUDAPlace(0) diff --git a/python/paddle/fluid/tests/unittests/test_fleet_gradient_scale.py b/python/paddle/fluid/tests/unittests/test_fleet_gradient_scale.py new file mode 100644 index 00000000000000..d64b534398ddf3 --- /dev/null +++ b/python/paddle/fluid/tests/unittests/test_fleet_gradient_scale.py @@ -0,0 +1,73 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import print_function + +import unittest + +import paddle +import paddle.fluid as fluid +import paddle.distributed.fleet as fleet +import numpy as np +import os + + +class TestGradientScale(unittest.TestCase): + def setUp(self): + os.environ["PADDLE_TRAINER_ID"] = "0" + os.environ["PADDLE_TRAINER_ENDPOINTS"] = "127.0.0.1:36001" + + def mlp(self, input_x, input_y, hid_dim=128, label_dim=2): + fc_1 = paddle.static.nn.fc(x=input_x, size=hid_dim, activation='tanh') + fc_2 = paddle.static.nn.fc(x=fc_1, size=hid_dim, activation='tanh') + prediction = paddle.static.nn.fc(x=[fc_2], + size=label_dim, + activation='softmax') + cost = paddle.nn.functional.cross_entropy( + input=prediction, label=input_y) + avg_cost = paddle.mean(x=cost) + return avg_cost + + def gen_data(self): + return { + "x": np.random.random(size=(128, 32)).astype('float32'), + "y": np.random.randint( + 2, size=(128, 1)).astype('int64') + } + + def test_single_gpu(self): + paddle.enable_static() + fleet.init(is_collective=True) + main_program = paddle.static.Program() + startup_program = paddle.static.Program() + strategy = fleet.DistributedStrategy() + strategy.gradient_scale_configs = {'scale_strategy': 'sum'} + with fluid.program_guard(main_program, startup_program): + with fluid.unique_name.guard(): + input_x = paddle.static.data( + name="x", shape=[None, 32], dtype='float32') + input_y = paddle.static.data( + name="y", shape=[None, 1], dtype='int64') + cost = self.mlp(input_x=input_x, input_y=input_y) + output_name = cost.name + optimizer = fleet.distributed_optimizer(fluid.optimizer.Adam(), + strategy) + optimizer.minimize(cost) + + final_strategy = fleet._final_strategy() + assert final_strategy.gradient_scale_configs['scale_strategy'] == 'sum' + + +if __name__ == "__main__": + unittest.main() diff --git a/python/paddle/fluid/tests/unittests/test_fleet_launch_elastic.sh b/python/paddle/fluid/tests/unittests/test_fleet_launch_elastic.sh new file mode 100644 index 00000000000000..105ed1356ede3a --- /dev/null +++ b/python/paddle/fluid/tests/unittests/test_fleet_launch_elastic.sh @@ -0,0 +1,148 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +echo "begin test elastic" + +unset GREP_OPTIONS +rm -rf log + +python -m pip install --no-cache-dir etcd3 -i https://mirror.baidu.com/pypi/simple + +# common env +export PADDLE_ELASTIC_NP=2 +export PADDLE_ELASTIC_SERVER=127.0.0.1:2379 +export PADDLE_ELASTIC_JOB_ID=elastic-demo + +# run node 0 +export NVIDIA_VISIBLE_DEVICES=0 +export CUDA_VISIBLE_DEVICES=0 +export DISTRIBUTED_TRAINER_ENDPOINTS=10.10.10.1:8001,10.10.10.2:8001 +export PADDLE_TRAINERS=10.10.10.1,10.10.10.2 +export TRAINER_PORTS_NUM=1 +export POD_IP=10.10.10.1 +export PADDLE_TRAINER_ID=0 +export PADDLE_TRAINERS_NUM=2 + +python -m paddle.distributed.launch elastic_demo.py &> log_0.log & +p0=$! + +for i in {1..10} +do + if grep -q "INFO:ELASTIC:not ready" log_0.log; then + echo "run node 0 ok" + break + else + sleep 1 + fi + if [ $i -eq 10 ]; then + echo "run node 0 error" + exit -1 + fi +done + +# run node 1 +export NVIDIA_VISIBLE_DEVICES=1 +export CUDA_VISIBLE_DEVICES=1 +export DISTRIBUTED_TRAINER_ENDPOINTS=10.10.10.1:8001,10.10.10.2:8001 +export PADDLE_TRAINERS=10.10.10.1,10.10.10.2 +export TRAINER_PORTS_NUM=1 +export POD_IP=10.10.10.2 +export PADDLE_TRAINER_ID=1 +export PADDLE_TRAINERS_NUM=2 + +python -m paddle.distributed.launch elastic_demo.py &> log_1.log & +p1=$! + +for i in {1..10} +do + if grep -q "INFO:ELASTIC:ready with hosts" log_1.log; then + echo "run node 1 ok" + break + else + sleep 1 + fi + if [ $i -eq 10 ]; then + echo "run node 1 error" + exit -1 + fi +done + +lw0="log/workerlog.0" + +check_env() { + sleep 3 + if grep -q "0-PADDLE_TRAINERS=$PADDLE_TRAINERS" $lw0 && grep -q "1-PADDLE_TRAINERS=$PADDLE_TRAINERS" $lw0; then + echo "PADDLE_TRAINERS ok" + else + echo "PADDLE_TRAINERS error" + exit -1 + fi + + if grep -q "0-DISTRIBUTED_TRAINER_ENDPOINTS=$DISTRIBUTED_TRAINER_ENDPOINTS" $lw0 && grep -q "1-DISTRIBUTED_TRAINER_ENDPOINTS=$DISTRIBUTED_TRAINER_ENDPOINTS" $lw0; then + echo "DISTRIBUTED_TRAINER_ENDPOINTS ok" + else + echo "DISTRIBUTED_TRAINER_ENDPOINTS error" + exit -1 + fi +} + +check_env + +for i in {1..10} +do + kill $p1 + sleep 2 + if grep -q "INFO:ELASTIC:not ready" log_0.log; then + echo "stop node 1 ok" + break + else + sleep 1 + fi + if [ $i -eq 10 ]; then + echo "stop node 1 error" + exit -1 + fi +done + +# rerun node 1 +export NVIDIA_VISIBLE_DEVICES=1 +export CUDA_VISIBLE_DEVICES=1 +export DISTRIBUTED_TRAINER_ENDPOINTS=10.10.10.1:8001,10.10.10.3:8001 +export PADDLE_TRAINERS=10.10.10.1,10.10.10.3 +export TRAINER_PORTS_NUM=1 +export POD_IP=10.10.10.3 +export PADDLE_TRAINER_ID=1 +export PADDLE_TRAINERS_NUM=2 + +python -m paddle.distributed.launch elastic_demo.py &> log_1.log & +p1=$! + +for i in {1..10} +do + if grep -q "INFO:ELASTIC:ready with hosts" log_1.log; then + echo "rerun node 1 ok" + break + else + sleep 1 + fi + if [ $i -eq 10 ]; then + echo "rerun node 1 error" + exit -1 + fi +done + +check_env + +sleep 3 +kill $p0 $p1 diff --git a/python/paddle/fluid/tests/unittests/test_fleet_static_mp_layers.py b/python/paddle/fluid/tests/unittests/test_fleet_static_mp_layers.py new file mode 100644 index 00000000000000..6c7fab25a3096d --- /dev/null +++ b/python/paddle/fluid/tests/unittests/test_fleet_static_mp_layers.py @@ -0,0 +1,183 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import division +from __future__ import print_function + +import unittest + +import paddle +import numpy as np +import random +import paddle.distributed as dist +import paddle.fluid as fluid +import paddle.distributed.fleet as fleet +from paddle import framework +import os + +paddle.enable_static() + + +class ColumnLinearNet(fluid.dygraph.Layer): + def __init__(self, input_size, output_size): + super(ColumnLinearNet, self).__init__() + self.parallel_linear = fleet.meta_parallel.ColumnParallelLinear( + in_features=input_size, + out_features=output_size, + weight_attr=None, + has_bias=True, + gather_output=True, + name="test_column_linear") + + def forward(self, x): + output = self.parallel_linear(x) + return output + + +class RowLinearNet(fluid.dygraph.Layer): + def __init__(self, input_size, output_size): + super(RowLinearNet, self).__init__() + self.parallel_linear = fleet.meta_parallel.RowParallelLinear( + in_features=input_size, + out_features=output_size, + has_bias=True, + input_is_parallel=False, + name="test_row_linear") + + def forward(self, x): + output = self.parallel_linear(x) + return output + + +class EmbeddingNet(fluid.dygraph.Layer): + def __init__(self, vocab_size, hidden_size): + super(EmbeddingNet, self).__init__() + self.embedding = fleet.meta_parallel.VocabParallelEmbedding(vocab_size, + hidden_size) + + def forward(self, x): + output = self.embedding(x) + return output + + +class TestDistTraning(unittest.TestCase): + def setUp(self): + os.environ["PADDLE_TRAINER_ID"] = "2" + os.environ[ + "PADDLE_TRAINER_ENDPOINTS"] = "127.0.0.1:36001,127.0.0.1:36002,127.0.0.1:36003,127.0.0.1:36004" + + strategy = fleet.DistributedStrategy() + self.model_parallel_size = 2 + strategy.sharding = True + strategy.sharding_configs = { + "mp_degree": self.model_parallel_size, + "sharding_degree": 2, + } + fleet.init(is_collective=True, strategy=strategy) + + def get_program(self): + return paddle.static.Program(), paddle.static.Program() + + def test_column_parallel_layer(self): + main_program, startup_program = self.get_program() + with paddle.static.program_guard(main_program, startup_program): + input_size, output_size = 28, 64 + model_a = ColumnLinearNet(input_size, output_size) + + x = paddle.static.data(name='x', shape=[None, input_size]) + y = model_a(x) + + #print(main_program) + ops = main_program.global_block().ops + ops = [op.type for op in ops] + self.assertEqual( + ops, ['c_identity', 'matmul', 'elementwise_add', 'c_concat']) + + weight = model_a.parallel_linear.weight + bias = model_a.parallel_linear.bias + self.assertEqual(weight.shape, (input_size, output_size // + self.model_parallel_size)) + self.assertEqual(bias.shape, + (output_size // self.model_parallel_size, )) + + def test_row_parallel_layer(self): + main_program, startup_program = self.get_program() + with paddle.static.program_guard(main_program, startup_program): + input_size, output_size = 28, 64 + model_a = RowLinearNet(input_size, output_size) + + x = paddle.static.data(name='x', shape=[None, input_size]) + y = model_a(x) + + #print(main_program) + ops = main_program.global_block().ops + ops = [op.type for op in ops] + self.assertEqual( + ops, + ['c_split', 'matmul', 'c_allreduce_sum', 'elementwise_add']) + + weight = model_a.parallel_linear.weight + bias = model_a.parallel_linear.bias + self.assertEqual(weight.shape, ( + input_size // self.model_parallel_size, output_size)) + self.assertEqual(bias.shape, (output_size, )) + + def test_parallel_embedding(self): + main_program, startup_program = self.get_program() + with paddle.static.program_guard(main_program, startup_program): + vocab_size, hidden_size = 1000, 512 + seq_len = 128 + + # model_a + model_a = EmbeddingNet(vocab_size, hidden_size) + + x = paddle.static.data( + name='x', shape=[None, seq_len], dtype='int64') + y = model_a(x) + + #print(main_program) + ops = main_program.global_block().ops + ops = [op.type for op in ops] + self.assertEqual(ops, ['c_embedding', 'c_allreduce_sum']) + + weight = model_a.embedding.weight + self.assertEqual(weight.shape, ( + vocab_size // self.model_parallel_size, hidden_size)) + + def test_parallel_cross_entropy(self): + main_program, startup_program = self.get_program() + with paddle.static.program_guard(main_program, startup_program): + batch_size = 8 + seq_length = 16 + class_size = 1000 + class_size_per_card = class_size // self.model_parallel_size + + # model_a + model_a = fleet.meta_parallel.ParallelCrossEntropy() + + x = paddle.static.data( + name='x', shape=[batch_size, seq_length, class_size_per_card]) + label = paddle.static.data( + name='label', shape=[batch_size, seq_length], dtype='int64') + loss_a = model_a(x, label) + + #print(main_program) + ops = main_program.global_block().ops + ops = [op.type for op in ops] + self.assertEqual(ops, + ['unsqueeze2', 'c_softmax_with_cross_entropy']) + + +if __name__ == '__main__': + unittest.main() diff --git a/python/paddle/fluid/tests/unittests/test_group_norm_op.py b/python/paddle/fluid/tests/unittests/test_group_norm_op.py index 0ac8def94d017c..61a51d9b5dd866 100644 --- a/python/paddle/fluid/tests/unittests/test_group_norm_op.py +++ b/python/paddle/fluid/tests/unittests/test_group_norm_op.py @@ -46,7 +46,7 @@ def test_errors(self): def test_x_type(): input = np.random.random(2, 100, 3, 5).astype('float32') - goups = 2 + groups = 2 fluid.layers.group_norm(input, groups) self.assertRaises(TypeError, test_x_type) diff --git a/python/paddle/fluid/tests/unittests/test_gru_rnn_op.py b/python/paddle/fluid/tests/unittests/test_gru_rnn_op.py index 9f18ec9843d7a4..77b88161d3a728 100644 --- a/python/paddle/fluid/tests/unittests/test_gru_rnn_op.py +++ b/python/paddle/fluid/tests/unittests/test_gru_rnn_op.py @@ -92,16 +92,6 @@ def rocm_rnn_get_place(): self._get_places = rocm_rnn_get_place - if self.is_bidirec: - for i in range(0, len(flat_w), 4): - flat_w[i + 1], flat_w[i + 2] = flat_w[i + 2], flat_w[i + 1] - - for i in range(len(flat_w)): - w = np.split(flat_w[i][1], 3, 0) - w = [w[1], w[0], w[2]] - w = np.concatenate(w) - flat_w[i] = (flat_w[i][0], w) - init_h = np.zeros((self.num_layers * self.direction_num, batch_size, self.hidden_size)).astype(self.dtype) diff --git a/python/paddle/fluid/tests/unittests/test_hybrid_parallel_topology.py b/python/paddle/fluid/tests/unittests/test_hybrid_parallel_topology.py index e4c469599d72c0..e8300113ddc42e 100644 --- a/python/paddle/fluid/tests/unittests/test_hybrid_parallel_topology.py +++ b/python/paddle/fluid/tests/unittests/test_hybrid_parallel_topology.py @@ -79,6 +79,99 @@ def test_topology(self): self.assertEqual(topo.get_dim_size("mp"), 2) self.assertEqual(topo.get_dim_size("pp"), 2) + def test_topology_4D(self): + topo = fleet.CommunicateTopology(["dp", "pp", "sharding", "mp"], + [2, 2, 2, 2]) + + # test get_comm_list + dp_comm_list = [[0, 8], [1, 9], [2, 10], [3, 11], [4, 12], [5, 13], + [6, 14], [7, 15]] + mp_comm_list = [[0, 1], [2, 3], [4, 5], [6, 7], [8, 9], [10, 11], + [12, 13], [14, 15]] + pp_comm_list = [[0, 4], [1, 5], [2, 6], [3, 7], [8, 12], [9, 13], + [10, 14], [11, 15]] + sharding_comm_list = [[0, 2], [1, 3], [4, 6], [5, 7], [8, 10], [9, 11], + [12, 14], [13, 15]] + + np.testing.assert_array_equal(dp_comm_list, topo.get_comm_list("dp")) + np.testing.assert_array_equal(mp_comm_list, topo.get_comm_list("mp")) + np.testing.assert_array_equal(pp_comm_list, topo.get_comm_list("pp")) + np.testing.assert_array_equal(sharding_comm_list, + topo.get_comm_list("sharding")) + + # test get_hybrid_group_names + parallel_names = ["dp", "pp", "sharding", "mp"] + np.testing.assert_array_equal(parallel_names, + topo.get_hybrid_group_names()) + + # test get_dims + np.testing.assert_array_equal(2, topo.get_dim("dp")) + np.testing.assert_array_equal(2, topo.get_dim("mp")) + np.testing.assert_array_equal(2, topo.get_dim("pp")) + np.testing.assert_array_equal(2, topo.get_dim("sharding")) + + # test world size + self.assertEqual(topo.world_size(), 16) + + # test get_rank + self.assertEqual(topo.get_rank(dp=0, pp=0, sharding=0, mp=0), 0) + self.assertEqual(topo.get_rank(dp=0, pp=0, sharding=0, mp=1), 1) + self.assertEqual(topo.get_rank(dp=0, pp=0, sharding=1, mp=0), 2) + self.assertEqual(topo.get_rank(dp=0, pp=0, sharding=1, mp=1), 3) + self.assertEqual(topo.get_rank(dp=0, pp=1, sharding=0, mp=0), 4) + self.assertEqual(topo.get_rank(dp=0, pp=1, sharding=0, mp=1), 5) + self.assertEqual(topo.get_rank(dp=0, pp=1, sharding=1, mp=0), 6) + self.assertEqual(topo.get_rank(dp=0, pp=1, sharding=1, mp=1), 7) + self.assertEqual(topo.get_rank(dp=1, pp=0, sharding=0, mp=0), 8) + self.assertEqual(topo.get_rank(dp=1, pp=0, sharding=0, mp=1), 9) + self.assertEqual(topo.get_rank(dp=1, pp=0, sharding=1, mp=0), 10) + self.assertEqual(topo.get_rank(dp=1, pp=0, sharding=1, mp=1), 11) + self.assertEqual(topo.get_rank(dp=1, pp=1, sharding=0, mp=0), 12) + self.assertEqual(topo.get_rank(dp=1, pp=1, sharding=0, mp=1), 13) + self.assertEqual(topo.get_rank(dp=1, pp=1, sharding=1, mp=0), 14) + self.assertEqual(topo.get_rank(dp=1, pp=1, sharding=1, mp=1), 15) + + # test get_coord + self.assertEqual(topo.get_coord(0), topo.coordinate(0, 0, 0, 0)) + self.assertEqual(topo.get_coord(1), topo.coordinate(0, 0, 0, 1)) + self.assertEqual(topo.get_coord(2), topo.coordinate(0, 0, 1, 0)) + self.assertEqual(topo.get_coord(3), topo.coordinate(0, 0, 1, 1)) + self.assertEqual(topo.get_coord(4), topo.coordinate(0, 1, 0, 0)) + self.assertEqual(topo.get_coord(5), topo.coordinate(0, 1, 0, 1)) + self.assertEqual(topo.get_coord(6), topo.coordinate(0, 1, 1, 0)) + self.assertEqual(topo.get_coord(7), topo.coordinate(0, 1, 1, 1)) + self.assertEqual(topo.get_coord(8), topo.coordinate(1, 0, 0, 0)) + self.assertEqual(topo.get_coord(9), topo.coordinate(1, 0, 0, 1)) + self.assertEqual(topo.get_coord(10), topo.coordinate(1, 0, 1, 0)) + self.assertEqual(topo.get_coord(11), topo.coordinate(1, 0, 1, 1)) + self.assertEqual(topo.get_coord(12), topo.coordinate(1, 1, 0, 0)) + self.assertEqual(topo.get_coord(13), topo.coordinate(1, 1, 0, 1)) + self.assertEqual(topo.get_coord(14), topo.coordinate(1, 1, 1, 0)) + self.assertEqual(topo.get_coord(15), topo.coordinate(1, 1, 1, 1)) + + # test get_axis_list + self.assertEqual(topo.get_axis_list("dp", 0), [0, 1, 2, 3, 4, 5, 6, 7]) + self.assertEqual( + topo.get_axis_list("dp", 1), [8, 9, 10, 11, 12, 13, 14, 15]) + self.assertEqual( + topo.get_axis_list("mp", 0), [0, 2, 4, 6, 8, 10, 12, 14]) + self.assertEqual( + topo.get_axis_list("mp", 1), [1, 3, 5, 7, 9, 11, 13, 15]) + self.assertEqual( + topo.get_axis_list("pp", 0), [0, 1, 2, 3, 8, 9, 10, 11]) + self.assertEqual( + topo.get_axis_list("pp", 1), [4, 5, 6, 7, 12, 13, 14, 15]) + self.assertEqual( + topo.get_axis_list("sharding", 0), [0, 1, 4, 5, 8, 9, 12, 13]) + self.assertEqual( + topo.get_axis_list("sharding", 1), [2, 3, 6, 7, 10, 11, 14, 15]) + + # test get_dim_size + self.assertEqual(topo.get_dim_size("dp"), 2) + self.assertEqual(topo.get_dim_size("mp"), 2) + self.assertEqual(topo.get_dim_size("pp"), 2) + self.assertEqual(topo.get_dim_size("sharding"), 2) + if __name__ == '__main__': unittest.main() diff --git a/python/paddle/fluid/tests/unittests/test_imperative_auto_mixed_precision.py b/python/paddle/fluid/tests/unittests/test_imperative_auto_mixed_precision.py index a56797971b5147..e3d2bda8921287 100644 --- a/python/paddle/fluid/tests/unittests/test_imperative_auto_mixed_precision.py +++ b/python/paddle/fluid/tests/unittests/test_imperative_auto_mixed_precision.py @@ -209,6 +209,34 @@ def test_nan_inf(self): self.assertTrue( np.array_equal(param.numpy(), params_init[param.name])) + def test_get_and_set(self): + with fluid.dygraph.guard(): + scaler = paddle.amp.GradScaler( + enable=True, + init_loss_scaling=1024, + incr_ratio=2.0, + decr_ratio=0.5, + incr_every_n_steps=1000, + decr_every_n_nan_or_inf=2, + use_dynamic_loss_scaling=True) + self.assertEqual(scaler.is_enable() == True, True) + self.assertEqual(scaler.get_init_loss_scaling() == 1024, True) + self.assertEqual(scaler.get_incr_ratio() == 2.0, True) + self.assertEqual(scaler.get_decr_ratio() == 0.5, True) + self.assertEqual(scaler.get_incr_every_n_steps() == 1000, True) + self.assertEqual(scaler.get_decr_every_n_nan_or_inf() == 2, True) + self.assertEqual(scaler.is_use_dynamic_loss_scaling() == True, True) + scaler.set_decr_every_n_nan_or_inf(4) + self.assertEqual(scaler.get_decr_every_n_nan_or_inf() == 4, True) + scaler.set_decr_ratio(0.1) + self.assertEqual(scaler.get_decr_ratio() == 0.1, True) + scaler.set_incr_every_n_steps(200) + self.assertEqual(scaler.get_incr_every_n_steps() == 200, True) + scaler.set_incr_ratio(3.0) + self.assertEqual(scaler.get_incr_ratio() == 3.0, True) + scaler.set_init_loss_scaling(100) + self.assertEqual(scaler.get_init_loss_scaling() == 100, True) + def reader_decorator(reader): def __reader__(): diff --git a/python/paddle/fluid/tests/unittests/test_math_op_patch.py b/python/paddle/fluid/tests/unittests/test_math_op_patch.py index b2afda9ed3f254..cef5adbc5d3e36 100644 --- a/python/paddle/fluid/tests/unittests/test_math_op_patch.py +++ b/python/paddle/fluid/tests/unittests/test_math_op_patch.py @@ -271,7 +271,6 @@ def test_astype(self): fetch_list=[b]) self.assertTrue(numpy.allclose(a_np.astype('float32'), b_np)) - @prog_scope() def test_bitwise_and(self): x_np = np.random.randint(-100, 100, [2, 3, 5]).astype("int32") y_np = np.random.randint(-100, 100, [2, 3, 5]).astype("int32") @@ -336,6 +335,28 @@ def test_bitwise_not(self): fetch_list=[z]) self.assertTrue(np.array_equal(out[0], out_np)) + @prog_scope() + def test_ndim(self): + a = paddle.static.data(name="a", shape=[10, 1]) + self.assertEqual(a.dim(), 2) + self.assertEqual(a.ndimension(), 2) + self.assertEqual(a.ndim, 2) + + @prog_scope() + def test_matmul(self): + a = paddle.static.data(name='a', shape=[2, 3], dtype='float32') + b = paddle.static.data(name='b', shape=[3, 5], dtype='float32') + c = a @b # __matmul__ + a_np = numpy.random.uniform(-1, 1, size=[2, 3]).astype('float32') + b_np = numpy.random.uniform(-1, 1, size=[3, 5]).astype('float32') + place = paddle.CPUPlace() + exe = paddle.static.Executor(place) + c_np = exe.run(paddle.static.default_main_program(), + feed={"a": a_np, + "b": b_np}, + fetch_list=[c]) + self.assertTrue(numpy.allclose(a_np @b_np, c_np)) + if __name__ == '__main__': unittest.main() diff --git a/python/paddle/fluid/tests/unittests/test_math_op_patch_var_base.py b/python/paddle/fluid/tests/unittests/test_math_op_patch_var_base.py index 7de6148fe73da2..0afc9ee6253ea6 100644 --- a/python/paddle/fluid/tests/unittests/test_math_op_patch_var_base.py +++ b/python/paddle/fluid/tests/unittests/test_math_op_patch_var_base.py @@ -575,6 +575,13 @@ def test_tensor_patch_method(self): self.assertTrue(inspect.ismethod(a.std)) self.assertTrue(inspect.ismethod(a.numel)) + def test_complex_scalar(self): + a_np = np.random.random(self.shape).astype(self.dtype) + with fluid.dygraph.guard(): + a = fluid.dygraph.to_variable(a_np) + res = 1J * a + self.assertTrue(np.array_equal(res.numpy(), 1J * a_np)) + if __name__ == '__main__': unittest.main() diff --git a/python/paddle/fluid/tests/unittests/test_paddle_save_load.py b/python/paddle/fluid/tests/unittests/test_paddle_save_load.py index 77aa4ae36b3ace..727ac368989fb7 100644 --- a/python/paddle/fluid/tests/unittests/test_paddle_save_load.py +++ b/python/paddle/fluid/tests/unittests/test_paddle_save_load.py @@ -95,6 +95,7 @@ def setUp(self): def test_large_parameters_paddle_save(self): # enable dygraph mode paddle.disable_static() + paddle.set_device("cpu") # create network layer = LayerWithLargeParameters() save_dict = layer.state_dict() @@ -103,11 +104,10 @@ def test_large_parameters_paddle_save(self): "layer.pdparams") protocol = 4 paddle.save(save_dict, path, protocol=protocol) - dict_load = paddle.load(path) + dict_load = paddle.load(path, return_numpy=True) # compare results before and after saving for key, value in save_dict.items(): - self.assertTrue( - np.array_equal(dict_load[key].numpy(), value.numpy())) + self.assertTrue(np.array_equal(dict_load[key], value.numpy())) class TestSaveLoadPickle(unittest.TestCase): diff --git a/python/paddle/fluid/tests/unittests/test_parallel_dygraph_dataparallel.py b/python/paddle/fluid/tests/unittests/test_parallel_dygraph_dataparallel.py index f3cd97ee1ec869..d15e55eb0fa146 100644 --- a/python/paddle/fluid/tests/unittests/test_parallel_dygraph_dataparallel.py +++ b/python/paddle/fluid/tests/unittests/test_parallel_dygraph_dataparallel.py @@ -124,6 +124,8 @@ def run_mnist_2gpu(self, target_file_name): break time.sleep(3) + +class TestDataParallelGradientCheck(TestMultipleGpus): def test_multiple_gpus_dynamic(self): self.run_mnist_2gpu('parallel_dygraph_gradient_check.py') diff --git a/python/paddle/fluid/tests/unittests/test_parallel_dygraph_sharding_parallel.py b/python/paddle/fluid/tests/unittests/test_parallel_dygraph_sharding_parallel.py new file mode 100644 index 00000000000000..b7e8e06029d937 --- /dev/null +++ b/python/paddle/fluid/tests/unittests/test_parallel_dygraph_sharding_parallel.py @@ -0,0 +1,31 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import print_function + +import unittest +import paddle.fluid as fluid + +from test_parallel_dygraph_dataparallel import TestMultipleGpus + + +class TestHybridParallel(TestMultipleGpus): + + # check sharding logic as well as the accuracy with single mode + def test_hybrid_parallel_sharding_logic(self): + self.run_mnist_2gpu('hybrid_parallel_sharding_model.py') + + +if __name__ == "__main__": + unittest.main() diff --git a/python/paddle/fluid/tests/unittests/test_pass_builder.py b/python/paddle/fluid/tests/unittests/test_pass_builder.py index cd463ea0405f56..023ceeaa73acc2 100644 --- a/python/paddle/fluid/tests/unittests/test_pass_builder.py +++ b/python/paddle/fluid/tests/unittests/test_pass_builder.py @@ -23,6 +23,7 @@ import os import sys import math +import tempfile class TestPassBuilder(unittest.TestCase): @@ -98,17 +99,17 @@ def test_parallel_testing_with_new_strategy(self): pass_builder.remove_pass(len(pass_builder.all_passes()) - 1) self.assertEqual(origin_len + 1, len(pass_builder.all_passes())) - current_path = os.path.abspath(os.path.dirname(__file__)) - graph_viz_path = current_path + os.sep + 'tmp' + os.sep + 'test_viz_pass' - viz_pass.set("graph_viz_path", graph_viz_path) - - self.check_network_convergence( - use_cuda=core.is_compiled_with_cuda(), - build_strategy=build_strategy) - try: - os.stat(graph_viz_path) - except os.error: - self.assertFalse(True) + with tempfile.TemporaryDirectory(prefix="dot_path_") as tmpdir: + graph_viz_path = os.path.join(tmpdir, 'test_viz_pass.dot') + viz_pass.set("graph_viz_path", graph_viz_path) + + self.check_network_convergence( + use_cuda=core.is_compiled_with_cuda(), + build_strategy=build_strategy) + try: + os.stat(graph_viz_path) + except os.error: + self.assertFalse(True) if __name__ == '__main__': diff --git a/python/paddle/fluid/tests/unittests/test_rnn_op.py b/python/paddle/fluid/tests/unittests/test_rnn_op.py index 22e07b0bc48c04..763ec3e7038a45 100644 --- a/python/paddle/fluid/tests/unittests/test_rnn_op.py +++ b/python/paddle/fluid/tests/unittests/test_rnn_op.py @@ -95,16 +95,6 @@ def rocm_rnn_get_place(): self._get_places = rocm_rnn_get_place - if self.is_bidirec: - for i in range(0, len(flat_w), 4): - flat_w[i + 1], flat_w[i + 2] = flat_w[i + 2], flat_w[i + 1] - - for i in range(len(flat_w)): - w = np.split(flat_w[i][1], 4, 0) - w = [w[0], w[1], w[3], w[2]] - w = np.concatenate(w) - flat_w[i] = (flat_w[i][0], w) - init_h = np.zeros((self.num_layers * self.direction_num, batch_size, hidden_size)).astype(self.dtype) init_c = np.zeros((self.num_layers * self.direction_num, batch_size, diff --git a/python/paddle/fluid/tests/unittests/test_roll_op.py b/python/paddle/fluid/tests/unittests/test_roll_op.py index b20293adf4c406..99121d2953a14f 100644 --- a/python/paddle/fluid/tests/unittests/test_roll_op.py +++ b/python/paddle/fluid/tests/unittests/test_roll_op.py @@ -63,6 +63,7 @@ def input_data(self): def test_roll_op_api(self): self.input_data() + paddle.enable_static() # case 1: with program_guard(Program(), Program()): x = fluid.layers.data(name='x', shape=[-1, 3]) diff --git a/python/paddle/fluid/tests/unittests/test_share_data_op.py b/python/paddle/fluid/tests/unittests/test_share_data_op.py new file mode 100644 index 00000000000000..1e6f0ef693c3da --- /dev/null +++ b/python/paddle/fluid/tests/unittests/test_share_data_op.py @@ -0,0 +1,87 @@ +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import unittest +import numpy as np +from op_test import OpTest +from paddle.fluid import core +from paddle.fluid.op import Operator + + +class TestShareDataOp(OpTest): + def setUp(self): + self.op_type = "share_data" + input = np.random.rand(2, 3, 5).astype("float32") + self.inputs = {'X': input} + self.outputs = {'Out': input} + + def test_check_output(self): + self.check_output() + + +class TestShareDataOpOnDifferentPlaces(unittest.TestCase): + def get_places(self): + places = [core.CPUPlace()] + if core.is_compiled_with_cuda(): + places.append(core.CUDAPlace(0)) + return places + + def check_with_tensor(self, place): + scope = core.Scope() + np_array = np.random.rand(2, 3, 5).astype("float32") + + # initialize input and output variable + x = scope.var('X').get_tensor() + x.set(np_array, place) + out = scope.var("Out").get_tensor() + + op = Operator("share_data", X="X", Out="Out") + op.run(scope, place) + self.assertTrue(np.allclose(np_array, out)) + + def check_with_selected_rows(self, place): + scope = core.Scope() + x_rows = [0, 1, 5, 4, 19] + x_height = 20 + row_numel = 2 + np_array = np.ones((len(x_rows), row_numel)).astype("float32") + + # initialize input variable + x = scope.var('X').get_selected_rows() + x.set_rows(x_rows) + x.set_height(x_height) + x_tensor = x.get_tensor() + x_tensor.set(np_array, place) + + # initialize the Out variable + out = scope.var("Out").get_selected_rows() + out_tensor = out.get_tensor() + + op = Operator("share_data", X="X", Out="Out") + op.run(scope, place) + + out_height = out.height() + out_rows = out.rows() + self.assertTrue(np.allclose(np_array, out_tensor)) + self.assertEqual(x_height, out_height) + self.assertEqual(x_rows, out_rows) + + def test_check_output(self): + for place in self.get_places(): + self.check_with_selected_rows(place) + self.check_with_tensor(place) + + +if __name__ == '__main__': + unittest.main() diff --git a/python/paddle/fluid/tests/unittests/test_simple_rnn_op.py b/python/paddle/fluid/tests/unittests/test_simple_rnn_op.py index 63688cbce24195..d7e24b6308e5d3 100644 --- a/python/paddle/fluid/tests/unittests/test_simple_rnn_op.py +++ b/python/paddle/fluid/tests/unittests/test_simple_rnn_op.py @@ -19,6 +19,7 @@ from op_test import OpTest import paddle import paddle.fluid as fluid +import paddle.fluid.core as core import paddle.fluid.layers as layers import random import sys @@ -44,8 +45,10 @@ def get_weight_names(self): def setUp(self): self.op_type = "rnn" - self.dtype = np.float64 - self.sequence_length = np.array([12, 11, 10, 9, 8], dtype=np.int32) + self.dtype = "float32" if core.is_compiled_with_rocm() else "float64" + self.sequence_length = None if core.is_compiled_with_rocm( + ) else np.array( + [12, 11, 10, 9, 8], dtype=np.int32) self.num_layers = 1 self.is_bidirec = False self.is_test = False @@ -76,7 +79,8 @@ def setUp(self): time_major=True, direction=direction, dropout=self.dropout, - nonlinearity=self.mode) + nonlinearity=self.mode, + dtype=self.dtype) flat_w = get_params_for_net(rnn1) diff --git a/python/paddle/fluid/tests/unittests/test_var_base.py b/python/paddle/fluid/tests/unittests/test_var_base.py index 98bc79fc7cb4ee..644e46f1081589 100644 --- a/python/paddle/fluid/tests/unittests/test_var_base.py +++ b/python/paddle/fluid/tests/unittests/test_var_base.py @@ -246,6 +246,9 @@ def _test_place(place): _test_place("gpu_pinned") _test_place(core.CUDAPlace(0)) _test_place("gpu:0") + if core.is_compiled_with_npu(): + _test_place(core.NPUPlace(0)) + _test_place("npu:0") def test_to_tensor_not_change_input_stop_gradient(self): with paddle.fluid.dygraph.guard(core.CPUPlace()): diff --git a/python/paddle/fluid/tests/unittests/test_variable.py b/python/paddle/fluid/tests/unittests/test_variable.py index c1956545f55ad1..a998d58fdbc607 100644 --- a/python/paddle/fluid/tests/unittests/test_variable.py +++ b/python/paddle/fluid/tests/unittests/test_variable.py @@ -305,7 +305,6 @@ def test_fake_interface_only_api(self): b = default_main_program().current_block() var = b.create_var(dtype="float64", lod_level=0) with fluid.dygraph.guard(): - self.assertRaises(AssertionError, var.detach) self.assertRaises(AssertionError, var.numpy) self.assertRaises(AssertionError, var.backward) self.assertRaises(AssertionError, var.gradient) @@ -345,6 +344,60 @@ def _test(): self.assertRaises(Exception, _test) + def test_size(self): + prog = paddle.static.Program() + with paddle.static.program_guard(prog): + x = paddle.assign(np.random.rand(2, 3, 4).astype("float32")) + exe = paddle.static.Executor(fluid.CPUPlace()) + exe.run(paddle.static.default_startup_program()) + + output = exe.run(prog, fetch_list=[x.size()]) + self.assertEqual(output[0], [24]) + + def test_detach(self): + b = default_main_program().current_block() + x = b.create_var(shape=[2, 3, 5], dtype="float64", lod_level=0) + detach_x = x.detach() + self.assertEqual(x.persistable, detach_x.persistable) + self.assertEqual(x.shape, detach_x.shape) + self.assertEqual(x.dtype, detach_x.dtype) + self.assertEqual(x.type, detach_x.type) + self.assertTrue(detach_x.stop_gradient) + + xx = b.create_var(name='xx', type=core.VarDesc.VarType.STEP_SCOPES) + self.assertRaises(AssertionError, xx.detach) + + startup = paddle.static.Program() + main = paddle.static.Program() + scope = fluid.core.Scope() + with paddle.static.scope_guard(scope): + with paddle.static.program_guard(main, startup): + x = paddle.static.data( + name='x', shape=[3, 2, 1], dtype='float32') + x.persistable = True + feed_data = np.ones(shape=[3, 2, 1], dtype=np.float32) + detach_x = x.detach() + exe = paddle.static.Executor(paddle.CPUPlace()) + exe.run(startup) + result = exe.run(main, + feed={'x': feed_data}, + fetch_list=[x, detach_x]) + self.assertTrue((result[1] == feed_data).all()) + self.assertTrue((result[0] == result[1]).all()) + + modified_value = np.zeros(shape=[3, 2, 1], dtype=np.float32) + detach_x.set_value(modified_value, scope) + result = exe.run(main, fetch_list=[x, detach_x]) + self.assertTrue((result[1] == modified_value).all()) + self.assertTrue((result[0] == result[1]).all()) + + modified_value = np.random.uniform( + -1, 1, size=[3, 2, 1]).astype('float32') + x.set_value(modified_value, scope) + result = exe.run(main, fetch_list=[x, detach_x]) + self.assertTrue((result[1] == modified_value).all()) + self.assertTrue((result[0] == result[1]).all()) + class TestVariableSlice(unittest.TestCase): def _test_item_none(self, place): diff --git a/python/paddle/fluid/tests/unittests/xpu/test_batch_norm_op_xpu.py b/python/paddle/fluid/tests/unittests/xpu/test_batch_norm_op_xpu.py index 1cdec863b2ac3d..8132a78f696756 100644 --- a/python/paddle/fluid/tests/unittests/xpu/test_batch_norm_op_xpu.py +++ b/python/paddle/fluid/tests/unittests/xpu/test_batch_norm_op_xpu.py @@ -141,7 +141,7 @@ def setUp(self): else: raise ValueError( "Unsupported data layout! Only NCHW and NHWC is supported, but received " - + data_layout) + + self.data_layout) np.random.seed(1024) self.x_np = np.random.random_sample(self.shape).astype(self.dtype) self.scale_np = np.random.random_sample( diff --git a/python/paddle/fluid/trainer_desc.py b/python/paddle/fluid/trainer_desc.py index 92a900e6c37158..4eca3a494e25a4 100644 --- a/python/paddle/fluid/trainer_desc.py +++ b/python/paddle/fluid/trainer_desc.py @@ -17,7 +17,7 @@ import os __all__ = [ 'TrainerDesc', 'MultiTrainer', 'DistMultiTrainer', 'PipelineTrainer', - 'HeterXpuTrainer', 'HeterBoxTrainer' + 'HeterXpuTrainer' ] @@ -346,30 +346,6 @@ def _gen_trainer_desc(self): self._device_worker._gen_worker_desc(self.proto_desc) -class HeterBoxTrainer(TrainerDesc): - """ - Implement of HeterBoxTrainer. - It's for Distributed training. - """ - - def __init__(self): - super(HeterBoxTrainer, self).__init__() - pass - - def _set_program(self, program): - super(HeterBoxTrainer, self)._set_program(program) - self._program = program - - def _gen_trainer_desc(self): - super(HeterBoxTrainer, self)._gen_trainer_desc() - self.proto_desc.class_name = "HeterBoxTrainer" - if self._program == None: - raise RuntimeError("None Program") - self._device_worker._set_infer(self._infer) - self._device_worker._set_program(self._program) - self._device_worker._gen_worker_desc(self.proto_desc) - - class PSGPUTrainer(TrainerDesc): """ Implement of PSGPUTrainer. diff --git a/python/paddle/fluid/trainer_factory.py b/python/paddle/fluid/trainer_factory.py index 95379a34c22144..7912ffca84ba41 100644 --- a/python/paddle/fluid/trainer_factory.py +++ b/python/paddle/fluid/trainer_factory.py @@ -22,7 +22,7 @@ local_logger = get_logger( __name__, logging.INFO, fmt='%(asctime)s-%(levelname)s: %(message)s') -from .trainer_desc import MultiTrainer, DistMultiTrainer, PipelineTrainer, HeterXpuTrainer, HeterBoxTrainer, PSGPUTrainer +from .trainer_desc import MultiTrainer, DistMultiTrainer, PipelineTrainer, HeterXpuTrainer, PSGPUTrainer from .device_worker import Hogwild, DownpourSGD, Section, DownpourSGDOPT from .framework import Variable from multiprocessing import Process, Manager diff --git a/python/paddle/hapi/callbacks.py b/python/paddle/hapi/callbacks.py index 5f1f3834382871..cae3bbfd49015a 100644 --- a/python/paddle/hapi/callbacks.py +++ b/python/paddle/hapi/callbacks.py @@ -126,7 +126,8 @@ def on_batch_end(self, mode, step=None, logs=None): class Callback(object): """ - Base class used to build new callbacks. + Base class used to build new callbacks. And new callbacks could also + terminate training by setting `model.stop_training=True`. Examples: @@ -685,7 +686,8 @@ def on_train_batch_end(self, step, logs=None): class EarlyStopping(Callback): - """Stop training when the given monitor stopped improving during evaluation. + """Stop training when the given monitor stopped improving during evaluation + by setting `model.stop_training=True`. Args: monitor(str): Quantity to be monitored. Default: 'loss'. mode(str|None): Mode should be one of 'auto', 'min' or 'max'. In 'min' diff --git a/python/paddle/hapi/model.py b/python/paddle/hapi/model.py index c9b6c0098e2823..25081a64e24de3 100644 --- a/python/paddle/hapi/model.py +++ b/python/paddle/hapi/model.py @@ -25,23 +25,18 @@ import time import socket import contextlib -from collections import Iterable import paddle from paddle import fluid from paddle.fluid import core from paddle.fluid.framework import in_dygraph_mode from paddle.fluid.framework import Variable -from paddle.fluid.framework import ParamBase -from paddle.fluid.framework import _current_expected_place from paddle.fluid.framework import _get_paddle_place from paddle.fluid.framework import _current_expected_place as _get_device from paddle.fluid.executor import global_scope from paddle.fluid.io import is_belong_to_optimizer from paddle.fluid.dygraph.base import to_variable from paddle.fluid.dygraph.parallel import ParallelEnv -from paddle.fluid.dygraph.dygraph_to_static.program_translator import ProgramTranslator -from paddle.fluid.dygraph.dygraph_to_static.program_translator import FunctionSpec from paddle.fluid.dygraph.io import INFER_MODEL_SUFFIX from paddle.fluid.dygraph.io import INFER_PARAMS_SUFFIX from paddle.fluid.layers.utils import flatten @@ -50,9 +45,6 @@ from paddle.io import DataLoader from paddle.io import Dataset from paddle.io import DistributedBatchSampler -from paddle.fluid.executor import scope_guard -from paddle.fluid.executor import Executor -from paddle.fluid.dygraph.layers import Layer from paddle.metric import Metric from paddle.static import InputSpec as Input import paddle.distributed as dist @@ -298,10 +290,11 @@ def mode(self): def mode(self, value): self.model.mode = value - def train_batch(self, inputs, labels=None): + def train_batch(self, inputs, labels=None, update=True): assert self.model._optimizer, \ "model not ready, please call `model.prepare()` first" self.mode = 'train' + assert update is True, "Does not support `update == False` in static mode by now." return self._run(inputs, labels) def eval_batch(self, inputs, labels=None): @@ -701,7 +694,7 @@ def mode(self, value): self.model.mode = value # TODO multi device in dygraph mode not implemented at present time - def train_batch(self, inputs, labels=None): + def train_batch(self, inputs, labels=None, update=True): assert self.model._optimizer, \ "model not ready, please call `model.prepare()` first" self.model.network.train() @@ -729,12 +722,14 @@ def train_batch(self, inputs, labels=None): if self._amp_level != "O0": scaled = scaler.scale(final_loss) scaled.backward() - scaler.minimize(self.model._optimizer, scaled) - self.model.network.clear_gradients() + if update: + scaler.minimize(self.model._optimizer, scaled) + self.model.network.clear_gradients() else: final_loss.backward() - self.model._optimizer.minimize(final_loss) - self.model.network.clear_gradients() + if update: + self.model._optimizer.minimize(final_loss) + self.model.network.clear_gradients() metrics = [] for metric in self.model._metrics: @@ -1017,9 +1012,10 @@ def __init__(self, network, inputs=None, labels=None): else: self._adapter = StaticGraphAdapter(self) - def train_batch(self, inputs, labels=None): + def train_batch(self, inputs, labels=None, update=True): """ - Run one training step on a batch of data. + Run one training step on one batch of data. And using `update` indicates + whether optimizer update gradients computing by this batch. Args: inputs (numpy.ndarray|Tensor|list): Batch of input data. It could @@ -1029,6 +1025,8 @@ def train_batch(self, inputs, labels=None): a numpy array or paddle.Tensor, or a list of arrays or tensors (in case the model has multiple labels). If has no labels, set None. Default is None. + update (bool): Whether update parameters after loss.backward() computing. + Using it to accumulate gradients. Default is True. Returns: A list of scalar training loss if the model has no metrics, @@ -1062,7 +1060,7 @@ def train_batch(self, inputs, labels=None): loss = model.train_batch([data], [label]) print(loss) """ - loss = self._adapter.train_batch(inputs, labels) + loss = self._adapter.train_batch(inputs, labels, update) if fluid.in_dygraph_mode() and self._input_info is None: self._update_inputs() return loss @@ -1536,7 +1534,8 @@ def fit( drop_last=False, shuffle=True, num_workers=0, - callbacks=None, ): + callbacks=None, + accumulate_grad_batches=1, ): """ Trains the model for a fixed number of epochs. If `eval_data` is set, evaluation will be done at the end of each epoch. @@ -1579,7 +1578,10 @@ def fit( callbacks (Callback|None): A list of `Callback` instances to apply during training. If None, `ProgBarLogger` and `ModelCheckpoint` are automatically inserted. Default: None. - + accumulate_grad_batches (int): The number of batches to accumulate gradident + during training process before optimizer updates. It can mimic large batch + size. Default: 1. + Returns: None @@ -1700,6 +1702,8 @@ def fit( do_eval = eval_loader is not None self._test_dataloader = eval_loader + self._accumulate = accumulate_grad_batches + steps = self._len_data_loader(train_loader) cbks = config_callbacks( callbacks, @@ -2004,7 +2008,12 @@ def _save_inference_model(self, path): model_filename=model_filename, params_filename=params_filename) - def _run_one_epoch(self, data_loader, callbacks, mode, logs={}): + def _run_one_epoch( + self, + data_loader, + callbacks, + mode, + logs={}, ): outputs = [] for step, data in enumerate(data_loader): # data might come from different types of data_loader and have @@ -2028,8 +2037,14 @@ def _run_one_epoch(self, data_loader, callbacks, mode, logs={}): callbacks.on_batch_begin(mode, step, logs) if mode != 'predict': - outs = getattr(self, mode + '_batch')(data[:len(self._inputs)], - data[len(self._inputs):]) + + _inputs = [data[:len(self._inputs)], data[len(self._inputs):]] + if mode == 'train': + _inputs.append((step + 1) % self._accumulate == 0 or + step + 1 == len(data_loader)) + + outs = getattr(self, mode + '_batch')(*_inputs) + if self._metrics and self._loss: metrics = [[l[0] for l in outs[0]]] elif self._loss: diff --git a/python/paddle/nn/layer/container.py b/python/paddle/nn/layer/container.py index ad41535f44ad6a..48697aa8f50909 100644 --- a/python/paddle/nn/layer/container.py +++ b/python/paddle/nn/layer/container.py @@ -14,7 +14,7 @@ from collections import OrderedDict from ...fluid.dygraph.layers import Layer -from six.moves import collections_abc +from collections.abc import Iterable, Mapping __all__ = [] @@ -276,12 +276,11 @@ def update(self, sublayers): """ assert isinstance( - sublayers, collections_abc.Iterable + sublayers, Iterable ), "The type of sublayers is not iterable of key/value pairs, the type of sublayers is " + type( sublayers).__name__ - if isinstance(sublayers, - (OrderedDict, LayerDict, collections_abc.Mapping)): + if isinstance(sublayers, (OrderedDict, LayerDict, Mapping)): for key, layer in sublayers.items(): self.add_sublayer(key, layer) else: diff --git a/python/paddle/nn/layer/transformer.py b/python/paddle/nn/layer/transformer.py index 891177532a4389..5aba8ae85ad1b3 100644 --- a/python/paddle/nn/layer/transformer.py +++ b/python/paddle/nn/layer/transformer.py @@ -161,6 +161,12 @@ def __init__(self, weight_attr=None, bias_attr=None): super(MultiHeadAttention, self).__init__() + + assert embed_dim > 0, ("Expected embed_dim to be greater than 0, " + "but recieved {}".format(embed_dim)) + assert num_heads > 0, ("Expected num_heads to be greater than 0, " + "but recieved {}".format(num_heads)) + self.embed_dim = embed_dim self.kdim = kdim if kdim is not None else embed_dim self.vdim = vdim if vdim is not None else embed_dim @@ -501,6 +507,15 @@ def __init__(self, self._config.pop("__class__", None) # py3 super(TransformerEncoderLayer, self).__init__() + + assert d_model > 0, ("Expected d_model to be greater than 0, " + "but recieved {}".format(d_model)) + assert nhead > 0, ("Expected nhead to be greater than 0, " + "but recieved {}".format(nhead)) + assert dim_feedforward > 0, ( + "Expected dim_feedforward to be greater than 0, " + "but recieved {}".format(dim_feedforward)) + attn_dropout = dropout if attn_dropout is None else attn_dropout act_dropout = dropout if act_dropout is None else act_dropout self.normalize_before = normalize_before @@ -797,6 +812,15 @@ def __init__(self, self._config.pop("__class__", None) # py3 super(TransformerDecoderLayer, self).__init__() + + assert d_model > 0, ("Expected d_model to be greater than 0, " + "but recieved {}".format(d_model)) + assert nhead > 0, ("Expected nhead to be greater than 0, " + "but recieved {}".format(nhead)) + assert dim_feedforward > 0, ( + "Expected dim_feedforward to be greater than 0, " + "but recieved {}".format(dim_feedforward)) + attn_dropout = dropout if attn_dropout is None else attn_dropout act_dropout = dropout if act_dropout is None else act_dropout self.normalize_before = normalize_before @@ -1196,6 +1220,14 @@ def __init__(self, custom_decoder=None): super(Transformer, self).__init__() + assert d_model > 0, ("Expected d_model to be greater than 0, " + "but recieved {}".format(d_model)) + assert nhead > 0, ("Expected nhead to be greater than 0, " + "but recieved {}".format(nhead)) + assert dim_feedforward > 0, ( + "Expected dim_feedforward to be greater than 0, " + "but recieved {}".format(dim_feedforward)) + if isinstance(bias_attr, (list, tuple)): if len(bias_attr) == 1: encoder_bias_attr = [bias_attr[0]] * 2 diff --git a/python/paddle/nn/quant/__init__.py b/python/paddle/nn/quant/__init__.py index c7f9a5073def83..8973761ab69443 100644 --- a/python/paddle/nn/quant/__init__.py +++ b/python/paddle/nn/quant/__init__.py @@ -21,5 +21,6 @@ from .functional_layers import transpose # noqa: F401 from .functional_layers import concat # noqa: F401 from .functional_layers import flatten # noqa: F401 +from .quant_layers import QuantStub # noqa: F401 __all__ = [] diff --git a/python/paddle/fluid/contrib/slim/quantization/imperative/quant_nn.py b/python/paddle/nn/quant/quant_layers.py similarity index 95% rename from python/paddle/fluid/contrib/slim/quantization/imperative/quant_nn.py rename to python/paddle/nn/quant/quant_layers.py index fd1f7f423ff8f4..c069b3147115e6 100644 --- a/python/paddle/fluid/contrib/slim/quantization/imperative/quant_nn.py +++ b/python/paddle/nn/quant/quant_layers.py @@ -26,21 +26,103 @@ from paddle.fluid.log_helper import get_logger __all__ = [ - 'FakeQuantMovingAverageAbsMax', 'FakeQuantAbsMax', + 'FakeQuantMovingAverageAbsMax', 'FakeQuantChannelWiseAbsMax', 'QuantizedConv2D', 'QuantizedLinear', - 'QuantizedNoweightLayer', 'MovingAverageAbsMaxScale', 'MAOutputScaleLayer', 'FakeQuantMAOutputScaleLayer', + 'QuantStub', ] _logger = get_logger( __name__, logging.INFO, fmt='%(asctime)s-%(levelname)s: %(message)s') +class FakeQuantAbsMax(layers.Layer): + r""" + FakeQuantAbsMax layer does the abs_max quant and then dequant. + Its computational formula is described as below: + + :math:`scale = max(abs(X))` + :math:`range = 2^{bit\_length - 1} - 1` + :math:`Out = round(X / scale * range) * scale / range` + """ + + def __init__(self, + name=None, + quant_bits=8, + dtype='float32', + quant_on_weight=False): + super(FakeQuantAbsMax, self).__init__() + self._quant_bits = quant_bits + self._name = name + scale_prefix = "{}.scale".format( + name) if name else 'quant_dequant.scale' + self._scale_name = unique_name.generate(scale_prefix) + if quant_on_weight: + scale_attr = ParamAttr( + name=self._scale_name, + initializer=Constant(0.0), + trainable=False) + self._scale = self.create_parameter( + shape=[1], attr=scale_attr, dtype=self._dtype) + self._scale.stop_gradient = True + else: + self._scale = None + + def forward(self, input): + if in_dygraph_mode(): + attrs = ('bit_length', self._quant_bits) + quant_out = _varbase_creator( + type=input.type, + name="{}.quantized.dequantized".format(input.name), + shape=input.shape, + dtype=input.dtype, + persistable=False) + out_scale = self._scale + if not out_scale: + out_scale = _varbase_creator( + type=core.VarDesc.VarType.LOD_TENSOR, + name=self._scale_name, + shape=[1], + dtype=self._dtype, + persistable=False) + out_scale.stop_gradient = True + out, _, = core.ops.fake_quantize_dequantize_abs_max( + input, quant_out, out_scale, *attrs) + return out + + check_variable_and_dtype(input, 'input', ['float32'], "FakeQuantAbsMax") + attrs = {'bit_length': self._quant_bits} + inputs = {"X": [input]} + quant_out = self._helper.create_variable( + name="{}.quantized.dequantized".format(input.name), + dtype=input.dtype, + type=core.VarDesc.VarType.LOD_TENSOR, + persistable=False, + stop_gradient=False) + out_scale = self._scale + if not out_scale: + out_scale = self._helper.create_variable( + name=self._scale_name, + dtype=self._dtype, + type=core.VarDesc.VarType.LOD_TENSOR, + persistable=False, + stop_gradient=True) + outputs = {"Out": [quant_out], "OutScale": [out_scale]} + + self._helper.append_op( + type="fake_quantize_dequantize_abs_max", + inputs=inputs, + outputs=outputs, + attrs=attrs) + + return quant_out + + class FakeQuantMovingAverageAbsMax(layers.Layer): r""" FakeQuantMovingAverageAbsMax layer does the moving_average_abs_max quant and then dequant. @@ -64,7 +146,7 @@ def __init__(self, name) if name else 'quant_dequant.scale' scale_attr = ParamAttr( name=unique_name.generate(scale_prefix), - initializer=Constant(0.001), + initializer=Constant(0.), trainable=False) self._scale = self.create_parameter( shape=[1], attr=scale_attr, dtype=dtype) @@ -74,7 +156,7 @@ def __init__(self, name) if name else 'quant_dequant.state' state_attr = ParamAttr( name=unique_name.generate(state_prefix), - initializer=Constant(1), + initializer=Constant(0), trainable=False) self._state = self.create_parameter( shape=[1], attr=state_attr, dtype=dtype) @@ -84,7 +166,7 @@ def __init__(self, name) if name else 'quant_dequant.accum' accum_attr = ParamAttr( name=unique_name.generate(accum_prefix), - initializer=Constant(1), + initializer=Constant(0), trainable=False) self._accum = self.create_parameter( shape=[1], attr=accum_attr, dtype=dtype) @@ -139,24 +221,21 @@ def forward(self, input): return quant_out -class FakeQuantAbsMax(layers.Layer): - r""" - FakeQuantAbsMax layer does the abs_max quant and then dequant. - Its computational formula is described as below: - - :math:`scale = max(abs(X))` - :math:`range = 2^{bit\_length - 1} - 1` - :math:`Out = round(X / scale * range) * scale / range` - """ - +class FakeQuantChannelWiseAbsMax(layers.Layer): def __init__(self, name=None, + channel_num=None, quant_bits=8, + quant_axis=0, dtype='float32', quant_on_weight=False): - super(FakeQuantAbsMax, self).__init__() + assert quant_on_weight == True, "Channel_wise only can be used on weight quantization." + super(FakeQuantChannelWiseAbsMax, self).__init__() self._quant_bits = quant_bits + self._quant_axis = quant_axis + self._dtype = dtype self._name = name + self._channel_num = channel_num scale_prefix = "{}.scale".format( name) if name else 'quant_dequant.scale' self._scale_name = unique_name.generate(scale_prefix) @@ -166,35 +245,39 @@ def __init__(self, initializer=Constant(0.0), trainable=False) self._scale = self.create_parameter( - shape=[1], attr=scale_attr, dtype=self._dtype) + shape=[self._channel_num], attr=scale_attr, dtype=self._dtype) self._scale.stop_gradient = True else: self._scale = None def forward(self, input): if in_dygraph_mode(): - attrs = ('bit_length', self._quant_bits) + attrs = ('bit_length', self._quant_bits, 'quant_axis', + self._quant_axis) quant_out = _varbase_creator( type=input.type, name="{}.quantized.dequantized".format(input.name), shape=input.shape, dtype=input.dtype, persistable=False) + out_scale = self._scale - if not out_scale: + if out_scale is None: out_scale = _varbase_creator( type=core.VarDesc.VarType.LOD_TENSOR, name=self._scale_name, - shape=[1], + shape=[self._channel_num], dtype=self._dtype, persistable=False) out_scale.stop_gradient = True - out, _, = core.ops.fake_quantize_dequantize_abs_max( + + out, _, = core.ops.fake_channel_wise_quantize_dequantize_abs_max( input, quant_out, out_scale, *attrs) return out - check_variable_and_dtype(input, 'input', ['float32'], "FakeQuantAbsMax") - attrs = {'bit_length': self._quant_bits} + check_variable_and_dtype(input, 'input', ['float32'], + "FakeQuantChannelWiseAbsMax") + attrs = {'bit_length': self._quant_bits, 'quant_axis': self._quant_axis} inputs = {"X": [input]} quant_out = self._helper.create_variable( name="{}.quantized.dequantized".format(input.name), @@ -213,7 +296,7 @@ def forward(self, input): outputs = {"Out": [quant_out], "OutScale": [out_scale]} self._helper.append_op( - type="fake_quantize_dequantize_abs_max", + type="fake_channel_wise_quantize_dequantize_abs_max", inputs=inputs, outputs=outputs, attrs=attrs) @@ -221,82 +304,83 @@ def forward(self, input): return quant_out -class FakeQuantChannelWiseAbsMax(layers.Layer): - def __init__(self, - name=None, - channel_num=None, - quant_bits=8, - quant_axis=0, - dtype='float32', - quant_on_weight=False): - assert quant_on_weight == True, "Channel_wise only can be used on weight quantization." - super(FakeQuantChannelWiseAbsMax, self).__init__() - self._quant_bits = quant_bits - self._quant_axis = quant_axis - self._dtype = dtype - self._name = name - self._channel_num = channel_num - scale_prefix = "{}.scale".format( - name) if name else 'quant_dequant.scale' - self._scale_name = unique_name.generate(scale_prefix) - if quant_on_weight: - scale_attr = ParamAttr( - name=self._scale_name, - initializer=Constant(0.0), - trainable=False) - self._scale = self.create_parameter( - shape=[self._channel_num], attr=scale_attr, dtype=self._dtype) - self._scale.stop_gradient = True - else: - self._scale = None +class MovingAverageAbsMaxScale(layers.Layer): + def __init__(self, name=None, moving_rate=0.9, dtype='float32'): + r""" + MovingAverageMaxScale layer is used to calculating the output quantization + scale of Layer. Its computational formula is described as below: + + :math:`scale = (moving\_rate*accum+max(abs(x)))/(moving\_rate*state+1)` + :math:`Out = X` + """ + super(MovingAverageAbsMaxScale, self).__init__() + self._moving_rate = moving_rate + + scale_prefix = '{}.scale'.format(name) if name else 'outscale.scale' + scale_name = unique_name.generate(scale_prefix) + scale_attr = ParamAttr( + name=scale_name, initializer=Constant(0), trainable=False) + self._scale = self.create_parameter( + shape=[1], attr=scale_attr, dtype=dtype) + self._scale.stop_gradient = True + + state_prefix = "{}.state".format(name) if name else 'outscale.state' + state_attr = ParamAttr( + name=unique_name.generate(state_prefix), + initializer=Constant(0), + trainable=False) + self._state = self.create_parameter( + shape=[1], attr=state_attr, dtype=dtype) + self._state.stop_gradient = True + + accum_prefix = "{}.accum".format(name) if name else 'outscale.accum' + accum_attr = ParamAttr( + name=unique_name.generate(accum_prefix), + initializer=Constant(0), + trainable=False) + self._accum = self.create_parameter( + shape=[1], attr=accum_attr, dtype=dtype) + self._accum.stop_gradient = True def forward(self, input): if in_dygraph_mode(): - attrs = ('bit_length', self._quant_bits, 'quant_axis', - self._quant_axis) + attrs = ('moving_rate', self._moving_rate, 'is_test', + not self.training) + state = self._state if self.training else None + accum = self._accum if self.training else None quant_out = _varbase_creator( type=input.type, - name="{}.quantized.dequantized".format(input.name), + name="{}.tmp".format(input.name), shape=input.shape, dtype=input.dtype, persistable=False) - out_scale = self._scale - if out_scale is None: - out_scale = _varbase_creator( - type=core.VarDesc.VarType.LOD_TENSOR, - name=self._scale_name, - shape=[self._channel_num], - dtype=self._dtype, - persistable=False) - out_scale.stop_gradient = True - - out, _, = core.ops.fake_channel_wise_quantize_dequantize_abs_max( - input, quant_out, out_scale, *attrs) + out, _, _, _ = core.ops.moving_average_abs_max_scale( + input, accum, state, quant_out, self._scale, state, accum, + *attrs) return out - check_variable_and_dtype(input, 'input', ['float32'], - "FakeQuantChannelWiseAbsMax") - attrs = {'bit_length': self._quant_bits, 'quant_axis': self._quant_axis} + check_variable_and_dtype(input, 'input', ['float32', 'float64'], + 'MovingAverageAbsMaxScale') + + attrs = {'moving_rate': self._moving_rate, 'is_test': not self.training} inputs = {"X": [input]} quant_out = self._helper.create_variable( - name="{}.quantized.dequantized".format(input.name), + name="{}.tmp".format(input.name), dtype=input.dtype, type=core.VarDesc.VarType.LOD_TENSOR, persistable=False, stop_gradient=False) - out_scale = self._scale - if not out_scale: - out_scale = self._helper.create_variable( - name=self._scale_name, - dtype=self._dtype, - type=core.VarDesc.VarType.LOD_TENSOR, - persistable=False, - stop_gradient=True) - outputs = {"Out": [quant_out], "OutScale": [out_scale]} + outputs = {"Out": [quant_out], "OutScale": [self._scale]} + + if self.training: + inputs['InState'] = [self._state] + inputs['InAccum'] = [self._accum] + outputs['OutState'] = [self._state] + outputs['OutAccum'] = [self._accum] self._helper.append_op( - type="fake_channel_wise_quantize_dequantize_abs_max", + type="moving_average_abs_max_scale", inputs=inputs, outputs=outputs, attrs=attrs) @@ -304,31 +388,7 @@ def forward(self, input): return quant_out -def _get_fake_quant_type(quant_type, **kwargs): - call_args = { - "name": kwargs.get("name", None), - "quant_bits": kwargs.get("quant_bits", 8), - "dtype": kwargs.get("dtype", "float32") - } - - if quant_type == 'abs_max': - call_args["quant_on_weight"] = kwargs.get("quant_on_weight", False) - elif quant_type == 'moving_average_abs_max': - call_args["moving_rate"] = kwargs.get("moving_rate", 0.9) - elif quant_type == 'channel_wise_abs_max': - call_args["quant_on_weight"] = kwargs.get("quant_on_weight", False) - call_args["channel_num"] = kwargs.get("channel_num", None) - call_args["quant_axis"] = kwargs.get("quant_axis", 0) - assert call_args["channel_num"] is not None, ( - "You need to input channel_num" - "when you use channel_wise_abs_max strategy.") - fake_quant_map = { - 'abs_max': FakeQuantAbsMax, - 'moving_average_abs_max': FakeQuantMovingAverageAbsMax, - 'channel_wise_abs_max': FakeQuantChannelWiseAbsMax - } - - return fake_quant_map[quant_type](**call_args) +QuantStub = MovingAverageAbsMaxScale class QuantizedConv2D(layers.Layer): @@ -489,117 +549,10 @@ def forward(self, input): return out -class QuantizedNoweightLayer(layers.Layer): - def __init__(self, - layer, - weight_bits=8, - activation_bits=8, - moving_rate=0.9, - *args, - **kwargs): - - super(QuantizedNoweightLayer, self).__init__() - self._layer = layer - self._fake_quant_input = _get_fake_quant_type( - 'moving_average_abs_max', - name=layer.full_name(), - moving_rate=moving_rate, - quant_bits=activation_bits, - dtype=self._dtype, - quant_on_weight=False) - - def forward(self, input): - return self._layer.forward(self._fake_quant_input(input)) - - -class MovingAverageAbsMaxScale(layers.Layer): - def __init__(self, name=None, moving_rate=0.9, dtype='float32'): - r""" - MovingAverageMaxScale layer is used to calculating the output quantization - scale of Layer. Its computational formula is described as below: - - :math:`scale = (moving\_rate*accum+max(abs(x)))/(moving\_rate*state+1)` - :math:`Out = X` - """ - super(MovingAverageAbsMaxScale, self).__init__() - self._moving_rate = moving_rate - - scale_prefix = '{}.scale'.format(name) if name else 'outscale.scale' - scale_name = unique_name.generate(scale_prefix) - scale_attr = ParamAttr( - name=scale_name, initializer=Constant(1), trainable=False) - self._scale = self.create_parameter( - shape=[1], attr=scale_attr, dtype=dtype) - self._scale.stop_gradient = True - - state_prefix = "{}.state".format(name) if name else 'outscale.state' - state_attr = ParamAttr( - name=unique_name.generate(state_prefix), - initializer=Constant(1), - trainable=False) - self._state = self.create_parameter( - shape=[1], attr=state_attr, dtype=dtype) - self._state.stop_gradient = True - - accum_prefix = "{}.accum".format(name) if name else 'outscale.accum' - accum_attr = ParamAttr( - name=unique_name.generate(accum_prefix), - initializer=Constant(1), - trainable=False) - self._accum = self.create_parameter( - shape=[1], attr=accum_attr, dtype=dtype) - self._accum.stop_gradient = True - - def forward(self, input): - if in_dygraph_mode(): - attrs = ('moving_rate', self._moving_rate, 'is_test', - not self.training) - state = self._state if self.training else None - accum = self._accum if self.training else None - quant_out = _varbase_creator( - type=input.type, - name="{}.tmp".format(input.name), - shape=input.shape, - dtype=input.dtype, - persistable=False) - - out, _, _, _ = core.ops.moving_average_abs_max_scale( - input, accum, state, quant_out, self._scale, state, accum, - *attrs) - return out - - check_variable_and_dtype(input, 'input', ['float32', 'float64'], - 'MovingAverageAbsMaxScale') - - attrs = {'moving_rate': self._moving_rate, 'is_test': not self.training} - inputs = {"X": [input]} - quant_out = self._helper.create_variable( - name="{}.tmp".format(input.name), - dtype=input.dtype, - type=core.VarDesc.VarType.LOD_TENSOR, - persistable=False, - stop_gradient=False) - outputs = {"Out": [quant_out], "OutScale": [self._scale]} - - if self.training: - inputs['InState'] = [self._state] - inputs['InAccum'] = [self._accum] - outputs['OutState'] = [self._state] - outputs['OutAccum'] = [self._accum] - - self._helper.append_op( - type="moving_average_abs_max_scale", - inputs=inputs, - outputs=outputs, - attrs=attrs) - - return quant_out - - class MAOutputScaleLayer(layers.Layer): """ - Calculate the scale (moving average abs max) for the output of the input layer. Add MovingAverageMaxScale layer to the behind of the input layer. + Calculate the scale (moving average abs max) for the output of the input layer. """ def __init__(self, layer=None, moving_rate=0.9, name=None, dtype='float32'): @@ -623,6 +576,10 @@ def forward(self, *inputs, **kwargs): class FakeQuantMAOutputScaleLayer(layers.Layer): + """ + Add FakeQuantMovingAverageAbsMax layer to the behind of the input layer. + """ + def __init__(self, layer, weight_bits=8, @@ -649,3 +606,30 @@ def forward(self, *inputs, **kwargs): return out else: return self._fake_quant_output(out) + + +def _get_fake_quant_type(quant_type, **kwargs): + call_args = { + "name": kwargs.get("name", None), + "quant_bits": kwargs.get("quant_bits", 8), + "dtype": kwargs.get("dtype", "float32") + } + + if quant_type == 'abs_max': + call_args["quant_on_weight"] = kwargs.get("quant_on_weight", False) + elif quant_type == 'moving_average_abs_max': + call_args["moving_rate"] = kwargs.get("moving_rate", 0.9) + elif quant_type == 'channel_wise_abs_max': + call_args["quant_on_weight"] = kwargs.get("quant_on_weight", False) + call_args["channel_num"] = kwargs.get("channel_num", None) + call_args["quant_axis"] = kwargs.get("quant_axis", 0) + assert call_args["channel_num"] is not None, ( + "You need to input channel_num" + "when you use channel_wise_abs_max strategy.") + fake_quant_map = { + 'abs_max': FakeQuantAbsMax, + 'moving_average_abs_max': FakeQuantMovingAverageAbsMax, + 'channel_wise_abs_max': FakeQuantChannelWiseAbsMax + } + + return fake_quant_map[quant_type](**call_args) diff --git a/python/paddle/optimizer/momentum.py b/python/paddle/optimizer/momentum.py index 85c5c60a34c500..38ca21a3df4c74 100644 --- a/python/paddle/optimizer/momentum.py +++ b/python/paddle/optimizer/momentum.py @@ -12,6 +12,8 @@ # See the License for the specific language governing permissions and # limitations under the License. +import warnings + from .optimizer import Optimizer from ..fluid import core from ..fluid import framework diff --git a/python/paddle/tensor/__init__.py b/python/paddle/tensor/__init__.py index bdefece122a0e7..2d4c97212be83d 100755 --- a/python/paddle/tensor/__init__.py +++ b/python/paddle/tensor/__init__.py @@ -66,6 +66,7 @@ from .manipulation import concat # noqa: F401 from .manipulation import expand # noqa: F401 from .manipulation import broadcast_to # noqa: F401 +from .manipulation import broadcast_tensors # noqa: F401 from .manipulation import expand_as # noqa: F401 from .manipulation import tile # noqa: F401 from .manipulation import flatten # noqa: F401 @@ -171,6 +172,7 @@ from .math import digamma # noqa: F401 from .math import neg # noqa: F401 from .math import lgamma # noqa: F401 +from .math import diagonal # noqa: F401 from .random import multinomial # noqa: F401 from .random import standard_normal # noqa: F401 @@ -355,12 +357,14 @@ 'shape', 'real', 'imag', + 'digamma', + 'diagonal' 'trunc' - 'digamma' 'bitwise_and', 'bitwise_or', 'bitwise_xor', 'bitwise_not', + 'broadcast_tensors', ] #this list used in math_op_patch.py for magic_method bind diff --git a/python/paddle/tensor/creation.py b/python/paddle/tensor/creation.py index b7c55ea424c710..734159422f6810 100644 --- a/python/paddle/tensor/creation.py +++ b/python/paddle/tensor/creation.py @@ -102,11 +102,10 @@ def to_tensor(data, dtype=None, place=None, stop_gradient=True): place = _get_paddle_place(place) if place is None: place = _current_expected_place() - elif not isinstance( - place, - (core.Place, core.CPUPlace, core.CUDAPinnedPlace, core.CUDAPlace)): + elif not isinstance(place, (core.Place, core.CPUPlace, core.CUDAPinnedPlace, + core.CUDAPlace, core.NPUPlace)): raise ValueError( - "'place' must be any of paddle.Place, paddle.CPUPlace, paddle.CUDAPinnedPlace, paddle.CUDAPlace" + "'place' must be any of paddle.Place, paddle.CPUPlace, paddle.CUDAPinnedPlace, paddle.CUDAPlace, paddle.NPUPlace" ) #Todo(zhouwei): Support allocate tensor on any other specified card diff --git a/python/paddle/tensor/manipulation.py b/python/paddle/tensor/manipulation.py index 1c33d19db4bdd0..6d6d2c9f9a74d5 100644 --- a/python/paddle/tensor/manipulation.py +++ b/python/paddle/tensor/manipulation.py @@ -120,6 +120,101 @@ def concat(x, axis=0, name=None): return paddle.fluid.layers.concat(input=x, axis=axis, name=name) +def broadcast_tensors(input, name=None): + """ + This OP broadcast a list of tensors following broadcast semantics + + .. note:: + If you want know more about broadcasting, please refer to :ref:`user_guide_broadcasting`. + + Args: + input(list|tuple): ``input`` is a Tensor list or Tensor tuple which is with data type bool, + float16, float32, float64, int32, int64. All the Tensors in ``input`` must have same data type. + Currently we only support tensors with rank no greater than 5. + + name (str, optional): The default value is None. Normally there is no need for user to set this property. + For more information, please refer to :ref:`api_guide_Name`. + + Returns: + list(Tensor): The list of broadcasted tensors following the same order as ``input``. + + Examples: + .. code-block:: python + + import paddle + x1 = paddle.rand([1, 2, 3, 4]).astype('float32') + x2 = paddle.rand([1, 2, 1, 4]).astype('float32') + x3 = paddle.rand([1, 1, 3, 1]).astype('float32') + out1, out2, out3 = paddle.broadcast_tensors(input=[x1, x2, x3]) + # out1, out2, out3: tensors broadcasted from x1, x2, x3 with shape [1,2,3,4] + """ + + num_inputs = len(input) + if in_dygraph_mode(): + return core.ops.broadcast_tensors(input, num_inputs) + + check_type(input, 'input', (list, tuple), 'broadcast_tensors') + if num_inputs < 1: + raise TypeError( + "At least 1 tensor is needed to perform broadcast_tensors") + + # Check input types + for id, x in enumerate(input): + check_variable_and_dtype( + x, 'input[' + str(id) + ']', + ['bool', 'float32', 'float64', 'int32', 'int64'], + 'broadcast_tensors') + if x.dtype != input[0].dtype: + raise TypeError( + "All the Tensors in the input must have the same data type.") + + # Check bcast semantics + output_shape_r_last_tensor_index = [] + output_shape_r = [] + + # Use while loop due to weird behaviour of "range()" + j = 0 + while j < len(input): + tensor = input[j] + shape = list(reversed(tensor.shape)) + + i = 0 + while i < len(shape): + if len(output_shape_r) <= i: + output_shape_r.append(shape[i]) + output_shape_r_last_tensor_index.append(j) + else: + invalid = (output_shape_r[i] != shape[i] and + output_shape_r[i] != 1 and shape[i] != 1) + if invalid: + last_index = output_shape_r_last_tensor_index[i] + raise TypeError( + "Input tensors to broadcast_tensors does not follow bcast semantics" + f"Tensor {last_index} conflicts with Tensor {j} in reversed dimension {i}" + ) + if output_shape_r[i] <= shape[i]: + output_shape_r[i] = shape[i] + output_shape_r_last_tensor_index[i] = j + i += 1 # while i < len(shape) + j += 1 # while j < len(input) + + helper = LayerHelper('broadcast_tensors', **locals()) + i = 0 + out = [] + while i < num_inputs: + out.append( + helper.create_variable_for_type_inference(dtype=helper.input_dtype( + ))) + i += 1 + + inputs = {'X': input} + helper.append_op( + type='broadcast_tensors', inputs=inputs, outputs={'Out': out}, + attrs={}) + + return out + + def flip(x, axis, name=None): """ Reverse the order of a n-D tensor along given axis in axis. @@ -364,28 +459,22 @@ def roll(x, shifts, axis=None, name=None): if axis: check_type(axis, 'axis', (list, tuple), 'roll') + else: + axis = [] + check_type(shifts, 'shifts', (list, tuple), 'roll') if in_dygraph_mode(): - if axis is None: - x = core.ops.reshape(x, 'shape', [-1, 1]) - axis = [0] - out = core.ops.roll(x, 'axis', axis, 'shifts', shifts) - return core.ops.reshape(out, 'shape', origin_shape) + return core.ops.roll(x, 'axis', axis, 'shifts', shifts) out = helper.create_variable_for_type_inference(x.dtype) - if axis is None: - x = reshape(x, shape=[-1, 1]) - axis = [0] - helper.append_op( type='roll', inputs={'X': x}, outputs={'Out': out}, attrs={'axis': axis, 'shifts': shifts}) - out = layers.reshape(out, shape=origin_shape) return out diff --git a/python/paddle/tensor/math.py b/python/paddle/tensor/math.py index 3f1f2b421476a5..7e85eb07a5b4d6 100755 --- a/python/paddle/tensor/math.py +++ b/python/paddle/tensor/math.py @@ -1696,6 +1696,114 @@ def __check_input(input, offset, dim1, dim2): outputs={'Out': [out]}) return out +def diagonal(x, offset=0, axis1=0, axis2=1, name=None): + """ + This OP computes the diagonals of the input tensor x. + + If ``x`` is 2D, returns the diagonal. + If ``x`` has larger dimensions, diagonals be taken from the 2D planes specified by axis1 and axis2. + By default, the 2D planes formed by the first and second axis of the input tensor x. + + The argument ``offset`` determines where diagonals are taken from input tensor x: + + - If offset = 0, it is the main diagonal. + - If offset > 0, it is above the main diagonal. + - If offset < 0, it is below the main diagonal. + + Args: + x(Tensor): The input tensor x. Must be at least 2-dimensional. The input data type should be bool, int32, int64, float16, float32, float64. + offset(int, optional): Which diagonals in input tensor x will be taken. Default: 0 (main diagonals). + axis1(int, optional): The first axis with respect to take diagonal. Default: 0. + axis2(int, optional): The second axis with respect to take diagonal. Default: 1. + name (str, optional): Normally there is no need for user to set this property. For more information, please refer to :ref:`api_guide_Name`. Default: None. + + Returns: + Tensor: a partial view of input tensor in specify two dimensions, the output data type is the same as input data type. + + Examples: + .. code-block:: python + + import paddle + + x = paddle.rand([2,2,3],'float32') + print(x) + # Tensor(shape=[2, 2, 3], dtype=float32, place=CUDAPlace(0), stop_gradient=True, + # [[[0.45661032, 0.03751532, 0.90191704], + # [0.43760979, 0.86177313, 0.65221709]], + + # [[0.17020577, 0.00259554, 0.28954273], + # [0.51795638, 0.27325270, 0.18117726]]]) + + out1 = paddle.diagonal(x) + print(out1) + #Tensor(shape=[3, 2], dtype=float32, place=CUDAPlace(0), stop_gradient=True, + # [[0.45661032, 0.51795638], + # [0.03751532, 0.27325270], + # [0.90191704, 0.18117726]]) + + out2 = paddle.diagonal(x, offset=0, axis1=2, axis2=1) + print(out2) + #Tensor(shape=[2, 2], dtype=float32, place=CUDAPlace(0), stop_gradient=True, + # [[0.45661032, 0.86177313], + # [0.17020577, 0.27325270]]) + + out3 = paddle.diagonal(x, offset=1, axis1=0, axis2=1) + print(out3) + #Tensor(shape=[3, 1], dtype=float32, place=CUDAPlace(0), stop_gradient=True, + # [[0.43760979], + # [0.86177313], + # [0.65221709]]) + + out4 = paddle.diagonal(x, offset=0, axis1=1, axis2=2) + print(out4) + #Tensor(shape=[2, 2], dtype=float32, place=CUDAPlace(0), stop_gradient=True, + # [[0.45661032, 0.86177313], + # [0.17020577, 0.27325270]]) + + """ + def __check_input(input, offset, dim1, dim2): + check_dtype(x.dtype, 'Input', + ['bool', 'int32', 'int64', 'float16', 'float32', 'float64'], + 'diagonal') + + input_shape = list(x.shape) + assert len(input_shape) >= 2, \ + "The x must be at least 2-dimensional, " \ + "But received Input x's dimensional: %s.\n" % \ + len(input_shape) + + axis1_ = axis1 if axis1 >= 0 else len(input_shape) + axis1 + axis2_ = axis2 if axis2 >= 0 else len(input_shape) + axis2 + + assert axis1_ < len(input_shape), \ + "The argument axis1 is out of range (expected to be in range of [%d, %d], but got %d).\n" \ + % (-(len(input_shape)), len(input_shape) - 1, axis1) + + assert axis2_ < len(input_shape), \ + "The argument axis2 is out of range (expected to be in range of [%d, %d], but got %d).\n" \ + % (-(len(input_shape)), len(input_shape) - 1, axis2) + + assert axis1_ != axis2_, \ + "axis1 and axis2 cannot be the same axis." \ + "But received axis1 = %d, axis2 = %d\n"%(axis1, axis2) + + if in_dygraph_mode(): + return core.ops.diagonal(x, 'offset', offset, 'axis1', axis1, 'axis2', axis2) + + __check_input(input, offset, axis1, axis2) + helper = LayerHelper('diagonal', **locals()) + out = helper.create_variable_for_type_inference(dtype=x.dtype) + + helper.append_op( + type='diagonal', + inputs={'Input': [x]}, + attrs={'offset': offset, + 'axis1': axis1, + 'axis2': axis2}, + outputs={'Out': [out]}) + return out + + @templatedoc(op_type="kron") def kron(x, y, name=None): """ diff --git a/python/paddle/tests/test_model.py b/python/paddle/tests/test_model.py index 0ced69c0f2ea96..a970489b92a879 100644 --- a/python/paddle/tests/test_model.py +++ b/python/paddle/tests/test_model.py @@ -718,6 +718,36 @@ def test_dygraph_export_deploy_model_about_inputs(self): model.save(save_dir, training=False) shutil.rmtree(save_dir) + def test_accumulate(self, ): + dim = 20 + data = np.random.random(size=(4, dim)).astype(np.float32) + label = np.random.randint(0, 10, size=(4, 1)).astype(np.int64) + net = MyModel() + optim = fluid.optimizer.SGD(learning_rate=0.001, + parameter_list=net.parameters()) + inputs = [InputSpec([None, dim], 'float32', 'x')] + labels = [InputSpec([None, 1], 'int64', 'label')] + + for amp_cfg in [None, 'O1']: + model = Model(net, inputs, labels) + model.prepare( + optim, + loss=CrossEntropyLoss(reduction="sum"), + amp_configs=amp_cfg) + losses, grads = [], [] + for stat in [False, False, True]: + loss, = model.train_batch([data], [label], update=stat) + losses.append(loss) + grads.append([p.grad.numpy() for p in net.parameters()]) + + for grad1, grad2, grad3 in zip(*grads): + np.testing.assert_almost_equal(grad1 * 2, grad2, decimal=4) + np.testing.assert_almost_equal( + grad3, np.zeros_like(grad3), decimal=4) + + np.testing.assert_almost_equal(losses[0], losses[1], decimal=4) + np.testing.assert_almost_equal(losses[0], losses[2], decimal=4) + class TestModelWithLRScheduler(unittest.TestCase): def test_fit_by_step(self): diff --git a/python/paddle/vision/transforms/transforms.py b/python/paddle/vision/transforms/transforms.py index 27eca19c28be6c..c09748913f9dad 100644 --- a/python/paddle/vision/transforms/transforms.py +++ b/python/paddle/vision/transforms/transforms.py @@ -555,6 +555,7 @@ class RandomHorizontalFlip(BaseTransform): def __init__(self, prob=0.5, keys=None): super(RandomHorizontalFlip, self).__init__(keys) + assert 0 <= prob <= 1, "probability must be between 0 and 1" self.prob = prob def _apply_image(self, img): @@ -589,6 +590,7 @@ class RandomVerticalFlip(BaseTransform): def __init__(self, prob=0.5, keys=None): super(RandomVerticalFlip, self).__init__(keys) + assert 0 <= prob <= 1, "probability must be between 0 and 1" self.prob = prob def _apply_image(self, img): diff --git a/python/requirements.txt b/python/requirements.txt index 31523e90506210..e9da2aa24d6cb2 100644 --- a/python/requirements.txt +++ b/python/requirements.txt @@ -2,7 +2,7 @@ requests>=2.20.0 numpy>=1.13 ; python_version>="3.5" and platform_system != "Windows" numpy>=1.13, <=1.19.3 ; python_version>="3.5" and platform_system == "Windows" protobuf>=3.1.0 -gast>=0.3.3 ; platform_system != "Windows" +gast>=0.3.3, <=0.4.0 ; platform_system != "Windows" gast==0.3.3 ; platform_system == "Windows" Pillow six diff --git a/tools/CrossStackProfiler/NetFileReader.py b/tools/CrossStackProfiler/NetFileReader.py index 29c2ae85e60458..fe900fab2ad24b 100755 --- a/tools/CrossStackProfiler/NetFileReader.py +++ b/tools/CrossStackProfiler/NetFileReader.py @@ -17,6 +17,7 @@ import glob import logging import pandas as pd +import multiprocessing from multiprocessing import Process diff --git a/tools/analysisPyXml.py b/tools/analysisPyXml.py index db3d6887853f4e..5d6a5ac459408e 100644 --- a/tools/analysisPyXml.py +++ b/tools/analysisPyXml.py @@ -25,7 +25,10 @@ def analysisPyXml(rootPath, ut): xml_path = '%s/build/pytest/%s/python-coverage.xml' % (rootPath, ut) - ut_map_file = '%s/build/ut_map/%s/%s.txt' % (rootPath, ut, ut) + related_ut_map_file = '%s/build/ut_map/%s/related_%s.txt' % (rootPath, ut, + ut) + notrelated_ut_map_file = '%s/build/ut_map/%s/notrelated_%s.txt' % (rootPath, + ut, ut) tree = ElementTree.parse(xml_path) root = tree.getroot() error_files = [] @@ -46,16 +49,27 @@ def analysisPyXml(rootPath, ut): '@', '\'\'\'', 'logger', '_logger', 'logging', 'r"""', 'pass', 'try', 'except', 'if __name__ == "__main__"' )) == False: - #print(line_hits, line_number) pattern = "(.*) = ('*')|(.*) = (\"*\")|(.*) = (\d)|(.*) = (-\d)|(.*) = (None)|(.*) = (True)|(.*) = (False)|(.*) = (URL_PREFIX*)|(.*) = (\[)|(.*) = (\{)|(.*) = (\()" #a='b'/a="b"/a=0 if re.match(pattern, output.strip()) == None: pyCov_file.append(clazz_filename) - os.system('echo %s >> %s' % - (clazz_filename, ut_map_file)) + coverageMessage = 'RELATED' break + else: + coverageMessage = 'FILTER' #hit filter logic + else: + coverageMessage = 'FILTER' else: + coverageMessage = 'ERROR' error_files.append(clazz_filename) break + else: + coverageMessage = 'NOT_RELATED' + if coverageMessage in ['NOT_RELATED', 'ERROR', 'FILTER']: + os.system('echo %s >> %s' % + (clazz_filename, notrelated_ut_map_file)) + elif coverageMessage == 'RELATED': + os.system('echo %s >> %s' % (clazz_filename, related_ut_map_file)) + print("============len(pyCov_file)") print(len(pyCov_file)) print("============error") diff --git a/tools/check_api_approvals.sh b/tools/check_api_approvals.sh index 97d97e8c0a26ad..40a0a618fb066d 100644 --- a/tools/check_api_approvals.sh +++ b/tools/check_api_approvals.sh @@ -39,21 +39,29 @@ function add_failed(){ api_spec_diff=`python ${PADDLE_ROOT}/tools/diff_api.py ${PADDLE_ROOT}/paddle/fluid/API_DEV.spec.api ${PADDLE_ROOT}/paddle/fluid/API_PR.spec.api` if [ "$api_spec_diff" != "" ]; then - echo_line="You must have one RD (XiaoguangHu01 or lanxianghit) and one TPM (saxon-zh or jzhang533 or dingjiaweiww or Heeenrrry or TCChenlong) approval for the api change for the management reason of API interface.\n" + echo_line="You must have one RD (XiaoguangHu01 or lanxianghit) approval for API change.\n" + echo_line="${echo_line} and one TPM approval for API change: \n" + echo_line="${echo_line} jzhang533/ZhangJun, dingjiaweiww/DingJiaWei, Heeenrrry/LiKunLun, TCChenlong/ChenLong for general APIs\n" + echo_line="${echo_line} PangHua/XiangHui for distributed related APIs\n" + echo_line="${echo_line} twismon/WangYunKai, CheQiXiao/CheQiXiao for inference related APIs.\n" + check_approval 1 46782768 47554610 - echo_line="" - check_approval 1 2870059 29231 23093488 28379894 11935832 + check_approval 1 29231 23093488 28379894 11935832 2682285 12050047 50894398 fi api_doc_spec_diff=`python ${PADDLE_ROOT}/tools/diff_api.py ${PADDLE_ROOT}/paddle/fluid/API_DEV.spec.doc ${PADDLE_ROOT}/paddle/fluid/API_PR.spec.doc` if [ "$api_doc_spec_diff" != "" ]; then - echo_line="You must have one TPM (saxon-zh or jzhang533 or dingjiaweiww or Heeenrrry or TCChenlong) approval for the api change for the management reason of API document.\n" - check_approval 1 2870059 29231 23093488 28379894 11935832 + echo_line="You must have one TPM approval for API documents change: \n" + echo_line="${echo_line} jzhang533/ZhangJun, dingjiaweiww/DingJiaWei, Heeenrrry/LiKunLun, TCChenlong/ChenLong for general API docs\n" + echo_line="${echo_line} PangHua/XiangHui for distributed related API docs\n" + echo_line="${echo_line} twismon/WangYunKai, CheQiXiao/CheQiXiao for inference related API docs.\n" + + check_approval 1 29231 23093488 28379894 11935832 2682285 12050047 50894398 fi -api_spec_diff=`python ${PADDLE_ROOT}/tools/check_api_source_without_core_ops.py ${PADDLE_ROOT}/paddle/fluid/API_DEV.source.md5 ${PADDLE_ROOT}/paddle/fluid/API_PR.source.md5` -if [ "$api_spec_diff" != "" ]; then - echo_line="APIs without core.ops: \n${api_spec_diff}\n" +api_src_spec_diff=`python ${PADDLE_ROOT}/tools/check_api_source_without_core_ops.py ${PADDLE_ROOT}/paddle/fluid/API_DEV.source.md5 ${PADDLE_ROOT}/paddle/fluid/API_PR.source.md5` +if [ "$api_src_spec_diff" != "" ]; then + echo_line="APIs without core.ops: \n${api_src_spec_diff}\n" echo_line="${echo_line}You must have one RD (zhiqiu (Recommend) or phlrain) approval for the api change for the opreator-related api without 'core.ops'.\n" echo_line="${echo_line}For more details, please click [https://github.com/PaddlePaddle/Paddle/wiki/paddle_api_development_manual.md]\n" check_approval 1 6888866 43953930 @@ -61,8 +69,8 @@ fi op_type_spec_diff=`python ${PADDLE_ROOT}/tools/check_op_register_type.py ${PADDLE_ROOT}/paddle/fluid/OP_TYPE_DEV.spec ${PADDLE_ROOT}/paddle/fluid/OP_TYPE_PR.spec` if [ "$op_type_spec_diff" != "" ]; then - echo_line="You must have one RD (Aurelius84 (Recommend) or liym27 or zhhsplendid)approval for the data_type registration of new operator. More data_type of new operator should be registered in your PR. Please make sure that both float/double (or int/int64_t) have been registered.\n For more details, please click [https://github.com/PaddlePaddle/Paddle/wiki/Data-types-of-generic-Op-must-be-fully-registered].\n" - check_approval 1 9j301846 33742067 7913861 + echo_line="You must have one RD (Aurelius84 (Recommend) or zhhsplendid)approval for the data_type registration of new operator. More data_type of new operator should be registered in your PR. Please make sure that both float/double (or int/int64_t) have been registered.\n For more details, please click [https://github.com/PaddlePaddle/Paddle/wiki/Data-types-of-generic-Op-must-be-fully-registered].\n" + check_approval 1 9301846 7913861 fi op_desc_diff=`python ${PADDLE_ROOT}/tools/check_op_desc.py ${PADDLE_ROOT}/paddle/fluid/OP_DESC_DEV.spec ${PADDLE_ROOT}/paddle/fluid/OP_DESC_PR.spec` @@ -84,10 +92,13 @@ if [ -n "${echo_list}" ];then echo -e "${echo_list[@]}" echo "There are ${failed_num} approved errors." echo "****************" -fi -python ${PADDLE_ROOT}/tools/diff_api.py ${PADDLE_ROOT}/paddle/fluid/API_DEV.spec ${PADDLE_ROOT}/paddle/fluid/API_PR.spec -python ${PADDLE_ROOT}/tools/check_op_register_type.py ${PADDLE_ROOT}/paddle/fluid/OP_TYPE_DEV.spec ${PADDLE_ROOT}/paddle/fluid/OP_TYPE_PR.spec -if [ -n "${echo_list}" ]; then + # L40 L48 L62 has fetch the result out, but there are splitted. + if [ "${api_spec_diff}" != "" -o "${api_doc_spec_diff}" != "" ] ; then + python ${PADDLE_ROOT}/tools/diff_api.py ${PADDLE_ROOT}/paddle/fluid/API_DEV.spec ${PADDLE_ROOT}/paddle/fluid/API_PR.spec + fi + if [ "${op_type_spec_diff}" != "" ] ; then + echo "op_type_spec_diff: ${op_type_spec_diff}" + fi exit 6 fi diff --git a/tools/check_file_diff_approvals.sh b/tools/check_file_diff_approvals.sh index 92e59675dad16a..b43e2280294886 100644 --- a/tools/check_file_diff_approvals.sh +++ b/tools/check_file_diff_approvals.sh @@ -54,6 +54,7 @@ API_FILES=("CMakeLists.txt" "python/paddle/fluid/tests/unittests/white_list/no_grad_set_white_list.py" "tools/print_signatures.py" "tools/sampcd_processor.py" + "tools/check_pr_approval.py" "paddle/scripts/paddle_build.bat" "tools/windows/run_unittests.sh" "tools/parallel_UT_rule.py" @@ -146,6 +147,9 @@ for API_FILE in ${API_FILES[*]}; do elif [ "${API_FILE}" == "tools/print_signatures.py" ];then echo_line="test_print_signatures.py will be executed for changed print_signatures.py.\n" run_tools_test test_print_signatures.py + elif [ "${API_FILE}" == "tools/checkout_pr_approval.py" ];then + echo_line="test_checkout_pr_approval.py will be executed for changed checkout_pr_approval.py.\n" + run_tools_test test_checkout_pr_approval.py elif [ "${API_FILE}" == "python/paddle/distributed/fleet/__init__.py" ]; then echo_line="You must have (fuyinno4 (Recommend), raindrops2sea) approval for ${API_FILE} changes" check_approval 1 35824027 38231817 diff --git a/tools/check_file_suffix.py b/tools/check_file_suffix.py new file mode 100644 index 00000000000000..1d422dd6c4fe02 --- /dev/null +++ b/tools/check_file_suffix.py @@ -0,0 +1,47 @@ +# Copyright (c) 2018 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +import sys +import json + + +def check_suffix(): + suffix_arr = [".pyc"] + json_buff = "" + for line in sys.stdin: + json_buff = "".join([json_buff, line]) + json_obj = json.loads(json_buff) + if not isinstance(json_obj, list): + print('Json String Should be a list Object\n') + return + files_with_invalid_suffix = [] + for i in range(len(json_obj)): + file_name = json_obj[i]["filename"] + if file_name == None: + continue + for suffix in suffix_arr: + if file_name.endswith(suffix): + files_with_invalid_suffix.append(file_name) + break + if len(files_with_invalid_suffix) != 0: + print('Error: Find file(s): [\n') + for i in range(len(files_with_invalid_suffix)): + print('\t' + files_with_invalid_suffix[i] + '\n') + print( + ' ] end(s) with invalid suffix, Please check if these files are temporary.' + ) + + +if __name__ == "__main__": + check_suffix() diff --git a/tools/check_pr_approval.py b/tools/check_pr_approval.py index 937b0be7562fab..c242afd06e760a 100644 --- a/tools/check_pr_approval.py +++ b/tools/check_pr_approval.py @@ -12,7 +12,6 @@ # See the License for the specific language governing permissions and # limitations under the License. -from __future__ import print_function import sys import json @@ -24,17 +23,24 @@ def check_approval(count, required_reviewers): json_resp = json.loads(json_buff) approves = 0 approved_user_ids = [] + approved_user_logins = set() for review in json_resp: if review["state"] == "APPROVED": approves += 1 approved_user_ids.append(review["user"]["id"]) + approved_user_logins.add(review["user"]["login"]) # convert to int required_reviewers_int = set() + required_reviewers_login = set() for rr in required_reviewers: - required_reviewers_int.add(int(rr)) + if rr.isdigit(): + required_reviewers_int.add(int(rr)) + else: + required_reviewers_login.add(rr) - if len(set(approved_user_ids) & required_reviewers_int) >= count: + if len(set(approved_user_ids) & required_reviewers_int) + len( + approved_user_logins & required_reviewers_login) >= count: print("TRUE") else: print("FALSE") diff --git a/tools/dockerfile/Dockerfile.release16 b/tools/dockerfile/Dockerfile.release16 new file mode 100644 index 00000000000000..7effa2e4ed5e84 --- /dev/null +++ b/tools/dockerfile/Dockerfile.release16 @@ -0,0 +1,163 @@ +# A image for building paddle binaries +# Use cuda devel base image for both cpu and gpu environment +# When you modify it, please be aware of cudnn-runtime version +FROM nvidia/cuda: +MAINTAINER PaddlePaddle Authors + +# ENV variables +ARG WITH_GPU +ARG WITH_AVX + +ENV WITH_GPU=${WITH_GPU:-ON} +ENV WITH_AVX=${WITH_AVX:-ON} + +ENV HOME /root +# Add bash enhancements +COPY paddle/scripts/docker/root/ /root/ + +# Prepare packages for Python +RUN apt-get update && \ + apt-get install -y make build-essential libssl-dev zlib1g-dev libbz2-dev \ + libreadline-dev libsqlite3-dev wget curl llvm libncurses5-dev libncursesw5-dev \ + xz-utils tk-dev libffi-dev liblzma-dev + +RUN apt-get update && \ + apt-get install -y --allow-downgrades --allow-change-held-packages \ + patchelf git python-pip python-dev python-opencv openssh-server bison \ + wget unzip unrar tar xz-utils bzip2 gzip coreutils ntp \ + curl sed grep graphviz libjpeg-dev zlib1g-dev \ + python-matplotlib \ + automake locales clang-format swig \ + liblapack-dev liblapacke-dev \ + net-tools libtool module-init-tools vim && \ + apt-get clean -y + +RUN wget https://github.com/koalaman/shellcheck/releases/download/v0.7.1/shellcheck-v0.7.1.linux.x86_64.tar.xz -O shellcheck-v0.7.1.linux.x86_64.tar.xz && \ + tar -xf shellcheck-v0.7.1.linux.x86_64.tar.xz && cp shellcheck-v0.7.1/shellcheck /usr/bin/shellcheck && \ + rm -rf shellcheck-v0.7.1.linux.x86_64.tar.xz shellcheck-v0.7.1 + +# Downgrade gcc&&g++ + + +# install cmake +WORKDIR /home +RUN wget -q https://cmake.org/files/v3.16/cmake-3.16.0-Linux-x86_64.tar.gz && tar -zxvf cmake-3.16.0-Linux-x86_64.tar.gz && rm cmake-3.16.0-Linux-x86_64.tar.gz +ENV PATH=/home/cmake-3.16.0-Linux-x86_64/bin:$PATH + + +# Install Python3.7 +RUN mkdir -p /root/python_build/ && wget -q https://www.sqlite.org/2018/sqlite-autoconf-3250300.tar.gz && \ + tar -zxf sqlite-autoconf-3250300.tar.gz && cd sqlite-autoconf-3250300 && \ + ./configure -prefix=/usr/local && make -j8 && make install && cd ../ && rm sqlite-autoconf-3250300.tar.gz && \ + wget -q https://www.python.org/ftp/python/3.7.0/Python-3.7.0.tgz && \ + tar -xzf Python-3.7.0.tgz && cd Python-3.7.0 && \ + CFLAGS="-Wformat" ./configure --prefix=/usr/local/python3.7.0 --enable-shared > /dev/null && \ + make -j8 > /dev/null && make altinstall > /dev/null && ldconfig + +ENV PATH=/usr/local/python3.7.0/include:${PATH} +ENV PATH=/usr/local/python3.7.0/bin:${PATH} +ENV LD_LIBRARY_PATH=/usr/local/python3.7.0/lib:${LD_LIBRARY_PATH} +ENV CPLUS_INCLUDE_PATH=/usr/local/python3.7.0/include/python3.7:$CPLUS_INCLUDE_PATH +RUN ln -sf /usr/local/python3.7.0/bin/python3.7 /usr/local/bin/python3 && ln -sf /usr/local/python3.7.0/bin/python3.7 /usr/bin/python3 +RUN mv /usr/bin/python /usr/bin/python.bak && ln -s /usr/local/python3.7.0/bin/python3.7 /usr/local/bin/python && ln -s /usr/local/python3.7.0/bin/python3.7 /usr/bin/python + +RUN rm -r /root/python_build + +WORKDIR /home +RUN python3.7 -m pip uninstall -y pip setuptools +RUN wget https://files.pythonhosted.org/packages/a7/e0/30642b9c2df516506d40b563b0cbd080c49c6b3f11a70b4c7a670f13a78b/setuptools-50.3.2.zip && apt-get -y install unzip && unzip setuptools-50.3.2.zip +WORKDIR /home/setuptools-50.3.2 +RUN python3.7 setup.py build && python3.7 setup.py install +WORKDIR /home +RUN wget https://files.pythonhosted.org/packages/28/af/2c76c8aa46ccdf7578b83d97a11a2d1858794d4be4a1610ade0d30182e8b/pip-20.0.1.tar.gz && tar -zxvf pip-20.0.1.tar.gz +WORKDIR pip-20.0.1 +RUN python3.7 setup.py install + +WORKDIR /home +RUN rm setuptools-50.3.2.zip pip-20.0.1.tar.gz && \ + rm -r setuptools-50.3.2 pip-20.0.1 + +# Install Go and glide +WORKDIR /home +RUN wget -qO- https://paddle-ci.cdn.bcebos.com/go1.8.1.linux-amd64.tar.gz | \ + tar -xz -C /usr/local && \ + mkdir /root/gopath && \ + mkdir /root/gopath/bin && \ + mkdir /root/gopath/src +ENV GOROOT=/usr/local/go GOPATH=/root/gopath +# should not be in the same line with GOROOT definition, otherwise docker build could not find GOROOT. +ENV PATH=${PATH}:${GOROOT}/bin:${GOPATH}/bin +# install glide +RUN curl -s -q https://glide.sh/get | sh + +# Install TensorRT +# following TensorRT.tar.gz is not the default official one, we do two miny changes: +# 1. Remove the unnecessary files to make the library small. TensorRT.tar.gz only contains include and lib now, +# and its size is only one-third of the official one. +# 2. Manually add ~IPluginFactory() in IPluginFactory class of NvInfer.h, otherwise, it couldn't work in paddle. +# See https://github.com/PaddlePaddle/Paddle/issues/10129 for details. + +# Downgrade TensorRT +COPY tools/dockerfile/build_scripts /build_scripts +RUN bash /build_scripts/install_nccl2.sh +RUN rm -rf /build_scripts + +# git credential to skip password typing +RUN git config --global credential.helper store + +# Fix locales to en_US.UTF-8 +RUN localedef -i en_US -f UTF-8 en_US.UTF-8 + +# specify sphinx version as 1.5.6 and remove -U option for [pip install -U +# sphinx-rtd-theme] since -U option will cause sphinx being updated to newest +# version(1.7.1 for now), which causes building documentation failed. +RUN pip3.7 --no-cache-dir install -U wheel py-cpuinfo==5.0.0 && \ + pip3.7 --no-cache-dir install -U docopt PyYAML sphinx==1.5.6 && \ + pip3.7 --no-cache-dir install sphinx-rtd-theme==0.1.9 recommonmark + +RUN pip3.7 --no-cache-dir install 'pre-commit==1.10.4' 'ipython==5.3.0' && \ + pip3.7 --no-cache-dir install 'ipykernel==4.6.0' 'jupyter==1.0.0' + +#For docstring checker +RUN pip3.7 --no-cache-dir install pylint pytest astroid isort + +RUN pip3.7 --no-cache-dir install coverage + +COPY ./python/requirements.txt /root/ +RUN pip3.7 --no-cache-dir install -r /root/requirements.txt + +# To fix https://github.com/PaddlePaddle/Paddle/issues/1954, we use +# the solution in https://urllib3.readthedocs.io/en/latest/user-guide.html#ssl-py2 +RUN apt-get install -y libssl-dev libffi-dev && apt-get clean -y && \ + pip3.7 install --upgrade pip && \ + pip3.7 --no-cache-dir install certifi urllib3[secure] + +# ar mishandles 4GB files +# https://sourceware.org/bugzilla/show_bug.cgi?id=14625 +# remove them when apt-get support 2.27 and higher version +RUN wget -q https://launchpad.net/ubuntu/+archive/primary/+sourcefiles/binutils/2.27-9ubuntu1/binutils_2.27.orig.tar.gz && \ + tar -xzf binutils_2.27.orig.tar.gz && \ + cd binutils-2.27 && \ + ./configure && make -j && make install && cd .. && rm -rf binutils-2.27 binutils_2.27.orig.tar.gz + +RUN apt-get install libprotobuf-dev -y + +# Older versions of patchelf limited the size of the files being processed and were fixed in this pr. +# https://github.com/NixOS/patchelf/commit/ba2695a8110abbc8cc6baf0eea819922ee5007fa +# So install a newer version here. +RUN wget -q https://paddle-ci.cdn.bcebos.com/patchelf_0.10-2_amd64.deb && \ + dpkg -i patchelf_0.10-2_amd64.deb + +# Configure OpenSSH server. c.f. https://docs.docker.com/engine/examples/running_ssh_service +RUN mkdir /var/run/sshd && echo 'root:root' | chpasswd && sed -ri 's/^PermitRootLogin\s+.*/PermitRootLogin yes/' /etc/ssh/sshd_config && sed -ri 's/UsePAM yes/#UsePAM yes/g' /etc/ssh/sshd_config +CMD source ~/.bashrc + +# ccache 3.7.9 +RUN wget https://paddle-ci.gz.bcebos.com/ccache-3.7.9.tar.gz && \ + tar xf ccache-3.7.9.tar.gz && mkdir /usr/local/ccache-3.7.9 && cd ccache-3.7.9 && \ + ./configure -prefix=/usr/local/ccache-3.7.9 && \ + make -j8 && make install && \ + ln -s /usr/local/ccache-3.7.9/bin/ccache /usr/local/bin/ccache + + +EXPOSE 22 diff --git a/tools/dockerfile/Dockerfile.release18 b/tools/dockerfile/Dockerfile.release18 new file mode 100644 index 00000000000000..ddae9e1c32aef1 --- /dev/null +++ b/tools/dockerfile/Dockerfile.release18 @@ -0,0 +1,125 @@ +# A image for building paddle binaries +# Use cuda devel base image for both cpu and gpu environment +# When you modify it, please be aware of cudnn-runtime version +FROM nvidia/cuda: +MAINTAINER PaddlePaddle Authors + +# ENV variables +ARG WITH_GPU +ARG WITH_AVX + +ENV WITH_GPU=${WITH_GPU:-ON} +ENV WITH_AVX=${WITH_AVX:-ON} +ENV DEBIAN_FRONTEND=noninteractive +ENV LD_LIBRARY_PATH=/usr/local/cuda-11.2/targets/x86_64-linux/lib:$LD_LIBRARY_PATH + +ENV HOME /root +# Add bash enhancements +COPY paddle/scripts/docker/root/ /root/ + +RUN apt-get update && \ + apt-get install -y software-properties-common && add-apt-repository ppa:deadsnakes/ppa && \ + apt-get update && \ + apt-get install -y curl wget vim git unzip unrar tar xz-utils libssl-dev bzip2 gzip \ + coreutils ntp language-pack-zh-hans python-qt4 libsm6 libxext6 libxrender-dev libgl1-mesa-glx \ + bison graphviz libjpeg-dev zlib1g-dev automake locales swig net-tools libtool module-init-tools + +# Downgrade gcc&&g++ +WORKDIR /usr/bin +COPY tools/dockerfile/build_scripts /build_scripts +RUN bash /build_scripts/install_trt.sh +RUN bash /build_scripts/install_gcc.sh gcc82 && rm -rf /build_scripts +RUN cp gcc gcc.bak && cp g++ g++.bak && rm gcc && rm g++ +RUN ln -s /usr/local/gcc-8.2/bin/gcc /usr/local/bin/gcc +RUN ln -s /usr/local/gcc-8.2/bin/g++ /usr/local/bin/g++ +RUN ln -s /usr/local/gcc-8.2/bin/gcc /usr/bin/gcc +RUN ln -s /usr/local/gcc-8.2/bin/g++ /usr/bin/g++ +ENV PATH=/usr/local/gcc-8.2/bin:$PATH + + +# install cmake +WORKDIR /home +RUN wget -q https://cmake.org/files/v3.16/cmake-3.16.0-Linux-x86_64.tar.gz && tar -zxvf cmake-3.16.0-Linux-x86_64.tar.gz && rm cmake-3.16.0-Linux-x86_64.tar.gz +ENV PATH=/home/cmake-3.16.0-Linux-x86_64/bin:$PATH + + +RUN apt-get update && \ + apt-get install -y python3.7 python3.7-dev && \ + mv /usr/bin/python /usr/bin/python.bak && ln -s /usr/bin/python3.7 /usr/bin/python && \ + mv /usr/bin/python3 /usr/bin/python3.bak && ln -s /usr/bin/python3.7 /usr/bin/python3 + + +WORKDIR /home +RUN wget https://files.pythonhosted.org/packages/a7/e0/30642b9c2df516506d40b563b0cbd080c49c6b3f11a70b4c7a670f13a78b/setuptools-50.3.2.zip && apt-get -y install unzip && unzip setuptools-50.3.2.zip +WORKDIR /home/setuptools-50.3.2 +RUN python3.7 setup.py build && python3.7 setup.py install +WORKDIR /home +RUN wget https://files.pythonhosted.org/packages/28/af/2c76c8aa46ccdf7578b83d97a11a2d1858794d4be4a1610ade0d30182e8b/pip-20.0.1.tar.gz && tar -zxvf pip-20.0.1.tar.gz +WORKDIR pip-20.0.1 +RUN python3.7 setup.py install + +WORKDIR /home +RUN rm setuptools-50.3.2.zip pip-20.0.1.tar.gz && \ + rm -r setuptools-50.3.2 pip-20.0.1 +RUN rm /usr/local/bin/pip && ln -s /usr/local/bin/pip3.7 /usr/local/bin/pip && \ + rm /usr/local/bin/pip3 && ln -s /usr/local/bin/pip3.7 /usr/local/bin/pip3 + + +# remove them when apt-get support 2.27 and higher version +RUN wget -q https://ftp.gnu.org/gnu/binutils/binutils-2.33.1.tar.gz && \ + tar -xzf binutils-2.33.1.tar.gz && \ + cd binutils-2.33.1 && \ + ./configure && make -j && make install && cd .. && rm -rf binutils-2.33.1 binutils-2.33.1.tar.gz + + +# Install Go and glide +RUN wget -qO- https://paddle-ci.cdn.bcebos.com/go1.8.1.linux-amd64.tar.gz | \ + tar -xz -C /usr/local && \ + mkdir /root/gopath && \ + mkdir /root/gopath/bin && \ + mkdir /root/gopath/src +ENV GOROOT=/usr/local/go GOPATH=/root/gopath +# should not be in the same line with GOROOT definition, otherwise docker build could not find GOROOT. +ENV PATH=${PATH}:${GOROOT}/bin:${GOPATH}/bin +# install glide +RUN curl -s -q https://glide.sh/get | sh + +# git credential to skip password typing +RUN git config --global credential.helper store + +# Fix locales to en_US.UTF-8 +RUN localedef -i en_US -f UTF-8 en_US.UTF-8 + +RUN pip3.7 --no-cache-dir install pre-commit==1.10.4 ipython==5.3.0 && \ + pip3.7 --no-cache-dir install ipykernel==4.6.0 wheel + +#For docstring checker +RUN pip3.7 --no-cache-dir install pylint pytest astroid isort + +COPY ./python/requirements.txt /root/ +RUN pip3.7 --no-cache-dir install -r /root/requirements.txt + + +# Older versions of patchelf limited the size of the files being processed and were fixed in this pr. +# https://github.com/NixOS/patchelf/commit/ba2695a8110abbc8cc6baf0eea819922ee5007fa +# So install a newer version here. +RUN wget -q https://paddle-ci.cdn.bcebos.com/patchelf_0.10-2_amd64.deb && \ + dpkg -i patchelf_0.10-2_amd64.deb + +# Configure OpenSSH server. c.f. https://docs.docker.com/engine/examples/running_ssh_service +#RUN mkdir /var/run/sshd && echo 'root:root' | chpasswd && sed -ri 's/^PermitRootLogin\s+.*/PermitRootLogin yes/' /etc/ssh/sshd_config && sed -ri 's/UsePAM yes/#UsePAM yes/g' /etc/ssh/sshd_config +#CMD source ~/.bashrc + +# ccache 3.7.9 +RUN wget https://paddle-ci.gz.bcebos.com/ccache-3.7.9.tar.gz && \ + tar xf ccache-3.7.9.tar.gz && mkdir /usr/local/ccache-3.7.9 && cd ccache-3.7.9 && \ + ./configure -prefix=/usr/local/ccache-3.7.9 && \ + make -j8 && make install && \ + ln -s /usr/local/ccache-3.7.9/bin/ccache /usr/local/bin/ccache + +# clang-form 3.8.0 +RUN wget https://paddle-ci.cdn.bcebos.com/clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz && \ + tar xf clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz && cd clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04 && \ + cp -r * /usr/local && cd .. && rm -rf clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04 && rm -rf clang+llvm-3.8.0-x86_64-linux-gnu-ubuntu-16.04.tar.xz + +EXPOSE 22 diff --git a/tools/dockerfile/ubuntu16_release.sh b/tools/dockerfile/ubuntu16_release.sh new file mode 100755 index 00000000000000..9d5d2881ccdd16 --- /dev/null +++ b/tools/dockerfile/ubuntu16_release.sh @@ -0,0 +1,118 @@ +#!/bin/bash + +# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +docker_name=$1 + + +function ref_whl(){ + if [[ ${WITH_GPU} == "ON" ]]; then + ref_gpu=gpu-cuda${ref_CUDA_MAJOR}-cudnn${CUDNN_MAJOR} + install_gpu="_gpu" + else + ref_gpu="cpu-avx" + install_gpu="" + fi + + if [[ ${WITH_MKL} == "ON" ]]; then + ref_mkl=mkl + else + ref_mkl=openblas + fi + + if [[ ${WITH_GPU} != "ON" ]]; then + ref_gcc="" + elif [[ ${gcc_version} == "8.2.0" ]];then + ref_gcc=-gcc8.2 + fi + + if [[ ${ref_CUDA_MAJOR} == "11.0" ]];then + ref_version=.post110 + elif [[ ${ref_CUDA_MAJOR} == "11.2" ]];then + ref_version=.post112 + elif [[ ${ref_CUDA_MAJOR} == "10" ]];then + ref_version=.post100 + elif [[ ${ref_CUDA_MAJOR} == "10.1" ]];then + ref_version=.post101 + elif [[ ${ref_CUDA_MAJOR} == "10.2" && ${PADDLE_VERSION} == "develop" ]];then + ref_version=.post102 + elif [[ ${ref_CUDA_MAJOR} == "10.2" && ${PADDLE_VERSION} != "develop" ]];then + ref_version="" + elif [[ ${ref_CUDA_MAJOR} == "9" ]];then + ref_version=.post90 + fi + + ref_dev=2.1.0.dev0 + + ref_web="https://paddle-wheel.bj.bcebos.com/${PADDLE_BRANCH}-${ref_gpu}-${ref_mkl}${ref_gcc}" + + if [[ ${PADDLE_VERSION} == "develop" && ${WITH_GPU} == "ON" ]]; then + ref_paddle37_whl=paddlepaddle${install_gpu}-${ref_dev}${ref_version}-cp37-cp37m-linux_x86_64.whl + elif [[ ${PADDLE_VERSION} == "develop" && ${WITH_GPU} != "ON" ]]; then + ref_paddle37_whl=paddlepaddle${install_gpu}-${ref_dev}-cp37-cp37m-linux_x86_64.whl + elif [[ ${PADDLE_VERSION} != "develop" && ${WITH_GPU} == "ON" ]]; then + ref_paddle37_whl=paddlepaddle${install_gpu}-${PADDLE_VERSION}${ref_version}-cp37-cp37m-linux_x86_64.whl + else + ref_paddle37_whl=paddlepaddle${install_gpu}-${PADDLE_VERSION}-cp37-cp37m-linux_x86_64.whl + fi +} + + +function install_whl(){ + dockerfile_line=`wc -l Dockerfile.tmp|awk '{print $1}'` + sed -i "${dockerfile_line}i RUN wget -q ${ref_web}/${ref_paddle37_whl} && pip3.7 install ${ref_paddle37_whl} && rm -f ${ref_paddle37_whl}" Dockerfile.tmp +} + + +function install_gcc(){ + if [ "${gcc_version}" == "8.2.0" ];then + sed -i 's##WORKDIR /usr/bin \ + COPY tools/dockerfile/build_scripts /build_scripts \ + RUN bash /build_scripts/install_gcc.sh gcc82 \&\& rm -rf /build_scripts \ + RUN cp gcc gcc.bak \&\& cp g++ g++.bak \&\& rm gcc \&\& rm g++ \ + RUN ln -s /usr/local/gcc-8.2/bin/gcc /usr/local/bin/gcc \ + RUN ln -s /usr/local/gcc-8.2/bin/g++ /usr/local/bin/g++ \ + RUN ln -s /usr/local/gcc-8.2/bin/gcc /usr/bin/gcc \ + RUN ln -s /usr/local/gcc-8.2/bin/g++ /usr/bin/g++ \ + ENV PATH=/usr/local/gcc-8.2/bin:$PATH #g' Dockerfile.tmp + else + sed -i 's##RUN apt-get update \ + WORKDIR /usr/bin \ + RUN apt install -y gcc g++ #g' Dockerfile.tmp + fi +} + + +# function install_jupyter() { +# if [[ ${WITH_NOTEBOOK} == "ON" ]];then +# # install jupyter notebook +# fi +# } + + +function make_dockerfile(){ + sed "s//${docker_name}/g" tools/dockerfile/Dockerfile.release16 >Dockerfile.tmp +} + + +function main(){ + make_dockerfile + install_gcc + # install_jupyter + ref_whl + install_whl +} + +main $@ diff --git a/tools/dockerfile/ubuntu18_release.sh b/tools/dockerfile/ubuntu18_release.sh new file mode 100755 index 00000000000000..216d8528200e57 --- /dev/null +++ b/tools/dockerfile/ubuntu18_release.sh @@ -0,0 +1,112 @@ +#!/bin/bash + +# Copyright (c) 2020 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +docker_name=$1 + + +function ref_whl(){ + if [[ ${WITH_GPU} == "ON" ]]; then + ref_gpu=gpu-cuda${ref_CUDA_MAJOR}-cudnn${CUDNN_MAJOR} + install_gpu="_gpu" + else + ref_gpu="cpu-avx" + install_gpu="" + fi + + if [[ ${WITH_MKL} == "ON" ]]; then + ref_mkl=mkl + else + ref_mkl=openblas + fi + + if [[ ${WITH_GPU} != "ON" ]]; then + ref_gcc="" + elif [[ ${gcc_version} == "8.2.0" ]];then + ref_gcc=-gcc8.2 + fi + + if [[ ${ref_CUDA_MAJOR} == "11.0" ]];then + ref_version=.post110 + elif [[ ${ref_CUDA_MAJOR} == "11.2" ]];then + ref_version=.post112 + elif [[ ${ref_CUDA_MAJOR} == "10" ]];then + ref_version=.post100 + elif [[ ${ref_CUDA_MAJOR} == "10.1" ]];then + ref_version=.post101 + elif [[ ${ref_CUDA_MAJOR} == "10.2" && ${PADDLE_VERSION} == "develop" ]];then + ref_version=.post102 + elif [[ ${ref_CUDA_MAJOR} == "10.2" && ${PADDLE_VERSION} != "develop" ]];then + ref_version="" + elif [[ ${ref_CUDA_MAJOR} == "9" ]];then + ref_version=.post90 + fi + + ref_dev=2.1.0.dev0 + + ref_web="https://paddle-wheel.bj.bcebos.com/${PADDLE_BRANCH}-${ref_gpu}-${ref_mkl}${ref_gcc}" + + if [[ ${PADDLE_VERSION} == "develop" && ${WITH_GPU} == "ON" ]]; then + ref_paddle37_whl=paddlepaddle${install_gpu}-${ref_dev}${ref_version}-cp37-cp37m-linux_x86_64.whl + elif [[ ${PADDLE_VERSION} == "develop" && ${WITH_GPU} != "ON" ]]; then + ref_paddle37_whl=paddlepaddle${install_gpu}-${ref_dev}-cp37-cp37m-linux_x86_64.whl + elif [[ ${PADDLE_VERSION} != "develop" && ${WITH_GPU} == "ON" ]]; then + ref_paddle37_whl=paddlepaddle${install_gpu}-${PADDLE_VERSION}${ref_version}-cp37-cp37m-linux_x86_64.whl + else + ref_paddle37_whl=paddlepaddle${install_gpu}-${PADDLE_VERSION}-cp37-cp37m-linux_x86_64.whl + fi +} + + +function install_whl(){ + dockerfile_line=`wc -l Dockerfile.tmp|awk '{print $1}'` + sed -i "${dockerfile_line}i RUN wget -q ${ref_web}/${ref_paddle37_whl} && pip3.7 install ${ref_paddle37_whl} && rm -f ${ref_paddle37_whl}" Dockerfile.tmp +} + + +function install_gcc(){ + if [ "${gcc_version}" == "8.2.0" ];then + sed -i 's##WORKDIR /usr/bin \ + COPY tools/dockerfile/build_scripts /build_scripts \ + RUN bash /build_scripts/install_trt.sh \ + RUN bash /build_scripts/install_gcc.sh gcc82 \&\& rm -rf /build_scripts \ + RUN cp gcc gcc.bak \&\& cp g++ g++.bak \&\& rm gcc \&\& rm g++ \ + RUN ln -s /usr/local/gcc-8.2/bin/gcc /usr/local/bin/gcc \ + RUN ln -s /usr/local/gcc-8.2/bin/g++ /usr/local/bin/g++ \ + RUN ln -s /usr/local/gcc-8.2/bin/gcc /usr/bin/gcc \ + RUN ln -s /usr/local/gcc-8.2/bin/g++ /usr/bin/g++ \ + ENV PATH=/usr/local/gcc-8.2/bin:$PATH #g' Dockerfile.tmp + else + sed -i 's##RUN apt-get update \ + WORKDIR /usr/bin \ + RUN apt install -y gcc g++ #g' Dockerfile.tmp + fi +} + + + +function make_dockerfile(){ + sed "s//${docker_name}/g" tools/dockerfile/Dockerfile.release18 >Dockerfile.tmp +} + + +function main(){ + make_dockerfile + install_gcc + ref_whl + install_whl +} + +main $@ diff --git a/tools/get_single_test_cov.py b/tools/get_single_test_cov.py index 088471364f21d0..421962bb584974 100644 --- a/tools/get_single_test_cov.py +++ b/tools/get_single_test_cov.py @@ -37,24 +37,47 @@ def getFNDAFile(rootPath, test): def analysisFNDAFile(rootPath, test): - ut_map_file = '%s/build/ut_map/%s/%s.txt' % (rootPath, test, test) - os.system('touch %s' % ut_map_file) + related_ut_map_file = '%s/build/ut_map/%s/related_%s.txt' % (rootPath, test, + test) + notrelated_ut_map_file = '%s/build/ut_map/%s/notrelated_%s.txt' % ( + rootPath, test, test) + os.system('touch %s' % related_ut_map_file) + os.system('touch %s' % notrelated_ut_map_file) fn_filename = '%s/build/ut_map/%s/fnda.tmp' % (rootPath, test) f = open(fn_filename) data = f.read().split('SF:') + related_file_list = [] for message in data: + message_list = message.split('\n') + clazz_filename = message_list[0] + if '/build/' in clazz_filename: + clazz_filename = clazz_filename.replace('/build', '') + if '.pb.h' in clazz_filename: + clazz_filename = clazz_filename.replace('.pb.h', '.proto') + if '.pb.cc' in clazz_filename: + clazz_filename = clazz_filename.replace('.pb.cc', '.proto') if 'FNDA:' in message: - message_list = message.split('\n') - clazz_filename = message_list[0] - #if not clazz_filename.endswith('.h'): #filter .h's Analysis + OP_REGIST = True for i in range(1, len(message_list) - 1): fn = message_list[i] matchObj = re.match( r'(.*)Maker(.*)|(.*)Touch(.*)Regist(.*)|(.*)Touch(.*)JitKernel(.*)|(.*)converterC2Ev(.*)', fn, re.I) if matchObj == None: - os.system('echo %s >> %s' % (clazz_filename, ut_map_file)) + OP_REGIST = False break + if OP_REGIST == False: + related_file_list.append(clazz_filename) + os.system('echo %s >> %s' % + (clazz_filename, related_ut_map_file)) + else: + os.system('echo %s >> %s' % + (clazz_filename, notrelated_ut_map_file)) + else: + if clazz_filename != '': + if clazz_filename not in related_file_list: # xx.pb.cc in RELATED xx.pb.h not in RELATED + os.system('echo %s >> %s' % + (clazz_filename, notrelated_ut_map_file)) f.close() @@ -64,7 +87,7 @@ def getCovinfo(rootPath, test): 'cd %s && lcov --capture -d . -o coverage.info --rc lcov_branch_coverage=0 > /dev/null 2>&1' % ut_map_path) os.system( - "cd %s && lcov --extract coverage.info '/paddle/paddle/fluid/framework/*' '/paddle/paddle/fluid/imperative/*' '/paddle/paddle/fluid/inference/*' '/paddle/paddle/fluid/memory/*' '/paddle/paddle/fluid/operators/*' '/paddle/paddle/fluid/string/*' '/paddle/paddle/fluid/distributed/*' '/paddle/paddle/fluid/extension/*' '/paddle/paddle/fluid/platform/*' '/paddle/paddle/fluid/pybind/*' -o coverage.info.tmp --rc lcov_branch_coverage=0 > /dev/null 2>&1" + "cd %s && lcov --extract coverage.info '/paddle/paddle/fluid/framework/*' '/paddle/paddle/fluid/imperative/*' '/paddle/paddle/fluid/inference/*' '/paddle/paddle/fluid/memory/*' '/paddle/paddle/fluid/operators/*' '/paddle/paddle/fluid/string/*' '/paddle/paddle/fluid/distributed/*' '/paddle/paddle/fluid/extension/*' '/paddle/paddle/fluid/platform/*' '/paddle/paddle/fluid/pybind/*' '/paddle/build/*' -o coverage.info.tmp --rc lcov_branch_coverage=0 > /dev/null 2>&1" % ut_map_path) os.system('rm -rf %s/paddle' % ut_map_path) os.system('rm -rf %s/coverage.info' % ut_map_path) diff --git a/tools/get_ut_file_map.py b/tools/get_ut_file_map.py index 59325b91d8e0ea..eaa1f3c5405ce5 100644 --- a/tools/get_ut_file_map.py +++ b/tools/get_ut_file_map.py @@ -20,7 +20,7 @@ def get_all_paddle_file(rootPath): """get all file in Paddle repo: paddle/fluild, python""" - traverse_files = ['%s/paddle/fluid' % rootPath, '%s/python' % rootPath] + traverse_files = ['%s' % rootPath] all_file_paddle = '%s/build/all_file_paddle' % rootPath all_file_paddle_list = [] with open(all_file_paddle, 'w') as f: @@ -56,7 +56,7 @@ def remove_useless_file(rootPath): def handle_ut_file_map(rootPath): - utNotSuccess = '' + utNotSuccess_list = [] ut_map_path = "%s/build/ut_map" % rootPath files = os.listdir(ut_map_path) ut_file_map = {} @@ -67,7 +67,7 @@ def handle_ut_file_map(rootPath): print("ut %s: %s" % (count, ut)) coverage_info = '%s/%s/coverage.info.tmp' % (ut_map_path, ut) if os.path.exists(coverage_info): - filename = '%s/%s/%s.txt' % (ut_map_path, ut, ut) + filename = '%s/%s/related_%s.txt' % (ut_map_path, ut, ut) f = open(filename) lines = f.readlines() for line in lines: @@ -86,19 +86,33 @@ def handle_ut_file_map(rootPath): ut_file_map[source_file] = [] if ut not in ut_file_map[source_file]: ut_file_map[source_file].append(ut) - else: not_success_file.write('%s\n' % ut) - utNotSuccess = utNotSuccess + '^%s$|' % ut - + utNotSuccess_list.append(ut) not_success_file.close() + print("utNotSuccess:") + print(utNotSuccess_list) + + for ut in files: + if ut not in utNotSuccess_list: + filename = '%s/%s/notrelated_%s.txt' % (ut_map_path, ut, ut) + f = open(filename) + lines = f.readlines() + for line in lines: + line = line.replace('\n', '').strip() + if line == '': + continue + elif line.startswith('/paddle/build'): + source_file = line.replace('/build', '') + else: + source_file = line + if source_file not in ut_file_map: + ut_file_map[source_file] = [] + with open("%s/build/ut_file_map.json" % rootPath, "w") as f: json.dump(ut_file_map, f, indent=4) - print("utNotSuccess:") - print(utNotSuccess) - def notsuccessfuc(rootPath): utNotSuccess = '' @@ -153,10 +167,7 @@ def ut_file_map_supplement(rootPath): for filename in load_dict_old: if filename not in load_dict_new: - if filename.endswith(('.h')): - load_dict_new[filename] = [] - else: - load_dict_new[filename] = load_dict_old[filename] + load_dict_new[filename] = load_dict_old[filename] with open("/pre_test/ut_file_map.json", "w") as f: json.dump(load_dict_new, f, indent=4) @@ -182,6 +193,8 @@ def ut_file_map_supplement(rootPath): if ut in all_uts_paddle_list: if not os.path.exists(filename) and ut not in prec_delta_new_list: prec_delta_new_list.append(ut) + prec_delta_new_list.append( + 'test_py_reader_error_msg') #add a python case for pycoverage prec_delta_file = open("/pre_test/prec_delta", 'w') for ut in prec_delta_new_list: prec_delta_file.write(ut + '\n') @@ -189,6 +202,15 @@ def ut_file_map_supplement(rootPath): prec_delta_file.close() +def utmap_analysis(rootPath): + ut_file_map_new = "%s/build/ut_file_map.json" % rootPath + with open(ut_file_map_new, 'r') as load_f: + load_dict_new = json.load(load_f) + print(len(load_dict_new)) + for filename in load_dict_new: + print(filename, len(load_dict_new[filename])) + + if __name__ == "__main__": func = sys.argv[1] if func == 'get_not_success_ut': diff --git a/tools/handle_h_cu_file.py b/tools/handle_h_cu_file.py index eb66a3d1dc48d6..ea01a1d8d41514 100644 --- a/tools/handle_h_cu_file.py +++ b/tools/handle_h_cu_file.py @@ -85,7 +85,7 @@ def get_h_cu_file(file_path): filename = file_path[2] ut = filename.replace('^', '').replace('$', '').replace('.log', '') os.system( - "cat %s/%s | grep 'precise test map fileeee:'| uniq >> %s/build/ut_map/%s/%s.txt" + "cat %s/%s | grep 'precise test map fileeee:'| uniq >> %s/build/ut_map/%s/related_%s.txt" % (dir_path, filename, rootPath, ut, ut)) diff --git a/tools/parallel_UT_rule.py b/tools/parallel_UT_rule.py index dbb77d07d5accc..5108d34f7bf779 100644 --- a/tools/parallel_UT_rule.py +++ b/tools/parallel_UT_rule.py @@ -665,7 +665,6 @@ 'convert_model2dot_ernie', 'im2col_test', 'test_logical_op', - 'test_imperative_mnist', 'test_imperative_deepcf', 'test_cholesky_op', 'test_sample_logits_op', diff --git a/tools/print_signatures.py b/tools/print_signatures.py index 3f0a3e834f3d32..65e7c7e0efcb86 100644 --- a/tools/print_signatures.py +++ b/tools/print_signatures.py @@ -17,19 +17,14 @@ Usage: ./print_signature "paddle.fluid" > signature.txt """ -from __future__ import print_function -import importlib import inspect import collections import sys -import pydoc import hashlib -import platform -import functools import pkgutil import logging -import paddle +import argparse member_dict = collections.OrderedDict() @@ -79,9 +74,7 @@ def is_primitive(instance): ErrorSet = set() IdSet = set() -skiplist = [ - 'paddle.vision.datasets.DatasetFolderImageFolder', 'paddle.truncdigamma' -] +skiplist = [] def visit_all_module(mod): @@ -100,21 +93,20 @@ def visit_all_module(mod): if hasattr(mod, "__all__"): member_names += mod.__all__ for member_name in member_names: - if member_name.startswith('__'): + if member_name.startswith('_'): continue cur_name = mod_name + '.' + member_name + if cur_name in skiplist: + continue try: instance = getattr(mod, member_name) if inspect.ismodule(instance): visit_all_module(instance) else: - doc_md5 = md5(instance.__doc__) instance_id = id(instance) if instance_id in IdSet: continue IdSet.add(instance_id) - member_dict[cur_name] = "({}, ('document', '{}'))".format( - cur_name, doc_md5) if hasattr(instance, '__name__') and member_name != instance.__name__: print( @@ -135,6 +127,7 @@ def get_all_api(root_path='paddle', attr="__all__"): """ walk through the paddle package to collect all the apis. """ + import paddle global api_info_dict api_counter = 0 for filefinder, name, ispkg in pkgutil.walk_packages( @@ -157,7 +150,8 @@ def get_all_api(root_path='paddle', attr="__all__"): logger.info('%s: collected %d apis, %d distinct apis.', attr, api_counter, len(api_info_dict)) - return [api_info['all_names'][0] for api_info in api_info_dict.values()] + return [(sorted(list(api_info['all_names']))[0], md5(api_info['docstring'])) + for api_info in api_info_dict.values()] def insert_api_into_dict(full_name, gen_doc_anno=None): @@ -185,6 +179,7 @@ def insert_api_into_dict(full_name, gen_doc_anno=None): "id": fc_id, "object": obj, "type": type(obj).__name__, + "docstring": '', } docstr = inspect.getdoc(obj) if docstr: @@ -221,7 +216,84 @@ def process_module(m, attr="__all__"): return api_counter -def get_all_api_from_modulelist(): +def check_public_api(): + import paddle + modulelist = [ #npqa + paddle, + paddle.amp, + paddle.nn, + paddle.nn.functional, + paddle.nn.initializer, + paddle.nn.utils, + paddle.static, + paddle.static.nn, + paddle.io, + paddle.jit, + paddle.metric, + paddle.distribution, + paddle.optimizer, + paddle.optimizer.lr, + paddle.regularizer, + paddle.text, + paddle.utils, + paddle.utils.download, + paddle.utils.profiler, + paddle.utils.cpp_extension, + paddle.sysconfig, + paddle.vision, + paddle.vision.datasets, + paddle.vision.models, + paddle.vision.transforms, + paddle.vision.ops, + paddle.distributed, + paddle.distributed.fleet, + paddle.distributed.fleet.utils, + paddle.distributed.parallel, + paddle.distributed.utils, + paddle.callbacks, + paddle.hub, + paddle.autograd, + paddle.incubate, + paddle.inference, + paddle.onnx, + paddle.device + ] + + apinum = 0 + alldict = {} + for module in modulelist: + if hasattr(module, '__all__'): + old_all = module.__all__ + else: + old_all = [] + dirall = dir(module) + for item in dirall: + if item.startswith('__'): + continue + old_all.append(item) + apinum += len(old_all) + alldict.update({module.__name__: old_all}) + + old_all = [] + dirall = dir(paddle.Tensor) + for item in dirall: + if item.startswith('_'): + continue + old_all.append(item) + apinum += len(old_all) + alldict.update({'paddle.Tensor': old_all}) + + for module, allapi in alldict.items(): + for member_name in allapi: + cur_name = module + '.' + member_name + instance = eval(cur_name) + doc_md5 = md5(instance.__doc__) + member_dict[cur_name] = "({}, ('document', '{}'))".format(cur_name, + doc_md5) + + +def check_allmodule_callable(): + import paddle modulelist = [paddle] for m in modulelist: visit_all_module(m) @@ -229,15 +301,49 @@ def get_all_api_from_modulelist(): return member_dict +def parse_args(): + """ + Parse input arguments + """ + parser = argparse.ArgumentParser(description='Print Apis Signatures') + parser.add_argument('--debug', dest='debug', action="store_true") + parser.add_argument( + '--method', + dest='method', + type=str, + default='from_modulelist', + help="using get_all_api or from_modulelist") + parser.add_argument( + 'module', type=str, help='module', default='paddle') # not used + + if len(sys.argv) == 1: + args = parser.parse_args(['paddle']) + return args + # parser.print_help() + # sys.exit(1) + + args = parser.parse_args() + return args + + if __name__ == '__main__': - get_all_api_from_modulelist() + args = parse_args() + check_allmodule_callable() + if args.method == 'from_modulelist': + check_public_api() + for name in member_dict: + print(name, member_dict[name]) + elif args.method == 'get_all_api': + api_signs = get_all_api() + for api_sign in api_signs: + print("{0} ({0}, ('document', '{1}'))".format(api_sign[0], api_sign[ + 1])) - for name in member_dict: - print(name, member_dict[name]) if len(ErrorSet) == 0: sys.exit(0) - for erroritem in ErrorSet: - print( - "Error, new function {} is unreachable".format(erroritem), - file=sys.stderr) - sys.exit(1) + else: + for erroritem in ErrorSet: + print( + "Error, new function {} is unreachable".format(erroritem), + file=sys.stderr) + sys.exit(1) diff --git a/tools/remove_grad_op_and_kernel.py b/tools/remove_grad_op_and_kernel.py index 85bbf8cdddc29d..80314f2c3c583a 100644 --- a/tools/remove_grad_op_and_kernel.py +++ b/tools/remove_grad_op_and_kernel.py @@ -124,7 +124,7 @@ def update_operator_cmake(cmake_file): custom_pattern2 = custom_pattern2[:-1] all_matches = [] - with open(op_file, 'r') as f: + with open(op_file, 'r', encoding='utf-8') as f: content = ''.join(f.readlines()) op, op_count = remove_grad_op_and_kernel(content, op_pattern1, diff --git a/tools/sampcd_processor.py b/tools/sampcd_processor.py index 07f112a5614a32..3ec12c11a7045a 100644 --- a/tools/sampcd_processor.py +++ b/tools/sampcd_processor.py @@ -27,7 +27,6 @@ import multiprocessing import platform import inspect -import json import argparse import shutil import re @@ -443,7 +442,7 @@ def get_filenames(full_test=False): import paddle whl_error = [] if full_test: - get_full_api() + get_full_api_from_pr_spec() else: get_incrementapi() all_sample_code_filenames = {} @@ -513,7 +512,20 @@ def get_full_api_by_walk(): from print_signatures import get_all_api apilist = get_all_api() with open(API_DIFF_SPEC_FN, 'w') as f: - f.write("\n".join(apilist)) + f.write("\n".join([ai[0] for ai in apilist])) + + +def get_full_api_from_pr_spec(): + """ + get all the apis + """ + global API_PR_SPEC_FN, API_DIFF_SPEC_FN ## readonly + pr_api = get_api_md5(API_PR_SPEC_FN) + if len(pr_api): + with open(API_DIFF_SPEC_FN, 'w') as f: + f.write("\n".join(pr_api.keys())) + else: + get_full_api_by_walk() def get_incrementapi(): diff --git a/tools/static_mode_white_list.py b/tools/static_mode_white_list.py index 075d1a16927ad2..09029b6ad821ee 100644 --- a/tools/static_mode_white_list.py +++ b/tools/static_mode_white_list.py @@ -468,6 +468,7 @@ 'test_sign_op', 'test_similarity_focus_op', 'test_size_op', + 'test_share_data_op', 'test_smooth_l1_loss', 'test_smooth_l1_loss_op', 'test_softmax_with_cross_entropy_op', @@ -475,6 +476,8 @@ 'test_split_and_merge_lod_tensor_op', 'test_split_ids_op', 'test_split_op', + 'test_split_mkldnn_op', + 'test_split_bf16_mkldnn_op', 'test_spp_op', 'test_square_error_cost', 'test_squared_l2_norm_op', @@ -622,6 +625,7 @@ 'test_lrn_mkldnn_op', 'test_matmul_mkldnn_op', 'test_matmul_bf16_mkldnn_op', + 'test_matmul_v2_mkldnn_op', 'test_mul_int8_mkldnn_op', 'test_multi_gru_mkldnn_op', 'test_multi_gru_fuse_pass', diff --git a/tools/test_check_pr_approval.py b/tools/test_check_pr_approval.py new file mode 100644 index 00000000000000..f4c089ee0f8720 --- /dev/null +++ b/tools/test_check_pr_approval.py @@ -0,0 +1,120 @@ +#! /usr/bin/env python + +# Copyright (c) 2021 PaddlePaddle Authors. All Rights Reserved. +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +""" +TestCases for check_pr_approval.py +""" +import unittest +import subprocess +import sys + + +class Test_check_approval(unittest.TestCase): + def setUp(self): + self.codeset = 'UTF-8' + # only key info in it + self.jsonstr = """ +[ + { + "id": 688077074, + "node_id": "MDE3OlB1bGxSZXF1ZXN0UmV2aWV3Njg4MDc3MDc0", + "user": { + "login": "wadefelix", + "id": 1306724, + "type": "User", + "site_admin": false + }, + "body": "", + "state": "COMMENTED", + "author_association": "CONTRIBUTOR" + }, + { + "id": 688092580, + "node_id": "MDE3OlB1bGxSZXF1ZXN0UmV2aWV3Njg4MDkyNTgw", + "user": { + "login": "MingMingShangTian", + "id": 13469016, + "type": "User", + "site_admin": false + }, + "body": "LGTM", + "state": "APPROVED", + "author_association": "CONTRIBUTOR" + }, + { + "id": 689175539, + "node_id": "MDE3OlB1bGxSZXF1ZXN0UmV2aWV3Njg5MTc1NTM5", + "user": { + "login": "pangyoki", + "id": 26408901, + "type": "User", + "site_admin": false + }, + "body": "LGTM", + "state": "APPROVED", + "author_association": "CONTRIBUTOR" + } +] +""".encode(self.codeset) + + def test_ids(self): + cmd = [sys.executable, 'check_pr_approval.py', '1', '26408901'] + subprc = subprocess.Popen( + cmd, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + output, error = subprc.communicate(input=self.jsonstr) + self.assertEqual('TRUE', output.decode(self.codeset).rstrip()) + + def test_logins(self): + cmd = [sys.executable, 'check_pr_approval.py', '1', 'pangyoki'] + subprc = subprocess.Popen( + cmd, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + output, error = subprc.communicate(input=self.jsonstr) + self.assertEqual('TRUE', output.decode(self.codeset).rstrip()) + + def test_ids_and_logins(self): + cmd = [ + sys.executable, 'check_pr_approval.py', '2', 'pangyoki', '13469016' + ] + subprc = subprocess.Popen( + cmd, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + output, error = subprc.communicate(input=self.jsonstr) + #self.assertEqual('', error.rstrip()) + self.assertEqual('TRUE', output.decode(self.codeset).rstrip()) + + def test_check_with_required_reviewer_not_approved(self): + cmd = [ + sys.executable, 'check_pr_approval.py', '2', 'wadefelix', + ' 13469016' + ] + subprc = subprocess.Popen( + cmd, + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE) + output, error = subprc.communicate(input=self.jsonstr) + self.assertEqual('FALSE', output.decode(self.codeset).rstrip()) + + +if __name__ == '__main__': + unittest.main() diff --git a/tools/test_print_signatures.py b/tools/test_print_signatures.py index 7cbdbb56cb1b10..1ca1e4149fb7e5 100644 --- a/tools/test_print_signatures.py +++ b/tools/test_print_signatures.py @@ -23,13 +23,9 @@ """ import unittest import hashlib -import inspect import functools from print_signatures import md5 -from print_signatures import get_functools_partial_spec -from print_signatures import format_spec -from print_signatures import queue_dict -from print_signatures import member_dict +from print_signatures import is_primitive def func_example(param_a, param_b): @@ -65,30 +61,27 @@ def test_md5(self): digest = algo.hexdigest() self.assertEqual(digest, md5(func_example.__doc__)) - def test_get_functools_partial_spec(self): - partailed_func = functools.partial(func_example, 1) - # args = inspect.getargspec(partailed_func) - self.assertEqual('func_example(args=(1,), keywords={})', - get_functools_partial_spec(partailed_func)) - -class Test_format_spec(unittest.TestCase): - def test_normal_func_spec(self): - args = inspect.getargspec(func_example) - self.assertEqual( - '''ArgSpec(args=['param_a', 'param_b'], varargs=None, keywords=None, defaults=None)''', - format_spec(args)) - - def test_func_spec_with_partialedfunc_as_param_default(self): - # but there is no function belongs to this type in API_DEV.spec - args = inspect.getargspec(func_example_2) - self.assertEqual( - '''ArgSpec(args=['func'], varargs=None, keywords=None, defaults=('func_example(args=(1,), keywords={})',))''', - format_spec(args)) - - -class Test_queue_dict(unittest.TestCase): - pass +class Test_is_primitive(unittest.TestCase): + def test_single(self): + self.assertTrue(is_primitive(2)) + self.assertTrue(is_primitive(2.1)) + self.assertTrue(is_primitive("2.1.1")) + self.assertFalse( + is_primitive("hello paddle".encode('UTF-8'))) # True for python2 + self.assertFalse(is_primitive(1j)) + self.assertTrue(is_primitive(True)) + + def test_collection(self): + self.assertTrue(is_primitive([])) + self.assertTrue(is_primitive(tuple())) + self.assertTrue(is_primitive(set())) + self.assertTrue(is_primitive([1, 2])) + self.assertTrue(is_primitive((1.1, 2.2))) + self.assertTrue(is_primitive(set([1, 2.3]))) + self.assertFalse(is_primitive(range(3))) # True for python2 + self.assertFalse(is_primitive({})) + self.assertFalse(is_primitive([1, 1j])) if __name__ == '__main__': diff --git a/tools/test_sampcd_processor.py b/tools/test_sampcd_processor.py index 81710dae167642..8963ae35f6b44c 100644 --- a/tools/test_sampcd_processor.py +++ b/tools/test_sampcd_processor.py @@ -16,10 +16,7 @@ import unittest import os -import tempfile import shutil -import sys -import importlib import re import sampcd_processor from sampcd_processor import find_all