Add gcu llama readme#8950
Conversation
|
Thanks for your contribution! |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #8950 +/- ##
===========================================
+ Coverage 54.06% 54.14% +0.08%
===========================================
Files 650 650
Lines 103883 103883
===========================================
+ Hits 56164 56250 +86
+ Misses 47719 47633 -86 ☔ View full report in Codecov by Sentry. 🚨 Try these New Features:
|
| @@ -0,0 +1,162 @@ | |||
| ## 🚣♂️ 使用PaddleNLP在GCU下运行llama2-13b模型 🚣 | |||
|
|
|||
| PaddleNLP在燧原GCU（[了解燧原](https://www.enflame-tech.com/)）上对llama2-13B模型进行了深度适配和优化，实现了燧原GCU推理入口和GPU的基本统一，仅需修改device即可完成推理任务的迁移。 | |||
There was a problem hiding this comment.
已添加S60加速卡基本情况介绍，并附官方链接
| ### 1. 环境准备：(这将花费您10～20min时间) | ||
|
|
||
| 1. 初始化环境，安装驱动<br/> | ||
| **注：您可以联系燧原工程师以获取软件驱动包和其他帮助** |
| cd PaddleNLP | ||
| python -m pip install -r requirements.txt | ||
| # 源码编译安装 paddlenlp | ||
| python setup.py bdist_wheel && python -m pip uninstall paddlenlp -y && python -m pip install dist/paddlenlp* |
There was a problem hiding this comment.
推荐一个固定版本的PaddleNLP版本会不会更好，然后再提供 git clone & python setup.py install 的方式
There was a problem hiding this comment.
推荐使用PaddleNLP release版本v3.0.0-beta0，已更新步骤描述
| tar -zxf wikitext-103.tar.gz | ||
| ``` | ||
| ### 3. 推理：(这将花费您15~30min时间) | ||
| 下载权重文件后需要更改推理配置文件，以获取更大的性能提升。将```/root/.paddlenlp/models/__internal_testing__/sci-benchmark-llama-13b-5k/config.json```更改为下面的内容： |
There was a problem hiding this comment.
默认是不会提供 model config.json的？这里提供下怎么下载 internal_testing/sci-benchmark-llama-13b-5k 的模型方式？
There was a problem hiding this comment.
首次推理将自动下载权重和配置，已更新描述
EnflameGCU
force-pushed
the
gcu_llama_readme
branch
2 times, most recently
from
f3a7669 to
16f869c
Compare
EnflameGCU
force-pushed
the
gcu_llama_readme
branch
from
16f869c to
6116026
Compare
| 2. 拉取镜像 | ||
| ```bash | ||
| # 注意此镜像仅为paddle开发环境，镜像中不包含预编译的飞桨安装包、TopsPlatform安装包等 | ||
| docker pull registry.baidubce.com/paddlepaddle/paddle:latest-dev |
There was a problem hiding this comment.
这里建议限定可运行版本，latest-dev不是好的tag
| ``` | ||
| 3. 参考如下命令启动容器 | ||
| ```bash | ||
| docker run --name paddle-gcu-test -v /home:/home --network=host --ipc=host -it --privileged registry.baidubce.com/paddlepaddle/paddle:latest-dev /bin/bash |
There was a problem hiding this comment.
此处docker启动命令也需按照对应镜像名称修改。
3 participants
PR types
New features
PR changes
Docs
Description
Add gcu llama2-13b readme