Skip to content

[CI] Support multi-Python version build (3.10/3.11/3.12)#7230

Open
EmmonsCurse wants to merge 6 commits into
PaddlePaddle:developfrom
EmmonsCurse:add_multi_python
Open

[CI] Support multi-Python version build (3.10/3.11/3.12)#7230
EmmonsCurse wants to merge 6 commits into
PaddlePaddle:developfrom
EmmonsCurse:add_multi_python

Conversation

@EmmonsCurse
Copy link
Copy Markdown
Collaborator

Motivation

Support multi-Python version build (3.10/3.11/3.12)

Modifications

  • Add PYTHON_VERSION input parameter to _build_linux.yml
  • Switch Python env via LD_LIBRARY_PATH/PATH in container
  • Isolate CCache and pip cache by Python version

Usage or Command

N/A

Accuracy Tests

N/A

Checklist

  • Add at least a tag in the PR title.
    • Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
    • You can add new tags based on the PR content, but the semantics must be clear.
  • Format your code, run pre-commit before commit.
  • Add unit tests. Please write the reason in this PR if no unit tests.
  • Provide accuracy results.
  • If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

@paddle-bot
Copy link
Copy Markdown

paddle-bot Bot commented Apr 7, 2026

Thanks for your contribution!

@EmmonsCurse
Copy link
Copy Markdown
Collaborator Author

/skip-ci ci_iluvatar
/skip-ci ci_hpu
/skip-ci build_xpu

PaddlePaddle-bot

This comment was marked as outdated.

PaddlePaddle-bot

This comment was marked as outdated.

@PaddlePaddle-bot
Copy link
Copy Markdown

PaddlePaddle-bot commented May 12, 2026

🤖 Paddle-CI-Agent | ci_status_monitor | 2026-05-17 21:10:50

CI报告基于以下代码生成(30分钟更新一次):


1 任务总览

所有 Required 任务均已通过(2/2),PR 可合并。有 1 个 Optional 任务失败,仅供参考,不阻塞合并。

总执行(rerun次数) 总任务 ✅ 通过 ❌ 失败 ⏳ 运行中 ⏸️ 等待中 跳过
14(0) 14 13 1 0 0 0

2 任务状态汇总

2.1 Required任务 : 2/2 通过

必选任务阻塞合并,失败需优先处理。

状态 任务 耗时 根因 修复建议 日志 重跑
其余 2 个必选任务通过 - - - - -

2.2 可选任务 — 11/12 通过

可选任务不阻塞合并,失败仅供参考。

状态 任务 耗时 日志 重跑
Trigger Jenkins for PR 10m50s Job -
其余 11 个可选任务通过 - - -

3 失败详情(仅 required)

无 required 失败任务。

PaddlePaddle-bot

This comment was marked as outdated.

Copy link
Copy Markdown

@PaddlePaddle-bot PaddlePaddle-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Paddle-CI-Agent | pr_review | 2026-05-17 18:45:34

📋 Review 摘要

PR 概述:为 FastDeploy CI 新增多 Python 版本(3.10/3.11/3.12)并行构建能力,通过 PYTHON_VERSION 参数在 manylinux 容器内切换 Python 环境。
变更范围.github/workflows/(CI 构建工作流)
影响面 Tag[CI]

问题

级别 文件 概述
🟡 建议 _build_linux.yml:230 pip install 追加了 `
❓ 疑问 _build_linux.yml:186 Python 路径硬编码具体补丁版本(3.11.0 / 3.12.0),容器实际版本不符时静默用错 Python
❓ 疑问 ci_metax.yml:3 on: 关键字本身被注释掉,workflow 失去所有触发条件(含 workflow_dispatch),成为无效文件
🟡 建议 pr_build_and_test.yml 6 个测试 job(unit test / logprob / pre_ce / base / stable / 4-cards)全部注释掉,PR CI 仅剩构建,覆盖度大幅下降

注:_build_linux_cu129.yml_build_linux_cu130.yml 存在相同的路径硬编码和 || true 问题,修复时需同步。

📝 PR 规范检查

标题使用官方 [CI] tag,描述结构完整(Motivation / Modifications / Usage or Command / Accuracy Tests / Checklist 均存在),合规,无需修改。

总体评价

多 Python 版本构建思路清晰,但有 3 处可靠性问题需关注:pip 失败被静默吞掉会让排查困难;cpython 路径硬编码补丁版本号存在版本漂移风险;ci_metax.yml 的 on: 关键字被整体注释后工作流完全失效。建议修复后再合入,同时评估是否有计划恢复被注释的测试 job。

Copy link
Copy Markdown

@PaddlePaddle-bot PaddlePaddle-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

补充行内评论(对应上方 Review 中的 3 条问题)。

# Skip cp310-specific wheels for non-3.10 Python versions
if [[ "${PYTHON_VERSION}" != "3.10" ]]; then
grep -v "cp310-cp310" requirements.txt > requirements_filtered.txt
python -m pip install -r requirements_filtered.txt || true
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 建议 || true 会吞掉 pip install 的失败退出码,依赖未安装时构建静默继续,最终以难以定位的 ImportError 告终。建议去掉 || true 让安装失败直接终止构建;如果部分包在 3.11/3.12 下确实不可用,应将它们明确加入 grep 过滤条件。同样问题存在于 _build_linux_cu129.yml_build_linux_cu130.yml,需一并修复。

-e "no_proxy=localhost,127.0.0.1,0.0.0.0,bcebos.com,.bcebos.com,bj.bcebos.com,su.bcebos.com,paddle-ci.gz.bcebos.com,apiin.im.baidu.com,baidu-int.com,.baidu.com,aliyun.com,gitee.com,pypi.tuna.tsinghua.edu.cn,.tuna.tsinghua.edu.cn" \
--gpus "\"device=${gpu_id}\"" ${docker_image} /bin/bash -c '
# Switch Python environment based on PYTHON_VERSION
if [[ "${PYTHON_VERSION}" == "3.11" ]]; then
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ 疑问 路径 /opt/_internal/cpython-3.11.0/ 硬编码了具体补丁版本号。若镜像实际安装的是 3.11.5 等更高补丁版本,export 会指向不存在的路径,PATH 前缀静默失效后回落到镜像默认 Python,导致实际版本与预期不符且无任何报错。建议切换后立即校验:python3 --version | grep -q "3.11" || { echo "ERROR: wrong python version"; exit 1; }。同样问题存在于 cu129/cu130 变体。

branches:
- develop
- release/**
#on:
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ 疑问 on: 关键字本身被注释掉(#on:),而 ci_hpu/iluvatar/xpu 的做法是保留 on: 关键字只注释 pull_request 子项并保留 workflow_dispatch。ci_metax.yml 目前无任何 on: 顶层键,GitHub Actions 将其视为无效工作流,既无法自动触发也无法手动触发。如果只是临时禁用 PR 触发,建议改为:on: workflow_dispatch:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants