[TPU][Bugfix] fix OOM issue in CI test

[yaochengji]

Purpose

Fix the tpu ci test: tests/v1/tpu/test_basic.py. In #21340, it got merged without running the TPU CI test.

Test Plan

pytest -s -v tests/v1/tpu/test_basic.py

Test Result

Passed

(Optional) Documentation Update

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[gemini-code-assist]

Code Review

This pull request addresses an OOM error in a TPU CI test by increasing the gpu_memory_utilization from 0.7 to 0.95. While this change may resolve the immediate issue, setting the memory utilization to such a high value introduces a risk of test flakiness and future maintenance problems. My review highlights this concern and suggests considering more robust solutions.

[QiliangCui]

This change broke the CI test again.

on main branch, one PR before this https://buildkite.com/vllm/ci/builds/24936 passed all tests.

this change caused OOM issue.

ERROR 07-25 07:24:51 [core.py:634] RuntimeError: Bad StatusOr access: RESOURCE_EXHAUSTED: Error loading program: Attempting to reserve 1.20G at the bottom of memory. That was not possible. There are 772.53M free, 0B reserved, and 772.53M reservable.

I will prepare a change to rollback this.