[Hardware][IBM Z] Optimize s390x Dockerfile

[R3hankhan123]

Purpose

This PR optimizes the s390x Dockerfile to improve build efficiency and maintainability:

1. Simplified compiler toolchain: Replaced individual GCC packages with gcc-toolset-14 for a more streamlined installation
2. Updated PyTorch: Removed custom PyTorch 2.7.0 build stage and switched to official PyTorch 2.8.0 CPU wheels
3. Updated torchvision: Upgraded from v0.20.1to v0.23.0 to match PyTorch 2.8.0
4. Removed AWS-LC-sys patches: Eliminated patch for aws-lc as fix has gone into the repo
5. Dynamic version detection: Replaced hardcoded version strings with automatic extraction from requirements files (e.g., outlines_core version now reads from requirements/common.txt)
6. Extended platform support: Added s390x to the platform conditions for llguidance and xgrammar in requirements/common.txt, enabling these structured output dependencies on s390x architecture

Test Plan

1. Build the s390x Docker image: docker build -f docker/Dockerfile.s390x -t vllm-cpu-s390x .
2. Verify all build stages complete successfully
3. Run the container and test basic vLLM functionality

Test Result

[root@b314lp50 ~]# podman run -it --rm   -p 8000:8000   --name vllm-gpt2   --entrypoint vllm   quay.io/r3hankhan/vllm:torch-2.8.0   serve gpt2  --host 0.0.0.0 --port 8000
INFO 11-04 05:42:44 [importing.py:68] Triton not installed or not compatible; certain GPU-related functions will not be available.
(APIServer pid=1) INFO 11-04 05:42:50 [api_server.py:1952] vLLM API server version 0.11.1rc6.dev58+g7f4bdadb9.d20251103
(APIServer pid=1) INFO 11-04 05:42:50 [utils.py:253] non-default args: {'model_tag': 'gpt2', 'host': '0.0.0.0', 'model': 'gpt2', 'dtype': 'bfloat16'}
config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 665/665 [00:00<00:00, 6.30MB/s]
(APIServer pid=1) INFO 11-04 05:43:04 [model.py:657] Resolved architecture: GPT2LMHeadModel
(APIServer pid=1) INFO 11-04 05:43:05 [model.py:1975] Downcasting torch.float32 to torch.bfloat16.
(APIServer pid=1) INFO 11-04 05:43:05 [model.py:1752] Using max model len 1024
(APIServer pid=1) WARNING 11-04 05:43:05 [cpu.py:160] Environment variable VLLM_CPU_KVCACHE_SPACE (GiB) for CPU backend is not set, using 4 by default.
(APIServer pid=1) INFO 11-04 05:43:05 [arg_utils.py:1358] Chunked prefill is not supported for ARM and POWER, S390X and RISC-V CPUs; disabling it for V1 backend.
(APIServer pid=1) INFO 11-04 05:43:05 [arg_utils.py:1364] Prefix caching is not supported for ARM and POWER, S390X and RISC-V CPUs; disabling it for V1 backend.
tokenizer_config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████| 26.0/26.0 [00:00<00:00, 266kB/s]
vocab.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.04M/1.04M [00:00<00:00, 3.69MB/s]
merges.txt: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 456k/456k [00:00<00:00, 2.53MB/s]
tokenizer.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.36M/1.36M [00:00<00:00, 4.85MB/s]
generation_config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████| 124/124 [00:00<00:00, 1.32MB/s]
INFO 11-04 05:43:15 [importing.py:68] Triton not installed or not compatible; certain GPU-related functions will not be available.
(EngineCore_DP0 pid=21) INFO 11-04 05:43:21 [core.py:93] Initializing a V1 LLM engine (v0.11.1rc6.dev58+g7f4bdadb9.d20251103) with config: model='gpt2', speculative_config=None, tokenizer='gpt2', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=1024, download_dir=None, load_format=auto, tensor_parallel_size=1, pipeline_parallel_size=1, data_parallel_size=1, disable_custom_all_reduce=True, quantization=None, enforce_eager=False, kv_cache_dtype=auto, device_config=cpu, structured_outputs_config=StructuredOutputsConfig(backend='auto', disable_fallback=False, disable_any_whitespace=False, disable_additional_properties=False, reasoning_parser='', enable_in_reasoning=False), observability_config=ObservabilityConfig(show_hidden_metrics_for_version=None, otlp_traces_endpoint=None, collect_detailed_traces=None), seed=0, served_model_name=gpt2, enable_prefix_caching=False, chunked_prefill_enabled=False, pooler_config=None, compilation_config={'level': None, 'mode': 2, 'debug_dump_path': None, 'cache_dir': '', 'backend': 'inductor', 'custom_ops': ['none'], 'splitting_ops': None, 'use_inductor': None, 'compile_sizes': None, 'inductor_compile_config': {'enable_auto_functionalized_v2': False, 'dce': True, 'size_asserts': False, 'nan_asserts': False, 'epilogue_fusion': True}, 'inductor_passes': {}, 'cudagraph_mode': <CUDAGraphMode.NONE: 0>, 'use_cudagraph': True, 'cudagraph_num_of_warmups': 0, 'cudagraph_capture_sizes': [], 'cudagraph_copy_inputs': False, 'full_cuda_graph': False, 'cudagraph_specialize_lora': True, 'use_inductor_graph_partition': False, 'pass_config': {}, 'max_cudagraph_capture_size': None, 'local_cache_dir': None}
(EngineCore_DP0 pid=21) WARNING 11-04 05:43:22 [cpu.py:404] Pin memory is not supported on CPU.
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:164] auto thread-binding list (id, physical core): [(0, 0), (1, 0), (2, 1), (3, 1), (8, 4), (9, 4), (10, 5), (11, 5), (16, 8), (17, 8), (18, 9), (19, 9)]
[W1104 05:43:23.044149155 utils.cpp:57] Warning: numa_migrate_pages failed. errno: 1 (function init_cpu_threads_env)
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:70] OMP threads binding of Process 21:
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:70] 	OMP tid: 21, core 0
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:70] 	OMP tid: 33, core 1
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:70] 	OMP tid: 34, core 2
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:70] 	OMP tid: 35, core 3
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:70] 	OMP tid: 36, core 8
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:70] 	OMP tid: 37, core 9
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:70] 	OMP tid: 38, core 10
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:70] 	OMP tid: 39, core 11
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:70] 	OMP tid: 40, core 16
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:70] 	OMP tid: 41, core 17
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:70] 	OMP tid: 42, core 18
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:70] 	OMP tid: 43, core 19
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_worker.py:70] 
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
[Gloo] Rank 0 is connected to 0 peer ranks. Expected number of connected peer ranks is : 0
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [parallel_state.py:1325] rank 0 in world size 1 is assigned as DP rank 0, PP rank 0, TP rank 0, EP rank 0
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu_model_runner.py:67] Starting to load model gpt2...
(EngineCore_DP0 pid=21) INFO 11-04 05:43:23 [cpu.py:147] Using Torch SDPA backend.
model.safetensors: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 548M/548M [00:13<00:00, 41.7MB/s]
(EngineCore_DP0 pid=21) INFO 11-04 05:43:37 [weight_utils.py:440] Time spent downloading weights for gpt2: 13.691614 seconds
(EngineCore_DP0 pid=21) INFO 11-04 05:43:37 [weight_utils.py:480] No model.safetensors.index.json found in remote.
Loading safetensors checkpoint shards:   0% Completed | 0/1 [00:00<?, ?it/s]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00,  1.08s/it]
Loading safetensors checkpoint shards: 100% Completed | 1/1 [00:01<00:00,  1.08s/it]
(EngineCore_DP0 pid=21) 
(EngineCore_DP0 pid=21) INFO 11-04 05:43:38 [default_loader.py:314] Loading weights took 1.09 seconds
(EngineCore_DP0 pid=21) INFO 11-04 05:43:38 [kv_cache_utils.py:1229] GPU KV cache size: 116,496 tokens
(EngineCore_DP0 pid=21) INFO 11-04 05:43:38 [kv_cache_utils.py:1234] Maximum concurrency for 1,024 tokens per request: 113.77x
(EngineCore_DP0 pid=21) INFO 11-04 05:43:38 [cpu_model_runner.py:77] Warming up model for the compilation...
(EngineCore_DP0 pid=21) INFO 11-04 05:44:01 [cpu_model_runner.py:87] Warming up done.
(EngineCore_DP0 pid=21) INFO 11-04 05:44:01 [core.py:258] init engine (profile, create kv cache, warmup model) took 22.86 seconds
(EngineCore_DP0 pid=21) WARNING 11-04 05:44:05 [cpu.py:160] Environment variable VLLM_CPU_KVCACHE_SPACE (GiB) for CPU backend is not set, using 4 by default.
(APIServer pid=1) INFO 11-04 05:44:05 [api_server.py:1717] Supported tasks: ['generate']
(APIServer pid=1) INFO 11-04 05:44:06 [api_server.py:2021] Starting vLLM API server 0 on http://0.0.0.0:8000
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:38] Available routes are:
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /openapi.json, Methods: HEAD, GET
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /docs, Methods: HEAD, GET
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /docs/oauth2-redirect, Methods: HEAD, GET
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /redoc, Methods: HEAD, GET
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /health, Methods: GET
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /load, Methods: GET
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /ping, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /ping, Methods: GET
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /tokenize, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /detokenize, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /v1/models, Methods: GET
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /version, Methods: GET
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /v1/responses, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /v1/responses/{response_id}, Methods: GET
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /v1/responses/{response_id}/cancel, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /v1/messages, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /v1/chat/completions, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /v1/completions, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /v1/embeddings, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /pooling, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /classify, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /score, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /v1/score, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /v1/audio/transcriptions, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /v1/audio/translations, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /rerank, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /v1/rerank, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /v2/rerank, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /scale_elastic_ep, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /is_scaling_elastic_ep, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /invocations, Methods: POST
(APIServer pid=1) INFO 11-04 05:44:06 [launcher.py:46] Route: /metrics, Methods: GET
(APIServer pid=1) INFO:     Started server process [1]
(APIServer pid=1) INFO:     Waiting for application startup.
(APIServer pid=1) INFO:     Application startup complete.
(APIServer pid=1) INFO 11-04 05:44:16 [loggers.py:215] Engine 000: Avg prompt throughput: 0.3 tokens/s, Avg generation throughput: 2.5 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=1) INFO:     10.88.0.1:51662 - "POST /v1/completions HTTP/1.1" 200 OK
(APIServer pid=1) INFO 11-04 05:44:26 [loggers.py:215] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 2.1 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=1) INFO 11-04 05:44:36 [loggers.py:215] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%

inference result

[root@b314lp50 ~]# curl http://localhost:8000/v1/completions   -H "Content-Type: application/json"   -d '{
    "model": "gpt2",
    "prompt": "Once upon a time",
    "max_tokens": 50,
    "temperature": 0.7
  }' | jq
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   742  100   637  100   105     59      9  0:00:11  0:00:10  0:00:01   138
{
  "id": "cmpl-46bf74eae1734c67af590d166fe9abab",
  "object": "text_completion",
  "created": 1762235050,
  "model": "gpt2",
  "choices": [
    {
      "index": 0,
      "text": ", the blue paint with the lights a set the head, the state the government agencies, the pastor turned out of black people have been with cars to protect your Web.\n\"\n11: 1.5)\n\" the Eagles fans, to",
      "logprobs": null,
      "finish_reason": "length",
      "stop_reason": null,
      "token_ids": null,
      "prompt_logprobs": null,
      "prompt_token_ids": null
    }
  ],
  "service_tier": null,
  "system_fingerprint": null,
  "usage": {
    "prompt_tokens": 4,
    "total_tokens": 54,
    "completion_tokens": 50,
    "prompt_tokens_details": null
  },
  "kv_transfer_params": null
}

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[chatgpt-codex-connector]

💡 Codex Review

vllm/requirements/cpu.txt

Lines 9 to 12 in 9a2071c

	--extra-index-url https://download.pytorch.org/whl/cpu
	torch==2.8.0+cpu; platform_machine == "x86_64" or platform_machine == "s390x"
	torch==2.8.0; platform_system == "Darwin"
	torch==2.8.0; platform_machine == "ppc64le" or platform_machine == "aarch64"

Torch wheels unavailable for s390x

The Dockerfile now relies on pip install torch==2.8.0+cpu for the s390x build (the torch source build stage was removed and the requirement here now includes platform_machine == "s390x"). PyTorch does not publish prebuilt wheels for the s390x architecture and also does not ship an sdist, so pip will fail with “No matching distribution found for torch==2.8.0+cpu” when this image is built on s390x. As a result the CPU Docker image can no longer be built. The custom build step needs to be retained or a wheel must be provided for s390x.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[gemini-code-assist]

Code Review

This pull request provides a great optimization for the s390x Dockerfile. The changes, such as switching to a pre-built PyTorch wheel and simplifying the toolchain, significantly improve build efficiency and maintainability.

I have one major observation that could lead to further significant optimization. The Dockerfile includes a numba-builder stage which builds llvm, llvmlite, and numba from source. This is a very time-consuming process. However, requirements/cpu.txt explicitly excludes numba for the s390x architecture (numba == 0.61.2; platform_machine != "s390x"). This suggests that numba is not a required dependency for s390x.

If numba is indeed not needed, removing the numba-builder stage and the corresponding installation of llvmlite and numba wheels in the final vllm-cpu stage would drastically reduce build time and image size. While I cannot comment directly on the relevant lines due to review constraints, I strongly recommend investigating this discrepancy.

Apart from this point, the rest of the changes are excellent and well-implemented.