[Feature]: investigate if we can turn off the ahead-of-time autotuning block in vLLM

[zou3519]

🚀 The feature, motivation and pitch

This should make things less complicated. But we may have discussed this before and there may have been a good reason for it (I don't remember anymore).

Alternatives

No response

Additional context

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[baonudesifeizhai]

i add a switch VLLM_ENABLE_FLASHINFER_AUTOTUNE
when i turn that off

 /usr/bin/time -v lm_eval --model vllm \
  --model_args "pretrained=nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8,tensor_parallel_size=8,kv_cache_dtype=fp8,add_bos_token=true,trust_remote_code=true,max_model_len=4096" \
  --tasks gsm8k --num_fewshot 5 --limit 250 --batch_size auto




res:

vllm ({'pretrained': 'nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8', 'tensor_parallel_size': 8, 'kv_cache_dtype': 'fp8', 'add_bos_token': True, 'max_model_len': 4096}), gen_kwargs: ({}), limit: 250.0, num_fewshot: 5, batch_size: auto
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value|   |Stderr|
|-----|------:|----------------|-----:|-----------|---|----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.576|±  |0.0313|
|     |       |strict-match    |     5|exact_match|↑  |0.868|±  |0.0215|


res:
/usr/lib/python3.12/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
        Command being timed: "lm_eval --model vllm --model_args pretrained=nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-FP8,tensor_parallel_size=8,kv_cache_dtype=fp8,add_bos_token=true,trust_remote_code=true,max_model_len=4096 --tasks gsm8k --num_fewshot 5 --limit 250 --batch_size auto"
        User time (seconds): 215.48
        System time (seconds): 57.45
        Percent of CPU this job got: 218%
        Elapsed (wall clock) time (h:mm:ss or m:ss): 2:04.68
        Average shared text size (kbytes): 0
        Average unshared data size (kbytes): 0
        Average stack size (kbytes): 0
        Average total size (kbytes): 0
        Maximum resident set size (kbytes): 4077592
        Average resident set size (kbytes): 0
        Major (requiring I/O) page faults: 75
        Minor (reclaiming a frame) page faults: 4593245
        Voluntary context switches: 180543
        Involuntary context switches: 129018
        Swaps: 0
        File system inputs: 6816
        File system outputs: 446200
        Socket messages sent: 0
        Socket messages received: 0
        Signals delivered: 0
        Page size (bytes): 4096
        Exit status: 0

could successful run
but when i turn on ... https://paste.ubuntu.com/p/tVqsHz26XR/ goes to that error ...
also confirm that happen on main branch ... env: b300 flashinfer 0.6.2