Fix for Llama4 models

[vidyasiv]

Command

PT_HPU_LAZY_MODE=0 ./calibrate_model.sh \
 -m <>/Llama-4-Maverick-17B-128E-Instruct \
 -d <>/mlperf_inference/llama2/processed-data.pkl  \
 -o .  \
 -b 128 -t 8 -l 4096

Error

1/4 Preparing calibration dataset
Loading source dataset: /mnt/weka/data/mlperf_inference/llama2/processed-data.pkl
Creating calibration dataset...
Traceback (most recent call last):
  File "/root/8625/vllm-hpu-extension/calibration/step-1-prepare-calibration-dataset.py", line 93, in <module>
    main(args)
  File "/root/8625/vllm-hpu-extension/calibration/step-1-prepare-calibration-dataset.py", line 65, in main
    tmp_input = tokenizer.apply_chat_template(
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'bool' object has no attribute 'apply_chat_template'
Error in step 1

Similar issue

Closest issue to boolean value return: huggingface/transformers#35037

Proposed fix

Based on HF documentation: Llama4 AutoTokenizer should work for Llama4 Text only, in multimodel cases, we need to use Autoprocessor.

I noticed omitting use_fast=False or setting use_fast=True in AutoTokenizer.from_pretrained() helped get past the error.

One option (this PR) is to raise an error so we switch to use_fast=True, another option is to omit use_fast=false in default case.
I don't have enough context to gauge if second option is a good idea.

Please advise if this PR makes sense or we need some other enabling for Llama4 models to fix the issue.

Testing

• Tested with Llama-4-Scout-17B-16E-Instruct
• Testing with Llama-4-Maverick-17B-128E-Instruct