Skip to content

Commit a607312

Browse files
authored
[Doc] Update mllama example based on official doc (#11567)
Signed-off-by: Chen Zhang <[email protected]>
1 parent ac79799 commit a607312

File tree

1 file changed

+14
-1
lines changed

1 file changed

+14
-1
lines changed

examples/offline_inference_vision_language.py

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -308,7 +308,20 @@ def run_mllama(question: str, modality: str):
308308
disable_mm_preprocessor_cache=args.disable_mm_preprocessor_cache,
309309
)
310310

311-
prompt = f"<|image|><|begin_of_text|>{question}"
311+
tokenizer = AutoTokenizer.from_pretrained(model_name)
312+
messages = [{
313+
"role":
314+
"user",
315+
"content": [{
316+
"type": "image"
317+
}, {
318+
"type": "text",
319+
"text": f"{question}"
320+
}]
321+
}]
322+
prompt = tokenizer.apply_chat_template(messages,
323+
add_generation_prompt=True,
324+
tokenize=False)
312325
stop_token_ids = None
313326
return llm, prompt, stop_token_ids
314327

0 commit comments

Comments
 (0)