Error in converting huggingface models

[calvintwr]

I tried to use export_meta_llama_hf.bin.py. The first error I encountered was:

FileNotFoundError: [Errno 2] No such file or directory: '../Llama-2-13b-chat-hf/params.json'

But that's goes away after I rename the Llama-2-13b-chat-hf/config.json to Llama-2-13b-chat-hf/params.json.

However, the next error hits, which is:

Traceback (most recent call last):
  File "/root/llama2.c/export_meta_llama_hf_bin.py", line 117, in <module>
    load_and_export(model_path, output_path)
  File "/root/llama2.c/export_meta_llama_hf_bin.py", line 105, in load_and_export
    state_dict = concat_weights(models)
  File "/root/llama2.c/export_meta_llama_hf_bin.py", line 78, in concat_weights
    for name in list(models[0]):
IndexError: list index out of range

On closer inspection,models originate from

llama2.c/export_meta_llama_bin.py

Line 98 in bd18228

model_paths = sorted(list(Path(model_path).glob('consolidated.*.pth')))

and it calls .glob to get .pth like this:model_paths = sorted(list(Path(model_path).glob('consolidated.*.pth')))

But there are no .pth files in the huggingface models.

[Nick-infinity]

I forgot to mention that we have to conver the pickle file and dump it as consolidated.pth

Sample code

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

tokenizer = AutoTokenizer.from_pretrained("meta_llama_2_model")
model = AutoModelForCausalLM.from_pretrained("meta_llama_2_model")
model_path = 'consolidate.pth'

Save the model's state_dict

torch.save(model.state_dict(), model_path)

[karpathy]

try now?

[calvintwr]

The conversion works now, but I run into Segmentation fault with bins that I exported using version 1 and version 2 flags. With version 0 flag, the model does run but outputs gibberish.

I am trying to debug it but I can’t find out what’s causing the Segmentation fault.

I went ahead and tried @kroggen ’s cuda implementation #310

but ran into malloc failed. I believe they are the same issue?

I am able to run stories150m.bin with no issue.