Skip to content

gguf-py : do not align the data start offset#18291

Merged
CISC merged 1 commit intomasterfrom
compilade/fix-safetensors-unaligned
Dec 22, 2025
Merged

gguf-py : do not align the data start offset#18291
CISC merged 1 commit intomasterfrom
compilade/fix-safetensors-unaligned

Conversation

@compilade
Copy link
Copy Markdown
Collaborator

@compilade compilade commented Dec 22, 2025

The safetensors format doesn't require alignment. Fixes: #18282 (which was a regression caused by #15667).

I assumed wrong since GGUF does align its data offset, and the writer for safetensors aligns to 8 bytes (see https://github.com/huggingface/safetensors/blob/806426784adb43631e9a1102d4621126bb589347/safetensors/src/tensor.rs#L256-L258), and also because the data offset alignment was implemented in the same way in #12820. But apparently some models aren't aligned.

It seems like PyTorch and Numpy can handle unaligned tensors, but I'm not completely sure (is it only for shape transformations, or does it also support arithmetic on unaligned tensors? (would need an unaligned model which has some arithmetic in its modify_tensors transformations to test this)). Copying the tensor (with e.g. data.copy()) wouldn't necessarily always be sufficient, because that doesn't seem to align to 8 bytes when the dtype is np.uint8. I'll try to figure out how to make an aligned copy. But if it's not really necessary in practice, then this is ready.

EDIT: I've looked at the .data_ptr() addresses when using the safetensors library with an unaligned model, and it doesn't make an aligned copy (at least when using get_slice like since #8482). So the new behavior is pretty much the same as with the safetensors library.

Tested on https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B

Thanks @fairydreaming for finding this problem! (and finding the rationale behind why unaligned safetensors exist)

Make sure to read the contributing guidelines before submitting a PR

The safetensors format doesn't require alignment.
@compilade compilade requested a review from CISC as a code owner December 22, 2025 15:17
@github-actions github-actions bot added the python python script changes label Dec 22, 2025
@compilade compilade added the bugfix fixes an issue or bug label Dec 22, 2025
Copy link
Copy Markdown
Collaborator

@fairydreaming fairydreaming left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, works fine. I think we can worry about possible alignment problems (if any) when they appear.

@CISC CISC merged commit 8f48807 into master Dec 22, 2025
5 checks passed
Anico2 added a commit to Anico2/llama.cpp that referenced this pull request Jan 15, 2026
The safetensors format doesn't require alignment.
blime4 referenced this pull request in blime4/llama.cpp Feb 5, 2026
The safetensors format doesn't require alignment.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bugfix fixes an issue or bug python python script changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Misc. bug: Enforcing alignment of safetensors byte-buffer causes GGUF conversion failure in some models

3 participants