blt wip#38579

Merged

ArthurZucker merged 140 commits intomainfrom

blt_wip

Sep 19, 2025

Commits on Sep 18, 2025

blt wip

[email protected]
authored and
[email protected]
committed
cpu version

itazap
authored and
[email protected]
committed
cpu friendly with full entropy model (real time patching)

itazap
authored and
[email protected]
committed
adding config file instead of args file

[email protected]
authored and
[email protected]
committed
enable MPS

LysandreJik
authored and
[email protected]
committed
refactoring unused code

[email protected]
authored and
[email protected]
committed
single config class in config file
[email protected]
committed
inherit from PreTrainedModel

[email protected]
authored and
[email protected]
committed
refactor LMTransformer --> BLTPatcher

[email protected]
authored and
[email protected]
committed
add conversion script

[email protected]
authored and
[email protected]
committed
load from new checkpoing with form_pretrained

[email protected]
authored and
[email protected]
committed
fixed demo from_pretrained

[email protected]
authored and
[email protected]
committed
clean up

[email protected]
authored and
[email protected]
committed
clean a few comments

[email protected]
authored and
[email protected]
committed
cleanup folder

[email protected]
authored and
[email protected]
committed
clean up dir

[email protected]
authored and
[email protected]
committed
cleaned up modeling further

[email protected]
authored and
[email protected]
committed
rename classes

[email protected]
authored and
[email protected]
committed
adding transformers Attention class and RotaryEmbedding class

[email protected]
authored and
[email protected]
committed
exchanged blt modules for transformers modules: attention, rotary_emb, create_causal_mask, etc

[email protected]
authored and
[email protected]
committed
seperate out patcher config, update modeling and conversion script

[email protected]
authored and
[email protected]
committed
rename vars to be more transformers-like

[email protected]
authored and
[email protected]
committed
rm unused functions

[email protected]
authored and
[email protected]
committed
adding cross attention from transformers

[email protected]
authored and
[email protected]
committed
pass arg

[email protected]
authored and
[email protected]
committed
rename weights

[email protected]
authored and
[email protected]
committed
updated conversion script

[email protected]
authored and
[email protected]
committed
overwritten commit! fixing PR

[email protected]
authored and
[email protected]
committed
apply feedback

[email protected]
authored and
[email protected]
committed
adding BLTRMSNorm like Llama

[email protected]
authored and
[email protected]
committed
add repeat_kv and eager_attention_forward copied from

[email protected]
authored and
[email protected]
committed
BLTMLP identical to MllamTextMLP

[email protected]
authored and
[email protected]
committed
clean up some args'

[email protected]
authored and
[email protected]
committed
more like mllama, but busier inits

[email protected]
authored and
[email protected]
committed
BLTTransformerLayer config

[email protected]
authored and
[email protected]
committed
decoder, encoder, global configs

[email protected]
authored and
[email protected]
committed
wip working on modular file

[email protected]
authored and
[email protected]
committed
cleaning up patch and configs
[email protected]
committed
clean up patcher helpers
[email protected]
committed
clean up patcher helpers further

[email protected]
authored and
[email protected]
committed
clean up

[email protected]
authored and
[email protected]
committed
some config renaming

[email protected]
authored and
[email protected]
committed
clean up unused configs

[email protected]
authored and
[email protected]
committed
clean up configs

[email protected]
authored and
[email protected]
committed
clean up configs

[email protected]
authored and
[email protected]
committed
update modular

[email protected]
authored and
[email protected]
committed
clean

[email protected]
authored and
[email protected]
committed
update demo

[email protected]
authored and
[email protected]
committed
config more like mllama, seperated subconfigs from subdicts

[email protected]
authored and
[email protected]
committed
read from config instead of self args
[email protected]
committed
update demo file

[email protected]
authored and
[email protected]
committed
model weights to causal lm weights

[email protected]
authored and
[email protected]
committed
missed file

[email protected]
authored and
[email protected]
committed
added tied weights keys

[email protected]
authored and
[email protected]
committed
BLTForCausalLM
[email protected]
committed
adding files after add-new-model-like

[email protected]
authored and
[email protected]
committed
update demo
[email protected]
committed
working on tests

[email protected]
authored and
[email protected]
committed
first running integration tests

[email protected]
authored and
[email protected]
committed
added integration tests

[email protected]
authored and
[email protected]
committed
adding tokenization tests, integration tests, and cleaned up tokenization file, + ruff

[email protected]
authored and
[email protected]
committed
tokenizer clean up

[email protected]
authored and
[email protected]
committed
modular file

[email protected]
authored and
[email protected]
committed
fixing rebase

[email protected]
authored and
[email protected]
committed
ruff

[email protected]
authored and
[email protected]
committed
adding correct basemodel output and updating config with checkpoint vals (for testing)

[email protected]
authored and
[email protected]
committed
BLTModelTests git status

[email protected]
authored and
[email protected]
committed
enabling inputs_embeds, although won't be equal to input_ids since need ids for patching logic

[email protected]
authored and
[email protected]
committed
fix sdpa == causal tests

[email protected]
authored and
[email protected]
committed
fix small model test and some gradient checkpointing

[email protected]
authored and
[email protected]
committed
skip training GC tests

[email protected]
authored and
[email protected]
committed
fix test

[email protected]
authored and
[email protected]
committed
updated modular

[email protected]
authored and
[email protected]
committed
update modular

[email protected]
authored and
[email protected]
committed
ruff

[email protected]
authored and
[email protected]
committed
adding modular + modeling

[email protected]
authored and
[email protected]
committed
modular

[email protected]
authored and
[email protected]
committed
more modern is_casual check

[email protected]
authored and
[email protected]
committed
cleaning up modular

[email protected]
authored and
[email protected]
committed
more modular reduction

[email protected]
authored and
[email protected]
committed
ruff

[email protected]
authored and
[email protected]
committed
modular fix

[email protected]
authored and
[email protected]
committed
fix styling

[email protected]
authored and
[email protected]
committed
return 2

[email protected]
authored and
[email protected]
committed
return 2

[email protected]
authored and
[email protected]
committed
fix some tests

[email protected]
authored and
[email protected]
committed
fix bltcrossattention after modular break

[email protected]
authored and
[email protected]
committed
some fixes / feedback

[email protected]
authored and
[email protected]
committed
try cache generate fix

[email protected]
authored and
[email protected]
committed
try cache generate fix

[email protected]
authored and
[email protected]
committed
fix generate tests

[email protected]
authored and
[email protected]
committed
attn_impl workaround

[email protected]
authored and
[email protected]
committed
refactoring to use recent TransformersKwargs changes

[email protected]
authored and
[email protected]
committed
fix hidden_states shape test

[email protected]
authored and
[email protected]
committed
refactor to new outputs

[email protected]
authored and
[email protected]
committed
simplify outputs a bit

[email protected]
authored and
[email protected]
committed
rm unneeded decoderlayer overwriting

[email protected]
authored and
[email protected]
committed
rename blt

[email protected]
authored and
[email protected]
committed
forgot tokenizer test renamed

[email protected]
authored and
[email protected]
committed
Reorder

itazap
authored and
[email protected]
committed
Reorder

itazap
authored and
[email protected]
committed
working on modular

itazap
authored and
[email protected]
committed
updates from modular

itazap
authored and
[email protected]
committed
new modular

itazap
authored and
[email protected]
committed
ruff and such

itazap
authored and
[email protected]
committed
update pretrainedmodel modular

itazap
authored and
[email protected]
committed
using cohere2 apply_rotary_pos_emb

[email protected]
authored and
[email protected]
committed
small changes

[email protected]
authored and
[email protected]
committed
apply feedback r2

itazap
authored and
[email protected]
committed
fix cross_attention

[email protected]
authored and
[email protected]
committed
apply more feedback

[email protected]
authored and
[email protected]
committed
update modeling fix

[email protected]
authored and
[email protected]
committed
load submodules from pretrainedmodel

[email protected]
authored and
[email protected]
committed
set initializer_range to subconfigs

[email protected]
authored and
[email protected]
committed
rm cross_attnetion_states pass when not needed

[email protected]
authored and
[email protected]
committed
add 7b projection layer support

[email protected]
authored and
[email protected]
committed
check repo

itazap
authored and
[email protected]
committed
make copies

itazap
authored and
[email protected]
committed
lost cohere2 rotate_half

[email protected]
authored and
[email protected]
committed
ruff

[email protected]
authored and
[email protected]
committed
copies?

[email protected]
authored and
[email protected]
committed
don't tie weights for submodules
[email protected]
committed
tie weights setting

[email protected]
authored and
[email protected]
committed
check docstrings

[email protected]
authored and
[email protected]
committed
apply feedback

[email protected]
authored and
[email protected]
committed
rebase

[email protected]
authored and
[email protected]
committed
rebased modeling

itazap
authored and
[email protected]
committed
update docs

itazap
authored and
[email protected]
committed
applying feedback

[email protected]
authored and
[email protected]
committed
few more fixes

[email protected]
authored and
[email protected]
committed
fix can_record_outputs

[email protected]
authored and
[email protected]
committed
fast tokenizer
[email protected]
committed
no more modulelist

[email protected]
authored and
[email protected]
committed
tok auto

[email protected]
authored and
[email protected]
committed
rm tokenizersss
[email protected]
committed
fix docs
[email protected]
committed
ruff
[email protected]
committed
fix after rebase
[email protected]
committed
fix test, configs are not subscriptable
[email protected]
committed
Merge branch 'main' into blt_wip
itazap
authored

blt wip#38579ArthurZucker merged 140 commits intomainhuggingface/transformers:mainfrom blt_wiphuggingface/transformers:blt_wipCopy head branch name to clipboard

Commits

Commits on Sep 18, 2025

blt wip#38579
ArthurZucker merged 140 commits intomainfrom
blt_wip