Add MLLama#33703

Merged

ArthurZucker merged 399 commits intomainfrom

refactor-mlamma

Sep 25, 2024

This pull request is big! We're only showing the most recent 250 commits

Commits on Sep 9, 2024

current changes
ArthurZucker
committed
nit
zucchini-nlp
committed
Add cross_attenttion_mask to processor
pavel
committed
multi-image fixed
zucchini-nlp
committed
Add cross_attenttion_mask to processor

pavel
authored and
zucchini-nlp
committed
cross attn works in all cases
zucchini-nlp
committed
WIP refactoring function for image processor
pavel
committed
WIP refactoring image processor functions
pavel
committed
Refactor preprocess to use global loops instead of list nested list comps
pavel
committed
Docstrings
pavel
committed
Add channels unification
pavel
committed
fix dtype issues
zucchini-nlp
committed
Update docsrings and format
pavel
committed
Consistent max_image_tiles
pavel
committed
current script
ArthurZucker
committed
updates
ArthurZucker
committed
Add convert to rgb
pavel
committed
Add image processor tests
pavel
committed
updates!
ArthurZucker
committed
update
ArthurZucker
committed
god damn it I am dumb sometimes
ArthurZucker
committed
Precompute aspect ratios
pavel
committed

Commits on Sep 10, 2024

Commits on Sep 11, 2024

Commits on Sep 12, 2024

Clean up a bit for errors and typos

pavel
authored and
qubvel
committed
A bit more constant fixes

pavel
authored and
qubvel
committed
90B keys and shapes match

pavel
authored and
qubvel
committed
Fix for 11B model

pavel
authored and
qubvel
committed
Fixup, remove debug part

pavel
authored and
qubvel
committed
Docs

pavel
authored and
qubvel
committed

Commits on Sep 13, 2024

Make max_aspect_ratio_id to be minimal

pavel
authored and
qubvel
committed
Update image processing code to match new implementation

pavel
authored and
qubvel
committed
Adjust conversion for final checkpoint state

pavel
authored and
qubvel
committed
Change dim in repeat_interleave (accordig to meta code)

pavel
authored and
qubvel
committed
tmp fix for num_tiles

pavel
authored and
qubvel
committed
Fix for conversion (gate<->up, q/k_proj rope permute)
pavel
committed
nits
zucchini-nlp
committed
codestyle
zucchini-nlp
committed

Commits on Sep 16, 2024

Vision encoder fixes

pavel
authored and
qubvel
committed
pass cross attn mask further

zucchini-nlp
authored and
qubvel
committed
Refactor aspect ratio mask

pavel
authored and
qubvel
committed
Disable text-only generation

pavel
authored and
qubvel
committed
Fix cross attention layers order, remove q/k norm rotation for cross atention layers

pavel
authored and
qubvel
committed
Refactor gated position embeddings

pavel
authored and
qubvel
committed
fix bugs but needs test with new weights

zucchini-nlp
authored and
qubvel
committed
rope scaling should be llama3

zucchini-nlp
authored and
qubvel
committed
Fix rope scaling name

pavel
authored and
qubvel
committed
Remove debug for linear layer
pavel
committed
fix copies
zucchini-nlp
committed
Make mask prepare private func
pavel
committed
Remove linear patch embed
qubvel
committed
Make precomputed embeddings as nn.Embedding module
qubvel
committed
MllamaPrecomputedAspectRatioEmbedding with config init
qubvel
committed
Remove unused self.output_dim
qubvel
committed
nit, intermediate layers
qubvel
committed
Rename ln and pos_embed
qubvel
committed
vision_chunk_size -> image_size
qubvel
committed
return_intermediate -> intermediate_layers_indices
qubvel
committed
vision_input_dim -> hidden_size
qubvel
committed
Fix copied from statements
qubvel
committed
fix most tests
zucchini-nlp
committed
Fix more copied from
qubvel
committed
layer_id->layer_idx
qubvel
committed
Comment
qubvel
committed
Fix tests for processor
qubvel
committed
Copied from for _prepare_4d_causal_attention_mask_with_cache_position
qubvel
committed
Style fix
qubvel
committed
Add MllamaForCausalLM
qubvel
committed
Merge branch 'refactor-mlamma' into refactor-configuration
qubvel
committed
WIP fixing tests
qubvel
committed
Remove duplicated layers
qubvel
committed
Remove dummy file
qubvel
committed
Fix style
qubvel
committed
Fix consistency
qubvel
committed
Fix some TODOs
qubvel
committed
fix language_model instantiation, add docstring
qubvel
committed
Move docstring, remove todos for precomputed embeds (we cannot init them properly)
qubvel
committed
Add initial docstrings
qubvel
committed
Fix
qubvel
committed
Merge pull request #10 from huggingface/refactor-configuration
qubvel
authored
fix some tests
zucchini-nlp
committed
lets skip these
zucchini-nlp
committed
nits, remove print, style
qubvel
committed

Commits on Sep 17, 2024

Commits on Sep 20, 2024

Commits on Sep 23, 2024

Commits on Sep 24, 2024

Commits on Sep 25, 2024