llama: implement YaRN RoPE scaling#2268

Merged

cebtenzzre merged 36 commits intoggml-org:masterfrom

cebtenzzre:ntkv2

Nov 1, 2023

Commits on Aug 7, 2023

Commits on Sep 5, 2023

implement new YaRN algorithm
cebtenzzre
committed

Commits on Sep 21, 2023

Commits on Oct 11, 2023

fix missing import
cebtenzzre
committed

Commits on Oct 13, 2023

Merge branch 'master' of https://github.com/ggerganov/llama.cpp into ntkv2
cebtenzzre
committed

Commits on Oct 20, 2023

Fix YaRN inverted scaling and add "rope.scaling.type" to GGUF (#1)
jquesnelle
authored
fix YaRN ramp, make mscale conditional, add --yarn-orig-ctx (#2)
jquesnelle
authored

Commits on Oct 22, 2023

Merge branch 'master' of https://github.com/ggerganov/llama.cpp into ntkv2
cebtenzzre
committed

Commits on Oct 23, 2023

Merge branch 'master' of https://github.com/ggerganov/llama.cpp into ntkv2
cebtenzzre
committed

Commits on Oct 24, 2023

Merge branch 'master' of https://github.com/ggerganov/llama.cpp into ntkv2
cebtenzzre
committed

Commits on Oct 30, 2023

fix loading rope.scaling.original_context_length from GGUF (#3)

jquesnelle
and
cebtenzzre
authored

Commits on Nov 1, 2023