Skip to content

server : improve infill context reuse#9894

Merged
ggerganov merged 1 commit intomasterfrom
gg/infill-2
Oct 15, 2024
Merged

server : improve infill context reuse#9894
ggerganov merged 1 commit intomasterfrom
gg/infill-2

Conversation

@ggerganov
Copy link
Copy Markdown
Member

  • Utilize the prompt as current line prefix. Keep the actual prefix constant during line-only edits
  • No need to special-handle control tokens
  • Move state from slot.params -> slot
  • Better naming

@ggerganov ggerganov merged commit 223c25a into master Oct 15, 2024
@ggerganov ggerganov deleted the gg/infill-2 branch October 15, 2024 13:28
@ggerganov ggerganov mentioned this pull request Oct 15, 2024
7 tasks
drollings pushed a commit to drollings/llama.cpp that referenced this pull request Oct 18, 2024
dsx1986 pushed a commit to dsx1986/llama.cpp that referenced this pull request Oct 29, 2024
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Nov 15, 2024
arthw pushed a commit to arthw/llama.cpp that referenced this pull request Nov 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant