Skip to content

Commit 768dee1

Browse files
SStasclaude
andcommitted
Fix llama.cpp latent steps: request embeddings on every step
Setting logits[0]=1 only on the last step prevented llama_get_embeddings_ith from returning hidden states on intermediate steps, so the same initial hidden state was re-injected N-1 times instead of iteratively refining. Now matches the HuggingFace connector and avp-agent behavior. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 2f05ab6 commit 768dee1

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

src/avp/connectors/llamacpp.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -241,7 +241,7 @@ def think(
241241
emb_batch.pos[0] = n_past
242242
emb_batch.seq_id[0][0] = 0
243243
emb_batch.n_seq_id[0] = 1
244-
emb_batch.logits[0] = 1 if step == steps - 1 else 0
244+
emb_batch.logits[0] = 1
245245

246246
rc = lc.llama_decode(think_ctx, emb_batch)
247247
if rc != 0:

0 commit comments

Comments
 (0)