Skip to content

Commit 5ca19f7

Browse files
authored
[BUG] Fork log should error on incomplete manifest (#5771)
## Description of changes _Summarize the changes made by this PR._ - Improvements & Bug fixes - Throw error during `fork_log` if the first available offset in the copied manifest is ahead of the compaction cursor - New functionality - N/A ## Test plan _How are these changes tested?_ - [ ] Tests pass locally with `pytest` for python, `yarn test` for js, `cargo test` for rust ## Migration plan _Are there any migrations, or any forwards/backwards compatibility changes needed in order to make sure this change deploys reliably?_ ## Observability plan _What is the plan to instrument and monitor this change?_ ## Documentation Changes _Are all docstrings for user-facing APIs updated if required? Do we need to make documentation changes in the_ [_docs section](https://github.com/chroma-core/chroma/tree/main/docs/docs.trychroma.com)?_
1 parent 47287c4 commit 5ca19f7

File tree

1 file changed

+17
-6
lines changed

1 file changed

+17
-6
lines changed

rust/log-service/src/lib.rs

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1452,13 +1452,24 @@ impl LogServer {
14521452
Arc::clone(&storage),
14531453
target_prefix,
14541454
);
1455+
let new_manifest = log_reader
1456+
.manifest()
1457+
.await
1458+
.map_err(|err| {
1459+
Status::new(
1460+
err.code().into(),
1461+
format!("Unable to read copied manifest: {}", err),
1462+
)
1463+
})?
1464+
.ok_or_else(|| Status::internal("Unable to find copied manifest"))?;
1465+
let first_copied_offset = new_manifest.oldest_timestamp();
1466+
if offset < first_copied_offset {
1467+
return Err(Status::internal(format!(
1468+
"Compaction cursor [{offset:?}] is behind start of manifest [{first_copied_offset:?}]",
1469+
)));
1470+
}
14551471
// This is the next record to insert, so we'll have to adjust downwards.
1456-
let max_offset = log_reader.next_write_timestamp().await.map_err(|err| {
1457-
Status::new(
1458-
err.code().into(),
1459-
format!("Failed to read copied log: {}", err),
1460-
)
1461-
})?;
1472+
let max_offset = new_manifest.next_write_timestamp();
14621473
if max_offset < offset {
14631474
return Err(Status::new(
14641475
chroma_error::ErrorCodes::Internal.into(),

0 commit comments

Comments
 (0)