Merged
Conversation
- more javadocs to guide usage of `INDEX` vs. `DATA`, - the `METADATA` constant is removed, as metadata files should be opened with `Directory#openChecksumIndexInput`, which doesn't take hints, - configure `FileTypeHint` on more files of the default codec, - remove checks on `FileTypeHint` from `toReadAdvice` - the default impl should only look at `DataAccessHint` to determine the appropriate read advice.
github-project-automation
bot
moved this to Open
in OpenSearch Lucene & Core Performance Tracking
github-actions
bot
added
module:core/store
module:core/codecs
module:test-framework
labels
jpountz
commented
May 9, 2025
| vectorsStreamFN, | ||
| context.withHints( | ||
| FileTypeHint.DATA, FileDataHint.KNN_VECTORS, DataAccessHint.RANDOM)); | ||
| d.openInput(vectorsStreamFN, context.withHints(FileTypeHint.DATA, DataAccessHint.RANDOM)); |
Contributor
Author
There was a problem hiding this comment.
Note: I removed KNN_VECTORS since these are term vectors, not KNN vectors.
Contributor
Author
Contributor
|
Oops - copy/paste error |
github-project-automation
bot
moved this from Open
to Merged
in OpenSearch Lucene & Core Performance Tracking
jpountz
added a commit
that referenced
this pull request
May 11, 2025
- more javadocs to guide usage of `INDEX` vs. `DATA`, - the `METADATA` constant is removed, as metadata files should be opened with `Directory#openChecksumIndexInput`, which doesn't take hints, - configure `FileTypeHint` on more files of the default codec, - remove checks on `FileTypeHint` from `toReadAdvice` - the default impl should only look at `DataAccessHint` to determine the appropriate read advice.
weizijun
added a commit
to weizijun/lucene
that referenced
this pull request
May 16, 2025
* main: (31 commits) Fix termination condition in TestStressNRTReplication. (apache#14665) deps(java): bump com.gradle.develocity from 3.19 to 3.19.2 (apache#14662) Build: remove hard-coded Java versions from ecj.javadocs.prefs (apache#14651) Update verifier comment to show label (apache#14658) Catch and re-throw Throwable rather than using a success boolean (apache#14633) Mention label in changelog verifier comment (apache#14656) Enable PR actions in changelog verifier (apache#14644) Fix FuzzySet#getEstimatedNumberUniqueValuesAllowingForCollisions to properly account for hashCount (apache#14614) Don't perform additional KNN querying after timeout, fixes apache#14639 (apache#14640) Add instructions to help/IDEs.txt for VSCode and Neovim (apache#14646) build(deps): bump ruff from 0.11.7 to 0.11.8 in /dev-tools/scripts (apache#14603) deps(java): bump de.jflex:jflex from 1.8.2 to 1.9.1 (apache#14583) Use the preload hint on completion fields and memory terms dictionaries. (apache#14634) Clean up FileTypeHint a bit. (apache#14635) Expressions: Improve test to use a fully private class or method Remove deprecations in expressions (apache#14641) removing constructor with deprecated attribute 'onlyLongestMatch (apache#14356) Moving CHANGES entry for apache#14609 from 11.0 to 10.3 (apache#14638) Overrides rewrite in PointRangeQuery to optimize AllDocs/NoDocs cases (apache#14609) Adding benchmark for histogram collector over point range query (apache#14622) ... # Conflicts: # lucene/CHANGES.txt
|
Hi @thecoop @jpountz ,
In this PR we have removed the change in MMAPDirectory to apply read advice as normal. So if no data access hint is provided, it falls back to default read advice which is "random". So read advice for this file has changed from default to random ? Can you please confirm if I'm missing something and if not, then should we revert back the read advice to normal ? |
Contributor
|
The logic in |
|
@thecoop But that is reverted in this PR, isn't it ? am i missing something ? 
|
Contributor
|
Ah, yes it does. However, #15040 (also applied to 10.3) changes the default read advice to NORMAL |
|
Yes got it thanks :) I just noted this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
INDEXvs.DATA,METADATAconstant is removed, as metadata files should be opened withDirectory#openChecksumIndexInput, which doesn't take hints,FileTypeHinton more files of the default codec,FileTypeHintfromtoReadAdvice- the default impl should only look atDataAccessHintto determine the appropriate read advice.