Skip to content

Commit 1984914

Browse files
satymishfacebook-github-bot
authored andcommitted
Pass row filters to Hive Reader to filter rows (facebookresearch#4256)
Summary: Pass row filters to Hive Reader to filter rows. This is needed for filtering for is_high_priority=true for Unicorn dataset Reviewed By: junjieqi Differential Revision: D71874955
1 parent 6116d36 commit 1984914

1 file changed

Lines changed: 4 additions & 0 deletions

File tree

benchs/bench_fw/descriptors.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,10 @@ class DatasetDescriptor:
8888

8989
embedding_id_column: Optional[str] = None
9090

91+
# filters on the dataset where each filter is a
92+
# string rep of a filter expression
93+
filters: Optional[List[str]] = None
94+
9195
# unused in open-source
9296
splits_distribution: Optional[List[List[bytes]]] = None
9397

0 commit comments

Comments
 (0)