Filter manifest files based on partition summaries

[scott-routledge2]

Changes included in this PR

Filter manifest files based on partition summaries when constructing parquet infos. This can avoid a lot of overhead reading from slow storage e.g. s3 in the case where there are many manifest files but only a few that will match the filter.

Testing on S3Tables with ~2700 data files and 48 manifest files (but only 1 manifest file actually matches the filter), time to read a small section of the table went from 10s to 5s.

Testing strategy

S3tables benchmark, CI

User facing changes

Checklist

• Pipelines passed before requesting review. To run CI you must include [run CI] in your commit message.
• I am familiar with the Contributing Guide
• I have installed + ran pre-commit hooks.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 68.86%. Comparing base (c33fbb5) to head (df612a9).
⚠️ Report is 131 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #938      +/-   ##
==========================================
+ Coverage   66.68%   68.86%   +2.18%     
==========================================
  Files         186      195       +9     
  Lines       66795    67643     +848     
  Branches     9507     9611     +104     
==========================================
+ Hits        44543    46584    +2041     
+ Misses      19572    18227    -1345     
- Partials     2680     2832     +152     

[IsaacWarren]

Nice! Thanks Scott

[ehsantn]

Thanks @scott-routledge2 .