Skip to content

Forward port parquet metadata cache impl #715

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Enmk merged 13 commits into from
Apr 6, 2025
Merged

Forward port parquet metadata cache impl #715

Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
3ac5403
Merge pull request #586 from Altinity/metadata_cache_for_parquet_24_12_2
Enmk Feb 5, 2025
93ca792
fix flaky test
arthurpassos Feb 27, 2025
000d75e
use parquet metadata cache for parquetmetadata format as well
arthurpassos Feb 18, 2025
04399bc
add test, see what CICD says
arthurpassos Feb 18, 2025
cbc5c5f
simplify dependencies by forward declaring parquet structure
arthurpassos Feb 18, 2025
a649737
fix tests
arthurpassos Feb 19, 2025
0914a05
Metadata cache on by default part 2
arthurpassos Mar 6, 2025
707d2a4
Update SettingsChangesHistory.cpp
arthurpassos Mar 6, 2025
4877cd1
Update SettingsChangesHistory.cpp
Enmk Mar 6, 2025
89196b7
Update SettingsChangesHistory.cpp
Enmk Mar 6, 2025
f1de4b7
Merge branch 'antalya-25.2' into metadata_cache_impl
arthurpassos Apr 5, 2025
17e9f57
Merge branch 'antalya-25.2' into metadata_cache_impl
arthurpassos Apr 6, 2025
01e4f46
rebase
arthurpassos Apr 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions programs/server/Server.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -313,6 +313,7 @@ namespace ServerSetting
extern const ServerSettingsUInt64 max_prefixes_deserialization_thread_pool_size;
extern const ServerSettingsUInt64 max_prefixes_deserialization_thread_pool_free_size;
extern const ServerSettingsUInt64 prefixes_deserialization_thread_pool_thread_pool_queue_size;
extern const ServerSettingsUInt64 input_format_parquet_metadata_cache_max_size;
}

}
Expand Down Expand Up @@ -2310,6 +2311,7 @@ try
dns_cache_updater->start();

auto replicas_reconnector = ReplicasReconnector::init(global_context);
ParquetFileMetaDataCache::instance()->setMaxSizeInBytes(server_settings[ServerSetting::input_format_parquet_metadata_cache_max_size]);

/// Set current database name before loading tables and databases because
/// system logs may copy global context.
Expand Down
Loading