-
Notifications
You must be signed in to change notification settings - Fork 9
Fix remote call of s3Cluster function #583
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
ce467f0
fc0eef8
812d4b3
abc65f3
b5e6dde
ed31a9f
46fa2d1
8ac4341
9abc929
85495bd
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -440,7 +440,7 @@ def test_parallel_distributed_insert_select_with_schema_inference(started_cluste | |
| node.query( | ||
| """ | ||
| CREATE TABLE parallel_insert_select ON CLUSTER 'first_shard' (a String, b UInt64) | ||
| ENGINE=ReplicatedMergeTree('/clickhouse/tables/{shard}/insert_select_with_replicated', '{replica}') | ||
| ENGINE=ReplicatedMergeTree('/clickhouse/tables/{shard}/parallel_insert_select', '{replica}') | ||
| ORDER BY (a, b); | ||
| """ | ||
| ) | ||
|
|
@@ -665,3 +665,57 @@ def test_cluster_default_expression(started_cluster): | |
| ) | ||
|
|
||
| assert result == expected_result | ||
|
|
||
|
|
||
| def test_remote_hedged(started_cluster): | ||
| node = started_cluster.instances["s0_0_0"] | ||
| pure_s3 = node.query( | ||
| """ | ||
| SELECT * from s3( | ||
| 'http://minio1:9001/root/data/{clickhouse,database}/*', | ||
| 'minio', 'minio123', 'CSV', | ||
| 'name String, value UInt32, polygon Array(Array(Tuple(Float64, Float64)))') | ||
| ORDER BY (name, value, polygon) | ||
| LIMIT 1 | ||
| """ | ||
| ) | ||
| s3_distributed = node.query( | ||
| """ | ||
| SELECT * from remote('s0_0_1', s3Cluster( | ||
| 'cluster_simple', | ||
| 'http://minio1:9001/root/data/{clickhouse,database}/*', 'minio', 'minio123', 'CSV', | ||
| 'name String, value UInt32, polygon Array(Array(Tuple(Float64, Float64)))')) | ||
| ORDER BY (name, value, polygon) | ||
| LIMIT 1 | ||
| SETTINGS use_hedged_requests=True | ||
| """ | ||
| ) | ||
|
|
||
| assert TSV(pure_s3) == TSV(s3_distributed) | ||
|
|
||
|
|
||
| def test_remote_no_hedged(started_cluster): | ||
| node = started_cluster.instances["s0_0_0"] | ||
| pure_s3 = node.query( | ||
| """ | ||
| SELECT * from s3( | ||
| 'http://minio1:9001/root/data/{clickhouse,database}/*', | ||
| 'minio', 'minio123', 'CSV', | ||
| 'name String, value UInt32, polygon Array(Array(Tuple(Float64, Float64)))') | ||
| ORDER BY (name, value, polygon) | ||
| LIMIT 1 | ||
| """ | ||
| ) | ||
| s3_distributed = node.query( | ||
| """ | ||
| SELECT * from remote('s0_0_1', s3Cluster( | ||
| 'cluster_simple', | ||
| 'http://minio1:9001/root/data/{clickhouse,database}/*', 'minio', 'minio123', 'CSV', | ||
| 'name String, value UInt32, polygon Array(Array(Tuple(Float64, Float64)))')) | ||
| ORDER BY (name, value, polygon) | ||
| LIMIT 1 | ||
| SETTINGS use_hedged_requests=False | ||
| """ | ||
| ) | ||
|
|
||
| assert TSV(pure_s3) == TSV(s3_distributed) | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is the PR really s3Cluster specific?
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No, affects all *Cluster object storage functions. Suggest to make test for others?
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not sure about new tests, may be. Depends on your feeling how probably is to break something that worked before or accidentally create a "bridge" that e.g. bypasses security.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added test for iceberg, most actual for us. |
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
According to the comment in ClientInfo.h
when query_kind == INITIAL_QUERY
initial_query_id is equal to current.
Does not it contradict with the condition?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Technically it must be some new kind, something like
REMOTE_INITIAL_QUERY, but this breaks backward compatibility on protocol level.