Skip to content

Refactor clickhouse admin integration tests#7017

Merged
karencfv merged 20 commits intooxidecomputer:mainfrom
karencfv:refactor-clickhouse-admin-tests
Nov 14, 2024
Merged

Refactor clickhouse admin integration tests#7017
karencfv merged 20 commits intooxidecomputer:mainfrom
karencfv:refactor-clickhouse-admin-tests

Conversation

@karencfv
Copy link
Copy Markdown
Contributor

@karencfv karencfv commented Nov 8, 2024

Overview

This commit changes the way the clickhouse clusters are spun up for clickhouse-admin integration tests. Instead of multiple clusters being spun up for every test, now we start up a single cluster via nextest's "setup scripts" functionality.
This way, we no longer need to manually hard code so many ports per test.

Caveats

Sadly, we still have to hardcode some ports, but at least this way it's only a set of ports, and we won't need to add more every time we want to test something new.

In order to get the teardown function working, I had to limit the tests to run on a single thread. They're a bit slower than before, but nothing too terrible. Setting up the cluster, running the tests and tearing down takes ~7s. The time to run the tests themselves is ~3s

@karencfv karencfv changed the title WIP: Refactor clickhouse admin integration tests Refactor clickhouse admin integration tests Nov 13, 2024
@karencfv karencfv marked this pull request as ready for review November 13, 2024 08:25
Copy link
Copy Markdown
Contributor

@andrewjstone andrewjstone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. All seems safe since we are just testing query functions for upgrade safety wrt parsing.

let config = DeploymentConfig {
path,
base_ports,
cluster_name: "oximeter_cluster".to_string(),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we have a constant defined for oximeter_cluster somewhere, although maybe not accessible to clickhouse-admin.

.with_log(log);

let lgif = clickhouse_cli.lgif().await.unwrap();
let lgif = clickhouse_cli.lgif().await?;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's totally fine to use ? here, but using unwrap will give you the proper line number for the failure, which may help debugging later.

Copy link
Copy Markdown
Contributor Author

@karencfv karencfv Nov 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, I wonder why I changed this. I didn't change the .unwrap() for a ? in any of the other tests 😄

// This Source Code Form is subject to the terms of the Mozilla Public
// License, v. 2.0. If a copy of the MPL was not distributed with this
// file, You can obtain one at https://mozilla.org/MPL/2.0/.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a doc comment here about how this should only be used for testing that doesn't write data to clickhouse, as that could make tests flaky.

@karencfv karencfv enabled auto-merge (squash) November 13, 2024 22:16
Copy link
Copy Markdown
Contributor Author

@karencfv karencfv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @andrewjstone 🙇‍♀️

@karencfv karencfv merged commit 63bf8cc into oxidecomputer:main Nov 14, 2024
@iliana iliana mentioned this pull request Nov 14, 2024
@karencfv karencfv deleted the refactor-clickhouse-admin-tests branch November 14, 2024 03:32
iliana added a commit that referenced this pull request Nov 14, 2024
#7017 vs #7050 semantic merge conflict.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants