Skip to content

Commit aa11941

Browse files
preved911dhui
andauthored
Adding support clickhouse cluster (#568)
* added url query for creating schema_migrations table on cluster Signed-off-by: Ildar Valiullin <preved.911@gmail.com> * changed documentation by clickhouse databases Signed-off-by: Ildar Valiullin <preved.911@gmail.com> Co-authored-by: Dale Hui <dhui@users.noreply.github.com>
1 parent df6106d commit aa11941

2 files changed

Lines changed: 22 additions & 7 deletions

File tree

‎database/clickhouse/README.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
|------------|-------------|
77
| `x-migrations-table`| Name of the migrations table |
88
| `x-migrations-table-engine`| Engine to use for the migrations table, defaults to TinyLog |
9+
| `x-cluster-name` | Name of cluster for creating `schema_migrations` table cluster wide |
910
| `database` | The name of the database to connect to |
1011
| `username` | The user to sign in as |
1112
| `password` | The user's password |
@@ -18,4 +19,7 @@
1819
* The Clickhouse driver does not natively support executing multipe statements in a single query. To allow for multiple statements in a single migration, you can use the `x-multi-statement` param. There are two important caveats:
1920
* This mode splits the migration text into separately-executed statements by a semi-colon `;`. Thus `x-multi-statement` cannot be used when a statement in the migration contains a string with a semi-colon.
2021
* The queries are not executed in any sort of transaction/batch, meaning you are responsible for fixing partial migrations.
21-
* Using the default TinyLog table engine for the schema_versions table prevents backing up the table if using the [clickhouse-backup](https://github.com/AlexAkulov/clickhouse-backup) tool. If backing up the database with make sure the migrations are run with `x-migrations-table-engine=MergeTree`.
22+
* Using the default TinyLog table engine for the schema_versions table prevents backing up the table if using the [clickhouse-backup](https://github.com/AlexAkulov/clickhouse-backup) tool. If backing up the database with make sure the migrations are run with `x-migrations-table-engine=MergeTree`.
23+
* Clickhouse cluster mode not officially supported, because not covered by tests right now, but you can try enable `schema_migrations` table replication:
24+
* When `x-cluster-name` specified, `x-migrations-table-engine` also should be specify. Read about [replicated table engines](https://clickhouse.tech/docs/en/engines/table-engines/mergetree-family/replication/#table_engines-replication).
25+
* `x-cluster-name` param only specify `schema_migrations` table replication by given cluster. You should still write your migrations so that the application tables are replicated within the cluster.

‎database/clickhouse/clickhouse.go‎

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ var (
3030

3131
type Config struct {
3232
DatabaseName string
33+
ClusterName string
3334
MigrationsTable string
3435
MigrationsTableEngine string
3536
MultiStatementEnabled bool
@@ -98,6 +99,7 @@ func (ch *ClickHouse) Open(dsn string) (database.Driver, error) {
9899
MigrationsTable: purl.Query().Get("x-migrations-table"),
99100
MigrationsTableEngine: migrationsTableEngine,
100101
DatabaseName: purl.Query().Get("database"),
102+
ClusterName: purl.Query().Get("x-cluster-name"),
101103
MultiStatementEnabled: purl.Query().Get("x-multi-statement") == "true",
102104
MultiStatementMaxSize: multiStatementMaxSize,
103105
},
@@ -227,12 +229,21 @@ func (ch *ClickHouse) ensureVersionTable() (err error) {
227229
}
228230

229231
// if not, create the empty migration table
230-
query = fmt.Sprintf(`
231-
CREATE TABLE %s (
232-
version Int64,
233-
dirty UInt8,
234-
sequence UInt64
235-
) Engine=%s`, ch.config.MigrationsTable, ch.config.MigrationsTableEngine)
232+
if len(ch.config.ClusterName) > 0 {
233+
query = fmt.Sprintf(`
234+
CREATE TABLE %s ON CLUSTER %s (
235+
version Int64,
236+
dirty UInt8,
237+
sequence UInt64
238+
) Engine=%s`, ch.config.MigrationsTable, ch.config.ClusterName, ch.config.MigrationsTableEngine)
239+
} else {
240+
query = fmt.Sprintf(`
241+
CREATE TABLE %s (
242+
version Int64,
243+
dirty UInt8,
244+
sequence UInt64
245+
) Engine=%s`, ch.config.MigrationsTable, ch.config.MigrationsTableEngine)
246+
}
236247

237248
if strings.HasSuffix(ch.config.MigrationsTableEngine, "Tree") {
238249
query = fmt.Sprintf(`%s ORDER BY sequence`, query)

0 commit comments

Comments
 (0)