|
| 1 | +--- |
| 2 | +title: Deploy Data Migration Using DM Binary |
| 3 | +summary: Learn how to deploy a Data Migration cluster using DM binary. |
| 4 | +aliases: ['/docs/tidb-data-migration/dev/deploy-a-dm-cluster-using-binary/'] |
| 5 | +--- |
| 6 | + |
| 7 | +# Deploy Data Migration Using DM Binary |
| 8 | + |
| 9 | +This document introduces how to quickly deploy the Data Migration (DM) cluster using DM binary. |
| 10 | + |
| 11 | +> **Note:** |
| 12 | +> |
| 13 | +> In the production environment, it is recommended to [use TiUP to deploy a DM cluster](/dm/deploy-a-dm-cluster-using-tiup.md). |
| 14 | +
|
| 15 | +## Preparations |
| 16 | + |
| 17 | +Download the official binary using the download link in the following table: |
| 18 | + |
| 19 | +| Package name | OS | Architecture | SHA256 checksum | |
| 20 | +|:---|:---|:---|:---| |
| 21 | +| `https://download.pingcap.org/dm-{version}-linux-amd64.tar.gz` | Linux | amd64 | `https://download.pingcap.org/dm-{version}-linux-amd64.sha256` | |
| 22 | + |
| 23 | +> **Note:** |
| 24 | +> |
| 25 | +> `{version}` in the above download link indicates the version number of TiDB. For example, the download link for `v1.0.1` is `https://download.pingcap.org/dm-v1.0.1-linux-amd64.tar.gz`. You can check the published DM versions in the [DM Release](https://github.com/pingcap/tiflow/releases) page. |
| 26 | +
|
| 27 | +The downloaded files have two subdirectories, `bin` and `conf`. The `bin` directory contains the binary files of DM-master, DM-worker, and dmctl. The `conf` directory contains the sample configuration files. |
| 28 | + |
| 29 | +## Sample scenario |
| 30 | + |
| 31 | +Suppose that you deploy a DM cluster based on this sample scenario: |
| 32 | + |
| 33 | +Two DM-worker nodes and three DM-master nodes are deployed on five servers. |
| 34 | + |
| 35 | +Here is the address of each node: |
| 36 | + |
| 37 | +| Instance | Server address | Port | |
| 38 | +| :---------- | :----------- | :-- | |
| 39 | +| DM-master1 | 192.168.0.4 | 8261 | |
| 40 | +| DM-master2 | 192.168.0.5 | 8261 | |
| 41 | +| DM-master3 | 192.168.0.6 | 8261 | |
| 42 | +| DM-worker1 | 192.168.0.7 | 8262 | |
| 43 | +| DM-worker2 | 192.168.0.8 | 8262 | |
| 44 | + |
| 45 | +Based on this scenario, the following sections describe how to deploy the DM cluster. |
| 46 | + |
| 47 | +> **Note:** |
| 48 | +> |
| 49 | +> - If you deploy multiple DM-master or DM-worker instances in a single server, the port and working directory of each instance must be unique. |
| 50 | +> |
| 51 | +> - If you do not need to ensure high availability of the DM cluster, deploy only one DM-master node, and the number of deployed DM-worker nodes must be no less than the number of upstream MySQL/MariaDB instances to be migrated. |
| 52 | +> |
| 53 | +> - To ensure high availability of the DM cluster, it is recommended to deploy three DM-master nodes, and the number of deployed DM-worker nodes must be greater than the number of upstream MySQL/MariaDB instances to be migrated (for example, the number of DM-worker nodes is two more than the number of upstream instances). |
| 54 | +> |
| 55 | +> - Make sure that the ports among the following components are interconnected: |
| 56 | +> - The `8291` ports among the DM-master nodes are interconnected. |
| 57 | +> - Each DM-master node can connect to the `8262` ports of all DM-worker nodes. |
| 58 | +> - Each DM-worker node can connect to the `8261` port of all DM-master nodes. |
| 59 | +
|
| 60 | +### Deploy DM-master |
| 61 | + |
| 62 | +You can configure DM-master by using [command-line parameters](#dm-master-command-line-parameters) or [the configuration file](#dm-master-configuration-file). |
| 63 | + |
| 64 | +#### DM-master command-line parameters |
| 65 | + |
| 66 | +The following is the description of DM-master command-line parameters: |
| 67 | + |
| 68 | +```bash |
| 69 | +./bin/dm-master --help |
| 70 | +``` |
| 71 | + |
| 72 | +``` |
| 73 | +Usage of dm-master: |
| 74 | + -L string |
| 75 | + log level: debug, info, warn, error, fatal (default "info") |
| 76 | + -V prints version and exit |
| 77 | + -advertise-addr string |
| 78 | + advertise address for client traffic (default "${master-addr}") |
| 79 | + -advertise-peer-urls string |
| 80 | + advertise URLs for peer traffic (default "${peer-urls}") |
| 81 | + -config string |
| 82 | + path to config file |
| 83 | + -data-dir string |
| 84 | + path to the data directory (default "default.${name}") |
| 85 | + -initial-cluster string |
| 86 | + initial cluster configuration for bootstrapping, e.g. dm-master=http://127.0.0.1:8291 |
| 87 | + -join string |
| 88 | + join to an existing cluster (usage: cluster's "${master-addr}" list, e.g. "127.0.0.1:8261,127.0.0.1:18261" |
| 89 | + -log-file string |
| 90 | + log file path |
| 91 | + -master-addr string |
| 92 | + master API server and status addr |
| 93 | + -name string |
| 94 | + human-readable name for this DM-master member |
| 95 | + -peer-urls string |
| 96 | + URLs for peer traffic (default "http://127.0.0.1:8291") |
| 97 | + -print-sample-config |
| 98 | + print sample config file of dm-worker |
| 99 | +``` |
| 100 | + |
| 101 | +> **Note:** |
| 102 | +> |
| 103 | +> In some situations, you cannot use the above method to configure DM-master because some configurations are not exposed to the command line. In such cases, use the configuration file instead. |
| 104 | +
|
| 105 | +#### DM-master configuration file |
| 106 | + |
| 107 | +The following is the configuration file of DM-master. It is recommended that you configure DM-master by using this method. |
| 108 | + |
| 109 | +1. Write the following configuration to `conf/dm-master1.toml`: |
| 110 | + |
| 111 | + ```toml |
| 112 | + # Master Configuration. |
| 113 | + name = "master1" |
| 114 | + |
| 115 | + # Log configurations. |
| 116 | + log-level = "info" |
| 117 | + log-file = "dm-master.log" |
| 118 | + |
| 119 | + # The listening address of DM-master. |
| 120 | + master-addr = "192.168.0.4:8261" |
| 121 | + |
| 122 | + # The peer URLs of DM-master. |
| 123 | + peer-urls = "192.168.0.4:8291" |
| 124 | + |
| 125 | + # The value of `initial-cluster` is the combination of the `advertise-peer-urls` value of all DM-master nodes in the initial cluster. |
| 126 | + initial-cluster = "master1=http://192.168.0.4:8291,master2=http://192.168.0.5:8291,master3=http://192.168.0.6:8291" |
| 127 | + ``` |
| 128 | + |
| 129 | +2. Execute the following command in the terminal to run DM-master: |
| 130 | + |
| 131 | + {{< copyable "shell-regular" >}} |
| 132 | + |
| 133 | + ```bash |
| 134 | + ./bin/dm-master -config conf/dm-master1.toml |
| 135 | + ``` |
| 136 | + |
| 137 | + > **Note:** |
| 138 | + > |
| 139 | + > The console does not output logs after this command is executed. If you want to view the runtime log, you can execute `tail -f dm-master.log`. |
| 140 | + |
| 141 | +3. For DM-master2 and DM-master3, change `name` in the configuration file to `master2` and `master3` respectively, and change `peer-urls` to `192.168.0.5:8291` and `192.168.0.6:8291` respectively. Then repeat Step 2. |
| 142 | + |
| 143 | +### Deploy DM-worker |
| 144 | + |
| 145 | +You can configure DM-worker by using [command-line parameters](#dm-worker-command-line-parameters) or [the configuration file](#dm-worker-configuration-file). |
| 146 | + |
| 147 | +#### DM-worker command-line parameters |
| 148 | + |
| 149 | +The following is the description of the DM-worker command-line parameters: |
| 150 | + |
| 151 | +{{< copyable "shell-regular" >}} |
| 152 | + |
| 153 | +```bash |
| 154 | +./bin/dm-worker --help |
| 155 | +``` |
| 156 | + |
| 157 | +``` |
| 158 | +Usage of worker: |
| 159 | + -L string |
| 160 | + log level: debug, info, warn, error, fatal (default "info") |
| 161 | + -V prints version and exit |
| 162 | + -advertise-addr string |
| 163 | + advertise address for client traffic (default "${worker-addr}") |
| 164 | + -config string |
| 165 | + path to config file |
| 166 | + -join string |
| 167 | + join to an existing cluster (usage: dm-master cluster's "${master-addr}") |
| 168 | + -keepalive-ttl int |
| 169 | + dm-worker's TTL for keepalive with etcd (in seconds) (default 10) |
| 170 | + -log-file string |
| 171 | + log file path |
| 172 | + -name string |
| 173 | + human-readable name for DM-worker member |
| 174 | + -print-sample-config |
| 175 | + print sample config file of dm-worker |
| 176 | + -worker-addr string |
| 177 | + listen address for client traffic |
| 178 | +``` |
| 179 | + |
| 180 | +> **Note:** |
| 181 | +> |
| 182 | +> In some situations, you cannot use the above method to configure DM-worker because some configurations are not exposed to the command line. In such cases, use the configuration file instead. |
| 183 | +
|
| 184 | +#### DM-worker configuration file |
| 185 | + |
| 186 | +The following is the DM-worker configuration file. It is recommended that you configure DM-worker by using this method. |
| 187 | + |
| 188 | +1. Write the following configuration to `conf/dm-worker1.toml`: |
| 189 | + |
| 190 | + ```toml |
| 191 | + # Worker Configuration. |
| 192 | + name = "worker1" |
| 193 | + |
| 194 | + # Log configuration. |
| 195 | + log-level = "info" |
| 196 | + log-file = "dm-worker.log" |
| 197 | + |
| 198 | + # DM-worker address. |
| 199 | + worker-addr = ":8262" |
| 200 | + |
| 201 | + # The master-addr configuration of the DM-master nodes in the cluster. |
| 202 | + join = "192.168.0.4:8261,192.168.0.5:8261,192.168.0.6:8261" |
| 203 | + ``` |
| 204 | + |
| 205 | +2. Execute the following command in the terminal to run DM-worker: |
| 206 | + |
| 207 | + {{< copyable "shell-regular" >}} |
| 208 | + |
| 209 | + ```bash |
| 210 | + ./bin/dm-worker -config conf/dm-worker1.toml |
| 211 | + ``` |
| 212 | + |
| 213 | +3. For DM-worker2, change `name` in the configuration file to `worker2`. Then repeat Step 2. |
| 214 | + |
| 215 | +Now, a DM cluster is successfully deployed. |
0 commit comments