I remember wanting to create MySQL Cluster sandboxes several years ago. By then, however, MySQL-Sandbox technology was not flexible enough to allow an easy inclusion, and the cluster software itself was not as easy to install as it is today. Thus, I kept postponing the implementation, until I started working with dbdeployer.
I included the skeleton of support for MySQL Cluster since the beginning (by keeping a range of ports dedicated for this technology, but I didn’t do anything until June 2018, when I made public my intentions to add support for NDB in dbdeployer with issue #20 (Add support for MySQL Cluster)). The issue had just a bare idea, but I needed help from someone, as my expertise with NDB was limited, and outdated.
Help came in November, when Daniël van Eeden started giving me bits of information on how to put together a cluster sandbox. I still resisted forcing my hand at the implementation, because by then I had realised that my method of checking the database server version to know whether it supported a given feature was inadequate to support anything other than vanilla MySQL or fully complaint forks.
The game changer was the cooperation with TiDB that opened the way for supporting Percona XtraDB Cluster. Even though these technologies are way different from MySQL Cluster, they forced me to improve dbdeployer’s code, making it more flexible, easier to enhance.
When I finally decided to start working on NDB, it took me only a few days to implement it, because I had all the pieces ready for this technology to become part of dbdeployer.
Following Däniel’s instructions, I had a prototype working, which I submitted to #dbdeployer channel on MySQL community slack. In that channel, I got help again from Däniel van Eeden, and then Frédéric Descamps summoned two more experts (Ted Wennmark and Bernd Ocklin), who gave me feedback, advice, and some quick lessons on how the cluster should work, which allowed me to publish a release (dbdeployer 1.23.0) this past week-end.
The implementation has some peculiarities for both users of dbdeployer and MySQL Cluster. For the ones used to dbdeployer, the biggest change is that we are deploying two entities, of which the main one is an NDB cluster, with its own directories and processes, while the MySQL servers are just the visible part of the cluster, but are, in fact, only cluster clients. Still, the cluster works smoothly in dbdeployer paradigm: the cluster is deployed (like Group replication or PXC) as a replication topology, and as such we can run the standard replication test and expect to get the same result that we would see when checking another multi-source deployment.
For people used to NDB, though, seeing NDB as “replication” feels odd, because the cluster is seeing as a distinct entity, and replication is when we transfer data between two clusters. If we were developing a dedicated tool for NDB clusters, this is probably what we would have done, but since we want dbdeployer integration, we must play by the general rules of the tool, where “single” is a stand-alone server instance, and we can’t have dbdeployer deploy single --topology=ndb, because single instance don’t have a topology, which is a property of a group of entities. Therefore, the price to pay for dbdeployer support accepting to see a MySQL cluster deployment as replication.
Now that we have covered all the philosophical angle, it’s time to show an example. Unlike PXC, which is requires Linux, MySQL Cluster can also run on MacOS, which makes my testing much easier.
The first step to run a cluster in dbdeployer is to download a tarball from dev.mysql.com/downloads/cluster, and then expand it in our usual directory ($HOME/opt/mysql):
$ dbdeployer unpack --prefix=ndb --flavor=ndb \
~/Downloads/mysql-cluster-gpl-7.6.9-macos10.14-x86_64.tar.gz
Unpacking tarball $HOME/Downloads/mysql-cluster-gpl-7.6.9-macos10.14-x86_64.tar.gz to $HOME/opt/mysql/ndb7.6.9
[...]
Renaming directory $HOME/opt/mysql/mysql-cluster-gpl-7.6.9-macos10.14-x86_64 to $HOME/opt/mysql/ndb7.6.9
We can repeat the same operation for MySQL Cluster 8.0.14, and in the end we will two expanded tarballs named ndb7.6.9 and ndb8.0.14. With this we can install a few clusters in the same host:
$ dbdeployer deploy replication ndb7.6 --topology=ndb --concurrent
# ndb7.6 => ndb7.6.9
$HOME/sandboxes/ndb_msb_ndb7_6_9/initialize_nodes
MySQL Cluster Management Server mysql-5.7.25 ndb-7.6.9
2019-03-18 23:47:15 [ndbd] INFO -- Angel connected to 'localhost:20900'
2019-03-18 23:47:16 [ndbd] INFO -- Angel allocated nodeid: 2
2019-03-18 23:47:16 [ndbd] INFO -- Angel connected to 'localhost:20900'
2019-03-18 23:47:16 [ndbd] INFO -- Angel allocated nodeid: 3
executing 'start' on node 1
................ sandbox server started
executing 'start' on node 2
.. sandbox server started
executing 'start' on node 3
.. sandbox server started
NDB cluster directory installed in $HOME/sandboxes/ndb_msb_ndb7_6_9
run 'dbdeployer usage multiple' for basic instructions'
$ dbdeployer deploy replication ndb8.0 --topology=ndb --concurrent
# ndb8.0 => ndb8.0.14
$HOME/sandboxes/ndb_msb_ndb8_0_14/initialize_nodes
MySQL Cluster Management Server mysql-8.0.14 ndb-8.0.14-dmr
2019-03-18 23:45:53 [ndbd] INFO -- Angel connected to 'localhost:21400'
2019-03-18 23:45:53 [ndbd] INFO -- Angel allocated nodeid: 2
2019-03-18 23:45:53 [ndbd] INFO -- Angel connected to 'localhost:21400'
2019-03-18 23:45:53 [ndbd] INFO -- Angel allocated nodeid: 3
executing 'start' on node 1
........ sandbox server started
executing 'start' on node 2
... sandbox server started
executing 'start' on node 3
.. sandbox server started
NDB cluster directory installed in $HOME/sandboxes/ndb_msb_ndb8_0_14
run 'dbdeployer usage multiple' for basic instructions'
If we look at the sandbox directories, we will see a few more subdirectories than we usually have with other topologies. For example:
ndb_conf # cluster configuration
ndbnode1 # management node (1)
ndbnode2 # data node (2)
ndbnode3 # data node (3)
node1 # MySQL node 1
node2 # MySQL node 2
node3 # MySQL node 3
The clusters are well framed into dbdeployer’s architecture, and they respond to standard commands like any other sandbox:
$ dbdeployer sandboxes --full-info
.-------------------.------.-----------.----------------------------------------------.--------.-------.--------.
| name | type | version | ports | flavor | nodes | locked |
+-------------------+------+-----------+----------------------------------------------+--------+-------+--------+
| ndb_msb_ndb7_6_9 | ndb | ndb7.6.9 | [20900 27510 27511 27512 ] | ndb | 3 | |
| ndb_msb_ndb8_0_14 | ndb | ndb8.0.14 | [21400 28415 38415 28416 38416 28417 38417 ] | ndb | 3 | |
'-------------------'------'-----------'----------------------------------------------'--------'-------'--------'
$ dbdeployer global status
# Running "status_all" on ndb_msb_ndb7_6_9
MULTIPLE /Users/gmax/sandboxes/ndb_msb_ndb7_6_9
node1 : node1 on - port 27510 (27510)
node2 : node2 on - port 27511 (27511)
node3 : node3 on - port 27512 (27512)
# Running "status_all" on ndb_msb_ndb8_0_14
MULTIPLE /Users/gmax/sandboxes/ndb_msb_ndb8_0_14
node1 : node1 on - port 28415 (28415)
node2 : node2 on - port 28416 (28416)
node3 : node3 on - port 28417 (28417)
$ dbdeployer global test-replication
# Running "test_replication" on ndb_msb_ndb7_6_9
# master 1
# master 2
# master 3
# slave 1
ok - '3' == '3' - Slaves received tables from all masters
# slave 2
ok - '3' == '3' - Slaves received tables from all masters
# slave 3
ok - '3' == '3' - Slaves received tables from all masters
# pass: 3
# fail: 0
# Running "test_replication" on ndb_msb_ndb8_0_14
# master 1
# master 2
# master 3
# slave 1
ok - '3' == '3' - Slaves received tables from all masters
# slave 2
ok - '3' == '3' - Slaves received tables from all masters
# slave 3
ok - '3' == '3' - Slaves received tables from all masters
# pass: 3
# fail: 0
Like other topologies, also the NDB cluster has a script that shows the status of the nodes:
$ ~/sandboxes/ndb_msb_ndb7_6_9/check_nodes
+---------+-----------+---------------+--------+---------+-------------+-------------------+
| node_id | node_type | node_hostname | uptime | status | start_phase | config_generation |
+---------+-----------+---------------+--------+---------+-------------+-------------------+
| 2 | NDB | localhost | 58 | STARTED | 0 | 1 |
| 3 | NDB | localhost | 58 | STARTED | 0 | 1 |
| 1 | MGM | localhost | NULL | NULL | NULL | NULL |
| 4 | API | localhost | NULL | NULL | NULL | NULL |
| 5 | API | localhost | NULL | NULL | NULL | NULL |
| 6 | API | localhost | NULL | NULL | NULL | NULL |
| 7 | API | localhost | NULL | NULL | NULL | NULL |
+---------+-----------+---------------+--------+---------+-------------+-------------------+
Connected to Management Server at: localhost:20900
Cluster Configuration
---------------------
[ndbd(NDB)] 2 node(s)
id=2 @127.0.0.1 (mysql-5.7.25 ndb-7.6.9, Nodegroup: 0, *)
id=3 @127.0.0.1 (mysql-5.7.25 ndb-7.6.9, Nodegroup: 0)
[ndb_mgmd(MGM)] 1 node(s)
id=1 @127.0.0.1 (mysql-5.7.25 ndb-7.6.9)
[mysqld(API)] 4 node(s)
id=4 @127.0.0.1 (mysql-5.7.25 ndb-7.6.9)
id=5 @127.0.0.1 (mysql-5.7.25 ndb-7.6.9)
id=6 @127.0.0.1 (mysql-5.7.25 ndb-7.6.9)
id=7 (not connected, accepting connect from localhost)
It is possible that we will need more iterations to make the deployment more robust. When testing it, keep in mind that this deployment is only for testing, and it won’t probably have all the performance that you may find in a well deployed production cluster. Still, compared to other topologies, the replication tests performed faster than I expected.
No comments:
Post a Comment