Tungsten Replicator cookbook. Advanced replication topologies made easy

I have been asked many times to provide an easy way of deploying fan-in and star schema replication schemas. So far, I have been delayed by more pressing duties.

Now the time has come. Since we are about to release a new version of Tungsten Replicator, I made the effort of putting together the steps for an easy deployment.

Recipes

The package (with downloads and svn code available at Tungsten-Replicator Toolbox) includes some juicy goodies. There are recipes to install.

• Master/slave, the classic replication topology. Nothing fancy, but with the tools mentioned in the next section, it becomes as valuable as the other topologies.
• All-masters. This is the Tungsten no-SPOF topology. Every node is a master, and every node has a direct slave service to every other node. A bit heavy on the network, but quite robust.
• Fan-in. The legendary multiple-source replication, where one slave can get instant updates from many masters.
• Star schema. The most efficient multiple-master deployment, where all the nodes are connected through a central hub. Here the trade-off is less traffic in exchange for a SPOF.

For each topology, there is a NODES_xxxx.sh file, which you need to edit, to add the list of your nodes. The nodes must be reachable by the O.S. account used for the installation, using ssh wit an authentication key (and no password).

Once you have set the list of nodes (the README file has more details), you can run the installer using the corresponding ./cookbook/install_xxxxx.sh.

Easier administration

In addition to the recipes, there are some tools that come with the package. For each topology, there is a script that shows the cluster, one that performs a simple replication test, checking that data generated in the masters reaches all the slaves, and a script that removes all replication in one go. Again, the README file has all the details.

All the scripts are written in (hopefully simple) Bash shell language. You can use them as they are, or use them as a basis to create additional administration tools.

Happy hacking!

MySQL Sandbox and laziness

Laziness strikes again. MySQL Sandbox was created with the intent of avoiding repetitive work when creating and using several servers. Turns out that even the current framework, which many say that is really time saving and enhances productivity, was not enough. So my desire for laziness, which is, as everybody should know a chief virtue for a programmer has made me code a shortcut script, which can joggle sandboxes as never before.

Enter the sb script (available in version 3.0.03). Now I can shorten my typing experience with sandboxes quite a lot:

  $ sb 5135
  # same as calling 
  # $SANDBOX_HOME/msb_5_1_35/use

Nice, isn't it? It saves me quite a lot of keys. Not only that. If this sandbox is not active, the script will start it for me. That's yet a few keystrokes more that I am saving.
What if I want a replication sandbox? Easy:

  $ sb r5135
  # same as calling 
  # $SANDBOX_HOME/rsandbox_5_1_35/m

For a multiple sandbox instead of a replication one, use "m5135."
Now, for the really amazing stuff. Let's suppose that the sandbox doesn't exist yet.
No problem.

  $ sb 5.1.35
  # same as calling 
  # make_sandbox 5.1.35
  # and
  # $SANDBOX_HOME/msb_5_1_35/use

And if you don't have a pre-expanded directory for 5.1.35 under $SANDBOX_BINARY as I do? Also this is easily done:

  $ sb mysql-5.1.35-YOUR_OS.tar.gz
  # same as calling 
  # make_sandbox mysql-5.1.35-YOUR_OS.tar.gz
  # and
  # $SANDBOX_HOME/msb_5_1_35/use

Creating and using a replication sandbox is really easy as well:

  $ sb r5.1.35
  # same as calling 
  # make_replication_sandbox 5.1.35
  # and
  # $SANDBOX_HOME/rsandbox_5_1_35/m

The manual explains the above cases and some more (Yes: there is actually more).
Go on! get lazy!