Building MySQL 5.1 from a source tree on Mac OS X

The objective of this exercise is compiling MySQL from a source tree (BitKeeper) on Mac OS X.

I bought a new laptop, a MacBook running Mac OS X Tiger. I was captivated by the user interface, and I was willing to try this new experience, after a long and successful relationship with Linux (I gave up Windows many years ago). Everything went well, until the moment I tried to compile MySQL from source.
Using the default Xcode tools, I was able to compile the source packages provided with the GA release (5.0.x). But with 5.1 it was a different story. I was not even able to complete the compile part. something was breaking quite soon during the process.

After unsuccessfully trying all the tools made by the MySQL Build Department, I finally decided to take things into my own hands, and to create a reliable compiling environment.

I - Isolating the environment

I decided that the sanest course of action was to get the most recent build tools and then I checked whether I could get some of them via Fink or apt-get. Simply running "fink selfupdate" did most of the necessary work. After that I had a very recent version of gcc/g++.
The GNU tools for building were not available through the standard interface, and then I had to compile them from scratch. I was a bit reluctant, because I could disrupt some existing dependency, and hinder the functionality of the Mac OS X development tools. Thus, I set up a limited environment for building.

• create a directory $HOME/usr/local/bin

• change the PATH to include the new binary directory at the start.

export PATH=$HOME/usr/local/bin:$PATH

Now whenever I start a build task, programs in the new path will take precedence over the built-in ones. This way I can still use the development tools to make Mac specific applications, and at the same time I can use the most recent tools for my job related purposes.

II - Making the building tools

Once the path is set, it's time to build the tools.

• compile the new tools in the following order:
• autoconf-2.61 http://ftp.gnu.org/gnu/autoconf/autoconf-2.61.tar.gz
• libtool-1.5.22 http://ftp.gnu.org/gnu/libtool/libtool-1.5.22.tar.gz
• automake-1.10 http://ftp.gnu.org/gnu/automake/automake-1.10.tar.gz
• byacc-20050813 ftp://invisible-island.net/byacc/byacc.tar.gz
• bison-2.3 http://ftp.gnu.org/gnu/bison/bison-2.3.tar.gz

For each tool, I issued this command:

./configure --prefix=$HOME/usr/local && make && make check && make install

automake took almost 40 minutes (it has a huge regression test suite) but in the end, I managed to get all the tools in working shape. It's important to do the installation in the above order. Testing of automake fails if autoconf and libtool were not already compiled and installed. I know because I tried to build all of them in parallel and it failed.

III - Using them

Now that I have the tools, I can do something practical.

$ bk clone username@bk_repository/mysql-5.1 51
$ cd 51
$ BUILD/compile-dist
$ make test
$ make dist

And it works!

BTW, BUILD/compile-dist and make dist are the commands used by the MySQL build team to create a source distribution. You can do that as well from a development source tree.

Caveat

This material is my personal experience (which is quite limited with Mac OS X). The reason for installing so many packages from source is because the ones that are provided natively fail sooner or later during the task that I needed to perform.
If any Mac OS X expert has a better solution for this particular problem, I will listen.

Update

Being a newbie at Mac OS X, I missed the simple point that I could have installed the needed tools with Darwin Ports. I would have saved time and it would have taken care of the dependency issues. Good to know for next time!

What is a bug?

It happens sometimes, when I report a bug, that I have an argument with someone at the receiving end of the reporting chain. The raw happens over the definition of a bug. For instance, there is a new implementation of a consolidated tool. The new tool does almost all the old one did, except X. Therefore I file a bug report saying that X is missing. The ensuing argument runs along the lines of:
- This is not a bug, says the Verifier. You can't say it's a bug because it does not do what you want. It should be downgraded to feature request.
- This is the recommended replacement of the old tool, I retort, and as such it should do at least what the old tool did, plus the new stuff. I insist it is a bug.
- The manual does not mention feature X for the new tool, and then it is not a bug. It's a feature request"
The Verifier's reasoning is technically correct, and it is within the boundaries of his allowed action, so I see no point in arguing much more.
However, I can start a campaign of educating people on bug evaluation, in hope that these notes reach the ones in charge of writing the guidelines, so that this kind of requests is met more kindly next time.

So, what exactly is a Bug?

I start by borrowing some definitions from Ron Patton's excellent book on Software testing. According to that, it's a bug when ...

1. The software doesn’t do something that the product specification says it should do.
2. The software does something that the product specification says it shouldn’t do.
3. The software does something that the product specification doesn’t mention.
4. The software doesn’t do something that the product specification doesn’t mention but should.
5. The software is difficult to understand, hard to use, slow, or—in the software tester’s eyes—will be viewed by the end user as just plain not right.

(There is an online sample chapter, The realities of software testing containing the above definition)
Let's comment on each point.

Broken promise (positive)

The software doesn’t do something that the product specification says it should do.
This is a no brainer. If you said that the editor would save my file when I hit the "save" button, and it doesn't, then it's a bug.

Broken promise (negative)

The software does something that the product specification says it shouldn’t do.
This is as simple as the previous one. If you say that the program won't overwrite my painfully composed setup file and instead it silently restores the defaults, then it's a bug.

Collateral damage

The software does something that the product specification doesn’t mention.
Here we enter in tricky territory. There is an extra feature not mentioned in the manual. Sometimes you should be just happy and say thank you, for instance if your word processor can handle simple spreadsheet-like calculations in table cells. But if the same word processor makes a hidden copy of every document you edit for backup purposes, you may get quite angry. Either way, is anything of this sort happens, then it's a bug.

Forgotten specs

The software doesn’t do something that the product specification doesn’t mention but should.
At this point, we start bickering loudly. If your word processor's manual does not mention the "save" option, and such option is nowhere to be found in the menu and task bars, anyone faintly computer-literate would complain that this omission is inexcusable. If such a missing feature happens, then it's a bug.

Catch-all clause

The software is difficult to understand, hard to use, slow, or—in the software tester’s eyes—will be viewed by the end user as just plain not right.

And now the bickering becomes open fight. You, the user, have the right to comfortable usage. The specifications warns you that you may get unpleasant side effects when using a killer application. You understand it and run it anyway, but it happens that the mentioned side effects are causing real damage. Guess what? It's a bug. No matter if it was in the docs. It should have been under the "known bugs" section. For instance, the docs may warn you that a system monitor, to keep you informed of the health of a database, will poll the server using a non-existing user. That sounds fine, but later, when your intrusion detection system rings you at 3am saying that the number of failed login attempts has reached the critical level and as a safety measure the database server was shut down, then, I can't help it, it's a bug.

This latest rule must not be used as an excuse to pass any request as a bug report. It's hard to set a clear boundary, but sometimes, without this fifth rule, many critical bugs would remain latent.