Contributing to MySQL QA - Ideas wanted

MySQL has recently started a campaign of open contribution, inviting the community to participate to the MySQL project in many ways.

The next target, also considering the higer stakes coming from the MySQL Enterprise challenge, will be Quality Assurance.

Quality Assurance

Now what is Quality Assurance (QA)? If you think that it's just bug hunting, then you have a simplistic view of the software generation lifecycle. QA deals with all the steps in the software lifecycle, and at each steps there are actions that can affect the quality of the final outcome. QA components include (but are not limited to) failure testing, statistical control, process and performance control, best practice adoption.

MySQL AB has its own QA department. Those of you who attended the MySQL Users Conference may have had a chance to attend a presentation by senior QA manager Omer BarNir about Internal QA in Open Source Development, where he explains the challenge of being a QA professional in such a dynamic company (presentation slides ).

Software testing

Since MySQL is a software company, finding bugs is important, of course, and software testing is one of the main branches of QA, but the crucial part of finding bugs is not how many you can find, but when you find them. The earlier you find a bug, the less costly will be to fix it.

Looking for bugs as soon as a feature pops up in the development lifecycle is just part of the task. The way you look for bugs is different from person to person, depending on who you are:

• a developer with an intimate knowledge of the code may find low level bugs that nobody else can catch, with tools like code peer reviews and walkthroughs;
  
• a professional tester, even without development experience, may find functional bugs by testing the application with a methodical approach;
  
• the final user, who does not have to know any of the above, may find bugs by simply using the application and comparing the results with earlier expectations.

Of all these three methods of finding bugs, the first one is the most effective. Bugs found at that stage are the least expensive to fix. But of course one can't find all the bugs with code inspections. And then there are the other two levels, which catch bugs that will cost more effort to fix.

Obviously, the bugs that affect the final users are the ones for which you care most, and it would be of mutual benefit to find these bugs as early as possible.

Community involvement

From the above explanation, you can see what the problem is. No matter how skilled are the professionals at this job, finding all the bugs is impossible. You can take any book on this subject, and every one of them will tell you the same postulate: You will not find all bugs. One thought is especially discouraging for us. Although the QA professionals find a lot of bugs, those which you neve hear of, because they are fixed before you hit the download button to get the application, there are still the ones that affect you, the final user, the most important party involved.

Therefore, here is the idea. We want to involve the community of users in our Quality Assurance activities.

How can we do that? To tell you the truth, we have plenty of ideas on what we can do and how to promote it. After all, until one month ago I was still just one of the many community members, and I have some ideas.And there are many brilliant people in house who came up with promising ideas.

But at this stage I don't want to tell you what I think it should be done, but rather listening to what users propose.

The questions to answer are:

• how can MySQL involve the users in earlier bug finding?
  
• how can MySQL involve the users in its testing process?
  
• what else can community members do for QA? (hints: bugs verification, performance testing, standard compliance testing)
  
• What kind of incentive would make you (more) willing to cooperate? (hints: public recognition, free services, discount on services, contests with prizes)

If you have an idea related to this topic, even if it is not an answer to the above questions, write a comment to this post, or drop me a line (giuseppe {mind the spelling} at mysql dot com).

Thanks in advance!

Also published at O'Reilly Databases