Testing is as Easy as Alpha, Beta, RC

I’ve discussed the concept of vaporware on Sciencetext before. Vaporware is software promised by a company that never seems to be released. Games, operating systems, upgrades…dozens of companies are guilty of producing vaporware. Often it’s down to technical problems, occasionally it’s a marketing ploy.

What about the opposite of vaporware? Software that gets released on to an unsuspecting computer crowd well before it’s ready-by-date? Alpha versions with fractional release numbers like 0.1 and 0.2 etc are not so common these days, except maybe in the open source world. However, Windows users are currently languishing with a beta release candidate (RC) of the next version of their operating system – Windows 7. Likewise, Firefox fans are waiting expectantly for an upgrade from version 3.1 to the next major release 3.5 which is also at the beta release candidate stage.

What is a beta version?

“Beta” is a nickname for software which has passed the alpha testing stage of development and has been released to users for software testing before its official release. It is the prototype of the software that is released to the public. Beta testing allows the software to undergo usability testing with users who provide feedback, so that any malfunctions these users find in the software can be reported to the developers and fixed. Beta software can be unstable and could cause crashes or data loss.

What is a release candidate?

The term release candidate (RC) refers to a version with potential to be a final product, ready to release unless fatal bugs emerge. In this stage of quality control, all product features have been designed, coded and tested through one or more Beta version with no known showstopper-class bug.

Computer users who dig out the alpha, beta, and release candidate versions of computer software are usually slightly more savvy than the average user. They are well aware that a beta release may have a few bugs that remain to be ironed out. However, even after extensive beta testing and RC revelations, you can rest assured that the final release will still not be perfect. There are always likely to be minor (and occasionally major) problems after the official release.

So, how does a software producer make the decision as to what time in the product cycle to release their software? In one sense all software is always in the beta stage, there is never a perfect release that works on all computer configurations, doesn’t interfere with any other program, and doesn’t itself cause conflicts with hardware and software in all possible variations on use.

Software producers have their own formulas, both technical and commercial, for deciding when to jump from alpha to beta to RC and finally to the actual product. But, Tapan Bagchi of the Indian Institute of Technology Kharagpur, recently pointed out just what a risky business this is.

He has carried out a study to assess the risk of releasing defective software that cannot be exhaustively tested and of needlessly testing defect-free software. There is only so much beta testing that can be done, after all. He explains that if the testing procedures used are imperfect, then it is relatively easy for a company to release a program they think is near perfect, only to discover that real users working with the software in a real environment find one of those show-stopping bugs. An error in a 1990s version of the Excel spreadsheet is a case in point.

His analysis reveals, quite obviously really, that there is justification for raising test coverage, perfecting test methods, and adopting recent innovations and programming methods such as component-based design. Bagchi says that, “Your ‘protection’ rises proportionally to the rise in [testing] coverage.” So, it is worth maximizing testing coverage in software testing to cover all bases.

“I felt happy that the conclusions reached agree with the intuition of expert designers and experienced developers and testers of software,” Bagchi told Sciencetext.

This seems to be what Microsoft has been doing with Windows 7, because it released the time-limited beta to anyone who wants to try it. This has apparently led to as near a perfect version of the operating system release candidate as can be achieved; excluding critique from Mac Fanboys of course. We’ll have to wait until October 2009 to see how the fully fledged Win7 fares in the wild, of course.

Research Blogging Icon Bagchi, T. (2009). Risks in software development with imperfect testing International Journal of Advanced Operations Management, 1 (1) DOI: 10.1504/IJAOM.2009.026522

facebooktwittergoogle_plusredditpinterestlinkedintumblrmail