Firefox 3.6.0: behind the scenes with unittests and performance tests

As you all know by now, Firefox 3.6.0 shipped today. The new features and improvements are already being covered by other people in great detail, so I thought I’d focus on something totally different that happened behind the scenes for this release.

Firefox 3.6.0 is the first release where we ran all our automated unittests and talos performance suites on the officially branded release builds (you can see the test results for yourself by clicking on the image!).

As you know, we run unittests and performance tests on unbranded builds for each checkin, and also on the unbranded nightly builds. However, releases are branded builds – and until now, the best we could do was run unittests and performance suites on unbranded builds, and if all looked good, then branch from that same source code, rebuild with branding enabled, and hand those branded builds to QA for testing and release.

While this approach has worked “good enough” for years now, we had to carefully track the possible differences between unbranded and branded builds, and hope that compile changes caused by branding was covered by manual testing. To fix this required some fiddly reworking of unittests and talos performance suites. For details, the curious can read through bug#383136, bug#457753, bug#517212 and all of their linked bugs.

Big tip-o-the-hat to catlee for making this happen, and hope you enjoy your new Firefox!.