From a RelEng point of view, Firefox 3.6.3 was our fastest release yet.

• 05:28 PST 01-apr-2010: “go to build” after fix landed and builds/tests/talos all ran green
• 18:01 PST 01-apr-2010: downloads available on mozilla.org, updates available for users doing CheckForUpdates
• 12hours 30mins elapsed time

While this is great to see, this was not a perfect release for us, and we know we can do better:

• bandwidth problems between MPT and 650castro caused us wasted time Wednesday night, chasing burning builds only to figure out the bustage was not a code problem after all. This was before the “go to build”, so is not included above, but was a delay – and its unclear how much of a delay this caused.
• manual errors in the config files when starting release automation caused us to interrupt/stop/restart release-automation. The error was caught early, so no harm while recovering from that, but we forgot to manually retrigger the source tarball step. Its now all fixed, but still…
• we were caught really short handed. Most of the group was heading out on Easter vacation; one person was sick, one already out on vacation. This caused us to defer some other proposed releases for a few days. This also caused a delay in starting mirror push mid-Thursday because of handover from one RelEng person to another – something we try to avoid doing *during* a release.

We’ll go through this in more detail in our postmortem later today, and file bugs for any other gotchas… all to try and improve the *next* release.

Note: as usual, I do not include the time it takes to *develop* and *test* a fix for a bug. I’m totally happy with developers taking the time needed to develop a good fix, and QA taking the time needed to verify the fix works as expected. If you start rushing any of that, its easy to accidentally compromise the quality of the fix. Which is bad. However, the time taken from “go to build” to “release is out” is a measure of the efficiency of the release automation process; something we’re constantly working on improving *between* releases. The faster we can safely release, the better.

ps: The speed at which Mozilla got this bug fixed, tested and distributed to users was noticed here
– very cool!

UPDATE: fixed typo in URL. joduinn 05apr2010