Firefox3.5.5 was released on Thursday 05-nov-2009, at 16:00PST.
This was our fastest turnaround on a release. By far.
From “Dev says go†to “release is now available to public†was approx 3 days (3d 4h 45m) wall-clock time. Release Engineering took 13-16hours. By comparison, the next fastest release turnaround was FF3.5.3 (~37hours) and FF2.0.0.9 (~37hours).
11:13 02nov: Dev says “go†for FF3.5.5
13:06 02nov: FF3.5.5 builds started
17:05 02nov: FF3.5.3 linux, mac builds handed to QA
20:03 02nov: FF3.5.3 signed-win32 builds handed to QA
00:28 03nov: FF3.5.3 update snippets available on test update channel
22:00 04nov: Dev & QA says “go” for Beta, and for Release; Build already completed final signing, bouncer entries
07:30 05nov: mirror replication started
10:55 05nov: mirror absorption good enough for testing
14:40 05nov: website changes finalized and visible. Build given “go” to make updates snippets live.
14:51 05nov: update snippets available on live update channel
16:00 05nov: release announced
Notes:
1) As we continue streamlining this process, now the long pole is communication between the groups, and also how the websites release notes are assembled and published. For this release, there were 8.5 9.5 hours of waiting between “go to mirrors” and “mirror push started”. Most of Thursday was spent updating release notes on websites. Meanwhile, we populated the mirrors, which takes ~3.5 hours of watching mirrors, but only took two brief commands on our part.
3) Our blow-by-blow scribbles are public, so the curious can read all about it here. Those Build Notes also link to our tracking bug#525814.
This super-super fast release turnaround was handled calmly and efficiently. It was a real credit to the team to see how well everyone worked well together on this, including smooth handoffs back-and-forth across timezones so everyone still had a life ! 🙂
take care
John.
Great turnaround one this one, indeed.
I don’t think it’s correct to say there was “8.5 hours of waiting between ‘go to mirrors’ and ‘mirror push started'”. In the e-mail sent late on Nov 4 we were instructed to push to mirrors next morning, not that night. That is to say, there wasn’t 8 hours of unnecessary waiting here, as your original sentence implies.
hi Ben;
I talked with Sam (ss) about this since posting this blog.
I had (incorrectly) thought his “go to mirrors” email marked the end of QA / beta feedback, and marked our permission to start push to mirrors when RelEng next came to office EST. Hence my thinking we had room to improve. However, from talking with Sam, he clarified that he (and some others in QA) were still monitoring during the night and early morning, and could still potentially halt a release. We were just not notified when that monitoring stopped.
For future releases, Sam will send two emails:
1) one email, typically late at night, to forewarn RelEng of a likely “go to mirrors” the next morning, so we can plan accordingly.
2) next email, typically early next morning after Sam verifies no last minute QA / beta cycle problems are reported and RelEng should start mirror push asap.
The extra email prevents a possible situation where RelEng starts mirror push first thing in morning, thinking we already have permission, while at the same time, Sam is still investigating a newly found problem which could halt the release. It also helps clarify the wall-clock timing above, and will, I believe address your comment above.
[…] Engineering portion of that was 10h 15m. By comparison, our previous fastest release turnaround was FF3.5.5 (3d 4h 45m from start to finish, with Release Engineering taking 13-16hours). For FF3.6.6, the times […]