Interesting!! We had 2,636 pushes in January 2011. This is a significant jump from the last few months, and almost hit our previous record (2,707 pushes in August 2010). Also interesting that a few branches were really busy but most branches had zero checkins.
- Shipping Fennec4.0beta4, Firefox4.0beta9, Firefox4.0beta10 and now Firefox4.0beta11 in quick succession, and with very short lockdowns, seemed to help unjam checkins backlog this month. A great relief for everyone!
- This faster cadence seems to have helped focus efforts, with less need for working on a project branch while waiting for a clear time to land in m-c. Also, as we get closer to the actual shipping of Firefox 4.0, it feels like most of the bigger pieces are done, and the remaining fixes still landing are each smaller fixes, which do not need a project branch, and can be done on tryserver. Of course, that is just my interpretations… if you have other interpretations of the same data, let me know!
- The load on TryServer jumped to 53% of our overall load. Looks like more people are now doing TryServer run before landing, which means the patches that do land are less-risky, and a tree that stays green more often!
- The numbers for this month are:
- 2,636 code changes to our mercurial-based repos, which triggered 335,210 jobs:
- 49,971 build jobs, or ~67 jobs per hour.
- 158,121 unittest jobs, or ~213 jobs per hour.
- 127,118 talos jobs, or ~171 talos jobs per hour.
- We are still double-running unittests for some OS; running unittest-on-builder and also unittest-on-tester. This continues while developers and QA work through the issues. Whenever unittest-on-test-machine is live and green, we disable unittest-on-builders to reduce wait times for builds. Any help with these tests would be great!
- The entire series of these infrastructure load blogposts can be found here.
- We are still not tracking down any l10n repacks, nightly builds, release builds or any “idle-timer” builds.
Here’s how the math works out (Descriptions of build, unittest and performance jobs triggered by each individual push are here: