Infrastructure load for December 2010


There were 1,766 pushes in December 2010. This is a continued and significant drop from September (2,436 pushes) , October (2,360 pushes) and November (2,322 pushes). This continued drop in the number of checkins is expected, considering the prolonged lockdown for FF4.0beta8, immediately followed by the lockdown for FF4.0beta9, and then the Christmas/NewYears holidays.

The numbers for this month are:

  • 1,766 code changes to our mercurial-based repos, which triggered 220,238 jobs:
  • 33,232 build jobs, or ~45 jobs per hour.
  • 105,396 unittest jobs, or ~142 jobs per hour.
  • 81,610 talos jobs, or ~110 talos jobs per hour.


  • The long-running lockdown for FF4.0beta8, and then for FF4.0beta9 definitely took their hit on who was able to checkin, and where/when.
  • The load on TryServer reduced back to ~50% of our overall load. So far, I do not know why. Anyone got suggestions?
  • We are still double-running unittests for some OS; running unittest-on-builder and also unittest-on-tester. This continues while developers and QA work through the issues. Whenever unittest-on-test-machine is live and green, we disable unittest-on-builders to reduce wait times for builds. Any help with these tests would be great!
  • The entire series of these infrastructure load blogposts can be found here.
  • We are still not tracking down any l10n repacks, nightly builds, release builds or any “idle-timer” builds.

Detailed breakdown is :

Here’s how the math works out (Descriptions of build, unittest and performance jobs triggered by each individual push are here:

One thought on “Infrastructure load for December 2010

Leave a Reply