Infrastructure load for February 2011


(Sorry for the long-delay in posting these, things are starting to calm down again, so I’ll catch up.)

We had 2,060 pushes in February 2011 – a significant drop from the last few months. I believe this is because of the checkin restrictions during February (for Firefox4.0beta11, Firefox4.0beta12, Fennec4.0beta4, Fennec4.0beta5) as we got closer to the Firefox4.0 release, but honestly, that is just my interpretation. I have no way to prove that. If you have other suggestions, please let me know.


  • Load on TryServer is about the same (52% vs 53% in previous month). of our overall load. People continue to send their patches to TryServer before landing, so patches that do land are less-risky, and the tree stays green more often!
  • The numbers for this month are:
    • 2,060 code changes to our mercurial-based repos, which triggered 262,340 jobs:
    • 39,365 build jobs, or ~58 jobs per hour.
    • 123,603 unittest jobs, or ~184 jobs per hour.
    • 99,372 talos jobs, or ~148 talos jobs per hour.
  • We are still double-running unittests for some OS; running unittest-on-builder and also unittest-on-tester. This continues while developers and QA work through the issues. Whenever unittest-on-test-machine is live and green, we disable unittest-on-builders to reduce wait times for builds. Any help with these tests would be great!
  • The entire series of these infrastructure load blogposts can be found here.
  • We are still not tracking down any l10n repacks, nightly builds, release builds or any “idle-timer” builds.

Detailed breakdown is :

Here’s how the math works out (Descriptions of build, unittest and performance jobs triggered by each individual push are here: