Summary:

April 2010 had the 2nd-highest number of pushes since we started recording load in Jan 2009; slightly down from last month’s record high. Try Server usage continues to be 1/3 of our load – mozilla-central is 1/3, and all the other project branches combined make up the remaining 1/3.

The numbers for this month are:

• 1,746 code changes to our mercurial-based repos, which triggered 187,592 jobs:
• 21,194 build jobs, or ~29 jobs per hour.
• 82,824 unittest jobs, or ~115 jobs per hour.
• 83,574 talos jobs, or ~116 talos jobs per hour.

Details:

• The number of builds we generate per checkin changed this month: we turned off WinMO builds everywhere, and enabled maemo5gtk, maemo5qt builds on specific branches.
• Our Unittest and Talos load continues high, like last month, and we expect this to jump further as more OS are still being added to Talos.
• Once we start running Unittests on all the Talos OS, we expect load to jump again. Once live and green, we’ll disable unittest-on-builders, and I’ll update the math here. In advance of that, we’re spinning up more machines to handle this future spike in load.
• The trend of “what time of day is busiest” changed again this month. Not sure what this means, but worth pointing out that each month seems to be different. This makes finding a “good” time for a downtime almost impossible.
• The entire series of these infrastructure load blogposts can be found here.
• We are still not tracking down any l10n repacks, nightly builds, release builds or any “idle-timer” builds.

Detailed breakdown is :

Here’s how the math works out (Descriptions of build, unittest and performance jobs triggered by each individual push are here: