Infrastructure load for Apr 2009

(Updated to include TryServer data joduinn 01may2009)


  • We pushed 1,226 code changes to our mercurial-based repos (979 code changes, 247 try server changes). This translates into 12,619 build/unittest jobs, or 17.5 jobs per hour, over the month. Its worth noting that we had almost no pushes in the last week, so thats mostly in the first three weeks.
  • The dropoff of changes in the last week matches up with the all hands being held in Mountain View.
  • The mozilla-1.9.1 branch was more active then usual, as expected in the lead up to FF3.5beta4.
  • The high volume of activity on TryServer was quite interesting, and matches what we saw in terms of slaves being overrun, and long wait times. Short term, we dealt with the spike by “borrowing” slaves from the main production pool. Long term, we’ve started creating more slaves for TryServer to avoid hitting these delays again.
  • We’re not measuring load on Talos yet. I’m still looking into that.


As each of these pushes triggers multiple different types of builds/unittest jobs, the *theoretical* total amount of work done by the pool-of-slaves in April was 12,619 jobs. For each push, we do:

  • mozilla-central: 12 jobs per push (L/M/W opt, L/M/W leaktest, L/M/W unittest, linux64 opt, linux-arm, WinCE)
  • mozilla-1.9.1: 10 jobs per push (L/M/W opt, L/M/W leaktest, L/M/W unittest, linux64 opt)
  • tracemonkey: 7 jobs per push (L/M/W opt, L/M/W unittest, linux64 opt)
  • try: 9 jobs per push (L/M/W opt, L/M/W leaktest, L/M/W unittest)
  • theoretical total: (546 x 12) + (271 x 10) + (162 x 7) + (247 x 9) = 12,619 jobs per month = 17.5 jobs per hour.