Infrastructure load for March 2012

No Comments

  • #checkins-per-month: March set another new record with 4,508 checkins for the month. That is now 5 months in a row setting new records: February2012 (4,027 checkins), January2012 (3,962 checkins), December2011 (3,262 checkins), and November2011 (3,209 checkins).
  • #checkins-per-day: We set a new record of 238 checkins per-day on 21-mar-2012. Also worth noting was that, in March, all 22 of the 22 working days had between 135-238 checkins per day, and of those, 2 days had over 200 checkins per day (214 on 12mar, 238 on 21mar).
  • #checkins-per-hour: We set a new record of 9.8 checkins-per-hour. And if that wasnt enough, we did it twice. For the first time, we saw double peak load times in a day: 11:00-noon PDT and 14:00-15:00 PDT. Note these records includes weekends, which are mostly idle, so the real checkins-per-hour on work days is higher.

Overall load since Jan 2009

mozilla-inbound, fx-team:
mozilla-inbound continues to be heavily used as an integration branch, with 26% of all checkins. By comparison, however, the fx-team branch only had 2% of the checkins, much less than mozilla-central’s 4%.

  • In the chart above, note that the number of mozilla-central checkins (192 or 4%) remained about the same, while the number of checkins on mozilla-inbound (1186 or 26%) continues to increase.
  • In the past whenever we had to unwind a large backlog of pending checkins, or back out a complicated bustage, we’d keep the mozilla-central tree closed for all checkins for the duration… which blocked all landings. However, now with mozilla-inbound and fx-team as integration branches in addition to mozilla-central, it means that developers have the option to continue landing their unrelated patches on an open integration branch while the cleanup work continues on the closed branch. In theory, if you give smart humans a easy way to route around a blockage, they’ll quickly start to use it so they can continue to get things done. In reality, very cool to see it actually happening.

Infrastructure load by branch

mozilla-aurora, mozilla-beta:

  • ~2.5% of our total monthly checkins landed into mozilla-aurora, slightly down from ~3% last month.
  • We’re back down to ~1% of our total monthly checkins landed into mozilla-beta. This is a relatively large percentage, as our trend since we started has been a very consistent ~1% of monthly checkins landing into mozilla-beta. Not sure why February is an anomaly, but it looks like or if this is the start of a trend, so lets watch this.

(Standard disclaimer: I’m always glad whenever we catch a problem *before* we ship a release; it avoids us having to do a chemspill release and we ship better code to our Firefox users.)

misc other details:

  • Pushes per day
    #Pushes this month

  • Pushes by hour of day
    #Pushes per hour

Leave a Reply