• #checkins-per-month: We had 7,771 checkins in August 2013. This is 10% above last month’s record 7,051 checkins.
  
  

• #checkins-per-day: We had 443 checkins on 26aug; this new record blew past our previous single-day record of 370 checkins on 02jul. During August, yet again all working days were over 200 checkins per day… in fact our lowest weekday load was 243 checkins on 08aug. 21-of-31 days had over 250 checkins-per-day, 11-of-31 days had over 300 checkins-per-day. 5-of-31 days exceeded our previous record of 370 checkins. Our first time over the 400 checkins-per-day threshold was 443 checkins on 26aug. Wow.
  
• #checkins-per-hour: Checkins are still mostly mid-day PT/afternoon ET. For 11 of every 24 hours, we sustained over 12 checkins per hour. This month was the first month we went over 15 checkins-per-hour, and we did that for 3 of every 24 hours! Our heaviest load time this month was 1pm-2pm PT (15.4 checkins-per-hour – a new record), with close followup at 10am-11am (15.2 checkins-per-hour) and 2pm-3pm (15.0 checkins-per-hour).
  

mozilla-inbound, b2g-inbound, fx-team:

• mozilla-inbound continues to be heavily used as an integration branch. Its noteworthy that as developers start to use other *-inbound branches, we saw mozilla-inbound reduce significantly to 16.3% of all checkins, the lowest ever usage of mozilla-inbound. The use of multiple *-inbounds is clearly helping improve bottlenecks (see pie chart below) and the congestion on mozilla-inbound is being reduced significantly as people use switch to using other *-inbound branches instead. All very cool to see and a definite part of the reason we hit new records this month.
• b2g-inbound continues to be a great success, with 8.2% of this month’s checkins landing here, and as predicted last month, the percentage of checkins on this branch is stabilizing.
• As predicted, usage of fx-team branch jumped for August, as it now had official full-time sheriffs coverage, just like mozilla-inbound and b2g-inbound. At 6.2% of checkins this month, its clearly a very active third place for developers to land. Usage of “fx-team” might grow a little more, over next month or two, before stabilizing. Having sheriff coverage clearly made a difference.
• The combined total of these 3 integration branches is 30.7%, showing just how much our sheriffs are helping across the board.

  

mozilla-aurora, mozilla-beta, mozilla-b2g18, gaia-central:
Of our total monthly checkins:

• 2.0% landed into mozilla-central, slightly higher than last month. As usual, very few people land directly on mozilla-central these days, when there are sheriff-assisted branches available instead.
• 1.6% landed into mozilla-aurora, slightly lower than last month.
• 1.4% landed into mozilla-beta, slightly higher than last month.
• 0.5% landed into mozilla-b2g18, slightly lower then last month.
• Note: gaia-central, and all other gaia-* branches, are not counted here anymore. For details, see here.

misc other details:
As usual, our build pool handled the load well, with >95% of all builds consistently being started within 15mins. Our test pool continues to improve. All the hard work by RelEng, ATeam and IT is paying off, we’re seeing more test jobs being handled with better response times. The work on fixing/disabling any tests that are hidden-yet-still-being-run is also improving our test situation. The peak per-day test load for August was 52,000 test jobs on 27aug. Still more work to be done here, but very encouraging progress.

As always, if you know of any test suites that no longer need to be run per-checkin, please let us know so we can immediately reduce the load a little. Also, if you know of any test suites which are perma-orange, and hidden on tbpl.m.o, please let us know – thats the worst of both worlds – using up scarce CPU time and not being displayed for people to make use of. We’ll make sure to file bugs to get tests fixed – or disabled – every little bit helps put scarce test CPU to better use.