Month: August 2012

137 hours compute hours every ~6 minutes

joduinn10 Comments

Followup from today’s Dev platform meeting.

We talk a lot about how the number of checkins continue to rise – its always great to see developers being able to do more work – and the “Infrastructure load” series of posts is one way to measure that. However, its worth noting that the amount of work we trigger per checkin also continues to rise.

Some of this is good (new builds, new test suites), but some of this is bad (consistently failing tests; obsolete tests).

How many hours of builds and tests do we run per commit?

  • In 2009, each commit triggered ~40 compute hours
  • In Jan 2012, each commit triggered 110 compute hours
  • In Aug 2012, each commit triggered 137 compute hours

137 hours of builds and tests per checkin is a lot – especially when we do a checkin every ~6 minutes. Put another way, that’s ~3.5 years of builds and tests being run per day.

If you look at the “wait times” posts in dev.tree-management, you’ll see that we’re having a tough time keeping up. We’re busily venting excess production load over to AWS for nativefennec / B2G desktop builds / linux desktop …. and are then recycling existing hardware to help grow inhouse capacity for win32 and win64.

But you can help.

Please file a bug if you know of:

  • any build or test that is being run but no longer needed OR
  • any build or test being hidden on tbpl.m.o because it constantly fails

We’ll disable the job to stop wasting CPU time, which will help everyone’s wait times. As needed, we’ll also file bugs to track fixing the problem and then reenable once the build/test is running green.

While any one given test suite might not seem like a lot of time to save, we run all these suites literally thousands of times per day. Every little saving helps makes things faster for everyone!

Thank you!
===========================================
Full list (with duration of job in seconds)
Android Armv6 mozilla-central build 1523.29213483
Android Debug mozilla-central build 1461.34090909
Android Tegra 250 mozilla-central opt test crashtest-2 1509.39215686
Android Tegra 250 mozilla-central opt test crashtest-3 1627.30769231
Android Tegra 250 mozilla-central opt test jsreftest-1 1635.7047619
Android Tegra 250 mozilla-central opt test jsreftest-2 2023.08181818
Android Tegra 250 mozilla-central opt test jsreftest-3 1498.89215686
Android Tegra 250 mozilla-central opt test mochitest-1 1723.55445545
Android Tegra 250 mozilla-central opt test mochitest-2 1433.60909091
Android Tegra 250 mozilla-central opt test mochitest-3 1533.75247525
Android Tegra 250 mozilla-central opt test mochitest-4 1311.97222222
Android Tegra 250 mozilla-central opt test mochitest-5 1332.30392157
Android Tegra 250 mozilla-central opt test mochitest-6 1373.30188679
Android Tegra 250 mozilla-central opt test mochitest-7 1373.56190476
Android Tegra 250 mozilla-central opt test mochitest-8 1691.24528302
Android Tegra 250 mozilla-central opt test reftest-1 1481.94166667
Android Tegra 250 mozilla-central opt test reftest-2 1585.47572816
Android Tegra 250 mozilla-central opt test reftest-3 1606.37864078
Android Tegra 250 mozilla-central opt test robocop 1813.01941748
Android Tegra 250 mozilla-central talos remote-tdhtml 1506.13861386
Android Tegra 250 mozilla-central talos remote-tp4m_nochrome 1592.23529412
Android Tegra 250 mozilla-central talos remote-trobocheck 1398.76146789
Android Tegra 250 mozilla-central talos remote-trobocheck2 1584.97142857
Android Tegra 250 mozilla-central talos remote-trobocheck3 1460.77669903
Android Tegra 250 mozilla-central talos remote-trobopan 1567.03883495
Android Tegra 250 mozilla-central talos remote-troboprovider 1592.09345794
Android Tegra 250 mozilla-central talos remote-ts 1761.95145631
Android Tegra 250 mozilla-central talos remote-tsvg 1490.09803922
Android XUL Tegra 250 mozilla-central opt test crashtest-2 1647.91428571
Android XUL Tegra 250 mozilla-central opt test crashtest-3 1430.54368932
Android XUL Tegra 250 mozilla-central opt test jsreftest-1 1550.04761905
Android XUL Tegra 250 mozilla-central opt test jsreftest-2 1593.39805825
Android XUL Tegra 250 mozilla-central opt test jsreftest-3 1754.47058824
Android XUL Tegra 250 mozilla-central opt test mochitest-1 1409.15686275
Android XUL Tegra 250 mozilla-central opt test mochitest-2 1521.00980392
Android XUL Tegra 250 mozilla-central opt test mochitest-3 1230.08737864
Android XUL Tegra 250 mozilla-central opt test mochitest-4 1525.06796117
Android XUL Tegra 250 mozilla-central opt test mochitest-5 1431.87254902
Android XUL Tegra 250 mozilla-central opt test mochitest-6 1476.26923077
Android XUL Tegra 250 mozilla-central opt test mochitest-7 1272.8490566
Android XUL Tegra 250 mozilla-central opt test mochitest-8 1799.10576923
Android XUL Tegra 250 mozilla-central opt test reftest-1 1569.52427184
Android XUL Tegra 250 mozilla-central opt test reftest-2 1658.66666667
Android XUL Tegra 250 mozilla-central opt test reftest-3 1551.54205607
Android XUL mozilla-central build 1277.25
Android mozilla-central build 1486.5
B2G gb_armv7a_gecko mozilla-central build 1264.95652174
B2G gb_armv7a_gecko-debug mozilla-central build 1539.45977011
B2G ics_armv7a_gecko mozilla-central build 1498.29850746
B2G ics_armv7a_gecko-debug mozilla-central build 1439.40298507
Linux mozilla-central build 1429.60227273
Linux mozilla-central leak test build 1413.39772727
Linux x86-64 mozilla-central build 1362.26136364
Linux x86-64 mozilla-central leak test build 1859.20454545
OS X 10.7 64-bit mozilla-central leak test build 1418.80681818
OS X 10.7 mozilla-central build 1324.79545455
Rev3 Fedora 12 mozilla-central debug test crashtest 1793.67857143
Rev3 Fedora 12 mozilla-central debug test jetpack 1527.98809524
Rev3 Fedora 12 mozilla-central debug test jsreftest 1477.39285714
Rev3 Fedora 12 mozilla-central debug test mochitest-other 1443.39285714
Rev3 Fedora 12 mozilla-central debug test mochitests-1/5 1704.18823529
Rev3 Fedora 12 mozilla-central debug test mochitests-2/5 1576.8452381
Rev3 Fedora 12 mozilla-central debug test mochitests-3/5 1615.10714286
Rev3 Fedora 12 mozilla-central debug test mochitests-4/5 1448.51190476
Rev3 Fedora 12 mozilla-central debug test mochitests-5/5 1751.01190476
Rev3 Fedora 12 mozilla-central debug test reftest 1603.91666667
Rev3 Fedora 12 mozilla-central debug test xpcshell 1416.51190476
Rev3 Fedora 12 mozilla-central opt test crashtest 1610.98809524
Rev3 Fedora 12 mozilla-central opt test crashtest-ipc 1559.86904762
Rev3 Fedora 12 mozilla-central opt test jetpack 1546.75
Rev3 Fedora 12 mozilla-central opt test jsreftest 1443.92857143
Rev3 Fedora 12 mozilla-central opt test mochitest-other 1358.32142857
Rev3 Fedora 12 mozilla-central opt test mochitests-1/5 1444.32941176
Rev3 Fedora 12 mozilla-central opt test mochitests-2/5 1247.67857143
Rev3 Fedora 12 mozilla-central opt test mochitests-3/5 1574.0952381
Rev3 Fedora 12 mozilla-central opt test mochitests-4/5 1452.3452381
Rev3 Fedora 12 mozilla-central opt test mochitests-5/5 1345.52380952
Rev3 Fedora 12 mozilla-central opt test peptest 1388.17857143
Rev3 Fedora 12 mozilla-central opt test reftest 1365.30952381
Rev3 Fedora 12 mozilla-central opt test reftest-ipc 1832.04761905
Rev3 Fedora 12 mozilla-central opt test reftest-no-accel 1419.16666667
Rev3 Fedora 12 mozilla-central opt test xpcshell 1645.51764706
Rev3 Fedora 12 mozilla-central talos chromer 1427.22619048
Rev3 Fedora 12 mozilla-central talos dirty 1282.8
Rev3 Fedora 12 mozilla-central talos dirtypaint 1629.80952381
Rev3 Fedora 12 mozilla-central talos dromaeo 1365.82222222
Rev3 Fedora 12 mozilla-central talos dromaeojs 1465.46153846
Rev3 Fedora 12 mozilla-central talos jstests 1725.84444444
Rev3 Fedora 12 mozilla-central talos nochromer 1409.21428571
Rev3 Fedora 12 mozilla-central talos other 1230.77380952
Rev3 Fedora 12 mozilla-central talos svgr 1486.88095238
Rev3 Fedora 12 mozilla-central talos tpn 1460.95238095
Rev3 Fedora 12×64 mozilla-central debug test crashtest 1460.81176471
Rev3 Fedora 12×64 mozilla-central debug test jetpack 1299.35294118
Rev3 Fedora 12×64 mozilla-central debug test jsreftest 1620.12790698
Rev3 Fedora 12×64 mozilla-central debug test mochitest-other 1441.22727273
Rev3 Fedora 12×64 mozilla-central debug test mochitests-1/5 1636.98888889
Rev3 Fedora 12×64 mozilla-central debug test mochitests-2/5 1292.28235294
Rev3 Fedora 12×64 mozilla-central debug test mochitests-3/5 1520.34117647
Rev3 Fedora 12×64 mozilla-central debug test mochitests-4/5 1539.4
Rev3 Fedora 12×64 mozilla-central debug test mochitests-5/5 1563.67058824
Rev3 Fedora 12×64 mozilla-central debug test reftest 1460.58823529
Rev3 Fedora 12×64 mozilla-central debug test xpcshell 1292.23529412
Rev3 Fedora 12×64 mozilla-central opt test crashtest 1539.57647059
Rev3 Fedora 12×64 mozilla-central opt test jetpack 1716.28235294
Rev3 Fedora 12×64 mozilla-central opt test jsreftest 1619.31764706
Rev3 Fedora 12×64 mozilla-central opt test mochitest-other 1523.48837209
Rev3 Fedora 12×64 mozilla-central opt test mochitests-1/5 1461.10588235
Rev3 Fedora 12×64 mozilla-central opt test mochitests-2/5 1548.6744186
Rev3 Fedora 12×64 mozilla-central opt test mochitests-3/5 1630.16470588
Rev3 Fedora 12×64 mozilla-central opt test mochitests-4/5 1540.01176471
Rev3 Fedora 12×64 mozilla-central opt test mochitests-5/5 1639.5047619
Rev3 Fedora 12×64 mozilla-central opt test peptest 1845.95294118
Rev3 Fedora 12×64 mozilla-central opt test reftest 1786.08235294
Rev3 Fedora 12×64 mozilla-central opt test xpcshell 1321.78823529
Rev3 Fedora 12×64 mozilla-central talos chromer 1399.71764706
Rev3 Fedora 12×64 mozilla-central talos dirty 1582.13333333
Rev3 Fedora 12×64 mozilla-central talos dirtypaint 1609.03529412
Rev3 Fedora 12×64 mozilla-central talos dromaeo 1793.82222222
Rev3 Fedora 12×64 mozilla-central talos dromaeojs 1266.35
Rev3 Fedora 12×64 mozilla-central talos jstests 1830.17777778
Rev3 Fedora 12×64 mozilla-central talos nochromer 1310.87058824
Rev3 Fedora 12×64 mozilla-central talos other 1684.15294118
Rev3 Fedora 12×64 mozilla-central talos svgr 1864.89411765
Rev3 Fedora 12×64 mozilla-central talos tpn 1607.82352941
Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test crashtest 1567.03960396
Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test jetpack 1587.17821782
Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test jsreftest 1711.85148515
Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test mochitest-other 1638.87128713
Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test mochitests-1/5 1424.64356436
Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test mochitests-2/5 1589.06930693
Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test mochitests-3/5 1592.65346535
Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test mochitests-4/5 1477.56435644
Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test mochitests-5/5 1662.66336634
Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test peptest 1594.99009901
Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test reftest 1704.13861386
Rev3 MacOSX Leopard 10.5.8 mozilla-central opt test xpcshell 1524.73267327
Rev3 MacOSX Leopard 10.5.8 mozilla-central talos chromer 1540.58415842
Rev3 MacOSX Leopard 10.5.8 mozilla-central talos dirty 1663.2037037
Rev3 MacOSX Leopard 10.5.8 mozilla-central talos dirtypaint 1493.07920792
Rev3 MacOSX Leopard 10.5.8 mozilla-central talos dromaeo 1467.05555556
Rev3 MacOSX Leopard 10.5.8 mozilla-central talos dromaeojs 1631.04255319
Rev3 MacOSX Leopard 10.5.8 mozilla-central talos jstests 1470.88888889
Rev3 MacOSX Leopard 10.5.8 mozilla-central talos nochromer 1586.46534653
Rev3 MacOSX Leopard 10.5.8 mozilla-central talos other 1580.38613861
Rev3 MacOSX Leopard 10.5.8 mozilla-central talos svgr 1610.73267327
Rev3 MacOSX Leopard 10.5.8 mozilla-central talos tpn 1768.98019802
Rev3 WINNT 5.1 mozilla-central debug test crashtest 1346.4
Rev3 WINNT 5.1 mozilla-central debug test jetpack 1568.725
Rev3 WINNT 5.1 mozilla-central debug test jsreftest 1465.05
Rev3 WINNT 5.1 mozilla-central debug test mochitest-other 1893.2625
Rev3 WINNT 5.1 mozilla-central debug test mochitests-1/5 1669.0875
Rev3 WINNT 5.1 mozilla-central debug test mochitests-2/5 1697.8875
Rev3 WINNT 5.1 mozilla-central debug test mochitests-3/5 1615.8375
Rev3 WINNT 5.1 mozilla-central debug test mochitests-4/5 1627.75
Rev3 WINNT 5.1 mozilla-central debug test mochitests-5/5 1597.425
Rev3 WINNT 5.1 mozilla-central debug test reftest 1757.05
Rev3 WINNT 5.1 mozilla-central debug test xpcshell 1720.375
Rev3 WINNT 5.1 mozilla-central opt test crashtest 1719.84146341
Rev3 WINNT 5.1 mozilla-central opt test jetpack 1640.7804878
Rev3 WINNT 5.1 mozilla-central opt test jsreftest 1725.46341463
Rev3 WINNT 5.1 mozilla-central opt test mochitest-other 1554.07317073
Rev3 WINNT 5.1 mozilla-central opt test mochitests-1/5 1405.34146341
Rev3 WINNT 5.1 mozilla-central opt test mochitests-2/5 1378.73170732
Rev3 WINNT 5.1 mozilla-central opt test mochitests-3/5 1467.54878049
Rev3 WINNT 5.1 mozilla-central opt test mochitests-4/5 1545.7195122
Rev3 WINNT 5.1 mozilla-central opt test mochitests-5/5 1382.70731707
Rev3 WINNT 5.1 mozilla-central opt test peptest 1382.92682927
Rev3 WINNT 5.1 mozilla-central opt test reftest 1650.84146341
Rev3 WINNT 5.1 mozilla-central opt test xpcshell 1608.17073171
Rev3 WINNT 5.1 mozilla-central talos chromer 1694.86585366
Rev3 WINNT 5.1 mozilla-central talos dirty 1556.30952381
Rev3 WINNT 5.1 mozilla-central talos dirtypaint 1651.80487805
Rev3 WINNT 5.1 mozilla-central talos dromaeo 1415.38095238
Rev3 WINNT 5.1 mozilla-central talos dromaeojs 1341.225
Rev3 WINNT 5.1 mozilla-central talos jstests 1612.71428571
Rev3 WINNT 5.1 mozilla-central talos nochromer 1556.0
Rev3 WINNT 5.1 mozilla-central talos other 1326.64634146
Rev3 WINNT 5.1 mozilla-central talos svgr 1524.90243902
Rev3 WINNT 5.1 mozilla-central talos tpn 1711.64634146
Rev3 WINNT 6.1 mozilla-central debug test crashtest 1308.125
Rev3 WINNT 6.1 mozilla-central debug test jetpack 1633.7875
Rev3 WINNT 6.1 mozilla-central debug test jsreftest 1833.8625
Rev3 WINNT 6.1 mozilla-central debug test mochitest-other 1391.5375
Rev3 WINNT 6.1 mozilla-central debug test mochitests-1/5 1398.2625
Rev3 WINNT 6.1 mozilla-central debug test mochitests-2/5 1269.4875
Rev3 WINNT 6.1 mozilla-central debug test mochitests-3/5 1545.325
Rev3 WINNT 6.1 mozilla-central debug test mochitests-4/5 1451.45
Rev3 WINNT 6.1 mozilla-central debug test mochitests-5/5 1265.275
Rev3 WINNT 6.1 mozilla-central debug test reftest 1292.25
Rev3 WINNT 6.1 mozilla-central debug test xpcshell 1563.7875
Rev3 WINNT 6.1 mozilla-central opt test crashtest 1645.51219512
Rev3 WINNT 6.1 mozilla-central opt test jetpack 1549.59756098
Rev3 WINNT 6.1 mozilla-central opt test jsreftest 1926.76829268
Rev3 WINNT 6.1 mozilla-central opt test mochitest-other 1845.15853659
Rev3 WINNT 6.1 mozilla-central opt test mochitests-1/5 1392.75609756
Rev3 WINNT 6.1 mozilla-central opt test mochitests-2/5 1516.2804878
Rev3 WINNT 6.1 mozilla-central opt test mochitests-3/5 1643.31707317
Rev3 WINNT 6.1 mozilla-central opt test mochitests-4/5 1658.19512195
Rev3 WINNT 6.1 mozilla-central opt test mochitests-5/5 1658.73170732
Rev3 WINNT 6.1 mozilla-central opt test peptest 1412.69512195
Rev3 WINNT 6.1 mozilla-central opt test reftest 1557.63529412
Rev3 WINNT 6.1 mozilla-central opt test reftest-no-accel 1773.42682927
Rev3 WINNT 6.1 mozilla-central opt test xpcshell 1849.8902439
Rev3 WINNT 6.1 mozilla-central talos chromer 1764.8902439
Rev3 WINNT 6.1 mozilla-central talos dirty 1440.97619048
Rev3 WINNT 6.1 mozilla-central talos dirtypaint 1697.02439024
Rev3 WINNT 6.1 mozilla-central talos dromaeo 1511.4047619
Rev3 WINNT 6.1 mozilla-central talos dromaeojs 1442.2
Rev3 WINNT 6.1 mozilla-central talos jstests 1709.88095238
Rev3 WINNT 6.1 mozilla-central talos nochromer 1299.13414634
Rev3 WINNT 6.1 mozilla-central talos other 1595.7195122
Rev3 WINNT 6.1 mozilla-central talos svgr 1865.98780488
Rev3 WINNT 6.1 mozilla-central talos tpn 1861.63414634
Rev3 WINNT 6.1 mozilla-central talos xperf 1551.48809524
Rev3 WINNT 6.1 x64 mozilla-central opt test crashtest 1598.8375
Rev3 WINNT 6.1 x64 mozilla-central opt test jetpack 1258.225
Rev3 WINNT 6.1 x64 mozilla-central opt test jsreftest 1472.35
Rev3 WINNT 6.1 x64 mozilla-central opt test mochitest-other 1490.725
Rev3 WINNT 6.1 x64 mozilla-central opt test mochitests-1/5 1404.075
Rev3 WINNT 6.1 x64 mozilla-central opt test mochitests-2/5 1376.5875
Rev3 WINNT 6.1 x64 mozilla-central opt test mochitests-3/5 1592.975
Rev3 WINNT 6.1 x64 mozilla-central opt test mochitests-4/5 1463.95
Rev3 WINNT 6.1 x64 mozilla-central opt test mochitests-5/5 1449.9
Rev3 WINNT 6.1 x64 mozilla-central opt test peptest 1529.0
Rev3 WINNT 6.1 x64 mozilla-central opt test reftest 1672.975
Rev3 WINNT 6.1 x64 mozilla-central opt test xpcshell 1316.7625
Rev3 WINNT 6.1 x64 mozilla-central talos chromer 1588.0875
Rev3 WINNT 6.1 x64 mozilla-central talos dirty 1083.07142857
Rev3 WINNT 6.1 x64 mozilla-central talos dirtypaint 1317.36842105
Rev3 WINNT 6.1 x64 mozilla-central talos dromaeo 1551.97619048
Rev3 WINNT 6.1 x64 mozilla-central talos dromaeojs 1061.57894737
Rev3 WINNT 6.1 x64 mozilla-central talos nochromer 1117.725
Rev3 WINNT 6.1 x64 mozilla-central talos other 1464.9625
Rev3 WINNT 6.1 x64 mozilla-central talos svgr 1294.5875
Rev3 WINNT 6.1 x64 mozilla-central talos tpn 1587.9875
Rev4 MacOSX Lion 10.7 mozilla-central debug test crashtest 1413.51190476
Rev4 MacOSX Lion 10.7 mozilla-central debug test jetpack 1886.20238095
Rev4 MacOSX Lion 10.7 mozilla-central debug test jsreftest 1689.70238095
Rev4 MacOSX Lion 10.7 mozilla-central debug test mochitest-other 1419.03529412
Rev4 MacOSX Lion 10.7 mozilla-central debug test mochitests-1/5 1367.14285714
Rev4 MacOSX Lion 10.7 mozilla-central debug test mochitests-2/5 1447.96428571
Rev4 MacOSX Lion 10.7 mozilla-central debug test mochitests-3/5 1306.36904762
Rev4 MacOSX Lion 10.7 mozilla-central debug test mochitests-4/5 1560.03571429
Rev4 MacOSX Lion 10.7 mozilla-central debug test mochitests-5/5 1346.22619048
Rev4 MacOSX Lion 10.7 mozilla-central debug test reftest 1160.25
Rev4 MacOSX Lion 10.7 mozilla-central debug test xpcshell 1641.98809524
Rev4 MacOSX Lion 10.7 mozilla-central opt test crashtest 1567.85148515
Rev4 MacOSX Lion 10.7 mozilla-central opt test jetpack 1488.48514851
Rev4 MacOSX Lion 10.7 mozilla-central opt test jsreftest 1610.36633663
Rev4 MacOSX Lion 10.7 mozilla-central opt test mochitest-other 1559.95049505
Rev4 MacOSX Lion 10.7 mozilla-central opt test mochitests-1/5 1551.06930693
Rev4 MacOSX Lion 10.7 mozilla-central opt test mochitests-2/5 1439.67326733
Rev4 MacOSX Lion 10.7 mozilla-central opt test mochitests-3/5 1611.44554455
Rev4 MacOSX Lion 10.7 mozilla-central opt test mochitests-4/5 1471.16831683
Rev4 MacOSX Lion 10.7 mozilla-central opt test mochitests-5/5 1648.7029703
Rev4 MacOSX Lion 10.7 mozilla-central opt test peptest 1444.3960396
Rev4 MacOSX Lion 10.7 mozilla-central opt test reftest 1564.81553398
Rev4 MacOSX Lion 10.7 mozilla-central opt test xpcshell 1543.82178218
Rev4 MacOSX Lion 10.7 mozilla-central talos chromer 1400.03960396
Rev4 MacOSX Lion 10.7 mozilla-central talos dirty 1479.14814815
Rev4 MacOSX Lion 10.7 mozilla-central talos dirtypaint 1492.04950495
Rev4 MacOSX Lion 10.7 mozilla-central talos dromaeo 1390.87037037
Rev4 MacOSX Lion 10.7 mozilla-central talos dromaeojs 1378.08510638
Rev4 MacOSX Lion 10.7 mozilla-central talos jstests 1653.78181818
Rev4 MacOSX Lion 10.7 mozilla-central talos nochromer 1320.79207921
Rev4 MacOSX Lion 10.7 mozilla-central talos other 1492.05940594
Rev4 MacOSX Lion 10.7 mozilla-central talos svgr 1500.45544554
Rev4 MacOSX Lion 10.7 mozilla-central talos tpn 1516.94059406
Rev4 MacOSX Snow Leopard 10.6 mozilla-central debug test crashtest 1336.52380952
Rev4 MacOSX Snow Leopard 10.6 mozilla-central debug test jetpack 1568.20238095
Rev4 MacOSX Snow Leopard 10.6 mozilla-central debug test jsreftest 1532.78823529
Rev4 MacOSX Snow Leopard 10.6 mozilla-central debug test mochitest-other 1355.49425287
Rev4 MacOSX Snow Leopard 10.6 mozilla-central debug test mochitests-1/5 1452.45238095
Rev4 MacOSX Snow Leopard 10.6 mozilla-central debug test mochitests-2/5 1497.10714286
Rev4 MacOSX Snow Leopard 10.6 mozilla-central debug test mochitests-3/5 1482.0952381
Rev4 MacOSX Snow Leopard 10.6 mozilla-central debug test mochitests-4/5 1443.01190476
Rev4 MacOSX Snow Leopard 10.6 mozilla-central debug test mochitests-5/5 1237.79761905
Rev4 MacOSX Snow Leopard 10.6 mozilla-central debug test reftest 1343.89411765
Rev4 MacOSX Snow Leopard 10.6 mozilla-central debug test xpcshell 1545.88372093
Rev4 MacOSX Snow Leopard 10.6 mozilla-central opt test crashtest 1669.87128713
Rev4 MacOSX Snow Leopard 10.6 mozilla-central opt test jetpack 1679.46534653
Rev4 MacOSX Snow Leopard 10.6 mozilla-central opt test jsreftest 1621.41584158
Rev4 MacOSX Snow Leopard 10.6 mozilla-central opt test mochitest-other 1568.45098039
Rev4 MacOSX Snow Leopard 10.6 mozilla-central opt test mochitests-1/5 1644.94059406
Rev4 MacOSX Snow Leopard 10.6 mozilla-central opt test mochitests-2/5 1851.26732673
Rev4 MacOSX Snow Leopard 10.6 mozilla-central opt test mochitests-3/5 1618.50485437
Rev4 MacOSX Snow Leopard 10.6 mozilla-central opt test mochitests-4/5 1634.09803922
Rev4 MacOSX Snow Leopard 10.6 mozilla-central opt test mochitests-5/5 1557.76153846
Rev4 MacOSX Snow Leopard 10.6 mozilla-central opt test peptest 1636.86138614
Rev4 MacOSX Snow Leopard 10.6 mozilla-central opt test reftest 1602.41176471
Rev4 MacOSX Snow Leopard 10.6 mozilla-central opt test xpcshell 1590.94059406
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos chromer 1665.55882353
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos dirty 1248.79245283
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos dirtypaint 1670.63366337
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos dromaeo 1626.81481481
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos dromaeojs 1625.02083333
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos jstests 1739.57407407
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos nochromer 1436.52941176
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos other 1602.79207921
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos svgr 1676.52475248
Rev4 MacOSX Snow Leopard 10.6 mozilla-central talos tpn 1481.43564356
WINNT 5.2 mozilla-central build 1420.93258427
WINNT 5.2 mozilla-central leak test build 1364.53932584
WINNT 6.1 x86-64 mozilla-central build 1437.09090909
jetpack-mozilla-central-fedora-debug 2145.93333333
jetpack-mozilla-central-fedora-opt 1462.06896552
jetpack-mozilla-central-fedora64-debug 1806.75862069
jetpack-mozilla-central-fedora64-opt 1603.75862069
jetpack-mozilla-central-leopard-debug 1972.31034483
jetpack-mozilla-central-leopard-opt 2482.82758621
jetpack-mozilla-central-lion-debug 1446.65517241
jetpack-mozilla-central-lion-opt 1509.27586207
jetpack-mozilla-central-snowleopard-debug 1982.5862069
jetpack-mozilla-central-snowleopard-opt 1654.34482759
jetpack-mozilla-central-w764-debug 2471.62068966
jetpack-mozilla-central-w764-opt 1800.65517241
jetpack-mozilla-central-win7-debug 1794.37931034
jetpack-mozilla-central-win7-opt 1567.93103448
jetpack-mozilla-central-xp-debug 2147.37931034
jetpack-mozilla-central-xp-opt 1786.24137931
============================================

HOWTO: Use crutches

joduinn10 Comments

I spent the best part of 2 years on crutches after a venom bite and an unrelated fall. Thankfully, I’ve made a full recovery. With all that time on crutches, I built up some tips/tricks. Since then, whenever I see someone on the street who is obviously new-to-crutches, I strike up a casual conversation about how long I spent on crutches, empathize about the various awkward realities of crutches and generally try to be encouraging. If things feel ok, then I ask if its ok to share some tips I learned from my time on crutches. This happened recently with someone in the office, and afterwards, they encouraged me to blog this in case it is helpful to anyone reading who is also newly on crutches.

1) buy fingerless cycling gloves
Sore hands from pressure of the crutch handle is a problem. Some people prefer to pad/wrap the crutch handle, but I found this still left lots of friction / movement-under-pressure while using crutches. Over time this friction led to blisters on the palm of your hands, between your thumb and first finger – exactly the spot that you needed to press your weight on while using crutches. Painful.

Padded cycling gloves solved this problem perfectly. These are padded in exactly the right area… between the base-of-thumb and forefinger…and they’re cheap! Being fingerless, I could still type and answer my cellphone with them on. And the mesh backing kept my hands from getting too hot while wearing them indoors all the time, so I could just easily leave them on all day.

2) be aware of possible nerve damage to armpits
If you get sore hands (see above), its easy to then start resting your full weight on your armpits on the top of the crutches. Every now and then is typically ok. However, its easy to get into the habit of resting your full weight on your armpits on the crutches. Dont. Leaning your weight on your armpits on your crutches can lead to nerve damage in your armpits, which will seriously limit future use of your arms and hands.

From wikipedia, “A condition known as crutch paralysis, or crutch palsy can arise from pressure on nerves in the armpit, or axilla. Specifically, the brachial plexus in the axilla is often damaged from the pressure of a crutch…In these cases the radial is the nerve most frequently implicated; the ulnar nerve suffers next in frequency”. More details are also at  http://en.wikipedia.org/wiki/Axillary_nerve.

Instead of resting your bodyweight on your armpits on the top of your crutches, use your arms to hold up your body weight. As a fulcrum point, squeeze in your arms to pin the top of the crutch between your upperarm and ribs – without contact to your armpit. Then try to move around the room. I found my arms were not use to this much work and quickly got tired. Start with short distances. Over time, build up to longer sustained use of crutches as your arms build up strength.

On the bright side: when I finally came off crutches, my arms were in the best shape ever. And I had no nerve damage in my arms and hands!

3) buy a jogger fanny pack, specifically one that has holders for full-size water bottles.
One of my most frustrating episodes soon after being on crutches was my first attempt to “have a normal day at home”, by myself, making my own coffee and hoping to sit at the computer with my leg elevated. It was tricky to wrangle the machine/filter/coffee/water while on crutches, but the brew smelled great – and the first sip was wonderful.

Success! If I can make my own coffee, how bad can life be?! Then I realized I had no way of getting from the kitchen to the chair, on crutches, while holding my prized mug of coffee. Foiled!

Weeks later, I had a brainwave. The water bottle holders in fanny packs is the same diameter as most car coffee travel mugs. Coffee in a travel mug in a fanny pack doesn’t spill while you are on crutches.

Make coffee. Pour into travel mug. Put travel mug into fanny-pack-bottle-holder. Use crutches to get to chair. Sit. Drink coffee from travel mug. Rejoice.

ps: While you have this fanny pack, you might as well also pack it with:

  • house keys
  • powerbars
  • cellphone
  • tissues

Thats it. Hope those hints help someone out there… and of course, if you have other tips or hints, please email me or add them to comments below.

take care
John.

Infrastructure load for July 2012

joduinn4 Comments
  • #checkins-per-month: We had 5,635 checkins in July2012, another new record, and well above our previous record of 5,246 checkins in May2012.
  • #checkins-per-day: We had consistently high load across the month, and 19-of-30 days had over 200 checkins-per-day. Put another way, we had over 200 checkins-per-day every work day in July except Canada Day (02jul2012) and US Independence Day (04jul2012).
  • #checkins-per-hour: The peak this month was 11.35 checkins per hour, and throughout the month, we sustained over 10-checkins-per-hour for 5 out of 24 hours in a day.

mozilla-inbound, fx-team:
mozilla-inbound continues to be heavily used as an integration branch, with 26% of all checkins, far more then the other integration branches fx-team (1.5% of checkins) or mozilla-central (~3% of checkins). For comparison, I note that more people landed on mozilla-aurora then on mozilla-central.

mozilla-aurora, mozilla-beta:

  • 3.8% of our total monthly checkins landed into mozilla-aurora.
  • 2.1% of our total monthly checkins landed into mozilla-beta. This is higher then previous months, but I guess this to be related to the NativeFennec-landing-on-beta work this month.

(Standard disclaimer: I’m always glad whenever we catch a problem *before* we ship a release; it avoids us having to do a chemspill release and also we ship better code to our Firefox users in the first place.)

misc other details:

  • Pushes per day
    • You can clearly see weekends through the month, as well as the impact of the national holidays on 02jul2012, 04jul2012.

  • Pushes by hour of day
    • It is worth noting that for 5 hours in every 24 hour day, we did over 10 checkins-per-hour. Phrased another way, thats one checkin every 6mins for 5 hours.