Ted’s recent landing of bug#421534 was blogged about by Ben Hearsum here. While it might initially seem like a real snooze, its actually really important for our ability to support multiple active project branches.

Each build slave keeps copies of the srcdir/objdir directory tree. Fair enough. However, we keep multiple copies. By the time you keep a directory tree for incremental builds, debug builds, leak builds, nightly builds, and release builds… and do that for each of mozilla-central, tracemonkey and mozilla-1.9.1… suddenly that all adds up.

By far the worst offender here was the Mac, with the “-save-temp” workaround for a compiler problem forcing each tree we kept to be huge (10GB); with Ted’s fix in place, that same tree is 3GB. Ted’s work on DWARF buys us a bunch of breathing room so we can still create new project branches as needed right now, while we continue to look for a more scalable solution.(any ideas/suggestions very very very welcome!)

Given the number of directory trees, and the number of active branches, buying more disks is only a short term stopgap solution. With the number of machines we have, its also an expensive short term stopgap. In terms of building scalable infrastructure, it would be great if each slave be able to support 5-6 different types of builds, across 8 or more simultaneous different project branches. We might never quite reach a perfectly clean “leave-no-trace” build, but every time we reduce the amount of space a build tree consumes on the slaves, it helps us scale up the number of build types, and project branches, we can provide for developers.

Thanks Ted.