One of the housekeeping tasks we’ve been doing this month, is figuring what machines we no longer need to be running, and closing them down if possible.
For over a year now, some machines owned by Build have been running performance tests using Tinderbox framework. These machines were originally a short term workaround while Talos performance machines were being brought online, and performance test suites migrated to work within the Talos framework. At this point, over a year later:
- 3 test suites are being run in Talos *and* in Tinderbox (txul, ts, tdhtml). We can stop running these on Tinderbox at any time.
- 5 newer suites are being run in Talos *only* (Tp3, tgfx, tsvg, tjss, sunspider).
- 2 older test suites are being run on Tinderbox *only* (Tp, Tp2). Moving these last two suites over from Tinderbox to Talos means we can then shutdown these old Tinderbox machines.
RobCee and Alice are working on moving those Tp, Tp2 suites. The plan is to get these test suites running on Talos, then run the tests on Talos *and* Tinderbox for a week, just to make sure all is ok, then finally close down the Tinderbox machines. All the test results will still show up on the graph server, no change there. Its worth pointing out that the migrated test will be run on different hardware, with different framework, so will give different results. The new Talos-based results should track the previous Tinderbox based results, but they will be different. For any historical point people care about, we can recreate historical data. Otherwise, we were just planning to redo important milestones, and the last couple of weeks of test runs – it seems to be what most people use.
The details involved are tricky, but the curious can follow along in bug#413695, and the whole set of bugs linked to from that.
[…] Last week, I blogged about us looking at old running systems and figuring out which ones could be shutdown. One set of machines were some old tinderbox performance machines, set up as a quick workaround well over a year ago(!). […]
[…] First off there is the old-style performance machines, pre-talos. These often find regressions first, because they cycle fast. Apparently that is going to improve but until then these old machines have useful data. Unfortunately the graphs they produce are … well lets just say pig ugly! It didn’t take too long to knock up something very basic, but nicer looking and allowing multiple results on the same graph. You can see it graphing the live data (yes it will appear blank for the age it takes to draw). Unfortunately the graph kit I’m using seems a little slow and not all that functional, but that could be changed. Then again I have just found out that new work on the graphs server pretty much blows this out of the water! […]