Housekeeping: Moving performance tests from Tinderbox to Talos

2 Comments

One of the housekeeping tasks we’ve been doing this month, is figuring what machines we no longer need to be running, and closing them down if possible.

For over a year now, some machines owned by Build have been running performance tests using Tinderbox framework. These machines were originally a short term workaround while Talos performance machines were being brought online, and performance test suites migrated to work within the Talos framework. At this point, over a year later:

  • 3 test suites are being run in Talos *and* in Tinderbox (txul, ts, tdhtml). We can stop running these on Tinderbox at any time.
  • 5 newer suites are being run in Talos *only* (Tp3, tgfx, tsvg, tjss, sunspider).
  • 2 older test suites are being run on Tinderbox *only* (Tp, Tp2). Moving these last two suites over from Tinderbox to Talos means we can then shutdown these old Tinderbox machines.

RobCee and Alice are working on moving those Tp, Tp2 suites. The plan is to get these test suites running on Talos, then run the tests on Talos *and* Tinderbox for a week, just to make sure all is ok, then finally close down the Tinderbox machines. All the test results will still show up on the graph server, no change there. Its worth pointing out that the migrated test will be run on different hardware, with different framework, so will give different results. The new Talos-based results should track the previous Tinderbox based results, but they will be different. For any historical point people care about, we can recreate historical data. Otherwise, we were just planning to redo important milestones, and the last couple of weeks of test runs – it seems to be what most people use.
The details involved are tricky, but the curious can follow along in bug#413695, and the whole set of bugs linked to from that.

2 Comments (+add yours?)

Leave a Reply