We already send daily “wait times” posts to mozilla.dev.tree-mangement. There’s 3 different pools where we need to measure “waittimes”, so there’s 3 posts per day.

We do need a way to get that data into something that makes trend-graph-over-time easier to see. However, meanwhile, if you just look at the subject field of the posts over time, you can see the number of jobs per day increasing, and the wait times for *some* specific OS getting worse.

tc
John.

The same job “jetpack-mozilla-central-w764-debug” seems a little odd since it purports to run a test on a debug build for win64. But tinderbox doesn’t seem to create debug builds for win64. And if you look in the build log this job seems to fail with a file-not-found error.

It also looks odd that the jetpack tests seem to be run twice (both as “jetpack-mozilla-central-…” and “Rev3 … mozilla-central … test jetpack”). Both seem to run the “run_jetpack.py” script, although with different options.

I don’t think windows have the best performance when it comes to many small files. Have you tried enabling folder compression on the \talos-slave\ folder. It have helped read performance for me (on something unrelated).

I assume that at least mochitests doesn’t alter the files in the test folder, so perhaps it would be an idea to not unzip the test files, but map the zip as drive Presumably “Pismo Mount File” can do this, or you could distribute the tests to the slaves as a zipped iso-file, which should be fast to unzip and multiple programs can mount.

Some of those number seem wildly unlikely, too: “WINNT 5.2 mozilla-central build 1420.93258427″ would be 23 minutes for a Windows build, when every Windows build for the last two days took between 115 and 120 minutes. 10.6 opt mochitest-5 I noticed the other day takes between 6 and 9 minutes, but you’ve got it at nearly 26 minutes.

Then we can evaluate whether we’re turning tests around within an acceptable amount of time. At the moment, I don’t even know whether tests have actually gotten slower (maybe it’s just my imagination), and if they have gotten slower, how much faster they’d need to get to match our speed from a few months ago.

For example, in mochitest-1, probably the most expensive mochitest is the WebGL one. But it’s useless to run it on every checkin. It’s useful to run it on every checkin that touches either content/canvas, gfx/gl, or gfx/layers; and it’s useful to run it once a day for integration testing, but it’s not useful to run it on every checkin. In this way, the load incurred by this test could be reduced by over 90% without losing coverage and with only very little loss of immediacy of detecting regressions.