It can be setup in one of two ways:
1) the next Build would build B+C
or
2) the next Build would build B, and the build after that would build C.

–

@everyone:
‘build-on-checkin’ in Buildbot supports a ‘treeStableTimer’, which is the amount of silent (ie. no-checkins) required before building. This could be set to 5min, 1h, etc. So _if_ we end up going to a build-on-checkin’ system in the future the problem some of describe can be made less of one.

As for the question about stacked checkins, are you envisioning a system where each stacked commit would be built, or that if the build machines were running behind several commits would be batched? e.g.

10am: commit A
10:01am: builds A kick off
10:02am: commit B
10:03am: commit C
10:20am: after builds ‘A’ finish, would they start on a tree with checkin ‘B’ or skip directly to ‘C’?

And, just to be sure, you’re not talking about doing this with the perftest machines, are you, just the build machines? I think it is valuable to reduce noise to get as many perftest runs as possible, even if there have been no checkins.

For example, if nothing gets checked in, no builds are done. However, once there is a check-in, the build machine “waits” for a period of time (e.g. 1 hour), for all checkins that occur within that timeframe (1 hr) to get in before doing up a build. This way, a dev with 3 or 4 checkins have that time to check in all that he wants, including other devs who want to do so at the same time as well.

Parallel machines could also work this way.

Just an idea…

Beyond that, the chaos of continuous builds (all the tinderboxen asynchronous) has a big advantage: whenever you commit a patch, it’s likely that one client will pick up your checkin quickly. This means everyone gets reasonably fast feedback on build bustages (so long as it’s cross-platform). With build-on-checkin, the first committer gets quick feedback (on all platforms) and everyone else gets slow feedback (on all platforms). On a relative basis, I’d expect that the initial feedback time hit to be much more significant (300% perhaps) with build-on-checkin while the time to finish builds would be marginally better or worse (the first-committer only sees a 50% decrease in build-time on average).

The chaos of continuous builds also results in builds from different boxes building with different sets of patches (one might have A and then B+C and then D, while another might have A+B and then C+D). This is often helpful when diagnosing test failures. Build-on-checkin would decrease the heterogeneity (it would bias all the boxes to have A+B and C+D).

I’d expect parallel builds from a buildbot pool would resolve these problems so long as the pool is sufficiently large, but without that, I don’t see much advantage to build-on-checkin (a few people benefit, the rest are worse off). I’m also not quite sure how they would distinguish checkins (since CVS has no atomic commits).

What would be helpful is to measure the distribution of time-to-initial-feedback and time-to-finished-builds for both modes, either using historical data from tinderbox and buildbot clients, or perhaps simulating each mode with historical checkin data and hypothetical build times.

Finally, (depending on how jumpy they are) build-on-checkin might result in more mid-checkin bustages, since the client would (by design) be trying to jump in right after someone commits something.