Fun and Games with Major Updates

04 Mar 2009 John 15 comments

When we make a Major Update available, all users at least see the Major Update offer. Right? Well, actually, not always.

Short answer:

When we make a major update offer available, only some of our users see the major update offer at all.
For users who might see a major update offer, they only have the potential to see the major update offer *until* we next make a new minor update available. At that point, the major update offer is effectively disabled / turned off for all users.

For the longer answer, make some coffee and read on!

First, to set context, here’s a diagram showing what we did for FF2, including the 3 different major update offers.

With that in mind, lets start by looking at a straightforward case:

Mozilla makes FF2.0.0.15 -> FF2.0.0.16 updates available
User upgrades from FF2.0.0.15 -> FF2.0.0.16
Mozilla make FF2.0.0.16 -> FF3.0.1 major update available
User see MU offer from FF2.0.0.16 -> FF3.0.1

some users upgrade to FF3.0.1, some stay on FF2.0.0.16

All that worked as expected. Now, lets revisit that scenario to find some “gotchas”.

1) User never sees the major update offer because they are on an untargeted older (or newer!) dot release.

Mozilla makes FF2.0.0.16->FF3.0.1 major update available, targeting users on the latest available FF2.0.0.16 release.
By design, this major update is only visible to FF2.0.0.16 users, so FF2.0.0.15 users will not see this major update offer.
Once FF2.0.0.17 is available, and users move to FF2.0.0.17, there is no way for those FF2.0.0.17 users to go back to see the FF2.0.0.16->FF3.0.1 major update.

A user on FF2.0.0.17 needed to wait until we produced FF2.0.0.18, upgrade to FF2.0.0.18, wait for us to produce a new major update offer from FF2.0.0.18, and then react to that new major update offer before we release FF2.0.0.19.

Similarly, once FF2.0.0.17 is available, users on FF2.0.0.15 can only upgrade to FF2.0.0.17; they are forced to skip over FF2.0.0.16, and are never given a chance to see the major update offer.
Today (03mar2009), we currently only have major update visible to FF2.0.0.20 users. This is important because according to today’s metrics data, 49% of our FF2 users are not on FF2.0.0.20. This means that 49% of FF2 users cannot even see our major update offer to move to FF3.0.5:

06.1M (51%) FF2.0.0.20 users who can see major update offer
05.8M (49%) other FF2 users who cannot see major update offer
=====
11.9M total FF2 users

Summary: major update targets upgrading users from a specific dot release. If a user is not on that specific dot release, they don’t see the offer.

2) Some users never sees the major update offer, even if they are on the targeted dot release, because of a race condition.

Mozilla makes FF2.0.0.16->FF3.0.1 major update available

This major update is only visible to FF2.0.0.16 users, so FF2.0.0.15 users do not see this.

Mozilla makes FF2.0.0.16->FF2.0.0.17 updates available
Existing FF2.0.0.16 users no longer see major update offer, and can now only see the update to FF2.0.0.17.
Any FF2.0.0.16 users who had their browser turned off long enough would miss this time window, and never see the major update offer at all.

Summary: The only users who can see the major update offer were using FF2.0.0.16 between the time the major update was first made available, and the time FF2.0.0.17 was released.

Hopefully all that made sense. Its tricky to explain without hand-waving in front of a whiteboard, so please let me know if you have any questions.

Finally, I’d really love to hear any suggestions people have on what we could do differently, so that more users see these major update offers?

UPDATE: I accidentally dropped some important disclaimers. See this follow-on blogpost for details. Sorry for any confusion. John 06mar2009

Uncategorized none

Comments

Leave a comment Trackback

Written by Dan about 1 year ago. Reply

Well the obvious solution would be to always offer the latest stable version to users regardless of what version they’re running (unless they are running the latest stable or newer) but I’m guessing based on those pictures you’re not doing that for a reason… profile migration issues maybe?
Written by Daniel Einspanjer about 1 year ago. Reply

Could you talk a bit about why we can’t offer a MU to lots of different point releases?
I understand that for minor upgrades, we don’t offer 2.0.0.15, 2.0.0.16, and 2.0.0.17 all the option to upgrade to .18 because these upgrades involve partial installs that have to be tested for each individual platform/locale/version. But isn’t an MU just a simple pointer to the complete or full download?

When I think about the simplest thing for our users to see, I come up with the following paths:

1. a user on .y is offered an upgrade to .z. This upgrade path has been tested by QA to support a partial download to save the user time. If it fails for some reason though, they will fall back to a complete download.
2. users on .t, .u, .v, .w, and .x are notified that there is a .z available. If they initiate the upgrade, what will happen is in effect the same as if they had downloaded either a complete or a manual download of .z. Hopefully, this would minimize the amount of testing required and we could make the notification available to anyone on any version prior to .y (unless the user had disabled notifications of course).
3. users on .a-z would receive a notification that there is a .A available (MU). Same logic as #2.

I know that there are probably some good reasons why we can’t do this, but I’d like to see it spelled out exactly what they are.
Written by bwmaister about 1 year ago. Reply

Oh, so that’s why I have to download and install firefox for those of my friends’ friends that I run into that haven’t updated to 3.0.*.

Seriously, I understand/imagine that mozilla will have to do a bunch of extra QA work, but please please make the major update available to all 2.* and 1.* users. There’s a guy at columbia (university) who’s still on 1.something because he never got the update, or something, and his firefox is telling him that he’s at the latest version. And I run into somebody about every two months (and I don’t run into that many people) who’s on an old version of the fox.

thanks for explaining, this will make future encounters like that easier to explain/deal with.
Written by Michael Lefevre about 1 year ago. Reply

“Well the obvious solution would be to always offer the latest stable version to users regardless of what version theyâ€™re running”

Well, John is better placed to answer than I am, but that would mean if someone is on 2.0.19, and they refuse to update to 3.0 (because they don’t like the new features, have incompatible add-ons or whatever), they would never be offered the security update to 2.0.20.

Seems what is needed is a system which can handle offering a major update and a minor update at the same time, so if the user doesn’t want the major update they can at least get the minor update, but all users will be offered the major update at least once regardless of which version they are on (which I think is what Daniel said more clearly).

But I suppose a problem with any more complicated path that could be suggested is that it’s impossible (I assume) to change the update mechanism in existing versions.

On the other hand, do we even know if this is actually a big deal? If these users are not accepting 2.0.x updates for whatever reason, would they accept a major update even if it was offered?
Written by Chris about 1 year ago. Reply

I agree with the points above – 2.0.* should all receive the major update. In addition, shouldn’t the major upgrade ‘trump’ the minor one, so if a 2.0.16 user is given two upgrade paths, the default one is the major upgrade?
Written by marianoCuenze about 1 year ago. Reply

First, really clear explanation.

Just a thought: why not do an update in the “update system” to allow this?:

1- check for all the 2.* upgrades (and offer to the user go to the last one )
2- for every 2.* upgrade, check if there is a MU to “switch to” the 3.* line, if there is one (the first one), track all the upgrades until the “tip” of 3.* and offer that “tip version” to the user.

then the user would see the last version in their “line” and in the “best current line”, and he/she will choose in which line want to be.
Written by Curious about 1 year ago. Reply

I know this is off topic … but I’d like to know what you used to make those lil diagrams in this blog post.
Written by John O’Duinn’s Soapbox » Fun and Games with Major Updates (addendum) about 1 year ago. Reply

[...] In all the edits/revisions of my earlier “Fun and Games with Major Updates” blogpost, somehow I dropped the following by accident: [...]
Written by juanb about 1 year ago. Reply

Buried in the annals of Bugzilla, there are a few bugs explaining how the update mechanism is supposed to deal with both offers at the same time. We never did enable this scenario because we didn’t feel comfortable about the update mechanism changes that were done to deal with Vista, which at the time we were just going to start to support, and AUS changes to deal with major updates.

The idea was to offer both kinds of updates, minor having precedence over major, and have the user incrementally update his or her way up to the next big version of Firefox, with major update offers along the way. Minor updates were assumed to be smaller in size; they would offer stability and familiarity to the user, and we knew what to expect. Major updates were a different beast.

After several iterations of major updates we should be a little more confident. Perhaps we can dig out those old bugs and see where we left off, and maybe it will turn out we can offer both at the same time with, hopefully, small changes.
Written by jmdesp about 1 year ago. Reply

@7 : pen, paper, scanner

Really I think the solution is obvious. Enhance the update system, make it able to handle *branches* in update.

Do you want to update to :
- minor version 2.0.20
- major version 3.0.7 : Followed by a text enumerating a list of reasons to choose 3.0.7 …

I’d love to have that the nightlies, after the trunk has branched and I’m directed to the branch without an option to stay on the trunk, except to download a full version of the latest trunk.

Also the text with the reason to choose 3.0.7 is *important*. A significant part of 2.0 users are deliberately keeping 2.0, because the first time they tried 3.0 they had some problem. The person who has a stable 2.0 install has a very high level of stability/feature requirements for a new version, that a 3.0.0 version might not fulfill.
If you try to identify and solve them, then you’ll need to explain them you have found and solved *their* problem with Firefox 3.
Written by Ian Thomas (thelem) about 1 year ago. Reply

@chris “shouldnâ€™t the major upgrade â€˜trumpâ€™ the minor one”

No – If a user is on 2.0.0.16 and knows they don’t want to upgrade to 3.x (maybe they have incompatible add ons) then we still want them to upgrade to the latest 2.0.0.x release. You might want to consider a screen that offers them the choice of both, but I think that’s a much more complicated discussion.
Written by John O’Duinn’s Soapbox » Fun and Games with Major Updates (followup) about 1 year ago. Reply

[...] There were lots of comments and questions raised from the last two blog posts [1] [2]. As some people were asking variations of the same questions, I thought it would be less confusing to summarize the questions and answers all here instead of replying in comment threads. Please let me know if I missed any of the comments, or misunderstood the questions, ok? [...]
Written by John about 1 year ago. Reply

“I know this is off topic â€¦ but Iâ€™d like to know what you used to make those lil diagrams in this blog post.
Comment by Curious”

Heh, sure. I’m using OmniGraffle (http://omnigroup.com) on my Mac OSX 10.5. There might be other things that work better/faster, but this works really well for me.

For something as complex as all this “race-condition during update cycles” problem, I found the simple line drawings I could create with OmniGraffle worked best. And hey, they are the nearest I could get to the whiteboard-diagrams-with-handwaving that I’ve scribbled on various whiteboards and napkins around Mozilla whenever someone asks me questions about all this.

tc
John.
Written by John O’Duinn’s Soapbox » Fun and Games with Major Updates (some new options) about 1 year ago. Reply

[...] After the last few blog posts [1], [2], [3] explaining details of some gotchas, here are a few mechanical ideas which might help. I’m not sure which, if any, make sense to try, but wanted to point out some new options we have available to us now after infrastructure improvements over the last year, and would love to hear what people thought. [...]
Written by John O’Duinn’s Soapbox » Major update to Firefox 3.5 about 1 year ago. Reply

[...] (For details on race conditions where people dont see the major update dialog box and on the “update fatigue” debate, see: here, here, here, here, and finally here.) 3) Nick Thomas led a bunch of significant cleanup in RelEng infrastructure, so we can now create major updates quite easily and reliably. [...]

John O'Duinn's Soapbox

Fun and Games with Major Updates

Comments

Leave a Comment

Recent Posts

Meta