Month: January 2010

Major update to Firefox 3.6 (the day after)

joduinn49 Comments

Looking at the approx 4.5million people who moved to FF3.6.0 since yesterday morning:

  • ~66% (3million) downloaded the FF3.6.0 installer and installed from it
  • ~33% (1.5million) manually did Help->CheckForUpdates

This is very similar to what we saw with the FF3.5.0 release. As far as I can tell, we did not gain 3million *new* users yesterday. My belief, without data to prove it, is that most/all of those 3million people were *existing* Firefox users who chose to do a download install, instead of doing CheckForUpdate. I guess there are cases where a Firefox user might want to do a fresh install, but I would have thought that most people would prefer the easier in-place upgrade of Help->CheckForUpdates.

To me, I was hoping to see:

  • ~20% downloaded the FF3.6.0 installer and installed from it
  • ~80% manually did Help->CheckForUpdates

Is there something we could do differently, so that next time, most existing Firefox users do Help->CheckForUpdates, instead of a fresh download? Is it possible that people hear about the new release, but do not know about Help->CheckForUpdates? What do you think?

Firefox 3.6.0: behind the scenes with unittests and performance tests

joduinnLeave a comment

As you all know by now, Firefox 3.6.0 shipped today. The new features and improvements are already being covered by other people in great detail, so I thought I’d focus on something totally different that happened behind the scenes for this release.

Firefox 3.6.0 is the first release where we ran all our automated unittests and talos performance suites on the officially branded release builds (you can see the test results for yourself by clicking on the image!).

As you know, we run unittests and performance tests on unbranded builds for each checkin, and also on the unbranded nightly builds. However, releases are branded builds – and until now, the best we could do was run unittests and performance suites on unbranded builds, and if all looked good, then branch from that same source code, rebuild with branding enabled, and hand those branded builds to QA for testing and release.

While this approach has worked “good enough” for years now, we had to carefully track the possible differences between unbranded and branded builds, and hope that compile changes caused by branding was covered by manual testing. To fix this required some fiddly reworking of unittests and talos performance suites. For details, the curious can read through bug#383136, bug#457753, bug#517212 and all of their linked bugs.

Big tip-o-the-hat to catlee for making this happen, and hope you enjoy your new Firefox!.

Drawing lines in the sand

joduinn2 Comments

This 8minute video shows the winner of “Ukraine’s Got Talent” drawing pictures in sand with light from underneath shining up to a projector. Sounds kinda strange at first, but don’t be fooled. I found it mesmerizing and quite touching.

Before you click the video link, you need to read this explanation first.

Kseniya Simonova, 24, uses sand and music to show how ordinary Ukrainian people were affected by the German invasion during World War II. She begins by creating a scene showing a couple sitting holding hands on a bench under a starry sky, but then warplanes appear and the happy scene is obliterated. It is replaced by a woman’s face crying, but then a baby arrives and the woman smiles again. Once again war returns and Miss Simonova throws the sand into chaos from which a young woman’s face appears. She quickly becomes an old widow, her face wrinkled and sad, before the image turns into a monument to an Unknown Soldier. This outdoor scene becomes framed by a window as if the viewer is looking out on the monument from within a house. In the final scene, a mother and child appear inside and a man standing outside, with his hands pressed against the glass, saying goodbye.

The Great Patriotic War, as it is called in Ukraine, resulted in one in four of the population being killed with 8-11 million deaths out of a population of 42 million, and you can see the very personal impact of her work on the live audience.

Talos recalibration (status 18jan2010)

joduinn6 Comments

If you don’t care about Talos performance results, or Talos hardware, stop reading now!

Its been a hectic couple of weeks on the Talos front since my last post. Here’s a quick summary of whats been going on:

In RelEng, we’re using this recalibration as a chance to cleanup a few long-standing details of how Talos slaves are configured. These changes to the Talos ref images include:

  • setting up these new machines in the Build network, with the rest of the build machines, not in the QA network, where Talos has been running since its inception. This will allow us to clean up some VPN and firewall configurations within the colo.
  • changing accounts on Talos machines to be consistent with all the other build machines
  • making sure configuration management software is installed.
  • bumping up OS versions, which have been intentionally been unchanged since 2007!:
    • OSX10.4: we’re leaving on old machines for now. From initial glance, this doesnt work on the new minis, and is already de-supported on mozilla-central, so we might end up just leaving it. More on this as it emerges.

    • OSX10.5: upgraded from 10.5.2 to 10.5.8.
    • WinXP: no OS change
    • WinVista: replaced with Win7
    • ubuntu7: replaced with fedora12 (32bit)
    • ubuntu 64bit: replaced with fedora12 (64bit)

In IT:

  • The first batch of 100 minis arrived just before New Years. Last week, Matthew and Phong (with a little help from aki, jhford and myself) spent a day unboxing, removing wrappers, sorting power cables, putting on asset tags, scanning serial numbers, etc, etc, etc. Scroll down for photos!

  • the racks were delivered, installed and cabled. Also power upgraded and air conditioning prep’d by middle of last week.
  • as ref images were completed, and tested in staging, use them to image a set of minis.

As of Friday evening, 60 of the new slaves are imaged using the new OSX10.5.8, WinXP and Win7 reference images, racked and powered.

In the coming week, we’ll:

  • have the remaining 40 imaged with linux32, linux64.

  • schedule a downtime to have all these new slaves enabled in production, along side the existing production slaves.

    If all goes well, after about 2 weeks, we’ll take the old systems out of production, and declare that first phase done. Stay tuned for more details.

    (If you’ve read this far, and have questions about anything that I’ve missed, please let me know.)




Infrastructure load for December 2009

joduinn1 Comment

Summary:

  • Numbers were down significantly compared to the previous few months. I’d guess an indicator that Mozilla entered the end phase of the Firefox 3.6.0 and Fennec 1.0 releases, both around the same time. The holidays in the last half of the month also reduced the load.
  • The numbers for this month are:
    • 1,268 code changes to our mercurial-based repos, which triggered:
    • 15,345 build jobs, or ~65 jobs per hour.
    • 33,263 unittest jobs, or ~44 jobs per hour.
    • 31,372 talos jobs, or ~42 talos jobs per hour.
  • Yet again, we enabled new Talos suites, and also disabled some unittest suites at different times during the month. For simplicity, I’ve ignored those changes for now, and will include in next month’s data.
  • Interesting to note that the trend of “what time of day has most checkins” changed again this month. Not sure what this means, but worth pointing out that each month seems to be different. This makes finding a “good” time for a downtime almost impossible.
  • We are still not tracking down any l10n repacks, nightly builds, release builds or any “idle-timer” builds.

Here’s how it looks compared to the year so far:

Detailed breakdown is :

Here’s how the math works out:

The types of build, unittest and performance jobs triggered by each individual push are best described here.

Chaos for Breakfast by Robert A. Hall

joduinn2 Comments


Most “management” books focus on working in for-profit companies. By contrast, this was the first book I’ve read that specifically addressed non-profit organizations.

There was some of the usual stuff about working with managers and employees. But there were also sections on working with volunteers (as opposed to employees), working in mixed employee+volunteer settings, fund-raising, public relations, outreach, political lobbying and the use of metrics to improve how you work. In an era when so many for-profit companies are focused on short term quarterly profits, and not on longer term health of their organization, or their people, I found it refreshing to see these self-evident truths written down:

“…[it is your] responsibility to ensure that the association is still exists in 10 years time to do the good work”

“…[it is your] responsibility to leave organization in better shape then [you] found it.”

“keep everyone informed”. (Something I struggle with constantly, given the very distributed nature of RelEng here in Mozilla.)

At 99pages, its a quick read compared to all the weighty tomes you usually find in that section of the bookstore. Later, after some thought, I went back and read it again, skipping some sections, and carefully re-reading others slowly. He also referred to several other interesting books that I’d never heard of, making my amazon.com wishlist even longer! I found his writing style terse, informative, down-to-earth, and occasionally a little repetitive. I since discovered the author is a Mensan, former US Marine and former Massachusetts State Senator, which explains a lot! 🙂

Overall, it was repetitive in a few places, not relevant to Mozilla in a few places, but I still found it well worth the read, and the re-read!

Nation by Terry Pratchett

joduinn4 Comments

As a long time fan of his Discworld series, I thought I’d give one of his non-Discworld books a try. The first couple of pages of this book were interesting, so I bought this on impulse in the bookshop. It was only *after* I finished reading it that I discovered the book was intended for teen readers, and I still liked this book.

The book is set on a tropical island after a tsunami wipes out the local villages. The sole survivor is Mau, a teen boy who starts to rebuild his devastated village by himself. Gradually other survivors from different cultures arrive, all looking for refuge and together they rebuild the physical village. In the process, they learn to question the social norms of their elders, and merge their various cultural backgrounds to create a new blended society of their very own. All good food for thought. And being Terry Pratchett, it was all done in a way that would give teens and grownups food for thought about life in the “real” world, while still being entertaining!