Aki assembling a small mobile pool!

Heading home Friday evening, I stopped by Aki’s desk and took these quick photos. At first glance, these might not look all that exciting:

…but look closer… closer… see the little bits of paper taped on the machines??

This is actually really really big! This is the beginning of a pool of mobile devices slaves to automatically run unittest and talos on mobile builds.

Aki’s been using these two machines to figure out linux mobile hardware setup problems around the hardware setup, toolchain setup, memory limitations as well as how to get a buildbot slave running on the device, communicating to a buildbot master.
Since end of Sept, we’ve been producing linux-arm builds of fennec automatically on checkin and every night. However, humans were still needed to manually download specific builds onto the physical devices when they had time, and manually run some tests. There’s still no automated unittests, talos, graphserver for mobile.

The build automation was done by updating all the linux slaves in our existing pool of slaves, and then doing cross-compile builds. That allowed us to re-use a bunch of existing infrastruture. However, we need to use actual devices for testing and performance… which opens a whole set of problems we have to figure out for the first time. For example:

  • if the device hangs during a test, how do you powercycle it? (Most people first suggest removing battery and running on remote switchable a/c; however, these devices refuse to boot unless a battery is present, and it looks like they need a physical button pushed on after a power reset, so we’re still looking for a solution – all suggestions welcome!)
  • How do these devices communicate to other machines; how do we get builds onto, and test data off from, these devices? The buildbot master needs to tell slaves when and what to do. The slaves need to post results to buildbot master as well as ftp.m.o and graphserver. Something (nagios?) needs to monitor to see if devices are ok. If we use wifi, does that cause noisy deviations in our test results? If we use ethernet, what s/w drivers and cable converters do we need?
  • If we setup devices in office, for easier manual reboots, what ssh/firewall changes do we need to get access to other build systems in secured area in colo?
  • What toolchain do we need to install on these devices? Both of these devices were “polluted” when we got them, meaning no-one was exactly sure whats on them, and we’re fairly sure they’re different from each other. We’ve got some more machines on order, so will have to figure out clean-setup-instructions as part of this setup.
  • Is there enough memory in the device to run o.s. + buildbot + talos + talos-tools + fennec? Can far can we increase memory? Do we have to rewrite buildbot/talos to fit into low memory situations?
  • We can’t build on these devices, but unittests currently requires building first before testing.
  • What can we do about turnaround time? These devices are *SLOW*… Break testsuites into chunks that can be run in parallel?

And all that is before you get to the the fun stuff… the usually expected problems, like how many tests fail because of different display properties, memory availability, slower CPUs causing test timeouts, environment differences, etc. etc. etc.

While there’s still lots to solve along the way, its exciting to watch Aki making methodical, persistent and rapid progress. Very very cool.

How to search Thunderbird emails with Spotlight on a MacBookPro (OSX 10.5)

After upgrading from OSX10.4 -> OSX10.5, I was surprised to discover that Spotlight was no longer indexing Thunderbird emails.

I went back and rechecked all the steps in my earlier blog post, thinking maybe some files were lost in the upgrade, but all appeared ok. Using “/usr/bin/mdimport -L" I verified that the importer was present and still running. I even tried "/usr/bin/mdimportmdutil -E" to do a complete new re-index, in case it was somehow corrupted. Still no success.

Re-reading my earlier instructions, this step caught my eye:

  • Move Thunderbird.mdimporter to either “~/Library/Spotlight/” (which I did) or “/Library/Spotlight/” (as suggested in some other posts)

…so as an experiment, I moved the Thunderbird.mdimporter directory from “~/Library/Spotlight/” to “/Library/Spotlight/”, used "/usr/bin/mdimport -L" to verify that it was running in the new location. Immediately, Spotlight was finding matches within my Thunderbird emails.

The only change I did was to move the Thunderbird.mdimporter – could the location of this really be o.s. version specific? Maybe that explains why some people used one location, and some the other?

(If anyone reading this has done these same steps, I’d be very curious what location are you using for Thunderbird.mdimporter, and what version of OSX you are using!)

UPDATE: At first, Spotlight worked fine, but problems arose soon after the original blog post. Spotlight started giving garbage results pointing to unrelated files, showing confusing icons and filenames for matches, and eventually a system crash. I suspect that Spotlight from 10.5 got confused by the existing indexed data of Spotlight from 10.4, but have no data to back that theory.

To get Spotlight working with Thunderbird properly, I did the following:

  • move the Thunderbird.mdimporter directory from “~/Library/Spotlight/” to “/Library/Spotlight/”
  • use "/usr/bin/mdutil -E" to clean out the existing indices.
  • reboot the computer
  • use "/usr/bin/mdutil -E" to clean out the existing indices again. (maybe unneeded but by now I was being paranoid!)
  • used "/usr/bin/mdimport -L" to verify that it was running in the new “/Library/Spotlight” location.
  • wait a few mins, then try searching your email.
  • smile and celebrate with a cup of coffee.

Hope that helps – John. 04dec2008

Branch diagram, including the new FF3.1 project branch

Last Wednesday, we setup the new Firefox3.1 project branch. (Details on the machine-setup work coming in a separate blog post.) Meanwhile, I thought people might find this diagram useful to help understand the repository cloning/branching work that happened.


  • To keep terminology consistent, last week we created a “project branch”. We’ll be creating “release branches” on this project branch for each milestone.
  • Before we allow checkins to the new mozilla-1.9.1 project branch, we will need to carefully cross-check all the changes landed on mozilla-central since the mozilla-1.9.1 branch was created (The area in the dotted circle) and verify that those changes were also landed on mozilla1.9.1.
  • We want the mozilla-1.9.1 branch to start from FF3.1beta2. To speedup opening the mozilla-1.9.1 branch, we branched early. This means that before we allow checkins to the new mozilla-1.9.1 project branch, we will need to carefully cross-check all the changes that landed on mozilla-central since the mozilla-1.9.1 branch was created (The area in the dotted circle), up to the branch for FF3.1beta2, and verify that those changes were also landed on mozilla1.9.1. Any later FF3.1beta2 respin would require patches to also be landed on mozilla-1.9.1.
  • There’s been discussions recently on whether we’d have a FF3.1b3 or not. I included FF3.1b3 in the diagram so that, if we do decide to do Beta3, people could see where it would be branched from. This is not meant to state or imply that we are doing Beta3! 🙂
  • The diagram above shows logical active-branches-of-code. At the SCM repository level, some of these active-branches-of-code are implemented as in-repo-branches, and some are implemented as separate-cloned-repos. For the sake of clarity, I’ve omitted those details in this post, but would be happy to go into further detail, if anyone is interested.

Hope that helps?


UPDATE#1: added extra text in 2nd bullet point, describing the cross-branch synchronization, to clarify how we branched and why.

UPDATE#2: Sam Sidler and and Gavin Sharp both pointed out that the branching diagram had an error when describing how FF1.5 was branched from FF2.0. They are correct, and I thank them. The branch diagram above has been updated, and is now correct. I since found the same error in a presentation we gave here at the 2008 Mozilla Summit (diagram had same error), and a brownbag in Mountain View office (diagram had same error).

How to export Contact info from Palm Pilot/Treo to Apple iPhone 3G/4Gs

The basic technique here is to export all your Contact info from Palm’s format to Microsoft Address format, and then import from there into the Apple iPhone Contacts app. The best instructions I’ve found so far were here, but I’ve added extra gotchas below in case it helps others. Note: this only transfers Contact info, and does not transfer Calendaring, ToDo or anything else.

Before starting, you need to do the following:

  1. On a MS Windows PC, install palm desktop and iTunes. (If you primarily use iTunes on another computer, its ok to just install iTunes on the windows PC, do this one-off data transfer, and then throw away that iTunes installation).
  2. Start Programs->Accessories->AddressBook and make sure that the Microsoft Address book is empty. (Note: this is not to be confused with MS Outlook, which is very different!)
  3. On the Palm Pilot/Treo, look through all the contacts for the following gotchas:
    • If any contact has two entries of the same type (for example, a person with two mobile numbers, or two work phone numbers), you will need to manually remove/rename one of these numbers before you start. I noticed that any contacts with multiple entries of the same type ended up losing all but one.
    • If you have the same person listed multiple times in your Palm Pilot/Treo, this will cause an error later on. I discovered that I accidentally had the same person entered twice, in two different categories. These duplicate names caused errors later on in the export process, so its best to check and fix this before you start. Worst case, if you miss a duplicate, its quick and easy to just throw away all the conversion work and restart from the beginning again….but it sure it annoying!
    • Check for any occurrences of  ‘=  (single-quote followed by equals)  in your contacts, or attached notes. If you find this anywhere, you must change them to something else before you can continue. It seems that ‘= (single-quote followed by equals) is used as a delimiter somewhere in the conversion process, so anything after that gets cut from that specific person’s info, and never transferred.
    • “Custom fields” are not transferred, so you should either migrate that data to one of the “standard” fields, or make a note of them, and come back later to cleanup by manually copying from palm desktop into MS Address book.

OK, thats it. Now we’re ready to begin:

  1. Hotsync your Palm pilot/treo with the Palm Desktop one final time. Remember, backups are your friend!
  2. Start Palm Desktop and view the Address page. Select the contacts you want to bring over (ctrl-A on keyboard if you want them all), then choose File->ExportvCard… to save all your contacts into one single file on your desktop.
  3. Start Programs->Accessories->AddressBook
  4. Drag and drop the newly created file from your desktop into Microsoft AddressBook. You’ll be asked to press “ok” for each entry being imported into AddressBook. Annoying but its very quick.
  5. Unplug your Palm Pilot/Treo, and plug in your iPhone.
  6. Start iTunes. On the iPhone page, Info tab, choose to sync contacts with Microsoft Address Book and click Apply.
  7. Sync your iPhone with iTunes – this will in turn pull the contacts from the Microsoft Address Book.
  8. On iPhone, look in Contacts app, and verify that all your contacts transferred over correctly. Specifically, look for long attached notes to make sure nothing was truncated.
  9. Undo step (6) in iTunes.

Thats it!

Again, this only transfers Contact info; I’m still investigating exporting Calendaring, ToDo and other types of “legacy data” – any hints? 🙂

Big tip-o-the-hat to: http://www.fixya.com/support/t161691-downloading_contacts_from_palm_iphone and also http://www.dba2csv.com. Both were wonderful help!

All this left me wondering why Apple didnt provide an import utility to handle migrating from Palm Pilot to iPhone…or from BlackBerry to iPhone.  That would sure make it easier for business users to migrate over to iPhone. Oh well.

UPDATE: These same instructions worked great tonight when migrating a friend from a Palm Pilot to a new iPhone 4Gs. Added warning about custom fields, which I missed in this first blog. joduinn 04-dec-2011

Sorting letters at the Royal Mail

Found this article and had to share. It struck me as a great experiment, yet somehow I had never even considered this idea before.

An artist wanted to see just how dedicated the letter sorting people were in the Royal Mail in England the UK. So, she mailed letters addressed using puzzles instead of the usual name-and-address format. Of the 130 letters she mailed, only 10 never arrived. Really good delivery rate, imho, especially considering at least one of them required the postal workers to complete the crossword!!

For the full story, and other examples, click here.

Electionmania and Joe Six Pack!?

I’m delighted that voting today was really really busy.

I live a short walk from my polling station, so walked over before breakfast, hoping to beat the rush. No luck, it was mobbed. By 8.30, lines were shorter, but still… At this point, I just patiently waited in the line, and ended up way late for work. But very very proud to have voted and proud of how good humored everyone else in line was.

While buying supplies for the election coverage tonight at the office, I found Joe Six Pack:

Its real beer, from a local microbrewery. I just love that someone thought of doing this… and also that they used their “Mavericks” beers for this. Lets see which one people prefer!


ps: Continuing in the election theme – here’s a local prankster who tried to do some street renaming: