Otoro and Unagi builds now on tbpl.mozilla.org

In case you missed it, Catlee has quietly enabled Otoro builds on mozilla-beta. This means that now:

  • per checkin, we generate a full oroto build. This otoro build is on the same changeset as the unagi build, the gecko-compiled-for-arm-with-b2g-enabled builds, and all the usual desktop Firefox and mobile Fennec builds that we already generate per checkin. Having these builds on tbpl helps narrow down build regressions. However, as these full otoro builds are ~164MB each, and we do a *lot* of builds per day, we do not plan on uploading these until we have tests to run against them. Of course, if you have a need for these, please let us know, and we’ll enable them, its an easy change.
  • every night, we generate and publish an oroto build, an unagi build, the gecko-compiled-for-arm-with-b2g-enabled builds, b2g desktop builds, on the same identical changeset. As usual, all the desktop Firefox and mobile Fennec builds are built on the one changeset.

These new otoro builds have already been quickly evaluated by RelEng and by QA, and as far as we can tell, they look fine to us. However, if you have an Otoro phone, and have access to the private share, you can help! Please grab one of these new Otoro builds, which you’ll find alongside the previous otoro builds, install it on your Otoro phone and give it a try. We’re expecting to transition everyone over to these builds in the coming days, so if you see anything which makes you think we should stop rollout, please file a bug and we’ll get right on it.

RelEng production systems now in 3 AWS regions

tl;dr: We’re now running our build infrastructure across 3 different Amazon regions. This makes us more robust and *cheaper*?! Whats not to love?! 🙂



This is important for 3 reasons:

1) It means RelEng can keep up with load, and hence keep all trees open, even if an Amazon region goes offline, or if a VPN link fails. Amazon doesn’t lose a region often, but it can happen. Mozilla doesn’t lose a VPN link often, but it can happen. Using 3 different regions, with 3 different VPN links, makes it unlikely we’d lose all at one time. In fact, multi-region outages on AWS are so rare, that the most recent multi-region outage I could find was in June2008.

2) As our first “go hybrid on AWS VPC” is a clear success, we’re experimenting with VPCs that are further away (ie slightly slower connection) from our inhouse colos. This allows us to start using regions that are cheaper (good!), and not inside the same earthquake zone (also good!).

3) We have a month or so of realistic usage data on AWS, as all the builds which we can run on AWS are now running on AWS. (The only exceptions are recent requests for new B2G builds, which we’re still setting up). This means that we can now make decisions about bulk-purchase-in-advance instances (called “reserved instances” in AWS lingo), which buys us the same compute power for ~1/4 the price. As these reserved instances are region-specific, and cannot be refunded or swapped around to other regions later, it was worth bringing new cheaper regions online first before we start bulk purchases in those cheaper regions.

All in all, this is a big deal.

Oh, and it is worth noting for the record that these additional regions were brought online, and into production, without needing any downtime, and without any hiccups! Big thanks to catlee and rail and ravi for their work.

Khmer team at MozCamp Singapore

At MozCamp Singapore, I was delighted to meet up with some of the Khmer localization team again. From left to right, here’s Vannak Eng, Sokhem Khoem, John O’Duinn, Sophea Sok, Mark West and Piseth Kheng. (Javier Sola was unable to make the trip at the last minute, so is sadly missing from the photo.)

They are very modest, great fun, and super sharp. As a sneak surprise, it was great to see them hack together a Khmer version of FirefoxOS during a quick hackfest arranged by Stas at MozCamp.

Since we last met at my workshop in Phnom Penh in Jan 2012, we officially released Khmer as part of the Firefox13.0 release and government policy now has Khmer Firefox being used in government departments in Cambodia! Great stuff.

If you want to find out more about the Khmer team, or want to join them in localizing, please look here: https://wiki.mozilla.org/L10n:Teams:km or http://www.mozillakm.org/

HOWTO use an unlocked Android phone in Malaysia

Here what I used in my trip to Malaysia in Nov2012, in case others find this helpful:


Disclaimer:

  • In the US, buying a cellphone “out-of-contract” is not the same as buying a cellphone “unlocked”. All of the following only works for an unlocked phone. Make sure your phone is unlocked before you get on the plane.
  • Different cellphone companies have different policies on this. AT&T declared that, despite my being a multi-year customer, with no contract, they would not unlock my phone per policy. T-Mobile said upfront that they would need ~40days from date-of-purchase of “out-of-contact” phone before I could ask to have it unlocked. On the 40th day, when I asked T-Mobile to unlock my phone, they sent me the phone unlock codes within 48hours.
  • Make sure your phone supports GSM. Sounds obvious, but still needs to be said, as most countries use GSM.

  • NOTE: You need to show your passport, or national ID card, when buying a pay-as-you-go SIM, or a phone, in Malaysia.
  • Buy a “HotLink” pay-as-you-go SIM card. I bought mine at the airport in Kuala Lumpur, but they are also for sale on most small street corner stores. While there are several mobile companies selling pay-as-you-go, I went with HotLink because they had the best price for data at 4G-LTE speeds, great high speed coverage everywhere I went, and no hassle about using your cellphone as a hotspot. Oh, and comparable prices for voice calls and text messaging.
  • Disassemble your phone to swap out sim card, insert new HotLink sim card and power up the phone.
  • On the phone, enter “*122#″ and press dial (typically, the green handset button). This will connect you to an automated voice service which will tell you your balance.
  • Assuming that works, you should now attempt to call any local number. By habit, I now call the mobile phone of the person at the store selling me the SIM card.
  • Cultural tip: I never setup voicemail – as discovered in my other recent trips to SEasia, most people dont both leaving voice messages – if they cant reach you when they phone, they hangup and send you a text message instead.
  • At this point you should be able to make/receive calls.
  • To make my Android 2.2 phone transmit/receive data, I had to add the following APN settings:
    * on home screen, go into “settings”
    * go into “wireless & network settings”
    * go into “mobile networks”
    * go into “access point names”
    * if there is not already a “maxis” APN, then create one as follows:
    ** Name = maxis
    ** APN == bbnet
    ** Proxy == 202.75.133.49
    ** Port == 80
    ** Username == Not set
    ** Password == Not set
    ** Server == Not set
    ** MMSC == Not set
    ** MMS proxy == Not set
    ** MMS port == Not set
    ** MCC == 502
    ** MNC == 12
    ** Authentication Type == Not set
    ** APN Type == Not set
    …hit save, and go back to “Access Point Names”.

  • verify that this new “maxis” APK is present, and is selected.
  • verify that “Use only 2G networks” is not selected.
  • Reboot the phone to see if that helps.
  • Now that the phone is configured correctly, I selected the 500mb-per-day data plan, as follows:
    enter “*100*9*1#” and press dialer
    read menu, enter “2” and click “ok”
    read menu, enter “1” and click “ok”
    …thats it.

  • At this point you should be able to make/receive calls, send/receive text messages, surf the web, and use your cellphone as a wifi hotspot.
  • To check your account balance call *122#.
  • When you need additional credits, buy a one-time use scratch-refill “top up” card at almost any corner store, and follow the instructions on the back. Alternatively, their website says you can topup using PayPal as well as credit cards, but I never personally tried that. Either way, you’ll receive a text message with the new balance when the credits are added to your account.

Infrastructure load for October 2012

  • #checkins-per-month: We had 5,893 checkins in October2012. This is a new record, breaking the previous record in August of 5,803. Even more impressively, we handled this record load with >95% of all builds consistently being started within 15mins.
  • #checkins-per-day: During October, 19-of-30 days had over 200 checkins-per-day.
  • #checkins-per-hour: The peak across this month was 12.87 checkins per hour between 10-11am, a new record, closely followed by 12.35 checkins per hour between 1pm-2pm.

mozilla-inbound, fx-team:
Ratios of checkins remain fairly consistent. mozilla-inbound continues to be heavily used as an integration branch, with 28.6% of all checkins, far more then the other integration branches fx-team (0.7% of checkins) or mozilla-central (2.5% of checkins). As usual, very few people land directly on mozilla-central – in fact more people go through approval process to land on mozilla-aurora.

mozilla-aurora, mozilla-beta:

  • 5.7% of our total monthly checkins landed into mozilla-aurora. This is an increase over previous months, and I suspect is caused by the number of b2g changes being landed into aurora.
  • 2.6% of our total monthly checkins landed into mozilla-beta.

(Standard disclaimer: I’m always glad whenever we catch a problem *before* we ship a release; it avoids us having to do a chemspill release and also we ship better code to our Firefox users in the first place.)

misc other details:

  • Pushes per day
    • You can clearly see weekends through the month.

  • Pushes by hour of day

Infrastructure load for September 2012

  • #checkins-per-month: We had 5,315 checkins in September2012, a drop from August.
  • #checkins-per-day: We had a new record of 315 checkins in one day (25sep). Thats a checkin every 4.5mins for 24 hours. This is a significant jump above our previous record of 284 checkins per day. During September, 16-of-30 days had over 200 checkins-per-day, but it looks like beginning of September was quieter than end of September. Not sure if this is because of vacations at the start of the month, or because of B2G deadlines at the end of September… or a combination of both?!
  • #checkins-per-hour: The per-hour peak across this month was slightly down, at “only” 11.3 checkins per hour.

mozilla-inbound, fx-team:
Ratios of checkins remain consistent. mozilla-inbound continues to be heavily used as an integration branch, with 24.1% of all checkins, far more then the other integration branches fx-team (1.5% of checkins) or mozilla-central (2.8% of checkins). As usual, very few people land directly on mozilla-central – in fact more people go through approval process to land on mozilla-aurora.

mozilla-aurora, mozilla-beta:

  • 3.3% of our total monthly checkins landed into mozilla-aurora.
  • 2.2% of our total monthly checkins landed into mozilla-beta.

(Standard disclaimer: I’m always glad whenever we catch a problem *before* we ship a release; it avoids us having to do a chemspill release and also we ship better code to our Firefox users in the first place.)

misc other details:

  • Pushes per day
    • You can clearly see weekends through the month.

  • Pushes by hour of day

“Oh, the Places You’ll Go!” by Dr Seuss

After last week, I’ve proudly added this to my bookshelf.

The book is, of course, fun. But the biggest reason I’m was because I received it at LEAD. This 4-session LEAD series ended Friday, and has been more then just an experience; its “an inflection point”. The excellently designed “coursework” and setup/moderation by Athena, Kate, Debbie and Mihca provided the framework. But the intensity, passion and honesty that everyone engaged within LEAD left me stunned, and immensely proud to know each and every one of them. The book is a small talisman.

ps: (possible NSFW?) below is a really fun reading of the book done at Burning Man 2011.