Late last week, we finally completed a long running project: we found a new, better, home for our growing array of mobile phones in our continuous integration pool. Here’s what I presented at the weekly Mozilla Foundation call Monday morning about that project, hopefully it makes sense.

1) Each checkin gets:

• 40 hours of build/unittest/talos on a desktop computer, and

• 25 hours of unittest/talos on mobile phones.
• Instead of one phone doing 25 hours of testing, we could have 25 phones each do one hour of testing. Or 50 phones do half-hour of testing…
• we get 100-120 checkins per day – that is a lot of phones.

2) Phones are really sensitive to wireless noise

• interference causes intermittent orange/red test results, also manual hang/reset work for RelEng.
• Building K office was better, because we were far from anyone, but the new office is in downtown Mountain View, and has lots more wireless noise.
• To fix this, we built a “faraday cage“, a shielded room to eliminate outside wireless interference. Wikipedia has a great description and diagrams, if you follow that link.

• Hopefully this should give us better stability for tests results. Time will tell.

3) How to arrange the phones in the room

• lots of running devices, each transmitting radio signal. We’re planning for at least 400 phones all fitting inside the wireless room, so how to deal with cross interference. Not big enough to just arrange on desks / floor like we did while we were waiting for the Faraday cage to be built. Have to think 3D.

• however, hard to attach to walls, cannot puncture walls – cannot breach cage.
• whatever we do use within the room must be non-metallic, to minimize wireless disruption within the room
• phones cannot be *too* close to each other, early testing with cross-interference showed failures if bunched too close.
• as the number of phones increases, we might find more cross-interference, so whatever we do needs to be possible to move around.
• must not touch the front / sides of the devices, which might accidentally press a button
• must be easy to see the screen on the running phone
• tolerant of heat, either from the phone or the phone charger
• easy to take dead phones out, open, remove batteries, reimage and replace (we do this ~10 times per week)
• must be compatible with phones we haven’t bought yet

4) How did we solve it?

• BedBathandBeyond.com sell rolling mobile bamboo garment racks, which can be assembled with 20 screws each. These can be assembled in minutes.
• They also sell hanging shoe racks which velcro to the cross-rail, are made of heat-resistant fabric, have plastic as a firm base to each shelf and hold 20 phones each.
• Each garment rack can hang 40 phones (two shoe racks with 20 phones each). The power strips rest on the bottom wooden shelf and run cables up the back of the shoe racks. In this photo, you can see racks for 120 phones, of which 90+ are already in use.
• Bonus: each shoe phone is about 3-5 inches apart from another phone in adjacent shoe rack, which is the same spacing we had when the phones were just arranged on some empty desks in the office.

5) Brought to you by: aki, dmoore, jhford and mrz.

• They all did *tons* of invisible behind-the-scene work to make this happen. Every time you see a mobile phone reporting test results on TinderboxPushLog or draw a dot on GraphServer, give thanks to aki, dmoore, jhford and mrz.

• If you are curious for a video of the very quick presentation (3.5 minutes, according to Jono and his stopwatch), see Monday’s Mozilla Foundation weekly call here. I mention it in case the video is easier to follow the verbal with hand-waveing vs this blog post.