Mozilla’s Release Engineering now on Dr Dobbs!

No Comments

book coverLong time readers of this blog will remember when The Architecture of Open Source Applications (vol2) was published, containing a chapter describing the tools and mindsets used when re-building Mozilla’s Release Engineering infrastructure. (More details about the book, about the kindle and nook versions, and about the Russian version(!).

Dr Dobbs recently posted an article here which is an edited version of the Mozilla Release Engineering chapter. As a long time fan of Dr Dobbs, seeing this was quite an honor, even with the sad news here.

Obviously, Mozilla’s release automation continues to evolve, as new product requirements arise, or new tools help further streamline things. There is still lots of interesting work being done here – for me, top of mind is Task Cluster, and ScriptHarness (v0.1.0 and v0.2.0). Release Engineering at scale is both complex, and yet very interesting – so you should keep watching these sites for more details, and consider if they would also help in your current environment. As they are all open source, you can of course join in and help!

For today, I just re-read the Dr. Dobbs article with a fresh cup of coffee, and remembered the various different struggles we went through as we scaled Mozilla’s infrastructure up so we could quickly grow the company, and the community. And then in the middle of it all, found time with armenzg, catlee and lsblakk to write about it all. While some of the technical tools have changed since the chapter was written, and some will doubtless change again in the future, the needs of the business, the company and the community still resonate.

For anyone doing Release Engineering at scale, the article is well worth a quiet read.

“REAMDE” by Neal Stephenson


(At 1,044 pages, this book looks daunting. I’ve enjoyed other Neal Stephenson books, especially Cryptonomicon, so I didn’t let the size of the book deter me when I was buying it. But I find reading a long book with complex intertwined plots needs continuity – no point in picking it up and trying to resume after leaving it unopened for weeks! Even though I bought this book over a year ago, I only finally had time to read it in the last couple of weeks. Aside, in this day-and-age-of-laptops-and-kindles, I was amused by the odd sidelook I got whenever I settled into a nearby coffee shop and produced this weighty hardback ink-on-paper tomb!)

Wikipedia has a great summary here, but obviously be warned that it has lots of plot spoilers. Without giving too much plot away, I liked the book. From my perspective, I really enjoyed how Neal can interweave different stories. While there were many different interwoven stories here, the ones that are top of mind for me were:

  • the hacker-and-former-girlfriend-get-kidnapped story
  • the spy-tracking-jihadists story
  • the massive on-line game business story

All very different stories, yet the detailed coverage of each make me think Neal has a great understanding of hackers, encryption, different-business-market-economies-of-massive-on-line-games, Soviet-veterans-of-the-confict-in-Afganistan, internet-cafes-in-developing-worlds… the list goes on and on. I even found the way computer issues were covered to be accurately describes (typically a pet peeve for me!). In the midst of all the other drama, I was greatly amused by the image of a super-important invulnerable character (Egdod) walking in unattended mode back to home base, while various other T’Rain players were attacking him / defending him / rubber-necking the impossible sight of Egdod moving through their world. And somehow, someday, I need to find a way to use the throwaway joke about “Your org chart?”, “No, orc chart”.

The book was a great read, and I’d recommend it.

“APE – How to Publish a Book” by Guy Kawasaki

1 Comment

A few years ago, I first put my toes into the book publishing world by co-writing a portion of AOSAv2 about Mozilla’s RelEng infrastructure. Having never written part of a published book before, I had no idea what I was really getting into. It was a lot of work, in the midst of an already-busy-day-time-job and yet, I found it was strangely quite rewarding. Not financially rewarding – all proceeds from the book went to Amnesty – but rewarding in terms of getting us all to organize our thoughts to write down in a clear, easy to read way, explaining the million-and-one details that “we just knew instinctively”, and hopefully helping spread the word to other software companies on what we did when changing Mozilla’s release cadence.

While working on AOSAv2, clearly explaining the technology was hard work, as expected, but I was surprised by how much work went into “simple” mechanics – merging back reviewer feedback, tracking revisions, dealing with formatting of tables and diagrams, publishing in different formats… and remember, this was a situation where book publisher contracts, revenue and other “messy stuff” was already taken care of by others. I “just” had to write. I was super happy to have the great guidance and support of Greg Wilson and Amy Brown who had been-there-done-that, helped work through all those details, and kept us all on track.

Ever since then, I’ve been considering more writing, but daunted by all the various details above and beyond “just writing”. These blog posts help scratch that itch, in between my own real-life-work-deadlines, but the idea of writing a full book, by myself, still lingered. A while ago, I grabbed “APE: Author, Publisher, Entrepreneur-How to Publish a Book” by Guy Kawasaki. It looked like a good HowTo manual, I’ve enjoyed some of his other books, and he’s always a great presenter, so I was looking forward to some quiet time to read this book cover-to-cover.

This was well worth the time, and I re-read it a few times!

For me, some of the highlights were:

  • why are you writing a book?” I really like how Guy turned this question around to “why would someone else want to read your book”. Excellent mind-flip. I’ve met a few people who want to write a book, and even a few published authors, and I’ve talked with them about my own ideas about writing. But no-one, not one, ever reversed the question like this. It was instantly self-evident to me – it takes time to read a book, and we’re all busy. So, even if someone gave me a book for free, why would I want to skip work and/or social plans to read a book by someone I don’t know. Making it clear, immediately, why someone would find it worthwhile reading your book is a crucial step that I think many people skip past. As the author, keeping this in mind at all times while writing, will help keep you focused on the straight-and-narrow path to writing a book that people would actually want to read.
  • Money: Most publishers are super-secret about their contracts/terms/conditions, which can make a new time author feel like they’re going to be taken (The only exception I know of is Apress, who publish all their terms on their website, with a “no haggling” clause). To help educate potential authors, I respect how much full detail Guy & Shawn gave in small, easy to follow, words.
  • Tell the world you’re writing a book – not that you’re thinking of writing a book.” Again, an excellent mind-flip to help keep you motivated and writing, every single day, whether you want to or not. Also, they provided many links to writer’s clubs (writer support groups!?) who would help you keep motivated.
  • print-on-demand vs print-big-batch: This reminded me of how software release cycles are changing the software industry from old monolith release cadence to rapid-release cadence. “Old way”: a big-bang-release every unpredictable 18months, with a costly big print run, and lots of ways to handle financial risk of under/over selling; any corrections are postponed until the next big-bang-release if it looks like there is enough interest. “New way”: build infrastructure to enable print-on-demand. Do smaller, more frequent, releases, each with small print runs, (almost) no risk of under/over selling, corrections handled frequently and easily. Yes, at first glance, each printed book might seem more expensive this way, but when you factor in the lack-of-under/over selling, removed financial risk, and benefits of frequent updates to the almost-free electronic readers, it actually feels cheaper, more efficient and more appealing to me.
  • In addition to printed books, there’s a good description of pros/cons of the different popular electronic formats (PDF, MOBI, EPUB, DAISY, APK…) as well as related DRM.
  • The differences between ebook publishers (Amazon, Apple, Barnes & Noble, Google, Kobo, …), Author Publisher Services (Lulu, Blurb, Author Solutions, …) and Print-on-Demand (Walkerville Publishing, Lightning Source, …) was detailed and very helpful. Complex chapter, with lots of data, and ending with the reassuring “Don’t obsess about making the wrong choice, however, because most distribution decisions are changeable.”!
  • translations, audiobooks: normally, these are handled as edge cases. Guy & Shawn walk through some of the options (Audible/Amazon, Books-on-Tape/RandomHouse), as well as financial & legal realities.
  • Some fun examples of rejection responses by agents/publishers. My personal favorite was a rejection sent to George Orwell about Animal Farm “It is impossible to sell animal stories in the USA”.

All in all, I found the writing style personal, helpful, direct and super honest. Even the way they ended the book… “Thank you. Now go write a book! —Guy and Shawn”

Thank you both.

“xkcd: vol.0″ by Randall Munroe

1 Comment

“xkcd: volume 0” by Randall Munroe

What can I say? After all the years of reading, buying the book seemed like an obvious “huh, how did I not buy this already” moment.

This was a great wander down memory lane. I found a great many of my favorite xkcd comics, including bobby-drop-tables (therapeutic for anyone with an apostrophe in their surname!), locked-out-of-house, i’m-compiling-code and “the one that makes every Release Engineer I know cringe“.

Somehow, there were even some I’d never seen before, a very happy discovery: chess-coaster (which in turn inspired real life!), the why-i’m-barred-from-speaking-at-crypto-conferences series, girls-on-the-internet, ninjas-vs-stallman, counting sheep

All in all, a great fun read, and I found the “extra” sidebar cartoons equally fun… especially the yakshaver! If you like xkcd, and don’t already have this book, go get it.

ps: He’s got a new book coming out in a few days, a book tour in progress, and a really subtle turtles-all-the-way-down comic which nudges about the new book… if you look *really* closely! I’m looking forward to getting my hands on it!

“Lost Cat” by Caroline Paul

1 Comment

After all the fun reading “Meanwhile in San Francisco”, I looked to see if this duo had co-written any other books. Sure enough, they had.

“Lost Cat” tells the true story of how an urban cat owner (one of the authors) loses her cat, then has the cat casually walk back in the door weeks later healthy and well. The book details various experiments the authors did using GPS trackers, and tiny “CatCam” cameras to figure out where her cat actually went. Overlaying that data onto google maps surprised them both – they never knew their cats roamed so far and wide across the city. The detective work they did to track down and then meeting with “Cat StealerA” and “Cat Stealer B” made for a fun read… Just like “Meanwhile in San Francisco”, the illustrations are all paintings. Literally. My all-time favorite painting of any cat ever is on page7.

A fun read… and a great gift to any urban cat owners you know.

“Meanwhile in San Francisco” by Wendy MacNaughton

1 Comment

I stumbled across this book by accident recently, and really enjoyed it. One of the reasons I love to travel is because of the different cultural norms… what is “normal” in one location would be considered downright “odd/strange/unusual” in another location. Since I first moved to San Francisco, the different types of people, from different backgrounds, who each call this town “home” continue to fascinate me… and all in a small 7mile x 7mile area.

This book is painted (yes really!) by a San Francisco resident, and does an excellent job of describing the heart of many different aspects of this unique town: Mah Jong in Chinatown, the SF City Library’s fulltime employee who is a social worker for homeless people, Frank Chu, Critical Mass, dogwalkers, Mission Hipsters, Muni drivers … and of course, everything you need to know about a Mission burrito!

A fun read… and a great gift to anyone who has patiently listened while you’ve tried to explain what makes San Francisco so special.

AOSA(vol2) and Mozilla’s Release Engineering: now in Russian!

1 Comment

book coverMozilla’s Release Engineering was part of “The Architecture of Open Source Applications (vol2)” published in paperback in May2012 and then as electronic downloads from Amazon and Barnes&Noble in Sept2012. Earlier this week, Rail was delighted to discover that the book has now been translated into Russian. It is very cool to see this.

As best as I can tell, this translation work was led by А. Панин (A. Panin), and they did a great job. Even taking the time to recreate the images with embedded translated text. Tricky hard work, and very very great to see. Thanks to Mr Panin for making this happen.

You can download the entire book, or just the Mozilla RelEng portion. (As always, proceeds from book sales go to Amnesty International.)

This makes me wonder – are there any other translations, or translations-in-progress, out there?

“Journey to the Heart of Aikido” by Linda Holiday Sensei and Motomichi Anno Sensei


(This post is unusual, in that I am “reviewing” a book before reading the final print yet. Maybe “previewing” is more accurate?)

I’ve had the great fortune of repeatedly training on the mat with many world-class Aikido practitioners. Two of these, Linda Holiday Sensei (6th dan, runs Aikido of Santa Cruz dojo) and Motomichi Anno Sensei (8th dan, direct student of OSensei the founder of Aikido, recipient of Japan’s Distinguished Service Award, and ran the Kumano Juku Dojo in Shingu, Japan for ~40 years.) have just published a book they have been working on for literally *years*.

This is exciting.

Training with both of these authors has been pivotal for me, on and off the mat. Over the years, I’ve heard readings of various passages, and even been present for some interviews gathering source material. All random snippets, in various drafts, and out of sequence, which makes it hard to predict how the final form will pull together. What I’ve heard so far have been very meaningful to me, so I’m eager to get my hands on a signed 1st edition of this book on Saturday.

More info in the San Francisco Chronicle’s recent interview with Linda Holiday or the book’s official website. If you are interested, there’s a (free!) open-to-the-public book reading by Linda Holiday with live Aikido demonstrations in San Francisco this Saturday.

Oni gashi mas!

“Brag! The art of tooting your own horn without blowing it” by Peggy Klaus

1 Comment

Normally, a small book like this (193 pages) would be a quick read for me, but this book took me literally months. Not, I hasten to add, because of any problems with the book or the writing style, that was all fine. The problem was that this book uncovered a bunch of things I am personally working through. I found myself reading a few pages, highlighting some lines, then walking away thinking. Repeat a few times a week. Occasionally, I’d go back and re-read entire chapters.

For me, bragging has negative connotations and is something I avoid like the plague. Stereotypes of obnoxious, pretentious people, loudly telling all within range just how great they are. The very last thing I ever want to be. Whether that is cultural, learned from family, something I developed myself growing up, or a mixture, I don’t know. But it is part of who I am. This book is all about encouraging people to find a comfortable place in between these extremes. As Peggy is quick to note, this means different things to different people, so you need to pay attention to what is authentic for you, as that authenticity is important. People have generations of experience spotting fakes, and worst of all, deep down, you’ll know you are faking it too.

Because of the book title, it took several people pushing to get me to even start reading this book. Chapter#1 opened with a line that stopped me dead in my tracks.

“Myth#1: A job well done speaks for itself.”

I’ve always thought that if I did a good job, or handled a tricky situation well, people would notice. If I solved some complex problem, that people would understand the complexity, understand the importance of the achievement and appreciate the work. In those circumstances, having others recognize and complement the achievement was fine, but any attempt on my part to “brag” about my work would in some way “cheapen the victory”. After reading this book, I now think that is *sometimes* true but not always true. While the people working beside me in the same trenches, working side-by-side with me on the problem might understand the scale of the accomplishment, most people simply don’t know the details. Over time, people might eventually notice that a recurring problem hasn’t happened in a while, or they might simply forget about a previously-annoying problem because it hasn’t happened in a while… but they’d never stop and wonder why. Another common trend is for people to not notice one problem is fixed, but instead notice that a different problem has “appeared”. Oh, and meanwhile, people don’t know what you are working on. Over time, this becomes frustrating for everyone. After reading this book, I’ve learned that I need to make sure I inform people of the work I’m doing, and why it’s important to them. I don’t need to go into all the complexities of the project, unless they ask for more details, but it’s important to make sure others are aware of my work, and the impact it has on them and their work.

I found this a tough read, yet super worth the time. And, yes, I strongly recommend it.

“It ain’t bragging if you done it” (Dizzy Dean)

“Oh, the Places You’ll Go!” by Dr Seuss

1 Comment

After last week, I’ve proudly added this to my bookshelf.

The book is, of course, fun. But the biggest reason I’m was because I received it at LEAD. This 4-session LEAD series ended Friday, and has been more then just an experience; its “an inflection point”. The excellently designed “coursework” and setup/moderation by Athena, Kate, Debbie and Mihca provided the framework. But the intensity, passion and honesty that everyone engaged within LEAD left me stunned, and immensely proud to know each and every one of them. The book is a small talisman.

ps: (possible NSFW?) below is a really fun reading of the book done at Burning Man 2011.

Older Entries