The “Distributed” book-in-progress: Early Release#1 now available!

No Comments

My previous post described how O’Reilly does rapid releases, instead of waterfall-model releases, for book publishing. Since then, I’ve been working with the folks at O’Reilly to get the first milestone of my book ready.

As this is the first public deliverable of my first book, I had to learn a bunch of mechanics, asking questions and working through many, many details. Very time consuming, and all new-to-me, hence my recent silence. The level of detailed coordination is quite something – especially when you consider how many *other* books O’Reilly has in progress at the same time.

Book Cover for DistributedOne evening, while in the car to a social event with friends, I looked up the “not-yet-live” page to show to friends in the car – only to discover it was live. Eeeeek! People could now buy the 1st milestone drop of my book. Exciting, and scary, all at the same time. Hopefully, people like it, but what if they don’t? What if I missed an important typo in all the various proof-reading sessions? I barely slept at all that night.

In O’Reilly language, this drop is called “Early Release #1 (ER#1)”. Now that ER#1 is out, and I have learned a bunch about the release mechanics involved, the next milestone drop should be more routine. Which is good, because we’re doing these every month. Oh, and like software: anyone who buys ER#1 will be prompted to update when ER#2 is available later in Oct, and prompted again when ER#3 is available in Nov, and so on.

You can buy the book-in-progress by clicking here, or clicking on the thumbnail of the book cover. And please, do let me know what you think – Is there anything I should add/edit/change? Anything you found worked for you, as a “remotie” or person in a distributed team, which you wish you knew when you were starting? If you were going to setup a distributed team today, what would you like to know before you started?

To make sure that any feedback doesn’t get lost or caught in spam filters, I’ve setup a special email address (feedback at oduinn dot com) although I’ve already been surprised by feedback via twitter and linkedin. Thanks again to everyone for their encouragement, proof-reading help and feedback so far.

Now, it’s time to brew more coffee and get back to typing.


A Release Engineer’s view on rapid releases of books

No Comments

As a release engineer, I’ve designed and built infrastructure for companies that used the waterfall-model and for companies that used a rapid release model. I’m now writing a book, so recently I have been looking at a very different industry through this same RelEng lens.

In days of old, here’s a simplistic description of what typically happened. Book authors would write their great masterpiece on their computer, in MSWord/Scrivener/emacs/typewriter/etc. Months (or years!) later, when the entire book was written, the author would deliver the “complete” draft masterpiece to the book company. Editors at the book company would then start reading from page1 and wade through the entire draft, making changes, fixing typos, looking for plot holes, etc. Sometimes they could make these corrections themselves, sometimes the changes required sending the manuscript back to the author to rewrite portions. Later, external reviewers would also provide feedback, causing even more changes. If all this took a long time, sometimes the author had to update the book with new developments to make sure the book would be up-to-date when it finally shipped. Tracking all these changes was detailed, time-pressured and chaotic work. Eventually, a book would finally go to press. You could reduce risk as well as simplify some printing and shipping logistics by having bookshops commit to pre-buy bulk quantities – but this required hiring a staff of sales reps selling the unwritten books in advance to bookstores before the books were fully written. Even so, you could still over/under estimate the future demand. And lets not forget that once people start reading the book, and deciding if they like it, you’ll get word-of-mouth, book reviews and bestseller-lists changing the demand unpredictably.

If people don’t like the book, you might have lots of copies of an unsellable book on your hands, which represents wasted paper and sunk costs. The book company is out-of-pocket for all the salaries and expenses from the outset, as well as the unwanted printed books and would hope to recover those expenses later from other more successful books. Similar to the Venture Capital business model, the profits from the successful books help recoup the losses from the other unprofitable books.

If people do like the book, and you have a runaway success, you can recoup the losses of the other books so long as you avoid being out-of-stock, which could cause people to lose interest in the book because of back order delays. Also, any errors missed in copy-editing and proofreading could potentially lead to legal exposure and/or forced destruction of all copies of printed books, causing further back order delays and unexpected costs. Two great examples are The Sinner’s Bible and the Penguin cook book.

This feels like the classic software development “waterfall” model.

By contrast, one advantage with the rapid release model is that you get quick feedback on the portions you have shipped so far, allowing you to revisit and quickly adjust/fix as needed… even while you are still working on completing and shipping the remaining portions. By the time you ship the last portion of the project, you’ve already adjusted and fixed a bunch of surprise gotchas found in the earlier chunks. By the time you ship your last portion, your overall project is much healthier and more usable. After all, for the already shipped portions, any problems discovered in real-world-use have already been fixed up. By comparison, a project where you write the same code for 18-24 months and then publish it all at the same time will have you dealing with all those adjustments/fixes for all the different chunks *at the same time*. Delivering smaller chunks helps keep you honest about whether you are still on schedule, or let you quickly see whether you are starting to slip the schedule.

The catch is that this rapid release requires sophisticated automation to make the act of shipping each chunk as quick, speedy and reliable as possible. It also requires dividing a project into chunks that can be shipped separately – which is not as easy to do as you might hope. Doing this well requires planning out the project carefully in small shippable chunks, so you can ship when each chunk is written, tested and ready for public consumption. APIs and contractual interfaces need to be understood and formalized. Dependencies between each chunk needs to figured out. Work estimates for writing and testing each module need to be guessed. A calendar schedule is put together. The list goes on and on… Aside: You probably should do this same level of planning also for waterfall model releases, but its easy to miss hidden dependencies or impact of schedule slips until the release date when everything was supposed to come together on the day.

So far, nothing new here to any release engineer reading this.

One of the (many) reasons I went with O’Reilly as the publisher for this book was their decision to invest in their infrastructure and process. O’Reilly borrowed a page from Release Engineers and invested in automation to switch their business from a waterfall model to a rapid release model. This change helps keep schedules realistic, as schedule slips can be spotted early. It helps get early feedback which helps ship a better final product, so the ratio of successful book vs unprofitable books should improve. It helps judge demand, which helps the final printing production planning to reduce cost wasting with less successful books, and to improve profits and timeliness for successful books. This capability is a real competitive advantage in a very competitive business market. This is an industry game changer.

When you know you want “rapid release for books”, you discover a lot of tools were already there, with a different name and slightly-different-use-cases. Recent technologies advances help make the rest possible. The overall project (“table of contents”). The breaking up of the overall project into chunks (“book chapters”). Delivering usable portions as they are ready (print on demand, electronic books + readers, online updates). Users (“readers”) who get the electronic versions will get update notices when newer versions become available. At O’Reilly, the process now looks like this:

Book authors and editors agree on a table of contents and an approximate ship date (write a project plan with scope) before signing contracts.

Book authors “write their text” (commit their changes) into a shared hosted repository, as they are writing throughout the writing creative process, not just at the end. This means that O’Reilly editors can see the state of the project as changes happen, not just when the author has finished the entire draft manuscript. It also means that O’Reilly reduces risk of catastrophic failure if an author’s laptop crashes, or is stolen without a backup.

A hosted automation system reads from the repository, validates the contents and then converts that source material into every electronic version of the book that will be shipped, including what is sent to the print-on-demand systems for generating the ink-on-paper versions. This is similar to how one source code revision is used to generate binaries for different deliverables – OSX, Windows, linux, iOS, Android,…

O’Reilly has all the automation and infrastructure you would expect from a professional-grade Release Engineering team. Access controls, hosted repos, hosted automation, status dashboards, tools to help you debug error messages, teams to help answer any support questions as well as maintain and improve the tools. Even a button that says “Build!”!!
The only difference is that the product shipped from the automation is a binary that you view in an e-reader (or send to a print-on-demand printing press), instead of a binary that you invoke to run as an application on your phone/desktop/server. With this mindset, and all this automation, it is no surprise that O’Reilly also does rapid releases of books, for the same reasons software companies do rapid releases. Very cool to see.

I’ve been thinking about this a lot recently because I’m now putting together my first “early release” of my book-in-progress. More info on the early release in the coming days when it is available. As a release engineer and first time author, I’ve found the entire process both totally natural and self-evident and at the same time a little odd and scary. I’d be very curious to hear what people think… both of the content of the actual book-in-progress, and also of this “rapid release of books” approach.

Meanwhile, it’s time to refill my coffee mug and get back to typing.


“we are all remote” at Cultivate NYC

No Comments

It’s official!!

I’ll be speaking about remoties at the O’Reilly Cultivate conference in NYC!

Cultivate logoCultivate is being held on 28-29 Sept 2015, in the Javits conference center, in New York City. This is intentionally the same week, and same location, as the O’Reilly Strata+Hadoop World conference, so if you lead others in your organization, and are coming to Strata anyways, you should come a couple of days early to focus on cultivate-ing (!) your leadership skills. For more background on O’Reilly’s series of Cultivate conferences, check out this great post by Mike Loukides. I attended the Cultivate Portland conference last month, when it was co-located with OSCON, and found it insightful edge-of-my-seat stuff. I expect Cultivate NYC to be just as exciting.

Meanwhile, of course, I’m still writing like crazy on my book (and writing code when no-one is looking!), so have to run. As always, if you work remotely, or are part of a distributed team, I’d love to hear what does/doesn’t work for you and any wishes you have for topics to include in the book – just let me know.

Hope to see you in NYC next month.


The “we are all remoties” book!?!


I’ve been working in distributed teams, as well as talking, presenting, coaching and blogging about “remoties”‚ in one form or another for 8?9? years now. So, I’m excited to announce that I recently signed a contract with O’Reilly to write a book about how to successfully work in, and manage in, a geo-distributed world. Yes, I’m writing a “we are all remoties” book. If you’ve been in one of my ever-evolving “we are all remoties” sessions, you have an idea of what will be included.

If you’ve ever talked with me about the pros (and cons!) of working as a remote employee or of working in a distributed team, you already know how passionate I am about this topic. I care deeply about people being able to work well together, and having meaningful careers, while being physically or somehow otherwise remote from each other. Done incorrectly, this situation can be frustrating and risky to your career, as well as risky to employers. Done correctly, however, this could be a global change for good, raising the financial, technical and economic standards across all sorts of far flung places around the globe. Heady game-changing stuff indeed.

There are many “advocacy books” out there, explaining why working remote is a good / reasonable thing to do – typically written from the perspective of the solo person who is already remote. There are also many different tools becoming available to help people working in distributed teams – exciting to see. However, I found very few books, or blogposts, talking about the practical mechanics of *how* to use a combination of these tools and some human habits to allow humans to work together effectively in distributed teams, especially at any scale or over a sustained amount of time. Hence, my presentations, and now, this upcoming book.


  • if you are physically geo-distributed from the people you work with, I’d like to hear what does or doesn’t work for you. If you know someone who is in this situation, please share this post with them.
  • If you have experience working in distributed teams, is there something that you wish was already explained in a book? Something that you had to learn the hard way, but which you wish was clearly signposted to make it easier for others following to start working in distributed teams? Do you have any ideas that did / didn’t work for you?
  • If you have published something on the internet about remoties, please be tolerant of any questions I might ask. If you saw any of my “we are all remoties” presentations, is there anything that you would like to see covered in more/less detail? Anything that you wish was written up in a book to help make the “remote” path easier for those following behind?

…you can reach me on twitter (“@joduinn”) or on email (john at my-domain-name – and be sure to include “remoties” in the subject, to get past spam filters.)

Now, time to brew some coffee and get back to typing.

(updated 31jul2015 to add twitter + email address.)

“We are ALL Remoties” (Jun2015 edition)


Since my last post on “remoties”, I’ve done several more presentations, some more consulting work for private companies, and even started writing this down more explicitly (exciting news coming here soon!). While I am always refining these slides, this latest version is the first major “refactor” of this presentation in a long time. I think this restructuring makes the slides even easier to follow – there’s a lot of material to cover here, so this is always high on my mind.

Without further ado – you can get the latest version of these slides, in handout PDF format, by clicking on the thumbnail image.

Certainly, the great responses and enthusiastic discussions every time I go through this encourages me to keep working on this. As always, if you have any questions, suggestions or good/bad stories about working remotely or as part of a geo-distributed teams, please let me know (either by email or in the comments below) – I’d love to hear them.


The canary in the coal mine

1 Comment

After my recent “We are ALL Remoties” presentation at Wikimedia, I had some really great followup conversations with Arthur Richards at WikiMedia Foundation. Arthur has been paying a lot of attention to scrum and agile methodologies – both in the wider industry and also specifically in the context of his work at Wikimedia Foundation, which has people in different locations. As you can imagine, we had some great fun conversations – about remoties, about creating culture change, and about all-things-scrum – especially the rituals and mechanics of doing daily standups with a distributed team.

Next time you see a group of people standing together looking at a wall, and moving postit notes around, ask yourself: “how do remote people stay involved and contribute?” Taking photographs of the wall of postit notes, or putting the remote person on a computer-with-camera-on-wheeled-cart feels like a duct-tape workaround; a MacGyver fix done quickly, with the best of intentions, genuinely wanting to help the remote person be involved, but still not-a-great experience for remoties.

There has to be a better way.

We both strongly agree that having people in different locations is just a way to uncover the internal communication problems you didn’t know you already have… the remote person is the canary in the coal mine. Having a “we are all remoties” mindset helps everyone become more organized in their communications, which helps remote people *and* also the people sitting near each other in the office.

Arthur talked about this idea in his recent (and lively and very well attended!) presentation at the Annual Scrum Alliance “Global Scrum Gathering” event in Phoenix, Arizona. His slides are now visible here and here.

If you work in an agile / scrum style environment, especially with a geo-distributed team of humans, it’s well worth your time to read Arthur’s presentation! Thought provoking stuff, and nice slides too!

Interviews about “Work from home” policies at Facebook, Virgin and yes, Yahoo!

No Comments

When I talk about “remoties”, I frequently get asked my thoughts on Yahoo’s now (in)famous “no more work-from-home” policy.

Richard Branson (Virgin, link to first video) and the separate comments from Jackie Reses (Yahoo, 2.27 into the link to second video) confirm what I’d heard from multiple unofficial mutterings – that Yahoo’s now (in)famous “no more work from home” decree was actually intended as a way to jolt the company culture into action.

I also liked Sheryl Sandberg (Facebook) comments about how a successful remote workplace depends on having clear measures of successful results. Rather then valuing someone by how many hours they are seen working in the office, instead it is better to have a company culture where you measure people by results. This echoes comments I’ve seen from Jason Fried in his “Remote” book, comments I’ve made in my “we are all remoties” presentations and which I’ve heard again and again from various long-term remote workers.

These two interviews discuss these points really well. The entire article is well worth a read, and both videos are only a few minutes long, so worth the quick watch.


“We are ALL Remoties” (Apr2015 edition)

1 Comment

Last week, I had the great privilege of talking with people at Wikimedia Foundation about “we are all remoties”!

This was also the first presentation by a non-Wikimedia person in their brand new space, and was further complicated with local *and* remote attendees! Chip, Greg and Rachel did a great job of making sure everything went smoothly, quickly setting up a complex multi-display remote-and-local video configuation, debugging some initial audio issues, moderating questions from remote attendees, etc. We even had extra time to cover topics like “Disaster Recovery”, “interviewing tips for remoties” and “business remotie trends”. Overall, it was a long, very engaged, session but felt helpful, informative, great fun and seemed to be well received by everyone.

As usual, you can get the latest version of these slides, in handout PDF format, by clicking on the thumbnail image. I’ve changed the PDF format slightly as requested, so let me know if you think this format is better/worse.

As always, if you have any questions, suggestions or good/bad stories about working in a remote or geo-distributed teams, please let me know – I’d love to hear them.

ps: Oh, and by the way, Wikimedia are hiring – see here for current job openings. They are smart, nice people, literally changing the world – and yes, remoties ARE welcome. :-)

“why work doesnt happen at work” by Jason Fried on TEDx


While reading “Remote”, I accidentally found this TEDx talk by one of the authors, Jason Fried. Somehow I’d missed this when it first came out in 2010, so stopped to watch it. I’ve now watched this a few times in a row, found it just as relevant today as it was 4-5 years ago, so am writing this blogpost.

The main highlights for me were:

1) work, like sleep, needs solid uninterrupted time. However, most offices are designed to enable interrupts. Open plan layouts. Phones. Casual walk-by interrupts from managers asking for status. Unneeded meetings. They are not designed for uninterrupted focus time. No-one would intentionally plan to have frequently-interrupted-sleep every night and consider it “good”, so why set up our work environments like this?

2) Many people go into the office for the day, attempting to get a few hours uninterrupted work done, only to spend time reacting to interrupts all day, and then lament at the end of the day that “they didn’t get anything done”! Been there, lived through that. As a manager, he extols people to try things like “no-talking-Thursdays”, just to see if people can actually be more productive.

3) The “where do you go when you really want to get work done” part of his presentation nailed it for me. He’s been asking people this question for years, and the answers tend to fall into three categories:

  • place: “the kitchen”, “the spare room”, “the coffee shop”, …
  • moving object: plane, train, car… the commute
  • time: “somewhere really early or really late at night or on the weekend”

… and he noted that no-one said “the office during office hours”!! The common theme is that people use locations where they can focus, knowing they will not get interrupted. When I need to focus, I know this is true for me also.

All of which leads to his premise that organizing how people work together, with most communication done in a less interruptive way is really important for productivity. Anyone who has been at one of my remoties sessions knows I strongly believe this is true – especially for remoties! He also asked why businesses spend so much money on these counter-productive offices.

Aside: I found his “Facebook and twitter are the modern day smoke breaks” comment quite funny! Maybe thats just my sense of humor. Overall, its a short 15min talk, so instead of your next “facebook/twitter/smokebreak”, grab a coffee and watch this. You’ll be glad you did.

“We are ALL Remoties” (Feb2015 edition)


Since my last blog post on “remoties”, I’ve worked with ProCore, and Haas, UCBerkeley (again!) as well as smaller private discussions with some other companies. Every single time, I continue to find people eager for passionate conversations, and clear “ah-ha!” moments, which I find very encouraging. There’s also plenty of volunteering stories/ideas of what did/didnt work for them in their past. All this helps me continue to hone and refine these slides, which I hope makes them even more helpful to others.

You can get the latest version of these slides, in handout PDF format, by clicking on the thumbnail image.

Remoties are clearly something that people care deeply about. Geo-distributed teams are becoming more common in the workplace, and yet the challenges continue to be very real and potentially disruptive. Given how this topic impacts people’s jobs, and their lives, I’m not surprised by the passionate responses, and each time, the lively discussions encourage me to keep working on this even more.

As always, if you have any questions, suggestions or good/bad stories about working in a remote or geo-distributed teams, please let me know – I’d love to hear them.


Older Entries