15 Sep 2015
Books, Mozilla, Release Engineering, Remoties
As a release engineer, I’ve designed and built infrastructure for companies that used the waterfall-model and for companies that used a rapid release model. I’m now writing a book, so recently I have been looking at a very different industry through this same RelEng lens.
In days of old, here’s a simplistic description of what typically happened. Book authors would write their great masterpiece on their computer, in MSWord/Scrivener/emacs/typewriter/etc. Months (or years!) later, when the entire book was written, the author would deliver the “complete” draft masterpiece to the book company. Editors at the book company would then start reading from page1 and wade through the entire draft, making changes, fixing typos, looking for plot holes, etc. Sometimes they could make these corrections themselves, sometimes the changes required sending the manuscript back to the author to rewrite portions. Later, external reviewers would also provide feedback, causing even more changes. If all this took a long time, sometimes the author had to update the book with new developments to make sure the book would be up-to-date when it finally shipped. Tracking all these changes was detailed, time-pressured and chaotic work. Eventually, a book would finally go to press. You could reduce risk as well as simplify some printing and shipping logistics by having bookshops commit to pre-buy bulk quantities – but this required hiring a staff of sales reps selling the unwritten books in advance to bookstores before the books were fully written. Even so, you could still over/under estimate the future demand. And lets not forget that once people start reading the book, and deciding if they like it, you’ll get word-of-mouth, book reviews and bestseller-lists changing the demand unpredictably.
If people don’t like the book, you might have lots of copies of an unsellable book on your hands, which represents wasted paper and sunk costs. The book company is out-of-pocket for all the salaries and expenses from the outset, as well as the unwanted printed books and would hope to recover those expenses later from other more successful books. Similar to the Venture Capital business model, the profits from the successful books help recoup the losses from the other unprofitable books.
If people do like the book, and you have a runaway success, you can recoup the losses of the other books so long as you avoid being out-of-stock, which could cause people to lose interest in the book because of back order delays. Also, any errors missed in copy-editing and proofreading could potentially lead to legal exposure and/or forced destruction of all copies of printed books, causing further back order delays and unexpected costs. Two great examples are The Sinner’s Bible and the Penguin cook book.
This feels like the classic software development “waterfall” model.
By contrast, one advantage with the rapid release model is that you get quick feedback on the portions you have shipped so far, allowing you to revisit and quickly adjust/fix as needed… even while you are still working on completing and shipping the remaining portions. By the time you ship the last portion of the project, you’ve already adjusted and fixed a bunch of surprise gotchas found in the earlier chunks. By the time you ship your last portion, your overall project is much healthier and more usable. After all, for the already shipped portions, any problems discovered in real-world-use have already been fixed up. By comparison, a project where you write the same code for 18-24 months and then publish it all at the same time will have you dealing with all those adjustments/fixes for all the different chunks *at the same time*. Delivering smaller chunks helps keep you honest about whether you are still on schedule, or let you quickly see whether you are starting to slip the schedule.
The catch is that this rapid release requires sophisticated automation to make the act of shipping each chunk as quick, speedy and reliable as possible. It also requires dividing a project into chunks that can be shipped separately – which is not as easy to do as you might hope. Doing this well requires planning out the project carefully in small shippable chunks, so you can ship when each chunk is written, tested and ready for public consumption. APIs and contractual interfaces need to be understood and formalized. Dependencies between each chunk needs to figured out. Work estimates for writing and testing each module need to be guessed. A calendar schedule is put together. The list goes on and on… Aside: You probably should do this same level of planning also for waterfall model releases, but its easy to miss hidden dependencies or impact of schedule slips until the release date when everything was supposed to come together on the day.
So far, nothing new here to any release engineer reading this.
One of the (many) reasons I went with O’Reilly as the publisher for this book was their decision to invest in their infrastructure and process. O’Reilly borrowed a page from Release Engineers and invested in automation to switch their business from a waterfall model to a rapid release model. This change helps keep schedules realistic, as schedule slips can be spotted early. It helps get early feedback which helps ship a better final product, so the ratio of successful book vs unprofitable books should improve. It helps judge demand, which helps the final printing production planning to reduce cost wasting with less successful books, and to improve profits and timeliness for successful books. This capability is a real competitive advantage in a very competitive business market. This is an industry game changer.
When you know you want “rapid release for books”, you discover a lot of tools were already there, with a different name and slightly-different-use-cases. Recent technologies advances help make the rest possible. The overall project (“table of contents”). The breaking up of the overall project into chunks (“book chapters”). Delivering usable portions as they are ready (print on demand, electronic books + readers, online updates). Users (“readers”) who get the electronic versions will get update notices when newer versions become available. At O’Reilly, the process now looks like this:
Book authors and editors agree on a table of contents and an approximate ship date (write a project plan with scope) before signing contracts.
Book authors “write their text” (commit their changes) into a shared hosted repository, as they are writing throughout the writing creative process, not just at the end. This means that O’Reilly editors can see the state of the project as changes happen, not just when the author has finished the entire draft manuscript. It also means that O’Reilly reduces risk of catastrophic failure if an author’s laptop crashes, or is stolen without a backup.
A hosted automation system reads from the repository, validates the contents and then converts that source material into every electronic version of the book that will be shipped, including what is sent to the print-on-demand systems for generating the ink-on-paper versions. This is similar to how one source code revision is used to generate binaries for different deliverables – OSX, Windows, linux, iOS, Android,…
O’Reilly has all the automation and infrastructure you would expect from a professional-grade Release Engineering team. Access controls, hosted repos, hosted automation, status dashboards, tools to help you debug error messages, teams to help answer any support questions as well as maintain and improve the tools. Even a button that says “Build!”!!
The only difference is that the product shipped from the automation is a binary that you view in an e-reader (or send to a print-on-demand printing press), instead of a binary that you invoke to run as an application on your phone/desktop/server. With this mindset, and all this automation, it is no surprise that O’Reilly also does rapid releases of books, for the same reasons software companies do rapid releases. Very cool to see.
I’ve been thinking about this a lot recently because I’m now putting together my first “early release” of my book-in-progress. More info on the early release in the coming days when it is available. As a release engineer and first time author, I’ve found the entire process both totally natural and self-evident and at the same time a little odd and scary. I’d be very curious to hear what people think… both of the content of the actual book-in-progress, and also of this “rapid release of books” approach.
Meanwhile, it’s time to refill my coffee mug and get back to typing.
06 Aug 2015
Most engineers I know are good at writing software. Some engineers I know are good at writing software for extraordinarily complex projects. Some have a brain that naturally, instinctively, just worked this way while some took all sorts of courses on algorithms and structured programming in university. Regardless, when you are coding by yourself, these important coding and problem solving skills are probably good enough. However, when you transition from coding-by-yourself to coding-in-a-team, there is the need for an additional skill.
Now you need the skills to explain to other engineers why your code, your approach, is best. Maybe even why your approach is better then an alternate approach being proposed by someone else. It is a given that you can write good code – that is how you got on the team in the first place. The question is: can you explain why your code is better then any other alternatives available?
This skill might sound deceptively simple, but in reality it has a few super complex twists to it. Someone once explained it to me as follows: smart people, given the same start point, the same goal, and the same restrictions/limitations, would come up with similar-enough solutions. Maybe not identical, because all humans are slightly different, but at least similar in approach – and would solve the same problem. Given that, if I was to find myself in a situation where very different solutions are being discussed, maybe we have different understanding of what the starting point is? Or what the goal is? Or what restrictions/assumptions/limitations to work within? Double checking all those can quickly uncover missing (or invalid) problem statements, restrictions/assumptions/limitations or goals. If it turns out that we’re trying to solve different problems, then of course the proposed solutions would be very different!
If your skills in logic/reasoning/conflict are weak, and if you skip the verification step, then it is easy to get frustrated by your inability to “convince others” of the greatness of your code or design. Maybe you cannot explain it correctly. Maybe the others don’t want to listen to you – because they feel equally about their proposals. Maybe both. If you are passionate about your code, but cannot have others understand its greatness, the situation can easily turn to frustration (“I give up – why does no-one understand how great my code is?”) or even anger (“This code is too important to give up on – what do they know anyway – they’re just idiots”). This shift from working together to find the best code/design (something which can be objectively measured or stress-tested, after everyone agrees on what they all think is “best”.) to destructively debating the other humans (using hard-to-measure qualities) is dangerous. Even if the “right” code is actually chosen in the end, the manner in which the discussion was carried out can be toxic to the group.
Like many other engineers, I never had any formal training in logic/reasoning/conflict at any of my universities, so I had to learn these skills on-the-job, or at workshops I found over the years. Sometimes it went well. Sometimes it did not. I’m still working on this. Being able to talk through differences of opinions, in a constructive way, is a crucial skill set to develop as you work with others during your engineering career. And essential for *everyone* in a team to have, in order for the *team* to work together effectively.
I stumbled across this book in 2014, loved it, and was then delighted and surprised to discover that it was written by someone I knew from when I worked at Mozilla!! Ali’s book is focused on describing the different types of arguments used in logic debates, with very easy-to-understand examples. This will help you keep your logic and reasoning honest when you are a discussion. It will also make it easy to notice when someone else switches to using “Ad Hominem” or “No True Scotsman” or “Genetic Fallacy” or “Straw Man” or any other of these types of arguments on you.
I think many many people, including myself, find the idea of a book about arguments to be daunting, almost off-putting, so I really like how the simple one-page description and the full page cartoon images make the whole topic area more approachable, appealing and in turn, very easy to understand in a non-threatening way. At 56 pages, it is a super short book, and almost 50% of those pages are cartoons, so you could expect to be done in a few minutes. However, each page found me sitting rethinking different conflicts over the years, so this took a lot longer to read then I expected. And I re-read it often. In addition to the printed version of the book, there is also a fun https://bookofbadarguments.com/.
Try it, I think you will like it.
ps: This is one of two great books that I really like in this area – both of which I think should be required reading for engineers. Both of which I just discovered I had not yet blogged about, hence this post. Watch for another post coming soon.
30 Jul 2015
Books, Mozilla, Remoties
I’ve been working in distributed teams, as well as talking, presenting, coaching and blogging about “remoties”‚ in one form or another for 8?9? years now. So, I’m excited to announce that I recently signed a contract with O’Reilly to write a book about how to successfully work in, and manage in, a geo-distributed world. Yes, I’m writing a “we are all remoties” book. If you’ve been in one of my ever-evolving “we are all remoties” sessions, you have an idea of what will be included.
If you’ve ever talked with me about the pros (and cons!) of working as a remote employee or of working in a distributed team, you already know how passionate I am about this topic. I care deeply about people being able to work well together, and having meaningful careers, while being physically or somehow otherwise remote from each other. Done incorrectly, this situation can be frustrating and risky to your career, as well as risky to employers. Done correctly, however, this could be a global change for good, raising the financial, technical and economic standards across all sorts of far flung places around the globe. Heady game-changing stuff indeed.
There are many “advocacy books” out there, explaining why working remote is a good / reasonable thing to do – typically written from the perspective of the solo person who is already remote. There are also many different tools becoming available to help people working in distributed teams – exciting to see. However, I found very few books, or blogposts, talking about the practical mechanics of *how* to use a combination of these tools and some human habits to allow humans to work together effectively in distributed teams, especially at any scale or over a sustained amount of time. Hence, my presentations, and now, this upcoming book.
- if you are physically geo-distributed from the people you work with, I’d like to hear what does or doesn’t work for you. If you know someone who is in this situation, please share this post with them.
- If you have experience working in distributed teams, is there something that you wish was already explained in a book? Something that you had to learn the hard way, but which you wish was clearly signposted to make it easier for others following to start working in distributed teams? Do you have any ideas that did / didn’t work for you?
- If you have published something on the internet about remoties, please be tolerant of any questions I might ask. If you saw any of my “we are all remoties” presentations, is there anything that you would like to see covered in more/less detail? Anything that you wish was written up in a book to help make the “remote” path easier for those following behind?
…you can reach me on twitter (“@joduinn”) or on email (john at my-domain-name – and be sure to include “remoties” in the subject, to get past spam filters.)
Now, time to brew some coffee and get back to typing.
(updated 31jul2015 to add twitter + email address.)
06 Jul 2015
Books, Mozilla, Startup
This book just came out and I loved it. If you are starting a company, or thinking of it, you need to read this book. Period.
Dan covered a whole range of topics very succinctly, and in easy-to-follow language. When and how to raise funds. What all those terms mean. Who should (and should not!) be on your board, and why. How to allocate shares and ownership between co-founders. Where to incorporate your company (Dan has strong opinions on this!). How to create (and then also maintain) company culture. A great section on decision making. A section on “Hiring” in the context of the Manhattan project vs the moon shot Apollo project that I think every engineering hiring manager should read before building a team. Several true stories about startups where co-founders mismatches caused company threatening problems (trivia: 6 of 10 startups lose a co-founder in early days). And some good (and bad!) stories of how important trust was.
Some great quotes that resonated with me:
“You have limited resources of time and money. When they run out, you go bankrupt. The important thing is not cost/benefit: it’s opportunity cost.”
(in the context of how much travel was needed for all the in-person meetings with investors when raising funding) “…Alaska Airlines gave me MVP status for my efforts. In January.”
“Entrepreneurship is the pursuit of opportunity without regard to the resources currently controlled”. Prof Stevenson, Harvard.
In a variation of the “fail fast” mantra in developer circles, Dan notes that “…while it might seem like cold comfort now, the sooner you fail, the sooner you can try again.” Oh, and he’s not just saying it – that was the ending of a chapter where he detailed the failure of one of his startups.
His tolerance for large volumes of coffee and pointer to suggested reading “Coffee, CYP1A2 Genotype, and Risk of Myocardial Infarction” was a great and unexpected tangent for me personally. (More info here Journal of American Medical Association)
“Startups don’t out think their competitors; they out-execute them.”
“If leadership is the forest, then management is the trees. Day to day, it’s what consumes your time, and its imperative that you get it right.”
It takes skill and seasoned-experience-in-the-field to have one person cover all these different topics. Even more skill to do so clearly, and concisely. Putting them all together in a way that makes sense was great. Just great. If you are starting a company, or thinking of it, you need to read this book. Period.
Aside: Having this on my kindle app, on my trusty nexus5 phone was quite a good reading experience. The book was written in short, digestible chapters, which I could quickly complete standing a store line, or in the back of a taxi between meetings. It also encouraged me to think more about the chapter I just finished in the time before I got to stop and read some more. A nice way to digest the many lessons in here. I’m still experimenting with what books I find do work best on phone+kindle vs ink-on-paper, but at least for this book, reading on kindle worked for me.
(Disclaimer: I bought this book because I’m starting my own company, and that is the basis of the above review. As this book is published by O’Reilly Press, it feels important to disclose that I am also currently doing some work with O’Reilly… which did not influence anything I wrote here.)
27 Jun 2015
Books, Mozilla, Release Engineering
Long time readers of this blog will remember when The Architecture of Open Source Applications (vol2) was published, containing a chapter describing the tools and mindsets used when re-building Mozilla’s Release Engineering infrastructure. (More details about the book, about the kindle and nook versions, and about the Russian version(!).
Dr Dobbs recently posted an article here which is an edited version of the Mozilla Release Engineering chapter. As a long time fan of Dr Dobbs, seeing this was quite an honor, even with the sad news here.
Obviously, Mozilla’s release automation continues to evolve, as new product requirements arise, or new tools help further streamline things. There is still lots of interesting work being done here – for me, top of mind is Task Cluster, and ScriptHarness (v0.1.0 and v0.2.0). Release Engineering at scale is both complex, and yet very interesting – so you should keep watching these sites for more details, and consider if they would also help in your current environment. As they are all open source, you can of course join in and help!
For today, I just re-read the Dr. Dobbs article with a fresh cup of coffee, and remembered the various different struggles we went through as we scaled Mozilla’s infrastructure up so we could quickly grow the company, and the community. And then in the middle of it all, found time with armenzg, catlee and lsblakk to write about it all. While some of the technical tools have changed since the chapter was written, and some will doubtless change again in the future, the needs of the business, the company and the community still resonate.
For anyone doing Release Engineering at scale, the article is well worth a quiet read.
14 Mar 2015
Books, HortonWorks, Mozilla
(At 1,044 pages, this book looks daunting. I’ve enjoyed other Neal Stephenson books, especially Cryptonomicon, so I didn’t let the size of the book deter me when I was buying it. But I find reading a long book with complex intertwined plots needs continuity – no point in picking it up and trying to resume after leaving it unopened for weeks! Even though I bought this book over a year ago, I only finally had time to read it in the last couple of weeks. Aside, in this day-and-age-of-laptops-and-kindles, I was amused by the odd sidelook I got whenever I settled into a nearby coffee shop and produced this weighty hardback ink-on-paper tomb!)
Wikipedia has a great summary here, but obviously be warned that it has lots of plot spoilers. Without giving too much plot away, I liked the book. From my perspective, I really enjoyed how Neal can interweave different stories. While there were many different interwoven stories here, the ones that are top of mind for me were:
- the hacker-and-former-girlfriend-get-kidnapped story
- the spy-tracking-jihadists story
- the massive on-line game business story
All very different stories, yet the detailed coverage of each make me think Neal has a great understanding of hackers, encryption, different-business-market-economies-of-massive-on-line-games, Soviet-veterans-of-the-confict-in-Afganistan, internet-cafes-in-developing-worlds… the list goes on and on. I even found the way computer issues were covered to be accurately describes (typically a pet peeve for me!). In the midst of all the other drama, I was greatly amused by the image of a super-important invulnerable character (Egdod) walking in unattended mode back to home base, while various other T’Rain players were attacking him / defending him / rubber-necking the impossible sight of Egdod moving through their world. And somehow, someday, I need to find a way to use the throwaway joke about “Your org chart?”, “No, orc chart”.
The book was a great read, and I’d recommend it.
30 Nov 2014
Books, HortonWorks, Mozilla
A few years ago, I first put my toes into the book publishing world by co-writing a portion of AOSAv2 about Mozilla’s RelEng infrastructure. Having never written part of a published book before, I had no idea what I was really getting into. It was a lot of work, in the midst of an already-busy-day-time-job and yet, I found it was strangely quite rewarding. Not financially rewarding – all proceeds from the book went to Amnesty – but rewarding in terms of getting us all to organize our thoughts to write down in a clear, easy to read way, explaining the million-and-one details that “we just knew instinctively”, and hopefully helping spread the word to other software companies on what we did when changing Mozilla’s release cadence.
While working on AOSAv2, clearly explaining the technology was hard work, as expected, but I was surprised by how much work went into “simple” mechanics – merging back reviewer feedback, tracking revisions, dealing with formatting of tables and diagrams, publishing in different formats… and remember, this was a situation where book publisher contracts, revenue and other “messy stuff” was already taken care of by others. I “just” had to write. I was super happy to have the great guidance and support of Greg Wilson and Amy Brown who had been-there-done-that, helped work through all those details, and kept us all on track.
Ever since then, I’ve been considering more writing, but daunted by all the various details above and beyond “just writing”. These blog posts help scratch that itch, in between my own real-life-work-deadlines, but the idea of writing a full book, by myself, still lingered. A while ago, I grabbed “APE: Author, Publisher, Entrepreneur-How to Publish a Book” by Guy Kawasaki. It looked like a good HowTo manual, I’ve enjoyed some of his other books, and he’s always a great presenter, so I was looking forward to some quiet time to read this book cover-to-cover.
This was well worth the time, and I re-read it a few times!
For me, some of the highlights were:
- “why are you writing a book?” I really like how Guy turned this question around to “why would someone else want to read your book”. Excellent mind-flip. I’ve met a few people who want to write a book, and even a few published authors, and I’ve talked with them about my own ideas about writing. But no-one, not one, ever reversed the question like this. It was instantly self-evident to me – it takes time to read a book, and we’re all busy. So, even if someone gave me a book for free, why would I want to skip work and/or social plans to read a book by someone I don’t know. Making it clear, immediately, why someone would find it worthwhile reading your book is a crucial step that I think many people skip past. As the author, keeping this in mind at all times while writing, will help keep you focused on the straight-and-narrow path to writing a book that people would actually want to read.
- Money: Most publishers are super-secret about their contracts/terms/conditions, which can make a new time author feel like they’re going to be taken (The only exception I know of is Apress, who publish all their terms on their website, with a “no haggling” clause). To help educate potential authors, I respect how much full detail Guy & Shawn gave in small, easy to follow, words.
- “Tell the world you’re writing a book – not that you’re thinking of writing a book.” Again, an excellent mind-flip to help keep you motivated and writing, every single day, whether you want to or not. Also, they provided many links to writer’s clubs (writer support groups!?) who would help you keep motivated.
- print-on-demand vs print-big-batch: This reminded me of how software release cycles are changing the software industry from old monolith release cadence to rapid-release cadence. “Old way”: a big-bang-release every unpredictable 18months, with a costly big print run, and lots of ways to handle financial risk of under/over selling; any corrections are postponed until the next big-bang-release if it looks like there is enough interest. “New way”: build infrastructure to enable print-on-demand. Do smaller, more frequent, releases, each with small print runs, (almost) no risk of under/over selling, corrections handled frequently and easily. Yes, at first glance, each printed book might seem more expensive this way, but when you factor in the lack-of-under/over selling, removed financial risk, and benefits of frequent updates to the almost-free electronic readers, it actually feels cheaper, more efficient and more appealing to me.
- In addition to printed books, there’s a good description of pros/cons of the different popular electronic formats (PDF, MOBI, EPUB, DAISY, APK…) as well as related DRM.
- The differences between ebook publishers (Amazon, Apple, Barnes & Noble, Google, Kobo, …), Author Publisher Services (Lulu, Blurb, Author Solutions, …) and Print-on-Demand (Walkerville Publishing, Lightning Source, …) was detailed and very helpful. Complex chapter, with lots of data, and ending with the reassuring “Don’t obsess about making the wrong choice, however, because most distribution decisions are changeable.”!
- translations, audiobooks: normally, these are handled as edge cases. Guy & Shawn walk through some of the options (Audible/Amazon, Books-on-Tape/RandomHouse), as well as financial & legal realities.
- Some fun examples of rejection responses by agents/publishers. My personal favorite was a rejection sent to George Orwell about Animal Farm “It is impossible to sell animal stories in the USA”.
All in all, I found the writing style personal, helpful, direct and super honest. Even the way they ended the book… “Thank you. Now go write a book! —Guy and Shawn”
Thank you both.
05 Sep 2014
Books, HortonWorks, Mozilla
“xkcd: volume 0” by Randall Munroe
What can I say? After all the years of reading xkcd.com, buying the book seemed like an obvious “huh, how did I not buy this already” moment.
This was a great wander down memory lane. I found a great many of my favorite xkcd comics, including bobby-drop-tables (therapeutic for anyone with an apostrophe in their surname!), locked-out-of-house, i’m-compiling-code and “the one that makes every Release Engineer I know cringe“.
Somehow, there were even some I’d never seen before, a very happy discovery: chess-coaster (which in turn inspired real life http://xkcd.com/chesscoaster!), the why-i’m-barred-from-speaking-at-crypto-conferences series, girls-on-the-internet, ninjas-vs-stallman, counting sheep…
All in all, a great fun read, and I found the “extra” sidebar cartoons equally fun… especially the yakshaver! If you like xkcd, and don’t already have this book, go get it.
ps: He’s got a new book coming out in a few days, a book tour in progress, and a really subtle turtles-all-the-way-down comic which nudges about the new book… if you look *really* closely! I’m looking forward to getting my hands on it!
01 Sep 2014
Books, HortonWorks, Mozilla
After all the fun reading “Meanwhile in San Francisco”, I looked to see if this duo had co-written any other books. Sure enough, they had.
“Lost Cat” tells the true story of how an urban cat owner (one of the authors) loses her cat, then has the cat casually walk back in the door weeks later healthy and well. The book details various experiments the authors did using GPS trackers, and tiny “CatCam” cameras to figure out where her cat actually went. Overlaying that data onto google maps surprised them both – they never knew their cats roamed so far and wide across the city. The detective work they did to track down and then meeting with “Cat StealerA” and “Cat Stealer B” made for a fun read… Just like “Meanwhile in San Francisco”, the illustrations are all paintings. Literally. My all-time favorite painting of any cat ever is on page7.
A fun read… and a great gift to any urban cat owners you know.
12 May 2014
Books, HortonWorks, Mozilla
I stumbled across this book by accident recently, and really enjoyed it. One of the reasons I love to travel is because of the different cultural norms… what is “normal” in one location would be considered downright “odd/strange/unusual” in another location. Since I first moved to San Francisco, the different types of people, from different backgrounds, who each call this town “home” continue to fascinate me… and all in a small 7mile x 7mile area.
This book is painted (yes really!) by a San Francisco resident, and does an excellent job of describing the heart of many different aspects of this unique town: Mah Jong in Chinatown, the SF City Library’s fulltime employee who is a social worker for homeless people, Frank Chu, Critical Mass, dogwalkers, Mission Hipsters, Muni drivers … and of course, everything you need to know about a Mission burrito!
A fun read… and a great gift to anyone who has patiently listened while you’ve tried to explain what makes San Francisco so special.