Mihai Budiu’s Blog

Put side to side a laptop and a mainframe from 40 years ago and you will be amazed by the astonishing evolution.  Not only are hardware resources many orders of magnitude larger, but also the software is immensely more powerful and sophisticated.  Operating systems appeared in the ’60, for managing mainframe computers.  They handled tasks common to a variety of user programs, such as low-level device handling, scheduling the processor, allocating and naming storage, enforcing security.  An operating system main role is to transform the raw machine into a set of high-level abstraction, which can be used and shared by applications.  For example, instead of thinking which bytes to write or read from a disk, the applications access named files.

It may seem that we are close to have reached the last word in operating systems, and studying them will soon be part of history.  But an invisible revolution is ongoing, whose importance and magnitude is enormous. Despite its enormous scale, the revolution is mostly invisible.  Hidden from our eyes is slowly evolving is a set of enormous, planetary-scale operating systems.  We are still quite far from having reached this goal; to make a historic analogy, I estimate is that we are still in the pre-Unix era (1971) with respect to the planetary OS.  If you look carefully you can see how every year a few more pieces of the puzzle are being filled.

So, what is the computer that is to be managed by the planetary OS?  In its current incarnation it is the datacenter — a collection of tens of thousands of processors and disks — but soon it is going to be a collection of datacenters linked together into a seamless whole.  The goal of the planetary OS is to make millions of processors and disks to behave as a single machine, as easy and natural to use as a desktop.  Using a simple interface you will invoke the power of tens of thousands of processors, spread all over the world.  You will sift through petabytes of data with a simple mouse click.  You will manage millions of data streams from a single console.

You can read about some of the pieces of this enormous puzzle.  Here are some sample links: the Google File System, Amazon’s S3 and VMWare VMFS offer a single filesystem namespace, spanning reliably and transparently up to tens of thousands of machines.  Google’s Map-Reduce and Microsoft’s Dryad allow anyone to execute a program on thousands of machines, sifting through hundreds of terabytes of data in a matter of minutes.  Amazon’s EC2 and VMWare’s DRS offers transparent compute resource virtualization (extending the concept of process in operating systems to the cluster-level).  Microsoft’s Autopilot is the generalization to the cluster-level of the BIOS and software updates, monitoring and deploying automatically software.  Google’s BigTable transforms a cluster into a huge database.  User management is described in Google’s paper, but it is an ubiquitous piece in many other on-line services, from E-Bay to Yahoo IM.  Google’s Chubby provides cluster-level reliable interprocess communication.  Akamai’s network handles vast volumes of traffic for millions of clients.  And, of course, Web-based platforms are the new user-level APIs, which programmers can use to craft mash-ups (the extension of traditional applications).

And this list could continue.

Currently most of these pieces are still disjoint – not yet tied together into a coherent whole, and most of them do not scale up to planetary-size (they are confined to the cluster- or datacenter level).  But the trend is unmistakable: there is an ongoing race to build a unified, simple, single system, which will manage transparently, automatically, and effortlessly (to a large degree – not even a desktop operating system can work without supervision, so we can’t expect full autonomy at this scale) a gigantic computer spanning the whole globe.

It is certainly a very exciting time!

Updated: fixed a broken link.

I read a very interesting business book about how great companies are built.  (I am not the only one, the cover boasts "more than 1 million sold.")  The book is "Built to Last: Successful Habits of Visionary Companies."  I am providing the Amazon link, but you can also find a brief description at wikisummaries.  I liked the book because the authors — who are current or ex- academics from the Stanford school of Business — claim convincingly that their analysis is driven by data.  The methodology employed was as follows: the authors have first identified 18 "great" companies, using a survey.  These are companies that have thrived for a very long time, more than 100 years for most of them.  Then the authors set-up a control group, which contains similar companies, but slightly less successful.  Then the authors attempt to find a set of recipes that stand behind the great companies.  The data collection and analysis took 6 years.  Not surprising, taking into account that they had to dig into archives older than 150 years sometimes.

The results are quite surprising, and I highly encourage you to read the book for details.  The one-line summary (which you won’t find in the book in this form) is that all such great companies are really organized a lot like religious organizations.  (This great observation is due to Martin Abadi.)

But one thing that struck me while reading the book is that I recognized a lot of the traits of these organizations in the way Google seems to be structured.  My personal hypothesis is that the founders of Google have actually read this book carefully (it was published in 1996), and they set up deliberately to create a company which would follow the recipe in the book, and thus would be destined to become "great."  The way their stock and revenues has been working, they seem to have been vindicated, but it is way too soon to give a verdict.

Here I want to point out some of these striking similarities.

• A great company should have a core value, which it preserves no matter how many mutations it suffers.  Clearly, I am thinking about "Do no evil."
• The company should be driven by "big, hairy, audacious goals."  Goals like the moon landing, which motivate and drive for a long time, but are reachable.  (These are not the same as the core value.)  What else but "organize the world’s information?"
• The company ideology should be more than just profits.  In fact, short-term profit should be secondary to pursuing the long-term goals.  Doesn’t this fit with the lack of financial guidance offered by Google?
• Create a cult-like organization, which draws a clear line between the insiders and the outsiders.  There is a lot to quote here, from the free meals and other perks, which are designed to bond the workers together, making them as independent as possible from the "outside world", to their almost paranoid secrecy (in some respects) — for example their extremely strict non-disclosure agreement.
• Try a lot of stuff and keep what works.  I believe this one needs no explanation.  In fact, Marissa Mayer, google’s VP of search and UX gave a talk at Stanford on Google’s "9 notions of innovation".  One of the points is that the numbers rule: an idea is judged better if supported by measurable evidence.
• Good enough never is.  Constantly drive for improvement; you can never declare victory, business is a continuous process.  Even if the search engine and AdSense bring in a truckload of money, Google still introduces a whole set of products which take (or will take) advantage of its strength in advertising, such as gmail, maps, youtube, and the on-line office suite.

For some of the recipes in the book I could not find equivalents in Google’s structure, but it is still a very young organization.  For example "home-grown management:" the organization should prepare its own leaders, and should be able to survive unscathed the disappearance of its charismatic founders.  (Anecdotal reports seem to indicate that Google does not like at all the idea of management, preferring a very flat organization, where a lot of the decision power is held by the engineers.)

It is not clear whether these recipes are either necessary or sufficient for creating a great company.  But I will certainly be following with interest the trajectory of Google.

I consider myself fortunate to have met a few of the people that have profoundly influenced computing.  It is perhaps a triviality to claim that computing is the revolution of the society that we are actual witnessing, and I find it fascinating to understand more about some of the powerful personalities that are driving this revolution.

I will have more to say about my personal theory on who actually influences computing in another post.  I perceive a great deal of confusion around about who are the actual drivers of change: is it industry, academia, the open-source movement, underground hackers, industrial consortia, etc.  I believe that the answer should be sought on a case-by-case basis.  Otherwise said, each of these has important contributions to claim.

But let me come back to my main thread: I will try to share in this space insights that I manage to obtain from some of these personalities.

I am openly encouraging discussion, and I am allowing everybody to comment for now.  However, I will mercilessly remove comments which I find off-topic, or that I find inflammatory or derogatory.  My main goal is to provide an interesting discussion ground for everybody, including the text and comments.

I have always enjoyed writing about computers, so why not have a “blog” of my own? So here it is.

I wrote quite a few texts in Romanian, which were published in the PC Report, (later renamed Net Report) computer magazine. I have preserved on-line copies of all of them.  That stuff was for some (small) profit, but this time I am going to do it for free.

I have also made two interviews that turned out quite well:

• The first was done in 2000, with Brian Kernighan, then at Lucent Bell Labs.
• The second was done in 2001, with Bruce Maggs, a VP at Akamai and professor at CMU.