Podcast: Quad Core
Faster, Hotter, Scalier
Jill
David
Jason
Joyent Weblog
Quo vadis, Computing. A Joyent Manifesto.
The operating system is dead. Long live the operating system. If the future is network applications, the past is the operating system. It is being pushed inexorably down the stack. Down to the level of the router and the switch. With it will go the general purpose server. If developers don’t need operating systems, they won’t need general purpose servers, the come-along of the operating system. This doesn’t mean that CPUs, RAM, storage, the raw materials an operating system manages, go away. The operating system will still be around to manage them. But users and developers won’t interoperate with them the way they have in the past. Users will interact with data (e.g. png, matrix, rtf, text, xml, mp3, etc), developers will interact with their chosen development stack (e.g. Rails, PHP, Python, Java, etc). This is why we haven’t been, generally, proponents of technologies such as VMWare and Xen. They emphasize the operating system. At Joyent, we emphasize data and the development stack.
A Web operating system will not emerge. I’m sure this point will be argued vigorously. But I’m right. Because the operating system is pushed down the stack, doesn’t mean what replaces it is an operating system. Duh. And if users will interact with data and developers with their chosen development stack, then Web APIs are the transitional neanderthals of evolutionary progress. Nice, you can walk on two legs. But you need a brain. Braininess is open data standards and protocols, not free APIs that trap data and developers in the holding pens of big companies. Sorry, did that in the 1990s.
Greg Papadopolous, CTO of Sun Microsystems, has coined the terms “red shift companies” and “blue shift companies”. “Red” describes a company for whom technology is a competitive advantage. “Blue” describes companies for whom technology is a cost. Let’s leave aside the “blue” companies. Greg hypothesizes that “red” companies will either be customers of Sun Microsystems or develop into competitors. I think he’s being generous, to Sun. There’s a move afoot, my Joyeur friends, that spells doom for the arms merchants of technology. What’s that? First, most vendors of server technology are really just repacked (often ingenious and beautiful, but still repackaged) design sourced from China and Taiwan. Or Intel. Or AMD. Example: this is really this. The conclusion I draw, if my previous hypotheses are correct, is that companies like HP, Sun, IBM become increasingly irrelevant since “red” customers will increasingly buy directly from the manufacturers in China and Taiwan themselves. Foxconn servers. What happened to Sony in televisions will happen to the incumbents in servers. Who ever heard of Vizio? But they are the number one seller of LCD high-definition televisions in the United States. Why? Because consumers don’t want to deal with a “value adding” middle man. “Red shift” will inevitably lead to blood in the water. That’s why the “arms dealers” will hastily rush to deliver products into the market that deliver their hardware as-a-service to customers. These efforts will only cannibalize existing server sales. And operating system sales and service. Expect lots of Hamlet.
What happens to the client? Doesn’t the operating system thrive there? No. Not long term. There are any number of technologies that are working to transition client applications (in the new world: client data tools) away from an operating system dependence towards portable stack dependence. This doesn’t mean a client edge device won’t have an operating system. It just won’t matter. Changes have been afoot to usher in this new era. The web is one. Web applications have made it acceptable for user-experience to vary widely between applications (=data tools). The influence of web applications has made its way into the user experiences of operating systems accelerating the acceptance, by users, of user experiences that can be best developed using web application development stacks (or portable runtimes such as Flash, Slingshot, etc.). This will push the client operating system to a position of irrelevance in time. Yeah, I’ll need an OS to run my device, but I won’t care which OS because developers are moving up the stack on the edge device just as they’ve moved up the stack on the server. (BTW: does this mean Linux wins as it simply becomes a competent collection of device drivers?)
Commenting is closed for this article.
One of the most intelligent posts I’ve read on this topic so far. I think you really nail the fate of both the hardware sellers and the OS providers. It will take a good while, and many organizations (like Microsoft and Adobe) will survive in a very different form, but the irrelevance of their current business models seems almost inevitable at this point. Just look at what’s happening in the mobile device space — the diversity of OS platforms is not really very relevant to the functionality available, and it’s of almost no interest to most users. On the hardware side, IBM is much further along the transition to a services model than Sun, but Sun can still get some legs if it works quickly. Amazon’s model proves that, I believe.
— Curtis Carmack 386 days ago #David
Great post. This is exactly why I’m starting to look at SiliconMechanics.com for servers.
You just can’t beat their prices unless you build your own SuperMicro server yourself. I wish SuperMicro would just provide this service … just b/c it would make the server that much cheaper.
— Tim 386 days ago #So, if the OS and hardware aren’t important, why is Joyent standardising on (Open)Solaris and Sun hardware? Why not just stay with FreeBSD on whitebox server hardware if there’s no real difference?
— David Magda 385 days ago #While this is an pretty standard rant, your version lacks both substance and credibility. Since you are selling servers (and stacks) as a service, you are pretty obviously biased. In the credibility department, just two posts ago you were gushing over your exploitation of a new OS feature (Dtrace) to augment the visibility of rails. I won’t reference the many other entries on Open Solaris.
The reality of the situation is that we are seeing a similar trend in computing that is occurring in other industries (and even many societies): bifurcation. In computing, The low end is getting more DIY and the highend is going more high touch. Look at what is happening to Windows. They are getting hurt on the lowend by Linux and on the highend by Apple. Similarly, society is starting to show the biggest disparities in wealth since before the 1930’s depression. The lesson here: don’t get caught in the middle. Assuming that companies can move out of the middle (through innovation or a different business model), they should be alright. The company in real jeopardy is Dell – racing to the bottom is going to be difficult when you are assembling other people’s components.
Just as aside, did Sun pee in your cereal ? This is two ranting posts in a row against them, after months of sucking up. Did they stop funding you or refuse to buy you ?
— jim 385 days ago #The importance of the operating system depends on your perspective. If you are a developer or user, I’d have to say this is mostly if not completely correct, especially over the long haul.
If you are an operator or an architect, and more concerned with things like scalability, security, flexibility, maintainability and so forth, well, the OS is very important indeed.
Joyent’s own value proposition would look entirely different without OpenSolaris and technologies like dTrace, ZFS and Zones. The current and future value proposition many Red Shift companies like Joyent depend an OS that can effectively and efficiently use hundreds of CPU’s in a single OS instance. What other OS can use 1 or 2U machines with 64, 128 or more threads?
Does anyone still think that the one or two thread pizza box will be cost effective in more than a year from now? How many hundreds of threads do you think a 1U machine will have in two years? Now, which OS can really run well on such a machine? There’s only one I know of: Solaris and OpenSolaris.
The fact that Joyent’s users don’t know this doesn’t reduce the strategic value of the OS to Joyent, its customers or Sun’s customers.
— Louis F. Springer 385 days ago #It makes sense.
@David Magda: Well, since the OS is now just another layer in the stack, it is going to have to be standardized, and integrated into the stack. From my understanding, OpenSolaris at Joyent, is almost inseparable from the rest of the software stack.
@jim: Well, I don’t see a conflict between this postings, and others on OpenSolaris. Sun had to open-source OpenSolaris to make sure it was still relevant. Now, that it is out there, it can be tightly bound into the stack.
OpenSolaris + Dtrace hacks + Rails + pkgsrc stuff + other funky things —> JoyentOS.
— Tom S 385 days ago #@Louis: that’s the point. The OS is important to us. It’s not (becoming) important to developers and users.
— David Young 385 days ago #@jim: (anonymous jim) Tom S is right. Joyent loves OpenSolaris (our post about DTrace proves it). But our customers don’t care (except for DTrace, which is a competitive advantage for our customers). That’s going to have an impact on our core decisions as a company. And will impact the client. Finally, to be very clear, we’ve never been funded by Sun. As to acquisition: all things are possible. But Sun has Network.com, and I’d imagine, in time, they will use that to offer their hardware as-a-service. We’ll see how that goes. As I said: lots of Hamlet.
Re: your other points. Biggest disparities since the depression? Depressing, I suppose. But I’m not an economist. I’m just a technology prognosticator.
Jim, are you a Sun employee? It’s OK. Let’s talk you down. My post isn’t about Sun. It’s about the industry.
— David Young 385 days ago #@David regarding “The OS is important to us. It’s not (becoming) important to developers and users.” I completely agree, and this is a good thing.
This sort kind of bittersweet from Sun’s perspective. Most people who use the Internet don’t know what Sun is, or what its about, and I doubt it will ever get any better.
What was that company that doesn’t make things but makes things better? They must have spent millions on that commercial and I still don’t know who they are.
— Louis F. Springer 385 days ago #I love the bloody ocean reference for a couple of reasons. Most importantly, it reminded me that my copy of ‘Blue Ocean Strategy’ is downstairs at the leasing office.
BOS is a marketing book, but don’t let that dissuade you. It provides methodologies which companies, technology firms included, can use to swim to calmer waters.
— jonto 385 days ago #David,
I’m having a lovely chat on Twitter with @brendanb about this. He wrote a response here: http://www.smackfoo.com/2007/10/evolutionary
I understand your post to mean roughly this:
1. developers and end users will care less and less what particular OS they ultimately run on because it just won’t matter
2. developers and end users will instead care about a more highly abstracted level of services
3. providers such as Joyent care very deeply about the underlying OS, because not every OS is sufficiently advanced to allow you to enable the above
I think where Brendan and I are circling around is whether that abstracted higher level is or is not an OS in its own right. Is a set of high-level services and protocol properly called an OS or isn’t it?
— andrew barnett 385 days ago #@andrew yes that’s correct, and #2 is especially born out in the uptake of services like s3.
Why the uptake? Because it abstracts that kind of stuff away from people who simply don’t care about it, and who feel like such things get in their way.
To some degrees it takes away the operating system, filesystems, directory design and a number of typical POSIX/UNIX concerns one has to deal with when storing lots of files. What OS does S3 run and how exactly does it do the distributed buckets?
Who cares (with a period).
Who then cares? Only a competitive service.
— Jason A. Hoffman 385 days ago #@Jim, I don’t think we rant, it’s not in our nature.
Neither this post nor my last one about NetApp and Sun were rants against Sun, I simply wrote about our perspective as both a NetApp and Sun customer while those two are going at each other’s throats. David here is simply writing that for developers who don’t work at the OS level, the OS, in fact, does not matter for them (otherwise no one would ever develop on Windows right? ;-) )
And as David said, Sun doesn’t fund us, we fund Sun.
— Jason A. Hoffman 385 days ago #@andrew: your summary is correct. To make an analogy. When you plug into an electrical socket, you don’t care whether the electricity was made using coal, nuclear, wind, or water. Your care about the voltage. You care about the type of plug you need. The same is happening in the compute world. Everything behind the plug is for the “experts”.
— David Young 385 days ago #I agree that portable thin client runtimes are the future on the desktop and the OS becomes less important; most people who know something about technology say something similar, though they usually do so in the context of talking up web apps and the current web stack. I like that David didn’t really do that and just mentioned several possibilities for that runtime like flash or slingshot. However, while this client runtime future will undoubtedly come to pass, most people play down the uncomfortable fact that the runtimes that now exist are horrible and none of them are likely to be successful in the future. Whether it’s java or flash or silverlight or ajax, each has deep flaws (inefficiency, lack of openness or portability) that will not allow them to be the winning client platform. And who is working on the platform that will survive and flourish? As far as I can see, nobody. So, we can all see the future but it will not come to pass till someone actually does the work to build it.
— Ajay 384 days ago #Great title, though I only knew what “Quo Vadis” meant because I just finished the book a month ago. This is a very insightful post! I cant wait to get an accelerator or 3 from you guys in the next couple months!
— Rich Breton 384 days ago #@andrew and @david
Does this boil down to the browser as the ‘new’ thin client (or something that bootstraps web apps or similar into a ‘client side’ app, so they feel like apps rather than web pages, maybe like mozilla’s prism).
Computing where the internet is your application and data server, and the client device is irrelevant in the sense that it wont matter what client device you use?
— Ryan 384 days ago #@Ryan: I don’t know if I’m convinced the browser is the client. That’s certainly true, today. But I think pushing data out to the edge of the network is a growing requirement, and browsers aren’t there, yet.
Users will focus on data (types). Developers will focus on development stacks. The operating system won’t matter (except for service providers and edge device manufacturers).
— David Young 384 days ago #Think of browser as a type of service though… it has to be something OS independent, where the app used to access the data is not on the OS, or the OS remains just as important as ever, because to a user the OS is the user experience.
— Ryan 383 days ago #Re commoditization: You are underestimating the power of patents. Patents were invented to combat commoditization, and they are on every global politicians agenda to combat the Chinese (now, and whoever will next commoditize). When companies are nearing extinction, they will launch their patent ICBMs. The Sun vs. Netapp case is just a bit atypical because both companies are US-based. As with ICBMs, you can’t win, and everybody else gets the fallout.
— Carsten 378 days ago #I agree with the article 100%, but I think “the OS is dead” has a lot of connotations that don’t apply. Instead, it would be clearer to say “the OS is a commodity”.
Writing to an OS-specific API (like Win32) is now “low level” stuff that should be handled by libraries. That means choosing an OS is no longer a commitment: Swiching your deployment OS is as easy as switching your CPU from Itanium to Opteron.
The proof is that Developers these days routinely write on Windows/Mac and deploy on Linux/Sun without a second thought.
All the hair-splitting about “what is an OS” misses the point. Developers are no longer writing to an OS. The platform that they write to (RoR/Java/etc) is not an OS (nor will it somehow ‘become’ an OS). Think of the transition from writing to a hardware platform (PDP 11) to writing to an OS. Now we have a similar shift from writing to an OS to writing to a higher-level platform.
— Dan The Man 375 days ago #