<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	xmlns:georss="http://www.georss.org/georss" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:media="http://search.yahoo.com/mrss/"
	>

<channel>
	<title>Joyeur &#187; General</title>
	<atom:link href="http://joyeur.com/category/general/feed/" rel="self" type="application/rss+xml" />
	<link>http://joyeur.com</link>
	<description>Joyent&#039;s Weblog</description>
	<lastBuildDate>Sat, 04 Feb 2012 00:21:49 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.com/</generator>
<cloud domain='joyeur.com' port='80' path='/?rsscloud=notify' registerProcedure='' protocol='http-post' />
<image>
		<url>http://0.gravatar.com/blavatar/c37750016912bc7b25a17e3263f0fe23?s=96&#038;d=http%3A%2F%2Fs2.wp.com%2Fi%2Fbuttonw-com.png</url>
		<title>Joyeur &#187; General</title>
		<link>http://joyeur.com</link>
	</image>
	<atom:link rel="search" type="application/opensearchdescription+xml" href="http://joyeur.com/osd.xml" title="Joyeur" />
	<atom:link rel='hub' href='http://joyeur.com/?pushpress=hub'/>
		<item>
		<title>Cull.tv Makes Noise with Node.js</title>
		<link>http://joyeur.com/2011/09/23/cull-tv-makes-noise-with-node-js/</link>
		<comments>http://joyeur.com/2011/09/23/cull-tv-makes-noise-with-node-js/#comments</comments>
		<pubDate>Fri, 23 Sep 2011 22:12:41 +0000</pubDate>
		<dc:creator>Jackson West</dc:creator>
				<category><![CDATA[General]]></category>
		<category><![CDATA[node.js]]></category>
		<category><![CDATA[Video]]></category>

		<guid isPermaLink="false">http://joyeur.com/?p=3098</guid>
		<description><![CDATA[Cull.tv is &#8220;rebooting music videos for the 21st century&#8221; according to Co-Founder Katherine de León &#8212; though for those of us old enough to remember MTV back when all the network did was play music videos, the site is a refreshing throwback to the days when you could &#8220;turn it on, leave it on.&#8221; While [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=3098&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p><img src="http://joyeur.files.wordpress.com/2011/09/cull-tv.jpg?w=500&#038;h=284" alt="" title="cull-tv" width="500" height="284" class="aligncenter size-full wp-image-3099" /></p>
<p><a href="http://cull.tv/">Cull.tv</a> is &#8220;rebooting music videos for the 21st century&#8221; according to Co-Founder Katherine de León &#8212; though for those of us old enough to remember MTV back when all the network did was play music videos, the site is a refreshing throwback to the days when you could &#8220;<a href="http://www.youtube.com/watch?v=Hh94FG5Q_j8">turn it on, leave it on</a>.&#8221;  While <a href="http://gigaom.com/video/culltv-launch/">NewTeeVee points out that the startup isn&#8217;t alone in trying to re-engineer the music video experience</a>, it is unique in that the site is built with Node.js.</p>
<p><a href="http://blog.cull.tv/post/10557539851/nodejs-crawling-culling-machinelearning">John Hurliman, Cull&#8217;s CTO, wrote a blog post describing the basics of putting the site together</a>, and shares how Node.js shined even in a surprising place:</p>
<p><span id="more-3098"></span></p>
<blockquote><p>[O]ne area where we are treading new ground with node.js+JavaScript is with our real-time recommender. The common wisdom is to use node in computing-light, I/O-heavy workloads. Machine learning and recommendations are often thought of as very compute-heavy, CPU-bound tasks. In reality, when your working set grows large enough and you go beyond simple collaborative filtering to making content-based decisions (requiring access to more and more data) and applying several layers of rulesets as most production ready recommenders do, you are I/O-bound again and worrying more about building a massive data pipeline than shaving off the last 10% of clock cycles.</p></blockquote>
<p>If you&#8217;re looking for an example of a well-designed, well-implemented web application built entirely using Node.js, check it out. Of course, it&#8217;s also great if you&#8217;re just looking to check out some cool new music!</p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/joyeur.wordpress.com/3098/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/joyeur.wordpress.com/3098/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/joyeur.wordpress.com/3098/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/joyeur.wordpress.com/3098/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/joyeur.wordpress.com/3098/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/joyeur.wordpress.com/3098/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/joyeur.wordpress.com/3098/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/joyeur.wordpress.com/3098/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/joyeur.wordpress.com/3098/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/joyeur.wordpress.com/3098/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/joyeur.wordpress.com/3098/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/joyeur.wordpress.com/3098/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/joyeur.wordpress.com/3098/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/joyeur.wordpress.com/3098/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=3098&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://joyeur.com/2011/09/23/cull-tv-makes-noise-with-node-js/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	
		<media:content url="http://1.gravatar.com/avatar/7b38f1442adc67f3ffa9f20034cefc47?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">jacksonwest</media:title>
		</media:content>

		<media:content url="http://joyeur.files.wordpress.com/2011/09/cull-tv.jpg" medium="image">
			<media:title type="html">cull-tv</media:title>
		</media:content>
	</item>
		<item>
		<title>Lessons Learned at the GamesBeat and MobileBeat Conferences</title>
		<link>http://joyeur.com/2011/07/15/lessons-learned-at-the-gamesbeat-and-mobilebeat-conferences/</link>
		<comments>http://joyeur.com/2011/07/15/lessons-learned-at-the-gamesbeat-and-mobilebeat-conferences/#comments</comments>
		<pubDate>Fri, 15 Jul 2011 17:39:51 +0000</pubDate>
		<dc:creator>Nima</dc:creator>
				<category><![CDATA[General]]></category>
		<category><![CDATA[Mobile]]></category>
		<category><![CDATA[Partnerships]]></category>

		<guid isPermaLink="false">http://joyeur.com/?p=2742</guid>
		<description><![CDATA[This week, the Joyent Cloud team, including General Manager Steve Tuck and I attended the MobileBeat and GamesBeat conferences here in San Francisco. Held at the Palace Hotel, it was an amazing opportunity to meet the growing community of developers, entrepreneurs and business folks in the mobile and gaming space. As we dashed madly between [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=2742&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p><img src="http://joyeur.files.wordpress.com/2011/07/gamesbeat_mobilebeat_2011.png?w=500" alt="" title="gamesbeat_mobilebeat_2011"   class="alignleft size-full wp-image-2749" />This week, the Joyent Cloud team, including General Manager Steve Tuck and I attended the <a href="http://events.venturebeat.com/mobilebeat2011/">MobileBeat</a> and <a href="http://events.venturebeat.com/gamesbeat2011/">GamesBeat</a> conferences here in San Francisco. Held at the Palace Hotel, it was an amazing opportunity to meet the growing community of developers, entrepreneurs and business folks in the mobile and gaming space. As we dashed madly between the various conferences and breakout sessions, it became plain to us that the lines between mobile and gaming had never been so blurry. Conferences on the mobile track covered games, and vice versa, leaving me to realize that you can&#8217;t think about creating and promoting games without mobile in mind, and if you&#8217;re making mobile apps, you&#8217;d better spend time to learn the engagement, virility, monetization and user adoption tricks from the gaming folks.</p>
<p>Without going into too many details, I just wanted to share the highlights I took away from the conference:</p>
<p><span id="more-2742"></span></p>
<p><strong>&#8220;There is a panic to go mobile&#8221;</strong></p>
<p style="padding-left:30px;">Everybody had mobile on the brain, whether building games or social apps or enterprise-focused tools, the ubiquitous request was &#8220;can I use it on my iPhone?&#8221; Developers in the audience were split 50/50 between designing for the mobile web via HTML5 (easier for cross platform development) or natively (better performance, better UX, better user experience). It&#8217;s still too early to tell if one approach will win over the other, but one thing is for sure:  Whatever you develop, it&#8217;s going to have to have a mobile version too.</p>
<p><strong>&#8220;Don&#8217;t believe everything you hear about iOS and Android&#8221;</strong></p>
<p style="padding-left:30px;">Numbers on conversion, engagement and monetization on the two platforms were often confusing and contradictory. The velocity of growth in both platforms sometimes probably leads to lagging and stale info, which leads to FUD. If you develop mobile apps, you&#8217;re going to have to develop on both. So figure out how you can leverage development platforms that are suited to your needs and support one or both lead platforms.</p>
<p><strong>&#8220;If you have 4 minutes to demo your game/app, don&#8217;t waste 3 of it on how great your team is.&#8221;</strong></p>
<p style="padding-left:30px;">I sat in on the gaming and mobile demos, and if I took only one thing away, it was the look of boredom on the judge&#8217;s faces when presenters wasted time talking about their &#8220;bench strength&#8221; or a long winded story about why they&#8217;re doing what they&#8217;re doing. The few demos that were done well followed a simple model: 15 seconds to tell us who you are and what you do, 45 seconds on what the product is, 45 seconds why its important and how its different, 15 seconds on who uses the app and how/why, and 2 minutes showing a demo or screen shots of your application. The best &#8220;new idea&#8221; presentation I&#8217;ve ever seen wasn&#8217;t powerpoint &#8212; it was Ryan Dahl&#8217;s 5 minute live demo showing how to write a working chat app in less than 150 lines of code.</p>
<p><strong>&#8220;Hiring super talent is expensive, so leverage who and what you can, especially platforms&#8221;</strong></p>
<p style="padding-left:30px;">Everyone was talking about Peter Thiel&#8217;s &#8220;how do you hire employee #20&#8243; and grumbling about the high cost of talent in the Bay Area. That&#8217;s okay, because the conference was also buzzing with companies developing mobile and gaming platforms that could significantly reduce the cost, skill set required and time needed to develop, launch and market new ideas. One day IBM&#8217;s Watson may learn to code &#8211; until then, I&#8217;m going to put my money on platforms that simplify, automate and accelerate mobile/games development.</p>
<p>After two days of intense sessions, meetings and &#8220;socializing&#8221;, Steve and I left the conference with two big takeaways.</p>
<p style="padding-left:30px;"><strong>One:</strong> Doing mobile is hard and developers really need some tools to simplify and automate a lot of the back-end components such as messaging, API integration, email, single-signon, social network integration, synchronization, GEO-location and data storage.</p>
<p style="padding-left:30px;"><strong>Two:</strong> The major cloud vendors haven&#8217;t done a good job helping mobile developers find a good home in the cloud. In 4 years, there will be more mobile screens interfacing with cloud based apps than desktop/laptop devices. There is a HUGE opportunity here.</p>
<p>Over the next few weeks, we are going to take some significant steps to address the shortcomings we see in the marketplace relating to mobile development and hosting. Until then, keep your RSS reader firmly pointed to <a title="Joyeur" href="http://www.joyeur.com">www.joyeur.com.</a></p>
<p><em>Photo by VentureBeat&#8217;s <a href="http://www.perivision.net/">Chris Peri</a>.</em></p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/joyeur.wordpress.com/2742/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/joyeur.wordpress.com/2742/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/joyeur.wordpress.com/2742/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/joyeur.wordpress.com/2742/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/joyeur.wordpress.com/2742/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/joyeur.wordpress.com/2742/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/joyeur.wordpress.com/2742/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/joyeur.wordpress.com/2742/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/joyeur.wordpress.com/2742/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/joyeur.wordpress.com/2742/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/joyeur.wordpress.com/2742/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/joyeur.wordpress.com/2742/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/joyeur.wordpress.com/2742/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/joyeur.wordpress.com/2742/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=2742&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://joyeur.com/2011/07/15/lessons-learned-at-the-gamesbeat-and-mobilebeat-conferences/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	
		<media:content url="http://1.gravatar.com/avatar/9b6989f4202ca2c16526d4b428e804e4?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">badnima</media:title>
		</media:content>

		<media:content url="http://joyeur.files.wordpress.com/2011/07/gamesbeat_mobilebeat_2011.png" medium="image">
			<media:title type="html">gamesbeat_mobilebeat_2011</media:title>
		</media:content>
	</item>
		<item>
		<title>Friday Math Fun #24: &#8220;Checkers and Dominoes&#8221;</title>
		<link>http://joyeur.com/2011/06/24/friday-math-fun-24-checkers-and-dominoes/</link>
		<comments>http://joyeur.com/2011/06/24/friday-math-fun-24-checkers-and-dominoes/#comments</comments>
		<pubDate>Fri, 24 Jun 2011 23:00:44 +0000</pubDate>
		<dc:creator>Jackson West</dc:creator>
				<category><![CDATA[General]]></category>
		<category><![CDATA[math]]></category>

		<guid isPermaLink="false">http://joyeur.com/?p=2599</guid>
		<description><![CDATA[Every Friday, our resident mathematician, Tim Eller, puts out a math challenge to the company. Kudos to @kevinykchan and @erikcorry for finding the solution to last week&#8217;s quiz! Take a checkerboard, and from two opposite corners remove one square (e.g. upper left and lower right). Take 31 dominoes, each of which could cover exactly two [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=2599&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p><img src="http://joyeur.files.wordpress.com/2011/06/checkers_and_dominoes.png?w=500&#038;h=281" alt="" title="checkers_and_dominoes" width="500" height="281" class="aligncenter size-full wp-image-2600" /></p>
<p><em>Every Friday, our resident mathematician, Tim Eller, puts out a math challenge to the company. <a href="http://twitter.com/#!/mmayo/status/81952890381549568">Kudos to @kevinykchan and @erikcorry</a> for finding the solution to <a href="http://joyeur.com/2011/06/17/friday-math-fun-23-crossing-the-bridge/">last week&#8217;s quiz</a>!</em></p>
<p>Take a checkerboard, and from two opposite corners remove one square (e.g. upper left and lower right). Take 31 dominoes, each of which could cover exactly two squares on the checkerboard. Can you cover the (cut) checkerboard with these 31 dominoes?</p>
<p><em>Have the answer? Leave it in the comments or post a tweet with the hashtag #FridayMathFun.</em></p>
<p><em>Photos by <a href="http://www.flickr.com/photos/davichi/2524779164/">David Oliva</a> and <a href="http://www.flickr.com/photos/stevensnodgrass/3990957354/">Steve Snodgrass</a>.</em></p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/joyeur.wordpress.com/2599/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/joyeur.wordpress.com/2599/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/joyeur.wordpress.com/2599/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/joyeur.wordpress.com/2599/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/joyeur.wordpress.com/2599/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/joyeur.wordpress.com/2599/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/joyeur.wordpress.com/2599/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/joyeur.wordpress.com/2599/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/joyeur.wordpress.com/2599/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/joyeur.wordpress.com/2599/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/joyeur.wordpress.com/2599/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/joyeur.wordpress.com/2599/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/joyeur.wordpress.com/2599/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/joyeur.wordpress.com/2599/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=2599&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://joyeur.com/2011/06/24/friday-math-fun-24-checkers-and-dominoes/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	
		<media:content url="http://1.gravatar.com/avatar/7b38f1442adc67f3ffa9f20034cefc47?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">jacksonwest</media:title>
		</media:content>

		<media:content url="http://joyeur.files.wordpress.com/2011/06/checkers_and_dominoes.png" medium="image">
			<media:title type="html">checkers_and_dominoes</media:title>
		</media:content>
	</item>
		<item>
		<title>Friday Math Fun #23: &#8220;Crossing the Bridge&#8221;</title>
		<link>http://joyeur.com/2011/06/17/friday-math-fun-23-crossing-the-bridge/</link>
		<comments>http://joyeur.com/2011/06/17/friday-math-fun-23-crossing-the-bridge/#comments</comments>
		<pubDate>Fri, 17 Jun 2011 17:27:31 +0000</pubDate>
		<dc:creator>markjoyent</dc:creator>
				<category><![CDATA[General]]></category>
		<category><![CDATA[math]]></category>

		<guid isPermaLink="false">http://joyeur.com/?p=2530</guid>
		<description><![CDATA[Every Friday, our resident mathematician, Tim Eller, puts out a math challenge to the company. Starting today, we&#8217;re going to open this game up to our readers too. -Mark In the black of night, four people come to a rickety narrow bridge over a chasm. The bridge can hold at most two people at a [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=2530&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p><img src="http://joyeur.files.wordpress.com/2011/06/3498168942_6bfe69c217_o_jw.jpg?w=500&#038;h=302" alt="" title="3498168942_6bfe69c217_o_JW" width="500" height="302" class="aligncenter size-full wp-image-2536" /></p>
<p><em>Every Friday, our resident mathematician, Tim Eller, puts out a math challenge to the company. Starting today, we&#8217;re going to open this game up to our readers too.  -Mark</em></p>
<p>In the black of night, four people come to a rickety narrow bridge over a chasm. The bridge can hold at most two people at a time. They have a dim flashlight whose battery has 17 minutes of life left, and because of the perils, anybody on the bridge must be traveling with the flashlight. (So if there are two people on the bridge, they must be together with the flashlight.) Each of the four people travels at a different pace. The fastest person could cross the bridge in 1 minute, the next in 2, the third in 5, and the slowest would take 10 minutes. How can all four of them make it across the bridge safely?</p>
<p><em>Original photo by <a href="http://www.flickr.com/photos/raes_antics/3498168942/">Raelene Gutierrez</a>.</em></p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/joyeur.wordpress.com/2530/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/joyeur.wordpress.com/2530/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/joyeur.wordpress.com/2530/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/joyeur.wordpress.com/2530/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/joyeur.wordpress.com/2530/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/joyeur.wordpress.com/2530/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/joyeur.wordpress.com/2530/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/joyeur.wordpress.com/2530/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/joyeur.wordpress.com/2530/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/joyeur.wordpress.com/2530/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/joyeur.wordpress.com/2530/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/joyeur.wordpress.com/2530/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/joyeur.wordpress.com/2530/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/joyeur.wordpress.com/2530/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=2530&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://joyeur.com/2011/06/17/friday-math-fun-23-crossing-the-bridge/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
	
		<media:content url="http://0.gravatar.com/avatar/aa49c3efd797f3deded2f583fe9d98f8?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">markjoyent</media:title>
		</media:content>

		<media:content url="http://joyeur.files.wordpress.com/2011/06/3498168942_6bfe69c217_o_jw.jpg" medium="image">
			<media:title type="html">3498168942_6bfe69c217_o_JW</media:title>
		</media:content>
	</item>
		<item>
		<title>Network Storage in the Cloud: Delicious but Deadly</title>
		<link>http://joyeur.com/2011/04/25/network-storage-in-the-cloud-delicious-but-deadly/</link>
		<comments>http://joyeur.com/2011/04/25/network-storage-in-the-cloud-delicious-but-deadly/#comments</comments>
		<pubDate>Mon, 25 Apr 2011 14:00:44 +0000</pubDate>
		<dc:creator>Jackson West</dc:creator>
				<category><![CDATA[General]]></category>
		<category><![CDATA[Industry]]></category>
		<category><![CDATA[Infrastructure]]></category>
		<category><![CDATA[Joyent]]></category>
		<category><![CDATA[Software]]></category>

		<guid isPermaLink="false">http://joyeur.com/?p=2205</guid>
		<description><![CDATA[This post is one in a series discussing storage architectures in the cloud. Read Magical Block Store: When Abstractions Fail Us and On Cascading Failures and Amazon’s Elastic Block Store for more insight. By Bryan Cantrill One of the persistent themes of my career has been something of a love affair with understanding systems failure [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=2205&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p><img class="aligncenter size-full wp-image-2251" title="3336267067_8dc4823f40" src="http://joyeur.files.wordpress.com/2011/04/3336267067_8dc4823f40.jpg?w=500&#038;h=375" alt="" width="500" height="375" /></p>
<p><em>This post is one in a series discussing storage architectures in the cloud. Read <a href="http://joyeur.com/2011/04/24/magical-block-store-when-abstractions-fail-us/">Magical Block Store: When Abstractions Fail Us</a> and <a href="http://joyeur.com/2011/04/22/on-cascading-failures-and-amazons-elastic-block-store/">On Cascading Failures and Amazon’s Elastic Block Store</a> for more insight.</em></p>
<p><em>By <a href="http://dtrace.org/blogs/bmc/">Bryan Cantrill</a></em></p>
<p>One of the persistent themes of my career has been something of a love affair with understanding systems failure &#8212; I fervently believe that <a href="http://dtrace.org/blogs/bmc/2007/05/06/the-inculcation-of-systems-thinking/">systems have the most to teach us when they fail</a>. And while this interest in failure hasn&#8217;t made my own any easier to stomach, my career has been blessed with plenty of opportunity for study: For an infrastructure provider, failures can be devastating, resulting in downtime, data loss and worse. And for a cloud provider, failures are not only just as devastating, but much more potentially widespread in their impact. As such, <a href="http://justinsb.posterous.com/aws-down-why-the-sky-is-falling">the massive, systemic failure in Amazon&#8217;s US-EAST-1 region</a> (and the prolonged downtime that it induced), provides an occasion to reflect on the nature of failure on the cloud. While it is true that some level of systems failure is endemic (and that systems must be designed with these failures in mind), this cannot become an excuse for unreliable systems. Failures must be understood if we are to collectively learn from them.</p>
<p>For Joyent, this has been an opportunity to reflect back on some of our own failures &#8212; and specifically how they changed the way we think about the cloud. You may have seen <a href="http://joyeur.com/2011/04/22/on-cascading-failures-and-amazons-elastic-block-store/">Jason on cascading failure</a> or <a href="http://joyeur.com/2011/04/24/magical-block-store-when-abstractions-fail-us/">Mark on the perils of abstraction</a>. I wanted to add my own perspective on Joyent&#8217;s experience with network storage.</p>
<p><span id="more-2205"></span></p>
<p>I knew Jason and Mark (and Dave and Ben) long before <a href="http://dtrace.org/blogs/bmc/2010/07/30/hello-joyent/">I came to Joyent in August</a>, and I remember when they were not so jaundiced about network storage: I recall having a discussion with Jason in 2006 when he excitedly talked of his plans for using block-based network storage as the bedrock of a large-scale IaaS offering. He was enthusiastic &#8212; it all seemed so elegant and fool-proof! &#8212; and as I was <a href="http://dtrace.org/blogs/bmc/2008/11/10/fishworks-now-it-can-be-told/">developing a network storage appliance at the time</a>, so (naturally) was I. I do remember us both wondering aloud if having a copy-on-write filesystem backed by a LUN that was in fact a remote copy-on-write filesystem was going to have new performance pathologies, but like doomed hikers ignoring a distant storm cloud, we collectively shrugged our shoulders and we each pushed on down our respective paths&#8230;</p>
<p>Five years later, we&#8217;re all older and wiser &#8212; and we each independently learned some very tough lessons about the peril of network storage in a multi-tenant environment. Jason <a href="http://joyeur.com/2011/04/22/on-cascading-failures-and-amazons-elastic-block-store/">summarized that lesson euphemistically</a>:</p>
<blockquote><p>While there are many strengths to network storage: Failure modes, particularly in a multi-tenant environment of unknown workloads, is not one of them.</p></blockquote>
<p>Anyone who has stood anything up in this business knows that Jason&#8217;s admonishment practically drips with the cold sweat of a production nightmare; Mark <a href="http://joyeur.com/2011/04/24/magical-block-store-when-abstractions-fail-us/">provides additional detail</a>:</p>
<blockquote><p>After nearly 9 months of trying to make it work we decided to begin the engineering effort to re-work our provisioning architecture, server characteristics, and so on to be able to reliably support persistent data pools on the servers themselves while keeping as many of the features [...] as we could</p></blockquote>
<p>For me personally, however, the lesson came not from trying to operate a cloud, but rather as one of the enterprise vendors that Mark refers to: The Fishworks appliances found quite a bit of traction in the cloud storage space, and we had several customers who enthusiastically hung eye-watering numbers of their cloud customers off of a single (clustered) head.</p>
<p>The problem was that the scale at which these boxes were operating &#8212; tremendous amounts of data churn over many, many virtual machines &#8212; meant that any hiccup would be felt across their entire fleet. And there were plenty of hiccups, often (but not always) due to operator error or operator misunderstanding.</p>
<p>As a concrete example, I found myself berated by one particularly angry cloud storage customer because the box was &#8220;misreporting&#8221; several thousand writes per second when the customer &#8220;knew&#8221; that any writes were impossible &#8212; that the particular LUNs being identified were &#8220;never&#8221; written to. While I was able to prove to the customer that the writes were indeed happening (DTrace FTW), I couldn&#8217;t take it any further: The network divided the symptoms of the problem from whatever was inducing them in their cloud, making it difficult to understand who was doing it or why.</p>
<p>Several days later, I happened to notice that the number of writes per second had dropped precipitously; I inquired if they had discovered the root-cause, and the customer sheepishly admitted that there had been a &#8220;configuration error.&#8221; Obviously not satisfied with that answer after the tongue-lashing I had received on this issue, I pressed for details &#8212; and it was revealed that the file systems had all been mounted on the networked LUNs without &#8220;noatime&#8221; set. That is, any <em>access</em> to a file updated the access time &#8212; the <tt>atime</tt> &#8212; on the i-node, turning a gentle read-only workload into a fire-breathing metadata-writing monster.</p>
<p>Now, this is a mistake that anyone could make (certainly), but that the storage system was distributed made it unnecessarily arduous to root-cause. Further, the performance ramifications of this were significantly more acute due to the scale of the system: Because they had consolidated so many virtual machines on so few (centralized) spindles, their operations per spindle were much higher than they would have been had the storage been local. And need I remind: These are disks that we&#8217;re talking about here. They are still operating under the same principle as the <a href="http://en.wikipedia.org/wiki/IBM_305_RAMAC">IBM RAMAC</a> circa 1956 &#8212; and their seek times are still counted in the same time units! They do not like having load poured on them, and they punish such demands with queueing delays and non-linear latency. (And lest you feel like venting your frustration, they also <a href="http://www.youtube.com/watch?v=tDacjrSCeq4">don&#8217;t like being shouted at</a>.)</p>
<p>This whole experience &#8212; and many others like it &#8212; left me questioning the value of network storage for cloud computing. Yes, having centralized storage allowed for certain things &#8212; one could &#8220;magically&#8221; migrate a load from one compute node to another, for example &#8212; but it seemed to me that these benefits were more than negated by the concentration of load and risk in a single unit (even one that is putatively highly available).</p>
<p>When I began to talk to Joyent, I was relieved to hear that their experiences so closely mirrored mine &#8212; and they had made the decision to abandon the fantasia of network storage for local data (root file systems, databases, etc.), pushing that data instead back to the local spindles and focussing on making it reliable and available. Yes, this design decision was (and remains) a trade-off &#8212; when local data is local, compute nodes are no longer stateless &#8212; and Joyent has needed to invest in technologies to allow for replication, migration and backup between nodes.</p>
<p>But in the end, the loss of functionality has been more than made up for by the gain in resiliency. Now when we have I/O performance problems (and, let&#8217;s be honest, where there&#8217;s I/O there are I/O performance problems), we know that the failure is contained to a single node and a bounded number of tenants &#8212; and we have <a href="http://dtrace.org/blogs/brendan/2011/03/14/mysql-query-latency-with-the-dtrace-pid-provider/">rich tooling</a> to diagnose the root-cause (which is often up-stack).</p>
<p>Moreover, by keeping local I/O local, we have all of the necessary context to make decisions around I/O scheduling to assure Quality of Service &#8212; which has enabled us to take a swing at <a href="http://dtrace.org/blogs/wdp/2011/03/our-zfs-io-throttle/">multi-tenant I/O throttling</a>. This is not to say that network storage doesn&#8217;t have its uses of course &#8212; but frankly, serving up virtual devices to a cloud just isn&#8217;t one of them: When local storage is consolidated, risk is consolidated along with it, and component failure is more likely to balloon into systemic failure.</p>
<p>More generally, network storage embodies a key design issue in architecting and operating a cloud: The tension between economies of scale and overconsolidation of risk. The challenge of the cloud is delivering enough of the former to be viable without so much of the latter as to be deadly. We at Joyent have learned much of this the hard way, and we reflect that wisdom not only in the way we operate our public cloud, but also in our <a href="http://www.joyent.com/software/smartdatacenter/">SmartDataCenter</a> offering. We will no doubt continue to learn ways in which we may better strike that balance, but we take solace in all that we have already learned &#8212; and hope that future education need not be forged in the kiln of unspeakable pain!</p>
<p>Visit <a href="http://www.joyentcloud.com/">http://www.joyentcloud.com/</a> to learn about how Joyent&#8217;s public cloud, Joyent Cloud, is different from Amazon&#8217;s AWS.</p>
<p><em>Photo by <a href="http://www.flickr.com/photos/mdid/3336267067/">Michael Zimmer</a>.</em></p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/joyeur.wordpress.com/2205/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/joyeur.wordpress.com/2205/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/joyeur.wordpress.com/2205/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/joyeur.wordpress.com/2205/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/joyeur.wordpress.com/2205/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/joyeur.wordpress.com/2205/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/joyeur.wordpress.com/2205/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/joyeur.wordpress.com/2205/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/joyeur.wordpress.com/2205/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/joyeur.wordpress.com/2205/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/joyeur.wordpress.com/2205/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/joyeur.wordpress.com/2205/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/joyeur.wordpress.com/2205/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/joyeur.wordpress.com/2205/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=2205&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://joyeur.com/2011/04/25/network-storage-in-the-cloud-delicious-but-deadly/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
	
		<media:content url="http://1.gravatar.com/avatar/7b38f1442adc67f3ffa9f20034cefc47?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">jacksonwest</media:title>
		</media:content>

		<media:content url="http://joyeur.files.wordpress.com/2011/04/3336267067_8dc4823f40.jpg" medium="image">
			<media:title type="html">3336267067_8dc4823f40</media:title>
		</media:content>
	</item>
		<item>
		<title>Magical Block Store: When Abstractions Fail Us</title>
		<link>http://joyeur.com/2011/04/24/magical-block-store-when-abstractions-fail-us/</link>
		<comments>http://joyeur.com/2011/04/24/magical-block-store-when-abstractions-fail-us/#comments</comments>
		<pubDate>Sun, 24 Apr 2011 16:33:36 +0000</pubDate>
		<dc:creator>markjoyent</dc:creator>
				<category><![CDATA[General]]></category>
		<category><![CDATA[Industry]]></category>
		<category><![CDATA[Infrastructure]]></category>
		<category><![CDATA[Joyent]]></category>
		<category><![CDATA[Software]]></category>

		<guid isPermaLink="false">http://joyeur.com/?p=2156</guid>
		<description><![CDATA[This post is one in a series discussing storage architectures in the cloud. Read Network Storage in the Cloud: Delicious but Deadly and On Cascading Failures and Amazon’s Elastic Block Store for more insight. Like anyone who has anything to do with the business of running applications attached to the Internet, I can&#8217;t help but [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=2156&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p><img class="aligncenter size-full wp-image-2236" title="4302020815_b18f3e8d0b_o" src="http://joyeur.files.wordpress.com/2011/04/4302020815_b18f3e8d0b_o.jpg?w=500&#038;h=375" alt="" width="500" height="375" /></p>
<p><em>This post is one in a series discussing storage architectures in the cloud. Read <a href="http://joyeur.com/2011/04/25/network-storage-in-the-cloud-delicious-but-deadly/">Network Storage in the Cloud: Delicious but Deadly</a> and <a href="http://joyeur.com/2011/04/22/on-cascading-failures-and-amazons-elastic-block-store/">On Cascading Failures and Amazon’s Elastic Block Store</a> for more insight.</em></p>
<p>Like anyone who has anything to do with the business of running applications attached to the Internet, I can&#8217;t help but feel uneasy and stressed when an outage as severe and impacting as the one Amazon Elastic Block Store has just experienced happens. The fact that Amazon is a competitor doesn&#8217;t matter. It just sucks for everyone. This post, however, isn&#8217;t about EBS. Not really. It&#8217;s about abstractions. In particular, the abstraction of &#8220;the disk.&#8221;</p>
<p>Abstraction levels are something that have fascinated me for my entire career. We (developers, operators, the whole lot of us) heavily depend on multiple abstraction layers in our respective stacks simply because the technology we&#8217;re putting into play is so incredibly complex that there&#8217;s no way we could progress without the ability to hide details.</p>
<p>The trick, of course, is getting the abstraction right. It has to be understandable, observable, and predictable. A fantastic implementation is worthless if the abstraction is wrong. Change the behaviour of the abstraction you&#8217;re implementing too much, and the abstraction becomes worthless. You can tell where I&#8217;m going with this &#8211; questioning whether or not the abstraction of a reliable, super cheap, performant network attached block storage device is viable.</p>
<p><span id="more-2156"></span></p>
<p>Certainly there&#8217;s a lot of players in this space with a lot of skin in the game who want us to believe it is. I&#8217;m not talking about just Amazon &#8211; the top 5 storage vendors of EMC, HP, IBM, NetApp, and Dell <a href="http://www.eweekeurope.co.uk/news/disk-storage-sector-grows-in-2010-7481">shipped 3397 petabytes of the stuff</a> <em>last quarter</em>.</p>
<p><strong>Reality Check</strong></p>
<p>There&#8217;s a phrase about abstractions (I honestly don&#8217;t know if it was myself or someone else who coined it) that I remind our engineering staff about on a regular basis. It goes something like this:</p>
<blockquote><p>We need our abstractions, our black boxes. Black boxes are great and enabling when they work. But when something goes wrong, the walls of our abstractions need to go transparent as fast as possible.</p></blockquote>
<p>There&#8217;s a reason we so fanatically drive at this goal at Joyent, and it&#8217;s not because we&#8217;re smarter or better than anyone else. It&#8217;s because we&#8217;ve been burned by invalid abstractions of our own doing so many times that we&#8217;ve programmed a Jabber bot to constantly berate us with reminders of how our hubris around abstractions led us to failure.</p>
<p>We still get it wrong, often. The most recent example that comes to mind wasn&#8217;t a storage product, but our <a href="http://reasonablysmart.com/">Smart Platform</a>. A pure Javascript Platform as a Service. For me at least, it was the proof point that server side JS was going to have a big role in the future of web and mobile applications. Yet, even though this was <a href="https://twitter.com/#!/jamesaduncan">James&#8217;s</a> (the lead architect and engineer) third time building a developer-focused application platform, and even though we wanted the platform to have far less magic than, say, Google App Engine, we still strayed too far down the magic path and pushed the abstraction level too high.</p>
<p>The object store and multi-tenant message routing architecture resulted in a body of code and interfaces that (it was open source from day one) nobody but us could really understand well enough to contribute to. Or debug it. In fact, the object store itself tripped us up so many times dogfooding the platform that we finally shook free of its hold and reevaluated our approach. The implementation had issues, but the real culprit was the set of abstractions that Smart Platform presented.</p>
<p>Today we&#8217;re the driving force behind <a href="http://nodejs.org/">NodeJS</a>, which sits on a very different place on the abstraction curve than Smart did. We&#8217;ve pulled lots of ideas and lessons from Smart into our hosted NodeJS offering, <a href="http://no.de">no.de</a>, but we did drop as much of the magic as possible. The end result is a service that&#8217;s tremendously simpler and in many ways less feature rich, but it doesn&#8217;t make promises we can&#8217;t fulfill and our customers can understand it.</p>
<p>In fact, when something goes wrong they can log right in with a shell and see exactly exactly what&#8217;s going on. We&#8217;ve paired the transparency with <a href="http://dtrace.org/blogs/dap/2011/03/26/example-http-request-latency-and-garbage-collection/">ground-breaking observability tools</a>. NodeJS and no.de are huge successes, largely, I think, because we walked the fine line of abstraction level better this time. It&#8217;s only a black box when you want it to be. When you need to look inside, you can. You can evaluate the risk level of the technology and approach yourself.</p>
<p><strong>Magical Thinking</strong></p>
<p>The promise of network block storage is wonderful: Take a familiar abstraction (the disk), sprinkle on some magic cloud pixie dust so that it&#8217;s completely reliable, available over the same cheap network you&#8217;re using for app traffic, map it to any instance in a datacenter regardless of network topology, make it so cheap it&#8217;s practically free, and voila, we can have our cake and eat it too! It&#8217;s the holy grail many a storage vendor, most of whom with decades experience in storage systems and engineering teams thousands strong have chased for a long, long time. The disk that never dies. The disk that&#8217;s not a disk.</p>
<p>The reality, however, is that the disk has never been a great abstraction, and the long history of crappy implementations has meant that many behavioral workarounds have found their way far up the stack. The best case scenario is that a disk device breaks and it&#8217;s immediately catastrophic taking your entire operating system with it. Failure modes go downhill from there. Networks have their own set of special failure modes too. When you combine the two, and that disk you depend on is sitting on the far side of the network from where your operating system is, you get a combinatorial explosion of complexity.</p>
<p>To make matters far worse, you can&#8217;t easily flip the switch and make the walls of this Magic Block Store go transparent so you can see what&#8217;s happening under the covers. EBS is no different than the big storage vendors in this regard &#8212; neither Amazon nor EMC seem to want us to know what goes on inside their abstraction.</p>
<p>I certainly don&#8217;t claim to know how EBS works, but of course people go to bars and have beers and talk. It&#8217;s commonly believed that EBS is built on <a href="http://www.drbd.org/">DRBD</a> with a dose of S3-derived replication logic. As a former boss of mine so demeaningly stated to me once, in grown-up land when your cheap home grown DRBD solution falls over and loses all your data you open up your check book and pay EMC or NetApp to do the job right. Maybe Amazon is different. Maybe they did what a dozen billion dollar companies before them tried to do and never pulled off. Or maybe EBS is indeed bandaids and chicken wire. I have no idea. Which is a problem, as a user of EBS.</p>
<p>Back to the abstraction instead of the implementation. Do we need block devices in the cloud? If we&#8217;re just asking too much of this ancient abstraction, as I believe we are, can we as developers and operators stop demanding the vendors make these Magic Block Store solutions? We certainly fear having to evolve our trusted abstractions, and the disk is about as old of an abstraction as we&#8217;ve got.</p>
<p>All is not lost, however. In my experience, it isn&#8217;t that we need &#8220;disks&#8221;, it&#8217;s that our apps need POSIX&#8217;s read() and write() and friends combined with behaviour that roughly approximates a physical disk. For the times we actually need a block interface vs. filesystem semantics, I would argue, the abstraction sitting on top (the filesystem, usually) is even <em>more</em> sensitive to the behaviour of the disk device than a POSIX application like a database is.</p>
<p>My opinion is that the only reason the big enterprise storage vendors have gotten away with network block storage for the last decade is that they can afford to over-engineer the hell out of them and have the luxury of running enterprise workloads, which is a code phrase for &#8220;consolidated idle workloads.&#8221; When the going gets tough in enterprise storage systems, you do capacity planning and make sure your hot apps are on dedicated spindles, controllers, and network ports.</p>
<p>It was fantasy believing it was possible to pull off a centralized network block storage service in a multi-tenant cloud without any of the architecture shenanigans our enterprise brethren do and think that applications, databases, and business could depend on its being <em>perfect</em>. Honestly, we should have know better. We the applications developers asked what is perhaps the crappiest of all abstractions in computers to solve all of our availability problems for us. We asked for magic. Clearly, the vendor never should have made the promise of magic, but everyone is to blame for this continued expectation that such magic is possible.</p>
<p><strong>A Different Approach</strong></p>
<p>At Joyent, when VMware and Amazon were preaching centralized network block storage as the salvation, we tacked the other way, for reasons I&#8217;ll explain below. (Hint: We tried and failed.) <em>We simply provide a POSIX filesystem interface to storage</em>, which happens to be sitting on local, not network, block devices. We lean on <a href="http://en.wikipedia.org/wiki/ZFS">ZFS</a> for durability and a fighting chance when things &#8220;go byzantine.&#8221; We use fast SAS drives in RAID groups with multiple parity stripes. We don&#8217;t pretend that this &#8220;disk&#8221; can never, ever go away or survive all failure modes. What&#8217;s more important, I believe, is the set of technologies we put in place so that our customers can transparently see exactly how our filesystems are holding up.</p>
<p>As an example, we provide tools and <a href="https://gist.github.com/939336">scripts</a> that show you a distribution of how much latency your MySQL database is seeing from the abstraction directly beneath it, the filesystem.</p>
<p>You can run it yourself from the command line, the output looks like this:</p>
<p><code><br />
# ./mysqld_pid_fslatency.d -p `pgrep mysqld`<br />
Tracing PID 7357... Hit Ctrl-C to end.<br />
^C<br />
MySQL filesystem I/O: 21329; latency (ns):</code></p>
<p>read<br />
value &#8212;&#8212;&#8212;&#8212;- Distribution &#8212;&#8212;&#8212;&#8212;- count<br />
1024 | 0<br />
2048 |@@@@@@@@@@@@ 3837<br />
4096 |@@@@@@@@@@@@@@@ 4998<br />
8192 |@@@@@@@@@@@ 3540<br />
16384 |@@ 745<br />
32768 | 95<br />
65536 | 19<br />
131072 | 3<br />
262144 | 0<br />
524288 | 0<br />
1048576 | 1<br />
2097152 | 1<br />
4194304 | 0</p>
<p>write<br />
value &#8212;&#8212;&#8212;&#8212;- Distribution &#8212;&#8212;&#8212;&#8212;- count<br />
4096 | 0<br />
8192 |@@@@@@@@@ 1902<br />
16384 |@@@@@@@@@@@@@@@@@@@@@@@@@@ 5181<br />
32768 |@@@@ 798<br />
65536 |@ 184<br />
131072 | 9<br />
262144 | 1<br />
524288 | 6<br />
1048576 | 0<br />
2097152 | 0<br />
4194304 | 7<br />
8388608 | 2<br />
16777216 | 0</p>
<p>Or you can use our visualizations:</p>
<p><img class="aligncenter" title="Cloud Analytics" src="http://dtrace.org/blogs/brendan/files/2011/03/ca_demo2_2.png" alt="" width="498" height="239" /></p>
<p>Either way, it&#8217;s pretty awesome. Being able to confirm your DB is seeing thousands of cache hits with microsecond latency and only a few disk hits in the 1-10ms range is the kind of insight that lets DBAs sleep at night. You can also easily ask and receive an answer to a question like &#8220;<a href="https://gist.github.com/939344">how many of my slow queries are because of filesystem/disk latency</a> vs. a missing index?&#8221; Trying to use a tool like iostat against a shared, network provided block device to figure out what your level of service your database is getting from the filesystem below it is an exercise in frustration that will get you nowhere.</p>
<p>For Joyent, keeping the network out of the storage solution and safely persisting those filesystems across reboots on the servers added some complexity to our provisioning. We don&#8217;t have the luxury of treating local disk as ephermeral when selecting nodes to place workload on, for example. We lose some nice features like quick re-mounting to arbitrary vm instances, too. But <em>it allowed us to present a usable, observable abstraction</em> that we&#8217;re continuing to improve with <a href="http://dtrace.org/blogs/wdp/2011/03/our-zfs-io-throttle/"> innovative I/O throttling and QoS</a>. We&#8217;re building a better building block, in our opinion. The complexity of tackling this work with an oversubscribed network in the middle would have been, in our opinion, an impossible engineering challenge.</p>
<p>Joyent, like always, arrived at the conclusion that we could not build a multi-tenant cloud on network block storage the hard way: We tried to build a multi-tenant cloud on network block storage. When we launched the Joyent IaaS product, it was in fact built on replicated storage nodes that presented iSCSI targets to the server hosts that then in turn implemented RAID1 mirrors of the network block devices. The hosts then carved up the LUNs into ZFS datasets which mapped to the virtual machines.</p>
<p>It&#8217;s worth explicitly pointing out that we had none of the flexible provisioning APIs and features that EBS currently exposes. I&#8217;m certainly not pretending we had an EBS competitor back then. But on the backend, the implementation was probably not <em>wildly</em> different, and the core abstraction we attached our customers virtual machines to was nearly identical. Our MBS implementation was originally built on the <a href="http://hub.opensolaris.org/bin/view/Project+avs/WebHome">StorEdge Availability Suite</a>, a product that came out of StorageTek/Sun that works very similarly to DRBD. We were never able to get it to work well enough, largely a function of us not having enough resources to pour into enhancing the base product to cope with scenarios we were seeing a lot when competing I/O from multiple hot tenants caused the replication to fall behind and never catch up. We instead invested in filers from NetApp and EqualLogic (we didn&#8217;t want all our eggs in one basket) and that bought us time to take engineering and ops resources off keeping the storage nodes running and work on other parts of our stack.</p>
<p>Ultimately, however, the approach suffered the same fate as the homegrown and the performance curve would degrade so badly over time that we were at times only attaching and handful of servers to a replicated filer pair. Obviously, this was not only uneconomical but unviable from a predictable performance point of view; Our customers hated the I/O experience. After nearly 9 months of trying to make it work we decided to begin the engineering effort to re-work our provisioning architecture, server characteristics, and so on to be able to reliably support persistent data pools on the servers themselves while keeping as many of the features commonly associated with MBS as we could (snapshots we have today, async replication isn&#8217;t that far off).</p>
<p>I still believe that it was the right decision, and as we have <a href="http://cloudcomputing.sys-con.com/node/1535175">grown up as a company</a> and <a href="http://dtrace.org/blogs/bmc/2010/07/30/hello-joyent/">increased our engineering capabilities</a> we&#8217;re only now really beginning to see how liberating the decision to drop network storage out of the Joyent cloud was. I believe that applications can rise to the task and not require perfect, always on block storage, for the simple fact that they&#8217;re going to have to do it anyways to get to the goal of global deliverability and availability. We just can&#8217;t lean on the false abstraction of a perfect disk forever.</p>
<p>-Mark</p>
<p>Visit <a href="http://www.joyentcloud.com/">http://www.joyentcloud.com/</a> to learn about how Joyent&#8217;s public cloud, Joyent Cloud, is different from Amazon&#8217;s AWS.</p>
<p><em>Photo by <a href="http://www.flickr.com/photos/mcbarnicle/4302020815/">mcbarnicle</a>.</em></p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/joyeur.wordpress.com/2156/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/joyeur.wordpress.com/2156/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/joyeur.wordpress.com/2156/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/joyeur.wordpress.com/2156/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/joyeur.wordpress.com/2156/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/joyeur.wordpress.com/2156/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/joyeur.wordpress.com/2156/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/joyeur.wordpress.com/2156/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/joyeur.wordpress.com/2156/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/joyeur.wordpress.com/2156/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/joyeur.wordpress.com/2156/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/joyeur.wordpress.com/2156/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/joyeur.wordpress.com/2156/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/joyeur.wordpress.com/2156/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=2156&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://joyeur.com/2011/04/24/magical-block-store-when-abstractions-fail-us/feed/</wfw:commentRss>
		<slash:comments>21</slash:comments>
	
		<media:content url="http://0.gravatar.com/avatar/aa49c3efd797f3deded2f583fe9d98f8?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">markjoyent</media:title>
		</media:content>

		<media:content url="http://joyeur.files.wordpress.com/2011/04/4302020815_b18f3e8d0b_o.jpg" medium="image">
			<media:title type="html">4302020815_b18f3e8d0b_o</media:title>
		</media:content>

		<media:content url="http://dtrace.org/blogs/brendan/files/2011/03/ca_demo2_2.png" medium="image">
			<media:title type="html">Cloud Analytics</media:title>
		</media:content>
	</item>
		<item>
		<title>Top 5 Lists Featuring Joyent &#8212; Includes Bonus List!</title>
		<link>http://joyeur.com/2011/04/04/top-5-lists-featuring-joyent-includes-bonus-list/</link>
		<comments>http://joyeur.com/2011/04/04/top-5-lists-featuring-joyent-includes-bonus-list/#comments</comments>
		<pubDate>Mon, 04 Apr 2011 21:01:04 +0000</pubDate>
		<dc:creator>Jackson West</dc:creator>
				<category><![CDATA[General]]></category>
		<category><![CDATA[Industry]]></category>
		<category><![CDATA[Joyent]]></category>
		<category><![CDATA[Media Coverage]]></category>
		<category><![CDATA[Software]]></category>

		<guid isPermaLink="false">http://joyeur.com/?p=2086</guid>
		<description><![CDATA[Everyone loves listicles &#8212; sites love to publish them because people love to click on them! Joyeur is certainly no exception. Recently, we&#8217;ve been featured in a number of lists, so here they are in order of exclusivity &#8212; along with some tips that might help you get past some editorial velvet ropes. 5. 2011 [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=2086&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p><img class="aligncenter size-full wp-image-2087" title="3566071999_59f334f39c" src="http://joyeur.files.wordpress.com/2011/04/3566071999_59f334f39c.jpg?w=500&#038;h=375" alt="" width="500" height="375" /></p>
<p>Everyone loves listicles &#8212; sites love to publish them because people love to click on them!  <a href="http://joyeur.com/2011/02/17/top-10-tips-for-social-gaming-success/">Joyeur is certainly no exception</a>.  Recently, we&#8217;ve been featured in a number of lists, so here they are in order of exclusivity &#8212; along with some tips that might help you get past some editorial velvet ropes.</p>
<p><strong>5. <a href="http://alwayson.goingon.com/AOStory/Announcing-2011-OnDemand-100-Top-Private-Companies">2011 OnDemand 100 Top Private Companies</a></strong><br />
Nobody produces listicles quite like AlwaysOn, which besides including Joyent on the <a href="http://alwayson.goingon.com/AOStory/Announcing-2011-OnDemand-100-Top-Private-Companies">OnDemand Top 100 as a leader in Cloud Infrastructure</a> is also currently working on the <a href="http://alwayson.goingon.com/AOStory/AlwaysOn-OnMobile-100-Top-Company-Competition">OnMobile 100 Top Company Competition</a> and the <a href="http://alwayson.goingon.com/AOStory/AlwaysOn-Global-250-Top-Company-Competition">AlwaysOn Global 250 Top Company Competition</a>.  So if your company hasn&#8217;t made a list yet, might be time to find Founder and Editor Tony Perkins at one of AlwaysOn&#8217;s many networking events and hand him a business card.</p>
<p><span id="more-2086"></span></p>
<p><strong>4. <a href="http://www.crn.com/slide-shows/cloud/229400421/the-top-20-cloud-infrastructure-vendors-of-2011.htm?pgno=1">CRN: The Top 20 Cloud Infrastructure Vendors of 2011</a></strong><br />
One way to make lists is to appeal to a niche audience. CRN Technology News focuses on our industry audience &#8212; technology integrators and solution providers. We read them, and naturally they keep a close eye on us &#8212; and our competition &#8212; so it&#8217;s rewarding to be honored with a selection. <strong></strong></p>
<p><strong>3. <a href="http://www.mercurynews.com/business/ci_16930409?nclick_check=1">O’Brien: 11 Predictions for 2011</a></strong><br />
There is absolutely nothing wrong with rampant speculation in the press, especially when it&#8217;s from the likes of Chris O&#8217;Brien from the San Jose Mercury News. He predicts that &#8220;2011 will be the year that many larger players swoop in and start buying up these racehorses,&#8221; and calls us out as a &#8220;hot name&#8221; in the business.  You&#8217;ll have to be a site subscriber to <a href="http://www.mercurynews.com/business/ci_16930409?nclick_check=1">see into the rest of O&#8217;Brien&#8217;s crystal ball of a column</a>.  But we certainly like the view from his perspective.</p>
<p><strong>2. <a href="http://searchcloudcmputing.techtarget.com/feature/Top-10-cloud-computing-providers-of-2011">Top 10 Cloud Computing Providers of 2011</a></strong><br />
For the second year in a row, we&#8217;ve stayed in the top ten of SearchCloudComputing&#8217;s top list. Hey, is that Microsoft at number nine?   <a href="http://searchcloudcomputing.techtarget.com/video/Debating-the-10-best-cloud-providers-for-2011">Check out Jo Maitland and Carl Brooks&#8217;s reasoning for the rankings on CloudCoverTV</a>.</p>
<p><strong>1. <a href="http://gigaom.com/cloud/5-cloud-software-vendors-that-dell-should-buy/">5 Cloud Software Vendors Dell Should Buy</a></strong><br />
Last and most exclusive, Derrick Harris gives us a nod as a business fit for our friends at Dell with a handful of other companies.  We certainly enjoy <a href="http://www.joyent.com/2010/03/dell-chooses-joyent-software-to-power-cloud-computing/">working with them to help provide their Dell Cloud Computing Solution</a>.  And now we know how celebrities must feel when, after working closely together on a project, suddenly the tabloids are full of gossip.  All our publicist is saying for the record is that no, we are not dating Angelina Jolie or Brad Pitt. Or both Angelina and Brad Pitt.  Get those cameras out of our face!</p>
<p><strong>Bonus:</strong> <a href="http://www.crn.com/slide-shows/cloud/229301194/the-10-million-cloud-club-10-well-funded-cloud-computing-vendors.htm?pgno=11">CRN also selected us</a> for their &#8220;<a href="http://www.crn.com/slide-shows/cloud/229301194/the-10-million-cloud-club-10-well-funded-cloud-computing-vendors.htm?pgno=1">$10 Million Cloud Club: 10 Well-Funded Cloud Computing Vendors</a>&#8221; list.</p>
<p>So while we may not be on the guest lists at actual trendy nightclubs &#8212;  we are nerds, after all &#8212; we&#8217;d much rather be with this sort of  quality company in these industry VIP rooms, anyway!</p>
<p><em>Photo by <a href="http://www.flickr.com/photos/jeffmaysh/3566071999/">Jeff Maysh</a>.</em></p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/joyeur.wordpress.com/2086/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/joyeur.wordpress.com/2086/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/joyeur.wordpress.com/2086/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/joyeur.wordpress.com/2086/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/joyeur.wordpress.com/2086/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/joyeur.wordpress.com/2086/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/joyeur.wordpress.com/2086/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/joyeur.wordpress.com/2086/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/joyeur.wordpress.com/2086/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/joyeur.wordpress.com/2086/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/joyeur.wordpress.com/2086/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/joyeur.wordpress.com/2086/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/joyeur.wordpress.com/2086/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/joyeur.wordpress.com/2086/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=2086&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://joyeur.com/2011/04/04/top-5-lists-featuring-joyent-includes-bonus-list/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
	
		<media:content url="http://1.gravatar.com/avatar/7b38f1442adc67f3ffa9f20034cefc47?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">jacksonwest</media:title>
		</media:content>

		<media:content url="http://joyeur.files.wordpress.com/2011/04/3566071999_59f334f39c.jpg" medium="image">
			<media:title type="html">3566071999_59f334f39c</media:title>
		</media:content>
	</item>
		<item>
		<title>ps pipe grep: Episode 25 (Hugs)</title>
		<link>http://joyeur.com/2011/03/16/ps-pipe-grep-episode-25-hugs/</link>
		<comments>http://joyeur.com/2011/03/16/ps-pipe-grep-episode-25-hugs/#comments</comments>
		<pubDate>Wed, 16 Mar 2011 20:43:51 +0000</pubDate>
		<dc:creator>Jackson West</dc:creator>
				<category><![CDATA[General]]></category>
		<category><![CDATA[node.js]]></category>
		<category><![CDATA[Podcast]]></category>

		<guid isPermaLink="false">http://joyeur.com/?p=2025</guid>
		<description><![CDATA[Tom Hughes-Croucher dropped by Studio 112B to sit down with Jason, while Robot David Young, already having reached the technological singularity, chimed in from Planet Marin. Trans-Atlantic diplomatic relations, Robot David&#8217;s possibly factual accounts of Texas history and definitely factual accounts of Texas barbecue, plus the hows and whys of attending tech conferences are discussed. [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=2025&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p><img src="http://joyeur.files.wordpress.com/2010/12/pspipegrep_logo.png?w=150&#038;h=150" alt="" title="pspipegrep_logo" width="150" height="150" class="alignleft size-thumbnail wp-image-1340" />Tom Hughes-Croucher dropped by Studio 112B to sit down with Jason, while Robot David Young, already having reached the technological singularity, chimed in from Planet Marin.  Trans-Atlantic diplomatic relations, Robot David&#8217;s possibly factual accounts of Texas history and definitely factual accounts of Texas barbecue, plus the hows and whys of attending tech conferences are discussed. Also? Node.js.</p>
<p>If you&#8217;d like to subscribe, <a href="http://feeds.feedburner.com/pspipegrep">here&#8217;s the feed</a>.  Otherwise, here’s <a href="http://serve.castfire.com/audio/532273/532273_2011-03-16-162913.default.mp3">a direct link to the mp3</a>.</p>
<p>This month, David was eagerly awaiting the drop of the new Lupe Fiasco joint <em><a href="http://itunes.apple.com/us/album/lasers/id418674062#">Lasers</a></em>.  So the podcast kicks off with a sample of the single, &#8220;<a href="http://www.youtube.com/watch?v=btQKGvVRnZ8">Words I Never Said</a>,&#8221; featuring the hook voiced by <a href="http://www.myspace.com/skylargrey">Skylar Grey</a>.</p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/joyeur.wordpress.com/2025/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/joyeur.wordpress.com/2025/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/joyeur.wordpress.com/2025/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/joyeur.wordpress.com/2025/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/joyeur.wordpress.com/2025/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/joyeur.wordpress.com/2025/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/joyeur.wordpress.com/2025/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/joyeur.wordpress.com/2025/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/joyeur.wordpress.com/2025/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/joyeur.wordpress.com/2025/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/joyeur.wordpress.com/2025/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/joyeur.wordpress.com/2025/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/joyeur.wordpress.com/2025/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/joyeur.wordpress.com/2025/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=2025&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://joyeur.com/2011/03/16/ps-pipe-grep-episode-25-hugs/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
<enclosure url="http://serve.castfire.com/audio/532273/532273_2011-03-16-162913.default.mp3" length="34359922" type="audio/mpeg" />
	
		<media:content url="http://1.gravatar.com/avatar/7b38f1442adc67f3ffa9f20034cefc47?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">jacksonwest</media:title>
		</media:content>

		<media:content url="http://joyeur.files.wordpress.com/2010/12/pspipegrep_logo.png?w=150" medium="image">
			<media:title type="html">pspipegrep_logo</media:title>
		</media:content>
	</item>
		<item>
		<title>ps pipe grep: Episode 24 (Reverse Groupon)</title>
		<link>http://joyeur.com/2011/02/22/ps-pipe-grep-episode-24-reverse-groupon/</link>
		<comments>http://joyeur.com/2011/02/22/ps-pipe-grep-episode-24-reverse-groupon/#comments</comments>
		<pubDate>Wed, 23 Feb 2011 07:59:47 +0000</pubDate>
		<dc:creator>Jackson West</dc:creator>
				<category><![CDATA[General]]></category>
		<category><![CDATA[Industry]]></category>
		<category><![CDATA[Podcast]]></category>

		<guid isPermaLink="false">http://joyeur.com/?p=1833</guid>
		<description><![CDATA[Bryan Cantrill makes his first appearance with Ben still on the road, and along with David and Jason takes a long, circuitous route from computing history to the computing future by way of elegant nachos. A moment of silence for Ken Olsen, Bryan boasts he can take Watson in a fistfight, David buys a Blackberry [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=1833&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p><img src="http://joyeur.files.wordpress.com/2010/12/pspipegrep_logo.png?w=150&#038;h=150" alt="" title="pspipegrep_logo" width="150" height="150" class="alignleft size-thumbnail wp-image-1340" />Bryan Cantrill makes his first appearance with Ben still on the road, and along with David and Jason takes a long, circuitous route from computing history to the computing future by way of elegant nachos. A moment of silence for Ken Olsen, Bryan boasts he can take Watson in a fistfight, David buys a Blackberry and Jason threatens to pivot the company.</p>
<p>If you&#8217;d like to subscribe, <a href="http://feeds.feedburner.com/pspipegrep">here&#8217;s the feed</a>.  Otherwise, here’s <a href="http://serve.castfire.com/audio/517433/517433_2011-02-23-024533.default.mp3">a direct link to the mp3</a>.</p>
<p>David’s enthusiasm for Kanye West’s My Beautiful Dark Twisted Fantasy is, apparently, indefatigable.  For reference, that&#8217;s <a href="http://gilscottheron.net/">Gil Scott-Heron</a>&#8216;s voice you hear in <a href="http://www.youtube.com/watch?v=n2Wsy8jHPk4">Who Will Survive in America</a>.</p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/joyeur.wordpress.com/1833/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/joyeur.wordpress.com/1833/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/joyeur.wordpress.com/1833/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/joyeur.wordpress.com/1833/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/joyeur.wordpress.com/1833/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/joyeur.wordpress.com/1833/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/joyeur.wordpress.com/1833/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/joyeur.wordpress.com/1833/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/joyeur.wordpress.com/1833/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/joyeur.wordpress.com/1833/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/joyeur.wordpress.com/1833/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/joyeur.wordpress.com/1833/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/joyeur.wordpress.com/1833/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/joyeur.wordpress.com/1833/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=1833&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://joyeur.com/2011/02/22/ps-pipe-grep-episode-24-reverse-groupon/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
<enclosure url="http://serve.castfire.com/audio/517433/517433_2011-02-23-024533.default.mp3" length="34473607" type="audio/mpeg" />
	
		<media:content url="http://1.gravatar.com/avatar/7b38f1442adc67f3ffa9f20034cefc47?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">jacksonwest</media:title>
		</media:content>

		<media:content url="http://joyeur.files.wordpress.com/2010/12/pspipegrep_logo.png?w=150" medium="image">
			<media:title type="html">pspipegrep_logo</media:title>
		</media:content>
	</item>
		<item>
		<title>Top 10 Tips for Social Gaming Success</title>
		<link>http://joyeur.com/2011/02/17/top-10-tips-for-social-gaming-success/</link>
		<comments>http://joyeur.com/2011/02/17/top-10-tips-for-social-gaming-success/#comments</comments>
		<pubDate>Thu, 17 Feb 2011 19:00:25 +0000</pubDate>
		<dc:creator>Jackson West</dc:creator>
				<category><![CDATA[Games]]></category>
		<category><![CDATA[General]]></category>

		<guid isPermaLink="false">http://joyeur.com/?p=1802</guid>
		<description><![CDATA[By Carly Guarcello After working with hundreds of companies, including some of the largest social gaming firms, we’ve identified the top 10 essential tips to maximize your game’s success. So while you are brainstorming for the next big hit, keep these in mind. 10. Know your competition Do your research. See what’s out there. It’s [...]<img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=1802&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></description>
			<content:encoded><![CDATA[<p><img src="http://joyeur.files.wordpress.com/2011/02/2327243497_b0b6baede9_b.jpg?w=500&#038;h=295" alt="" title="2327243497_b0b6baede9_b" width="500" height="295" class="aligncenter size-full wp-image-1803" /></p>
<p><em>By Carly Guarcello</em></p>
<p>After working with hundreds of companies, including some of the largest <a href="http://joyeur.com/2011/02/08/joyent-powers-up-kingdoms-of-camelot/">social</a> <a href="http://joyeur.com/2011/02/14/joyent-powers-up-country-life/">gaming</a> <a href="http://joyeur.com/2011/02/07/joyent-powers-up-ufc-undisputed-fight-nation/">firms</a>, we’ve identified the top 10 essential tips to maximize your game’s success. So while you are brainstorming for the next big hit, keep these in mind.</p>
<p><strong>10. Know your competition</strong><br />
Do your research. See what’s out there. It’s likely that your first few ideas, or something similar, have already been done. But, there is always room for a new twist. What’s important is how you differentiate your game from the rest. That is what will make it succeed.</p>
<p><strong>9. Identify and set your goals</strong><br />
Whether you’re going for profit or for number of users, have numbers in mind to help you establish a timeline for your game’s success.</p>
<p><span id="more-1802"></span></p>
<p><strong>8. Pick a monetization strategy</strong><br />
With a monetization strategy in place you can calculate the usage required to achieve success. You will be able to maximize the return and meet the goals you have set for your game.</p>
<p><strong>7. Marketability</strong><br />
Create a strategy and tactical plan that you can implement to promote your game and attract as many players as possible. Your plan needs to include advertising, news mentions, and a presence on one or more social networks since that is where your audience will be found. So make sure you get the word out and make your game “The Game” that users want to play.</p>
<p><strong>6. Blazing speed</strong><br />
Your game has to have phenomenal performance and no latency. The fastest way to lose players is with a game that takes five minutes to load or freezes in the middle of an exciting moment.</p>
<p><strong>5. Inviting look and feel</strong><br />
The game experience should be even better than described in components of your promotion plan. Keep text to a minimum. Players won’t read it because they want to play the game, not read a novel.</p>
<p><strong>4. Achieve success on one platform and then, expand to others</strong><br />
Although you want to reach as many potential players as possible, the most successful games focus on one platform to start and then expand to others. By getting the kinks out of game logistics on one platform you will be able handle issues cost-effectively. Your focus will create a great game experience and you’ll be able to grow a loyal, rabid fan base. Expanding to additional platforms and devices should be your end goal.</p>
<p><strong>3. Provide great player rewards</strong><br />
Your game has to be rewarding in some way. Rewards can be emotional, social, or financial. The rewards have to compel gamers to keep playing. Be sure to monitor key metrics including duration and frequency of play. Understanding these metrics will help you keep engagement and retention rates high.</p>
<p><strong>2. Cultivate all players</strong><br />
Successful games welcome new players as well as nurture existing players. Social media platforms should play a large role in player cultivation. The most successful games have consistent player engagement both in the game and in discussions about the game. </p>
<p><em>And the number one success tip…</em></p>
<p><strong>1. Keep the space between your ears open</strong><br />
After you release your game, listen to the feedback from players. Not only will you get some great ideas for enhancements, but you’ll grow player loyalty.</p>
<p><em>Photo by <a href="http://www.flickr.com/photos/8136496@N05/2327243497/">Flickr user terren in Virginia</a>.</em></p>
<br />  <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gocomments/joyeur.wordpress.com/1802/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/comments/joyeur.wordpress.com/1802/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godelicious/joyeur.wordpress.com/1802/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/delicious/joyeur.wordpress.com/1802/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gofacebook/joyeur.wordpress.com/1802/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/facebook/joyeur.wordpress.com/1802/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gotwitter/joyeur.wordpress.com/1802/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/twitter/joyeur.wordpress.com/1802/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/gostumble/joyeur.wordpress.com/1802/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/stumble/joyeur.wordpress.com/1802/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/godigg/joyeur.wordpress.com/1802/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/digg/joyeur.wordpress.com/1802/" /></a> <a rel="nofollow" href="http://feeds.wordpress.com/1.0/goreddit/joyeur.wordpress.com/1802/"><img alt="" border="0" src="http://feeds.wordpress.com/1.0/reddit/joyeur.wordpress.com/1802/" /></a> <img alt="" border="0" src="http://stats.wordpress.com/b.gif?host=joyeur.com&amp;blog=13003674&amp;post=1802&amp;subd=joyeur&amp;ref=&amp;feed=1" width="1" height="1" />]]></content:encoded>
			<wfw:commentRss>http://joyeur.com/2011/02/17/top-10-tips-for-social-gaming-success/feed/</wfw:commentRss>
		<slash:comments>1</slash:comments>
	
		<media:content url="http://1.gravatar.com/avatar/7b38f1442adc67f3ffa9f20034cefc47?s=96&#38;d=identicon&#38;r=G" medium="image">
			<media:title type="html">jacksonwest</media:title>
		</media:content>

		<media:content url="http://joyeur.files.wordpress.com/2011/02/2327243497_b0b6baede9_b.jpg" medium="image">
			<media:title type="html">2327243497_b0b6baede9_b</media:title>
		</media:content>
	</item>
	</channel>
</rss>
