Talking storage systems with Sun’s ZFS team
The inventors of Sun Microsystems’ ZFS, Jeff Bonwick, distinguished engineer and storage CTO, and Bill Moore, hardware/software architect, tell me what was behind coming up with ZFS. It’s a file system. That sounds boring, right? But it’s not, since this technology is used in many of the world’s datacenters to keep files safely stored even when bad things happen. The conversation takes us all over the place, since these guys are experts on storage systems. Hard drives and all that. What does ZFS stand for? Zettabyte File System.
Tags: Sun Microsystems, ZFS, Jeff Bonwick, Bill Moore, Zettabyte
September 5th, 2007 at 4:34 pm
[…] A conversation with the inventors of Sun’s ZFS Smart dudes. 45 minutes. Talking about Sun Microsystems’ ZFS. File storage system that’s used on a ton of datacenter computers and is rumored to be used in a future version of OSX. […]
September 5th, 2007 at 7:42 pm
More smart guys please Scoble.
September 6th, 2007 at 12:43 am
Please can you do something about the sound levels on these. The start is loud but the actual interview is to low
Thanks
R Walker
September 6th, 2007 at 3:39 pm
Excellent Interview! This is the sort of content you can’t find anywhere else. I agree with Mr. Overend. More “smart folks” plz.
September 16th, 2007 at 8:18 am
[…] Smart dudes. 45 minutes. Talking about Sun Microsystems’ ZFS. File storage system that’s used on a ton of datacenter computers and is rumored to be used in a future version of OSX. […]
September 20th, 2007 at 12:19 am
[…] http://www.sun.com/software/solaris/zfs_learning_center.jsp […]
October 7th, 2007 at 9:04 pm
[…] Going Bollywood I just got back from a series of customer meetings with technologists from the telecommunications, media and entertainment industry (they are, after all, converging on the same market, monetizing consumers). I was joined by Greg, and a number of folks from Sun, notably Jeff Bonwick and Bill Moore, the two co-inventors of the ZFS file system. All three industries are largely underserved by innovation - their requirements and opportunities are vastly outstripping the rate at which the industry is innovating. (That’s code for, “their technology budgets are growing.”) There were a number of choice anecdotes from the sessions, my two favorites being the following. The first was relayed by the CTO of a major movie studio, who explained the value of very, very long term deep storage archives. His company had recently pulled a more than fifty year old movie out of a salt mine - where it was stored on 35mm film color separations. Before you ask, vinyl film outlasts the standards that the industry produces (a point on which I’ve written previously), and salt mines are environmentally more stable than datacenters). They pulled the separations to remaster and reissue the film on DVD. In so doing, the resulting movie actually increased in resolution for the viewer - the devices used to display the movie today (likely a laptop or high definition TV) offer better resolution than what was available to the original viewers (likely a 50’s era TV or movie theater). The movie improved with time - the modern viewer saw the movie in greater detail than the original viewers (suggesting one should always store higher resolution data than the display you see in front of you). After release of the digitally remastered version, the DVD rose to number 8 on the Amazon best seller list. Cost of production? Near $0. The effort was nearly pure profit. And they have a library of approximately 30,000 films. That’s a ton of value in a salt mine (if all the titles are as compelling, which is unlikely, but an interesting thought exercise, nonetheless). The second anecdote relates to the advent of very high resolution digital cameras - the highest, and most desirable are currently known as “4K,” offering 4096 x 3112 pixels (!) per frame - yielding cameras that spew 100’s of megabytes of imagery per second). The direcctor of one feature length film wanted to keep all the footage from a soon to be introduced new movie. He wanted to preserve outtakes and all, for the eventual “Behind the Making of…” or “Director’s Cut…” versions of the movie. The digital master for an average 4k film is roughly 9 Terabytes - that’s for the version you and I would see in a theater. But the total archive including out takes and secondary/tertiary angles (bits are a lot cheaper than film, so why not set up three or four cameras for every shot?) was roughly (drum roll please)… a PETABYTE (or a thousand terabytes, or roughly 500,000 iPods). Equivalent to about a million feet of 35mm film. That’s a lot of data. To be archived for… well, to the first anecdote, likely forever (like health records or airport surveillance). And now you know one of the (many) principle motivations behind a file system we built at Sun, ZFS. The focus of the ZFS team is both the scale, simplicity and quality of storage (on Mac OS X, BSD, Solaris and Linux). This is a great overview of ZFS by Jeff and Bill - of why ZFS matters to the media and entertainment industry, and likely anyone concerned with high quality, high scale and high productivity, storage. As I was told by Jeff (parroting a storage executive), there are only two types of disk drives in the industry. Drives that have failed, and drives that are about to fail. And now you know what inspired ZFS, and what’s inspiring interest in it. […]
October 8th, 2007 at 8:17 pm
[…] Talking storage systems with Sun’s ZFS team (tags: storage sun zfs) October 08th 2007 Posted to Links […]
October 9th, 2007 at 1:32 pm
Great segment Robert. Low level software has always been my passion (I work in the embedded firmware world) so it was fascinating to listen to a couple of very articulate engineers who are clearly as excited about that world as I am.
Sun as a company works on some stuff, like dtrace, that I find really cool. Hopefully you’ll continue to talk to them!
October 15th, 2007 at 8:33 am
So what’s this I hear in the interview about “CIFS support in the works?” I’m all ears, because we want to do ILM and disk pooling (SAM & ZFS) but our primary clients will be Mac and Windows systems via SMB (CIFS).
October 22nd, 2007 at 3:33 pm
[…] http://www.podtech.net/scobleshow/technology/1619/talking-storage-systems-with-suns-zfs-team# […]
October 30th, 2007 at 11:03 am
I bet this gets asked a lot, but Google didn’t help me this time. Is there an audio-only download option? Or some legit “hack” to extract the audio track from one of the video file formats? Thanks.
December 19th, 2007 at 8:52 am
The volume increases at around the 11 minute mark.