Filed under: Analysis / Opinion, Enterprise, Rumors, Apple
Rumor: Apple Enterprise Products to use ZFS
In the last 12 months, the storage demand at my workplace (a university of 10,000 students) has risen exponentially. The 2.4TB NAS purchased last summer was outstripped by the end of our second semester just a few weeks ago. We'll recover a lot of that space by deep-sixing unused and stale accounts--which we can do every semester--but this does not address the growing problem of long-term storage and archival of data generated by our students, faculty, and staff. Eventually, we'll need to figure out a way to keep some--if not most--of this data indefinitely. The good news is that storage costs continue to decline--one terabyte of data storage is about $1500-$2000 right now. The bad news is that managing these massive amounts of data only continues to get more and more complex.Microsoft's answer to this problem is WinFS, a new filesystem and storage manager that was to be included in Vista. WinFS would be the solution to some of these storage problems by providing a scalable filesystem built on top of a relational database. WinFS would have allowed for metadata tagging, datastore consolidation and sophisticated backup/restore, notifications, and access rules (ACL's). It's exactly what the Enterprise market needs right now, a sophisticated and stable solution for this growing problem. The problem is that Microsoft has yanked WinFS from Vista and will only be using pieces of it in Vista's Server version. Vista will run on the increasing old and creaky NTFS.
Some industry experts are suggesting that Apple may include the open source ZFS file system/content manager in their upcoming Leopard Server. ZFS was produced at Sun Microsystems about two years ago and was recently integrated into the most recent version of Solaris 10. An employee at Sun has posted to the Mac OS X Server mailing list that Apple may be interested in porting Mac OS X to run on ZFS.
If it's true that Apple may build Leopard Server to run on top of ZFS (instead of HFS+), it would place them in a strong position in the Enterprise market. Apple's X-Serve, X-RAID, and X-SAN are already some of the most affordable enterprise-level storage products on the market. Having a new, fast, scalable, reliable file system and content management system combined with Apple's traditionally easy-to-use admin tools might give Apple a powerful push into the coveted Enterprise storage market.
![TUAW [Cafepress]](http://www.blogsmithmedia.com/www.tuaw.com/media/tuaw-cafepress-promo.png)
Reader Comments (Page 1 of 1)
eM said 5:13PM on 6-27-2006
A few points:
A) WinFS is/was not a file system in the standard sense (the FS stands for Future Storage). It's a database system that sits on top of a regular file system. WinFS was never meant to replace NTFS -- it was meant to run on top of it.
B) ACLs have been in NTFS for years. The primary benefit of WinFS would have been the ability of software developers to use an SQL-based API for file operations.
C) It's non-sensical to compare WinFS and ZFS, as noted above. That said, ZFS is a very nice file system (the nicest I can think of, in fact). However, I can't see Apple using it in only OS X Server and leaving HFS+ on the desktop. If Apple does the work of porting ZFS, why would they possibly leave the severely dated HFS+ on consumer machines?
All in all, I think that the whole affair is pretty baseless speculation. As with kernel replacement, it would be a nice step, but I'm not holding my breath.
Reply
Allan White said 5:27PM on 6-27-2006
I think eM's comments are accurate. ZFS would represent a huge leap forward in filesystems, but I can't imagine the complexity of replacing the filesystem.
Xsan is affordable - until you realize how many Xserves you need to run it properly (3 minimum, I think, for video servers like I'm spec'ing now). Ouch.
I could imagine a scenario where OS X Server could use it, with it supported on the client. However, I doubt you'd get many of the benefits, so it would only be optimal if used in both places.
Check back on this topic in, like, 2 years.
Reply
Ben McElroy said 5:29PM on 6-27-2006
And now WinFS is dead. See Microsoft for the details. Just happened in the last couple of days. A real shame too because that was, to me, the most interesting aspect of Windows Vista.
Reply
switchtoamac said 5:34PM on 6-27-2006
I guess we'll know soon enough as WWDC is coming up in August.
Reply
John C. Randolph said 8:42PM on 6-27-2006
I wouldn't really describe OS X as "running on top of" any particular filesystem. You can install it on an HFS+ volume or a UFS volume today; ZFS would be just like UFS from that standpoint.
-jcr
Reply
Ian Z McFaul said 9:51PM on 6-27-2006
I just wanted to add that the article also seems a bit less credible and authoritative when the author doesn't use the correct product names: Xserve, Xserve RAID, and Xsan.
Reply
Michael Maggard said 10:04PM on 6-27-2006
WinFS has/had little bearing on your resource management interest, it is/was a metadata management layer on top of NTFS. It is/was more in line with Spotlight then a filesytem,
As to NTFS being "creaky", it is no more so then many other filesystems. Yes, there are always exciting newer ones coming along; ext3, Rieser, ZFS, etc. but taking advantage of them throughout the OS can be complex and time-consuming and any benefit must be weighed against the transition costs.
The same couild be said for HFS+; Apple is still plumbing in support for HFS+ five years after introducing MacOS X. Adopting ZFS would require a significent rengineering, requiring drawing resources away from other projects like kernel development, a rewritten Finder, etc.
Is it possible Apple is interested in ZFS? Sure. They keep an eye on all sorts of intersting developments. Is anyone seriously expecting their adoption of it? Not particularly.
At some point Apple is likely to move beyond HFS+ to another file system. However my own guess is that won't be until they've gotten some other rough edges polished first and then they'll be adopting a substantially different architecture, not just replacing their filesystem format.
Reply
stingerman said 3:26AM on 6-28-2006
OS X is not dependent on any file system, you could use Fat32 with OS X or NFS. You just need to provide the proper file system plugin. So, adding Sun's system would be just as straightforward. XSAN is a new file system as well.
Reply
Gary Rogers said 8:17AM on 6-28-2006
Has everyone forgotten Apple's hiring the mind behind BeFS? Now, I'll say ZFS would be a coup for Apple in conjunction with it's XSan and XRaid products, but I just don't see it sitting under the OS. Under NAS devices? Sure. It could potentially give Apple another feather in it's cap when going toe-to-toe with EMC, HP, and the like in the mid-sized storage market.
Reply
Jeff said 9:25AM on 6-28-2006
Nice... MacRumors had this posted in April http://www.macrumors.com/pages/2006/04/20060430200158.shtml
Reply
andrew said 9:54AM on 6-28-2006
Not sure what the hubbub is about apple having to "rewrite" a large portion of OS X to support ZFS. From Wikipedia:
"ZFS appears to applications as a standard POSIX file system; no application changes are needed to store data in ZFS."
More interestingly, according to quantum mechanics, it would require more energy to fill the massive capacity of a ZFS filesystem then it would to boil the oceans of the world (assuming 100% efficency, which today's computers can't even touch). Likewise, an ideal quantum storage device would have to weigh 136 billion kilograms to hold it (again, it would be exponentially higher in reality).
"Although we'd all like Moore's Law to continue forever, quantum mechanics imposes some fundamental limits on the computation rate and information capacity of any physical device. In particular, it has been shown that 1 kilogram of matter confined to 1 liter of space can perform at most 1051 operations per second on at most 1031 bits of information [see Seth Lloyd, "Ultimate physical limits to computation." Nature 406, 1047-1054 (2000)]. A fully-populated 128-bit storage pool would contain 2128 blocks (nibbles) = 2137 bytes = 2140 bits; therefore the minimum mass required to hold the bits would be (2140 bits) / (1031 bits/kg) = 136 billion kg.
To operate at the 1031 bits/kg limit, however, the entire mass of the computer must be in the form of pure energy. By E=mc2, the rest energy of 136 billion kg is 1.2x1028 J. The mass of the oceans is about 1.4x1021 kg. It takes about 4,000 J to raise the temperature of 1 kg of water by 1 degree Celsius, and thus about 400,000 J to heat 1 kg of water from freezing to boiling. The latent heat of vaporization adds another 2 million J/kg. Thus the energy required to boil the oceans is about 2.4x106 J/kg * 1.4x1021 kg = 3.4x1027 J. Thus, fully populating a 128-bit storage pool would, literally, require more energy than boiling the oceans."
Reply
Robin Harris said 9:57PM on 6-28-2006
AFAIK, Apple's storage products have only found favor in non-Enterprise apps, like media production and HPC. I've watched Apple storage for many years. For a long time they only did disk quals and not too many of those. Their current products are all low-investment projects. So ZFS, which eliminates the need for RAID controllers and a lot of storage management BS, would fit well with that philosophy. And provable end-to-end data storage integrity would get favorable attention from folks who now dismiss OS X as a toy.
HFS+ is cow dung. If Apple is serious about enabling consumers to use TB's of data - coming soon to a home near you! - they need a modern FS. Maybe ZFS isn't their choice, but they have to do *something*.
Reply
Robin Harris said 10:00PM on 9-16-2006
Now we know why Apple is working on ZFS: iTV. How else are you going to get people to spend hundreds or thousands of dollars on digital files that they can't backup? You have to have reliable, redundant storage. See the story http://storagemojo.com/?p=248 Means, Motive & Opportunity: Apple Kills the Media Center PC.
Oh, and they are working on it: there is a ZFS vnode in the current Leopard developer's release.
Reply