[Update 03/27] iOS 10.3 Update with APFS Now Available to public
Without a file system, your Mac, iPhone, iPad or even your Apple Watch would not work. A file system is used by an operating system like macOS and iOS to read from and write to some sort of storage, whether it’s a huge hard disk drive or a speedy solid-state drive (SSD). All Macs currently use a file system called HFS+ (Hierarchical File System Plus), an outgrowth of the original Hierarchical File System that was introduced in 1985 with Mac System 2.1. With the roots of the file system now over 30 years old, Apple just announced a new file system called APFS (Apple File System) that will be shipped on all new Apple devices starting in 2017. As new versions of Apple operating systems roll out later in 2016 — macOS Sierra, iOS 10, tvOS 10 and watchOS 3 — existing devices will use APFS. (Note: You can read Apple’s APFS documentation here and watch an introduction video here.)
The newly re-named macOS Sierra will give Mac users their first look at APFS soon, but if you are planning to be part of the public beta test for the new operating system, there are some things you’ll want to look out for. APFS volumes currently cannot be used as startup disks, Time Machine volumes, or even as part of a Fusion Drive. File Vault encryption is out of the question since APFS has its own encryption scheme built-in. There are also issues dealing with the current, early version of APFS as it is case-sensitive. HFS+ is unlike many Unix-based file systems in that it can be configured to be case-insensitive, and running OS X with case-sensitive HFS+ has been known to cause issues. Our recommendation? Only try APFS from a beta version of macOS Sierra on an external volume that doesn’t have critical information on it.
Related: See the Rocket Yard’s complete WWDC 2016 coverage here
APFS is what’s known as a modern file system, containing features that will make it usable on every device that Apple makes from the Apple Watch to the Mac Pro. Currently, iOS, watchOS and tvOS use HFS+ Case Sensitive (or what’s called HFSX) as a file system, but they’ll be using APFS as well. Let’s take a look at some of the new, modern features of APFS.
64-bit node number support
An inode is a data structure used to represent a file system object like a file or a directory, with each inode storing the attributes and disk block location(s) of the file system object’s data. These attributes can include object manipulation metadata like change, access or modify time, and owner and permission data. 64 bits can represent up to 18,446,744,073,709,551,615 objects, so Apple’s definitely looking at the future of big storage with APFS. By comparison, HFS+ can only address up to 4,294,967,295 objects.
Support for nanosecond time stamp granularity
File systems create a log of writes. HFS+ timestamps are only accurate to one second; that’s known as the granularity of the timestamps. APFS can track those timestamps with a granularity of one nanosecond — that’s one billionth of a second. This results in much better recovery from power-loss situations.
Crash protection
While APFS is writing data to a storage device, it makes a copy of metadata that describes the attributes of a file or folder. In the case of a power loss or severe system crash while the file system is writing to storage, crash protection ensures that what APFS writes to storage and writes to the the file system journal are completely synchronized.
Sparse file support
Sparse files are not supported by HFS+, and that’s not good. A sparse file is a computer file that tries to use file system space more efficiently when a file is mostly empty. With sparse files, storage is only allocated when needed, so large files can be created even if there’s insufficient free space available. APFS, like other modern file systems (many forms of Unix and NTFS), supports sparse files.
Trim support
Trim is used by an operating system to tell an SSD which blocks of data aren’t in use and can be wiped internally. OS X originally supported Trim for Apple SSDs, and third-party SSDs often needed special utilities. The combination of macOS Sierra and APFS will support Trim for all SSDs. (Learn more: With An OWC SSD, There’s No Need For TRIM)
Sharing APFS volumes on a network requires use of SMB
Since way back in the Mac OS System 6 days, Apple has used a proprietary network protocol for sharing files over networks — Apple File Protocol or AFP. OS X supports a variety of file sharing network protocols in addition to AFP, including SMB (Server Message Block), NFS (Network File System), FTP and WebDAV. APFS volumes shared on a network will need to use the SMB protocol.
Encryption improvements
Mac encryption can be done several ways right now; through File Vault for full-disk encryption or through third-party applications for per-file encryption. APFS allows native encryption scalable to individual files depending on the level of security you desire. A file or directory can have no encryption, single-key encryption, and “multi-key encryption with per-key files and another key for sensitive metadata”. Since all Apple devices — not just the Mac — will base storage of data on APFS in the future, they’ll all have the option of providing encryption to files.
Fast directory sizing
Back in the days of slower (PowerPC-based) Macs, getting a read on the amount of available disk space could take a while; doing a “Get Info” on a volume could take minutes while the Mac did stat calls to get an answer on just how much space was being used. Faster Macs have made that wait a lot shorter (seconds with a fast Mac), but APFS fast directory sizing will be almost instantaneous. The new file system gives macOS a method of querying the size of a directory and its child objects very quickly.
Snapshots
APFS can be asked to create a read-only instance of the file system — a snapshot — then add changed blocks to the snapshot as the file system’s state diverges from the original snapshot. What’s interesting is that this may result in a completely new way for how Time Machine will work. Currently, Apple’s OS X backup scheme uses an old-fashioned system of hard links that are built and maintained by Time Machine. A future version of Time Machine — perhaps in macOS Sierra — would just capture snapshots instead.
Clones
While snapshots are read-only, clones are read/write. APFS can create file or directory clones instantly without waiting for data to be copied. A clone of a directory or file stores all changes made between it and the original object, making the clone an editable, writable copy of a directory or file at any point in time. Clones will most likely be used by developers to capture document revisions and do versioning.
Write coalescing
Enterprise disk arrays have been able to do write coalescing for a long time, which gives file systems some leeway in how they commit data to storage. APFS will be able to take a number of short and potentially unrelated writes, then combine them into one long write that uses the physical storage in a much more efficient manner.
Space sharing
One of the more intriguing demonstrations during Tuesday’s APFS presentation was about space sharing, which allows multiple volumes to be created out of the same physical storage space. Rather than pre-sizing a partition before using it, it’s possible to create a volume that grows as space is added to it. Additional physical storage can be added later to keep up with volume growth without having to resize the original logical volume. This may have long-term ramifications for how macOS works with RAID storage.
Other than what we’ve heard about APFS in the WWDC presentation, we don’t know much about how it will affect the world of Mac storage. As APFS development continues alongside the various beta versions of macOS Sierra, we’ll provide more information about the new, modern macOS file system.
[Update 03/27] Have you noticed a difference in performance now that APFS is installed in iOS 10.3? Let us know in the comments.
OMG! “Sparse file support” was a part of PROTOS, an O/S developed by (long deceased) Computer Automation’s – Austin Development Center (way long gone). It was part of the ADC “Purple Arrows” Filesystem project (so-called because that’s how it was drawn on a whiteboard during the design phase.)
The “Unrealized File Regions” were Part 1 of “Purple Arrows”; the other part was “Duplicated Data@Single Physical Rep.” for databases. (Remember, this was the mid-80s when a 300Mb CDC Storage Module drive was MASSIVE!)
This “SFS” filesystem was running production at the CA-ADC facility when CA-ADC was closed: “too expensive…”
What is “empty byte”? Seriously can you use more CS terms to explain a bit more accurately?
What does APFS do that ZFS didn’t do? Nanosecond time stamps maybe. HFS+ is crappy ,but another proprietary file system isn’t the answer.
Actually, you can’t view Apple’s video unless you are on a mac since they no longer offer Safari for Windows
Apple seems basically oblivious to the fact that many professional users (e.g., scientists like me) must (by Federal regulation as well as cultural need) maintain data integrity and readability over decades. I don’t give a toss about their new toy if I can’t continue to access and use the huge data accumulated in the past, spread over many devices. I have been compelled to stick with OS 10.6.8 because the loss of the seemingly trivial scroll arrows is a fatal handicap for really precise graphics work.
APFS will support HFS. It’s written in the docs and was also specifically noted during the presentation at WWDC. How scroll bars are viewed/used is your decision in every version of OS X since 10.6.8. They didn’t go anywhere. It’s a property setting in System Preferences.
Apologies…I mis-read. You stated “scroll arrows” not “scrollbars”; so, never mind. A lot of people want those little arrows back. Don’t see how that impedes graphics work since that’s what I do, but to each his own.
I’m in a similar boat, but in an entirely different ocean Lew. We’re using an old, but brilliantly written OS 9 program, complete with annoying bugs and lack of what some consider necessary features. Why? Because we must maintain not only access, but the ability to edit many thousands of files created over decades. While we do have and enjoy the latest machines for other work we’re forced to keep an inventory of older G4s & G5s with OS 10.4 so we can run the “Classic” emulator. We even use Sheepshaver as sort of a backup on our newer Macs, but it’s really not even close to being an acceptable solution. It’s OK for very simple work but not sufficient for the vast majority of what we need to do.
I’ve tried reaching someone at the company who might be helpful, but just finding someone who might understand what I’m talking about has been impossible.
Apple is also creating a time-bomb in digital images. My children, who grew up in a world of digital obsolescence, are worried that their childhood pix, and the childhood pix of their kids, will vanish into the fog. Apple could actually do a serious and durable business in migration tools and hardware, but shows no understanding of the need.
100% true sir!
How will APFS make digital pictures obsolete?
Curious how this will affect current storage solutions on the market. I am considering purchasing an OWC Thunderbay 4 RAID and SoftRaid. Will it be compatible with the new APFS?
Hi, Mike. Great Question! The OWC ThunderBay 4 RAID and SoftRAID will work 100% with APFS.
Hi Jarrod. Good to hear the OWC ThunderBay 4 Raid and SoftRAID will work with APFS. I have one of these attached to a Mac Mini running OSX Server providing a network Time Machine backup drive for my home systems. Any idea if the Thunderbay and Softraid will support encrypted time machine backups at some point? Will APFS enable this?
Yes, SoftRAID is adding support for APFS in High Sierra. APFS encryption is fully expected to be supported. SoftRAID cannot promise Time Machine encryption yet, it is possible there may be an Apple related Time machine bug preventing it from working on SoftRAID volumes.
A public beta of SoftRAID for High Sierra should be available in the next couple weeks, we will post in the Rocket Yard when it is announced.
Any word on the soft raid beta to support AFPS on Thunderbay 4?, can I expect an increase in copy/read performance?.. FYI the disk utility has a “Convert to AFPS” tab for external drives.
It’s a bit concerning that Apple decided to checksum its own metadata but not user data. Apple engineers feel that Apple devices basically don’t return bogus data…***
huh? That’s contradictory. By checksumming metadata implies that Apple devices can return incorrect data to the OS. Some very strange decision making going on here…
Cloning is a concern to me, if the blocks representing multiple clones become corrupt, am I to assume that all clones are now lost as well?
Apple announced that they were to use ZFS back when 10.5 was in beta, but then Oracle came along and bought out Sun Microsystems who developed/open-sourced ZFS, so I’m glad to see that Apple have been working towards replacing HFS+ during this time.
I’ll be staying with ZFS on OSX for my data but do welcome the change.
*** Source:
http://dtrace.org/blogs/ahl/2016/06/19/apfs-part1/
Will NVMe PCIe be a part of this system? We have been incorporating that into our Linux systems were I work. It is unbelieveably fast!
“where I work”
Apple incorporates the NVMe host controller interface in their newest devices (e.g. iPhone 6S & MacBook Pro) right now, and the SSD storage on MacBook Pro models is attached via PCIe x4 v3.0 physically. So APFS will support NVMe.
ZFS?
How reasonable is it to assume some backward capability with APFSsuch that the 70k+ image files I have stored on a Drobo will still readable and rewritable? An original Drobo/Droboshare is attached already via SMB; the new Drobo N attaches automatically, so how I’ll macOS and APFS deal with my two NASs (which are attached via Ethernet to an OWC Thunderbolt box connected to a late 2011 MacBook Pro via FireWire).
If your drobos are connected via smb over the network, there’s no issue, APFS would be your local filesystem, and honey badger don’t care what FS your NAS uses.
From experience. I know that your greater risk is a volume failure on the Drobo drive. You do have more than one backup don’t you? Wish I did. It would have saved me from having to recreate all the critical stuff.
Will upgrading to Sierra require reformatting and clean install to convert disks to APFS? What backwards compatibility will be available to HFS+ (or FAT32) disks?
Hi, Chris –
At this time I haven’t seen any word on whether or not Sierra would require reformatting and a clean install. But I seriously doubt if Apple would require users – especially those who don’t have the tech ability to do a reformat/clean install – to do this just to be able to upgrade to Sierra. One possibility is that just new devices shipped in 2017 will come with storage set up as APFS and that existing devices will use HFS+ with Sierra.
I’m hoping that there’s some sort of “convert in place” capability that will happen during the upgrade to Sierra. Since at this time, macOS Sierra only works with APFS on an external, non-startup disk, we won’t know for a while.
Steve
According to Apple, converting HFS+ to APFS will be nondestructive. However it will not be possible to return an APFS volume to HFS+ without erasing the contents of the volume.
In the WWDC session about this it was mentioned that there will be an in-place conversion. The data will stay in place while the APFS metadata is created. This is done so it is an atomic operation, should a crash occur along the way your data is safe. Then the final conversion, whatever is needed, will be done. So, in-plalce, no backup/reformat/restore needed.
Also, they said it is coming in 2017, not 2016. Whether that means early in 2017 or in December and in the new macOS AFTER Sierra…. they didn’t say. :)