{"id":92053,"date":"2024-04-30T17:30:00","date_gmt":"2024-04-30T22:30:00","guid":{"rendered":"https:\/\/eshop.macsales.com\/blog\/?p=92053"},"modified":"2024-06-17T15:40:12","modified_gmt":"2024-06-17T20:40:12","slug":"how-to-setup-a-high-performance-storage-workflow-for-photography-and-video","status":"publish","type":"post","link":"https:\/\/eshop.macsales.com\/blog\/92053-how-to-setup-a-high-performance-storage-workflow-for-photography-and-video\/","title":{"rendered":"How to Setup a High Performance Storage Workflow for Photography and Video"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

In my last post, I discussed the pitfalls of cloud storage for photo libraries and how the best, most reliable method for organizing your library for future reference is trusting in tried-and-true files and folders<\/a>. But if you’re not depending on a cloud service and\/or app like Apple Photos for your organization and storage, what is the best method for reliably storing photography and video captures, alongside a high performance workflow?<\/p>\n\n\n\n

As a medium format<\/a> photographer or even \u201conly\u201d 35mm<\/a>, you might face storage challenges similar to mine. The size of my image library keeps growing. Single-shot RAW file captures with my 100MP Fujifilm GFX100 II<\/a> are up to 130MB each with a single focus stacked image<\/a> requiring from 3 to 30 frames. A single<\/em> composition might require 0.5 to 4GB of captures, plus the finished stacked image. Or large volume for other reasons, which lands you in the same place.<\/p>\n\n\n\n

Two key considerations stand out: (1) fast storage for performance-sensitive tasks, and (2) high-capacity overall storage. Sometimes these two can be the same thing, but often cost reasons preclude that due to the sheer volume of storage required\u2014SSDs are still far more expensive that hard drives on a per-terabyte basis.<\/p>\n\n\n\n

I covered my recommended backup method in a previous post<\/a>, but in a nutshell: (1) backup every day<\/strong> at least once to at least one always-attached backup. (2) maintain at least three additional \u201coffsite\u201d backups<\/strong> stored safely away from the computer, never having those backups all together with the computer at once, (3) consider cloud-based backup as an adjunct<\/em> <\/strong>if internet bandwidth makes it viable.<\/p>\n\n\n\n

The discussion here is based on what works: for 15 years or so I have consulted<\/a> for nearly a thousand clients about choosing and configuring a system for photography. You do not need technical knowledge to set up or use an outstanding system. What I cover here is the right solution for almost everyone.<\/p>\n\n\n\n

Primary storage vs. backup, ideal system<\/h2>\n\n\n\n

\u201c<\/strong>Primary storage<\/strong>\u201d as used here means the original master copies of all your stuff<\/strong>. As compared to backups, which are copies<\/em> of that primary storage. See Backup Best Practices: How to Make Fast Clone Backups on a Mac<\/em><\/a>.<\/p>\n\n\n\n

In my case, I have not graduated to a single large SSD, so I use two 8TB and one 16TB SSD. I would prefer a single 32TB SSD of course! But things evolve over time, and also cost money.<\/p>\n\n\n\n

\"\"
Primary Storage Volumes \u2014 one 32TB SSD would be nicer<\/figcaption><\/figure><\/div>\n\n\n\n

The ideal<\/em> primary storage solution is a very large SSD e.g. a 16TB or 32TB OWC Thunderblade X8<\/a> (see pricing<\/a>). But since that can be cost prohibitive, the next-best thing is a two-tier SSD + hard drive system<\/strong>. <\/p>\n\n\n\n

Don\u2019t Starve the GPU\/GPU <\/h2>\n\n\n\n

A \u201cbalanced\u201d system needs appropriate CPU and GPU processing power and memory, but without a fast SSD to deliver data, much of that potential is idled while the processing units wait for data. See The Ultimate Mac Buyer\u2019s Guide Parts 1\/2\/3\/4\/5\/6<\/em><\/a>.<\/p>\n\n\n\n

A prime example where SSD speed matters is the Adobe Lightroom<\/a> catalog. Any operation such as accessing previews or using brushes or controls, setting metadata on many files, etc., tends to become one where response time matters, particularly as the catalog grows. Those interactive<\/strong> operations directly impact productivity whereas batch<\/strong> operations might or might not matter to you, since you can walk away for a break.<\/p>\n\n\n\n

Drive speed starvation vs. ample<\/h3>\n\n\n\n

While the Lightroom catalog itself demands a fast SSD, RAW files stored on a reasonably fast hard drive have until recently<\/strong> had minimal impact on performance: for most operations, the time required to process<\/em> the RAW file (eg. to generate a preview, export as JPG, etc.) had been much longer than the time needed to read it<\/em> off disk. This is no longer true.<\/p>\n\n\n\n

CPU and GPU speeds have greatly improved, particularly in chips like the 24-core Apple M2 Ultra in my 2023 Mac Pro <\/a>and Mac Studio<\/a>. The M3 Max chip is also blazingly fast<\/a>. On my system, reading and converting and saving a 100-megapixel RAW file as JPG takes only 0.65 seconds<\/a>.<\/p>\n\n\n\n

If it takes only 2\/3 second to process a 100MP RAW file, but the hard drive takes 1\/2 second just to read the file<\/em> (let alone write the JPG), the workflow is seriously degraded. And it is not just one CPU in use; there may be 10 or 14 or 24 CPU cores that all need data simultaneously<\/strong> e.g. bandwidth requirements approaching 3000 MB\/sec \u2014 vs. 250MB\/sec or so best case for a hard drive. A hard drive system becomes a severe bottleneck<\/strong>. Even a 4-drive RAID-0 stripe delivering 900MB\/sec is a bottleneck. <\/p>\n\n\n\n

A workflow involving hard drive storage is a recipe for stuttering performance<\/strong>. <\/p>\n\n\n\n

A few years ago, it might have taken 5 to 10 seconds to render a large RAW file. Now it might only take a second or so. That changes the drive speed equation, depending on the workflow tasks. Hard drives have made only marginal speed gains over the past decade, making them an increasingly poor choice for high-performance computers when speed of workflow matters. Even a non-performance oriented SSD is now 4X the speed of the fastest hard drive.<\/p>\n\n\n\n

Big SSD vs. 2-tier storage with migration<\/h2>\n\n\n\n

Before SSDs became so fast and so capacious, I recommended a hard drive RAID-0 or RAID-5 system for performance. Here in 2024, that\u2019s a losing proposition; you are far<\/em> better off with a fast SSD. And even a compact portable SSD like the OWC Envoy Pro Elektron (now available in up to 4TB)<\/a> can easily outperform a 4-drive RAID-0 stripe of hard drives<\/a>.<\/p>\n\n\n\n

One large SSD (or two) are the ideal choice for primary storage. For those with very large capacity requirements, that might not fly for cost reasons. Thankfully, a 2-tier approach can get the job done just as well. Here’s how:<\/p>\n\n\n\n