2020 is shaping up to be an eventful year, and we’ve been hard at work in 2020 on new features for Halide, Spectre, and beyond.

Today we’re breaking our silence with our latest update: here’s Halide 1.16, a ‘groundwork’ update with several features that improve everything about RAW — it improves how Halide displays your RAW captures, how it saves RAW (and regular compressed) files, improves Halide’s file output, and makes your life a bit better. On top of that, we’ve improved Halide’s performance even more.

Let’s start with a feature that merits its own description: Rescue.

Rescue

There’s no worse of feeling than going into your camera roll after a day of shooting, only to discover that perfect shot is disappeared. This is extremely rare, but not impossible:

• You taking a photo at 5% battery, and suddenly the phone shuts down.
• You get a phone call in the middle of a shoot
• Or, a bug in iOS or (believe it or not) Halide causes an issue

Even it’s a one in a million chance, we cannot accept even the slightest chance of losing a photo. We treat data loss as a critical problem, and that’s why we built Rescue.

In the past, when you took a photo with Halide, we would simplify hand it off to iOS to upload to iCloud. We assumed everything worked as expected, so long as iOS doesn’t hand us back an error. In Halide 1.16 we trust but verify.

Now when you capture a photo, we keep a backup copy around, similar to how your email client might have an “outbox.” We don’t remove that backup copy until we’ve verified for ourselves that the photo exists in your library.

If you ever feel like, “I swore I took more photos,” and you can’t find it in your iCloud library, take a peak inside Image Rescue.

1. Open the photo reviewer.
2. Tap the “Grid View” icon.
3. Tap the “Lifesaver” icon in the upper right.

Rescue is a feature that we’ve put together thanks to feedback from our community, which includes photographers that have pushed iPhones to their limits in extremely remote locations or extreme conditions. We’re excited to release this update to show that Halide is the ultimate photography tool that everyone can truly rely on, even when the going gets tough.

Smart RAW is Smarter

We first introduced our Machine Learning based Smart RAW adjustment last year. We gather data continuously to improve camera settings for better RAW output with more editing headroom and lower noise.

Halide 1.16 packs a nice update to Smart RAW, based on iOS 13 behavior and our own research. You’ll notice better highlight recovery during day scenes.

Here’s a few sample photos which were captured with the parameters that Smart RAW now uses:

• 

• 

• 

• 

Flat RAW Previews

This is a big one.

RAW files deserve several posts of their own to explain just how they work, what the power of RAW is, and why you should — or shouldn’t — shoot in RAW. We did just that (so click here if you’d like to read up!) but it’s still not exactly intuitive.

https://lux.camera/the-power-of-raw-on-iphone-part-2-editing-raw/

Many people just getting started with RAW complain that at a glance, the image looks much worse than a picture taken with the default camera app.

Yikes, what’s happening with the sky? It looks like the highlights are blown out:

So what’s going on? The iOS RAW engine is trying to develop your RAW file for you. You see, a completely unadjusted RAW is pretty “flat” looking. When studying this RAW file, you’ll find that it actually has more detail in the sky and shadows than the picture straight out of the default camera app.

So why do RAWs look like that in most apps?

The standard iOS RAW engine adds a contrast curve and other adjustments to punch it up. Unfortunately, it doesn’t employ any sort of sophisticated computational photography algorithms— it twiddles some knobs and calls it a day. Very often this makes it appear as though your RAW has clipped data, making you think it’s worse than a JPEG.

Halide 1.16 renders flat RAWs in the Photo Reviewer. While the image appears less vibrant, it makes it much more obvious what areas are actually clipped, without having to jump into an editor. Images do appear a bit ‘bland’ and flat, but the beauty of RAW is what you can do with them in an editor. You can now even export the ‘Flat JPG’ straight to apps that support it.

From looking at the underlying RAW, it’s obvious we only need to stop down the image a little bit to recover more sky.

Faster Saves

While updating our plumbing to support Rescue, we improved the way we saved photos so narrow the window where bad things can happen. We make saving photos the highest priority task in the app, and throttle other behavior (e.g. fetching photos from the library) while there’s a queue of photos to be saved.

Reduced Memory

When we say this is a groundwork update, we mean it. One of the most essential things of iOS apps is performance and memory usage. We’ve reduced the chance of running out of memory in depth mode, causing a crash. We say “chance,” because crash-worthy memory usage is less predictable than you think. Want details? We’ve got them:

Warning: Technical Details Ahead.

Apple doesn’t share hard numbers on memory limits. You can force a crash on an iPhone 7 by deliberately eating up to 1.4 gigs of memory, but iOS can consume memory on its own in surprising way. For example, if your email client wakes up in the background to fetch new emails, it’s suddenly going to need more memory than when it’s chilling in the background.

Last year we measured that Halide typically used between 100mb and 150mb of memory. That seems reasonable when you consider a single full-resolution photo consumes nearly 50MB of memory.

However, in depth mode, Halide would peak a 576MB when generating our portrait effect — it has a lot of moving parts. We planned to get that number down, but traced most memory usage to internals of Apple’s Core Imageframework. Apple didn’t expose knobs for us to adjust to fix it, so this could take a while to find a solution. “Well, 576MB is still way below that 1.4 gig limit, so things are safe for now, right?” Well…

When any app uses the iPhone camera, it fires up an iOS daemon called mediaserverd. Normally this daemon consumes 230MB of memory. We found out that in depth mode, it would hover around 850MB, which sends iOS into a “Danger! Free up memory!” zone. Sometimes you’d be unlucky, and iOS would decide to kill Halide. We solved this experimenting with different camera configurations, and ultimately got Apple’s daemon to only use 515MB. We were back in a safe zone.

In this update, we’ve further optimized our Core Image usage, and reduced Halide’s peak memory use in Depth Mode to 300MB. As before, when shooting RAW, it peaks at 150MB. In addition to getting us into an even safer zone, it makes us a better iOS citizen.

Spectre

Today we’re also issuing a small Spectre update — it has been in solid shape since being awarded with Apple’s pick as iPhone App of the Year. This update brings it up to speed with Siri Shortcuts and fixes a few bugs.

The Future

Spectre will receive Rescue (and our optimizations) in an upcoming release. In fact, we’re working toward launching these sort of common features simultaneously on Halide and Spectre by sharing code.

This has slowed down some Halide feature development, but once we’re through those woods, it will be worth it.

In addition to this update, we’ve had our heads down on something really big. Stay tuned: this year will be a great one for Halide and Spectre users alike.

Stay healthy, stay safe, and happy snapping from the Halide team!