Lux Year 3: The End of the Beginning

Ben Sandofsky
May 20, 2020
10 min read

It’s Halide’s third birthday! Every year, we’re going to celebrate with a recap. Last year was big. Buckle up, because this one’s a rollercoaster.

In the last year, we’ve released five major updates to Halide. Our second app, Spectre, was chosen by Apple as 2019 iPhone App of the Year. While 2020 turned into a rather challenging year, we’re relieved to report things are going to be all right.

Even when things go well, one does well by remembering Ed Catmull’s advice: “Success hides problems.” That’s why we’re going to be fully transparent, sharing where we missed our marks, and a few of our anxieties. We’ll explain how we’re addressing these things, so they don’t grow into real problems.

But that’s not all: this year, we’re looking at the future. We’re looking at what we’re building next, and have one of our most exciting announcements yet — so stick around to the end.

The Lost Summer

This time last year, we had just finished a series of small updates to Spectre to settle the app into a solid state. Now we were ready to roll up our sleeves and work on our next huge Spectre update. Sadly, this story ends with our only misfire of 2019.

Spectre covers a lot of new territory for me, and I never studied the math behind computer vision while in school. Our next big update involved a lot more theory, and I underestimated the time spent decyphering funny symbols.

An example of the “easy” math in computer graphics.

It took two months to get a functional prototype up and running. When we finally tested it in the field, sometimes the results were amazing, and sometimes they sucked. We hold ourselves to very high standards, and we knew this wasn’t ready for prime time.

Meanwhile, a storm brewed on the horizon: iOS 13. At the start of summer, we tested Halide with the first iOS beta, and found significant issues. We filed tickets with Apple and hoped for the best, but as iOS 13 reached its last few betas in August, it was clear these issues weren’t going to be fixed in time. If we didn’t find workarounds, we expected users breaking down our doors with torches and pitchforks.

So we halted work on Spectre to find iOS 13 workarounds for Halide.

Halide 1.13 launched with little fanfare but no complaints, which is all we hoped for. It was now too late to finish the Spectre update we’d started over summer, because Fall means two things: pumpkin spice lattes and new iPhone hardware.

Fall

The iPhone 11 Pro introduced a three camera module. In our first prototype, we just set the zoom button to cycle between cameras, but it was obvious that having to cycle through all the lenses would be annoying.

We rethought the interaction and designed a new control, inspired by pie menus — but with the classic gestural flexibility and power that you’ve come to expect from Halide. Quick swipes enable faster lens switches than any other camera app around.

As we started playing with the experience, we thought, “You know what would be cool? A preview of the focal length crop!”

And so our lens guides were launched in Halide 1.14.

Before we could catch our breath and get back to work on Spectre, Apple had one last card up its sleeve with iOS 13.2: they had opened up Deep Fusion and Smart HDR to third party developers! These “smart processing” features have been a user request for some time, so we dropped everything to add them to Halide.

Whenever Apple releases a new device, we update our metering algorithms to make the most of the new sensors’ characteristics. So while I worked on Smart Processing, Sebastiaan explored the iPhone 11 sensor in the field. These featured all launched in Halide 1.15, alongside a deep dive into the new hardware.

The email

Just as things were calming down, and we were returning to work on Spectre, we got a mysterious email from Apple. The email said:

“ I have an opportunity I’d like to talk to you about that I think you’ll find interesting.”

In a follow-up call, we found out Apple selected Spectre as 2019’s iPhone App of the Year.

Apple invited us to a celebration in New York, to meet press, executives, and other award winners. They also asked us to put together a demo for the event. This was tricky — have you noticed all of Spectre’s marketing photos feature the outdoors?

We spent two weeks putting together a special build for the event, optimized for light painting on the iPhone 11 Pro.

It even worked with PixelStick, an LED bar that lets you flash custom patterns as you wave it in the air.

We never ended up using the PixelStick at the event, because it takes a little too much practice for casual folks who would only be there for a minute. Regardless, the event went off amazing!

*Record Scratch* Yup, that’s us.

The Winter of our Discontent

In the followings days, Sebastiaan and I faced an existential question. “Where do we see this going?”

There’s a million things we want to do, but the lost Spectre update was a warning sign that we’re bumping up against our limits. In the last year it’s felt like we’ve had to weigh the opportunity cost of everything. Does it make sense to spend two weeks building a demo for Apple’s event? (Absolutely.) Should we spend a few days writing this post? A few days doesn’t feel like much until you realize all product work grinds to a halt.

This is the point we’re supposed to hire someone else, but it’s scary. When it’s just the two of us, we have absolute freedom. If we make a dumb business move we only hurt ourselves. When you add an employee, now you’re risking someone else’s livelihood.

In the past, we’ve mostly planned a few months ahead. Our product vision boiled down to, “Wouldn’t it be cool if…”. That’s great for individual features, and even brought us to our second app. But if we’re going to bring on someone else, we need a long term vision. So we sat down and sketched it out.

(the Lux design department requires us to state for the record that these are not the actual names or icons of our upcoming products )

We have three projects on the horizon. There’s our big Spectre update, which has the working title Banner. There’s a second project, Stark, which is even larger in scope but with fewer technical risks. Finally, there’s our most ambitious project, Fury.

At this point, Banner has too many unknowns. For now, we’ll focus on Stark, which allows us to develop a better understanding of Banner’s challenges. We hope to launch Stark in 2020, Fury in 2021, and Bannersomewhere in between.

That brings us to worries for our first employee. If updates to our current apps slow as we approach the final months before launch, we should expect sales to slow. We need enough money in the bank to weather the storm. Fortunately, we had App of the Year.

In the past, Sebastiaan and I just split our profits every month. This time, we set aside a safety buffer for our first employee. Even if app sales drop to $0, we should be able to cover six month of salary.

As 2019 drew to a close, we had a plan. Convince an amazing developer to join our team, and focus all of our efforts on Stark.

Welcome to 2020

If I could pick one moment to summarize 2020, it was the call from my brother on a Sunday morning in January.

“Dad had a stroke.”

I dropped everything and hopped on the first flight to Boston. After his emergency surgery and aftermath, my brother and I spent the next month helping our dad settle back into daily life. I’m happy to report that he feels mostly back to normal.

By mid-February, I was able to return to work. We began figuring out the nitty-gritty details involved when hiring an employee. Then the world turned upside-down.

The Spring COVID-19 Slump

Worldwide shelter-in-place orders shook up the App Store. According to SensorTower, apps in the Travel and Navigation category saw dramatic drops, while apps in the Business category thrived.

We make camera apps, and we follow a pay-up-front model. People use camera apps when they’re out and about, and in frightening economic times, $3 can feel like a lot. In addition, the App Store is mostly promoting apps to help people through the crisis. By April, two months into the lockdown, our monthly revenue dropped 30%.

Sales picked up a bit when we shipped Halide 1.16. It added Image Rescue, a feature that helps recover photos otherwise lost by crashes or system bugs. We also refined Smart RAW.

A few weeks later, Apple shipped the new iPhone SE. On this new device, just like the iPhone XR, the first-party camera app only supports portrait mode photos for humans. And just like the XR, we released Halide 1.17 to allow you to take portrait photos of pets and inanimate objects. We also gave Spectre an update to work and look its best on this latest, shiniest, tiniest iPhone.

Between the new hardware launch and Halide updates, April revenue returned to pre-COVID19 levels. We doubt Apple will release a new phone every month, but it’s nice to get a taste of normalcy, however brief.

Other folks aren’t so lucky. That’s why last month we donated $5,000 to Compass Family Services, which assists San Francisco’s homeless and at-risk families, one of the hardest hit groups in this crisis. We also donated $5,000 to the Second Harvest Food Bank of Silicon Valley. If you have spare resources, please check out your local charities.

In April, we took our first dip in the toes of iPad with Esper. We built this prototype to flex Apple’s fancy new LIDAR technology. While it seems weird for us to spend a week on a prototype after complaining we’re low on resources, it’s extremely likely this technology will find its way into a future iPhone. The prototype taught us that LIDAR is amazing for room scanning, but doesn’t provide the resolution for use in photography apps, just yet.

Finally, we wrapped up April getting boring business stuff out of the way. To hire our first employee, we registered with the necessary state agencies, picked a payroll system, researched health insurance… the fun never stops.

We also decided it was time to convert our business entity from a partnership to a corporation. We are now officially employees of Lux Optics Incorporated. It’s been an incredible journey, and Sebastiaan and I are super pumped to work for Sebastiaan and me.

That brings us to the awkward situation of trying to hire someone during a global pandemic. This is not a great time to make dramatic career changes.

A few months ago, as the magnitude of COVID19 sank in, Sebastiaan and I decided six months of runway just doesn’t do it for us anymore. So Sebastiaan and I are paying ourselves $1 a week. Once we have enough cash reserves to cover one year of our first employee’s salary, we will resume paying ourselves a more normal (but modest) salary. With our anxieties quelled, we were ready to move forward.

Announcing Our First Hire

We didn’t have to look far to find an amazing iOS developer with a decade of experience. Most importantly, she’s really cool and makes an awesome friend. Everyone, please give a big hand for the third member of our team: Rebecca Slatkin! Rebecca, why don’t you take the mic?

Excluding the woodpecker/squirrel fight I broke up last week, joining the Lux Team is easily the biggest event to occur in my life in 2020.

I’ve used my time in quarantine to do my share of puzzles, cooking, and organizing the boxes of photos in shoeboxes, albums, and misc drawers. I’ve tried to organize my photos before and it always starts out with good confidence, but always suffers the same fate of every project I don’t have a clear goal from the start

A camera has always been a fixture in my life. My dad, a professional photographer, carried a camera around like an appendage. He regularly snapped photos of seemingly trivial occurrences often while dodging swats and protests from me and my siblings as we brushed our teeth, struggled with math homework, and cleaned dishes. I took on a similar role as my dad in college. Insisting my friends “act natural” and carry on with whatever they were doing as I snapped away.

Years later, surrounded with piles of photos, I am grateful to have in my possession hundreds of photos that are not mere stiff, hard, posed photographs; they are photographs with feeling in them, and sentiment, and capture who we truly were at that moment. A collection of photos of life that leaves out the little moments and enumerates only the big isn’t a proper picture of my life and the events that led me to where I am today.

As I begin my work on the Lux team I will use the love and affection for my collection of photos to build products and features for you to use capture moments to hang feelings on: little or big, majestic or trivial.

We’re incredibly excited that Rebecca is joining us and we can’t wait to show you what we’re all working on together!

In Closing

It’s been a crazy year, and none of it would be possible without the support of folks like you. From the bottom of our hearts, thank you so much for being there with us for three years and letting us do what we love.

Check out our new team page here, and expect big things from us all at Lux Optics in the future!