David Rose, MIT lecturer, inventor, and five-time entrepreneur, is a world-recognized expert in augmented reality (AR). In his quest to realize the potential of what he's termed "SuperSight," David has created an online eye testing business and virtual try-on system for Warby Parker and a memory mirror to compare outfits with Nieman Marcus. As part of this mission, David came to Cantina with a breakthrough idea to transform boating: the Waze for Water.

I am trying to augment the world and give people magical experiences like x-ray vision. But, inventing superpowers requires deep technical expertise. Cantina helped me quickly prototype augmented reality anchored to a lake surface to realize the vision of seeing through the water.

The Opportunity

David envisioned using AR to allow boaters and anglers to view 3D underwater bathymetric maps overlaid on the real world. This would help them discover great fishing spots, remember every great location with virtual fishing flags, and boat safer by avoiding hazards. This was the first time anyone had created a marine-focused AR experience because of the technical challenge of anchoring information to an undulating, non-fixed water surface.

Cantina’s work was instrumental in helping David bring the concept from idea to reality in record time. Available in the app store after less than a year, our work to create ClearWaterAR focused on four areas:

• Explore: Creative Technical Exploration
• Define: Prototyping and Testing
• Refine: Visual Design
• Build: App Development

The Challenge

While David foresees that ClearWater AR will eventually be used with wearable AR glasses, that market is not yet mature enough to target. Instead, we decided to focus on a phone-based AR app to explore and define an experience we could put in the hands of boaters today.

The challenge is that phone-based AR technologies make use of the camera and GPS to help anchor the AR experience. Similar to how a classic computer mouse works, a phone’s camera tracks tiny visual variations on the floor to help detect where the phone points as you look around. Unfortunately, on the water, this standard anchor-based approach on the water doesn’t work: water constantly shifts and is highly reflective. GPS also lacks accuracy and is too slow to provide the necessary updates for a smooth, believable experience.

Creative Technical Exploration, Prototyping, Design, and App Development

David has written the book on taking everyday experiences and making them magical. To deliver the necessary user experience he envisioned, we had to overcome some foundational technical challenges. The team rapidly explored several potential solutions, working in a two-week cadence to generate an engineering prototype that would allow us to explore and learn. Some of this effort involved getting out on the water to understand boaters' context of use better and to get a feel for the environment.

Overlaying a 3D underwater bathymetric map on the surface of moving water proved quite tricky. After multiple exploratory iterations, we identified a solution: a combination of gyroscopic anchoring, GPS, and error correction algorithms. With the right combination of sensor data, the phone could accurately sense its location not through sight but through movement and its place in space. And because we avoided using the camera, this approach also overcame the issues presented by water’s inherent reflectivity. While location sensing was critical to addressing core functionality, the error-correcting algorithms were the ‘magic sauce.’ They made the experience stable and believable.

We used the UNITY game engine to create a smooth, interactive experience that feels less like a traditional page-based app and more like a video game. As the team iteratively prototyped and tested the experience with users, we also designed a visual interface and user flows to meet the needs of boaters and anglers.

Outcome: From Murky Waters to Product Launch

ClearWater AR became publicly available in the App Store in less than one year. Through our proven methods of idea activation and innovation strategy, Cantina saw an entrepreneur’s vision through the murky waters of proof-of-concept and agile prototyping to enable a successful product launch.

Read more about this project and others in David’s most recent book, SuperSight: What Augmented Reality Means for Our Lives, Our Work, and the Way We Imagine the Future.