Case studies
Understanding the impact of a new mobile experience.
One of the largest museums of modern and contemporary art in the United States
Services: Mobile Evaluation, User Research, Product and Service Testing
Tools: Narrated visits, In-depth interviews
Delivered: 2017
The ask.
SFMOMA’s new digital programme featured a mobile app that offered an intimate and location-aware audio experience. But was it hitting the mark with visitors? We put on our research and evaluation hats to find out.
The answer.
The new app was launched following SFMOMA’s 2016 expansion. The team wanted to give people a unique insight into its collection by telling human-centred art stories. They had two main objectives.
- Share voices and perspectives not typically represented or heard in an art museum setting.
- Connect artworks and stories in larger narrative threads, rather than separate, individual descriptions.
The app delivered multiple immersive audio walks using indoor positioning, which gave visitors real-time navigation in the building. By doing away with the singular, omniscient voice of the museum, the app encouraged visitors to understand and connect with the artworks in new ways.
To understand the experience, we needed to evaluate the experience of the whole service and have a birds eye view what happened to visitors as they used the guide.
The approach.
To find out whether the app was working with visitors, we did a large-scale survey to measure levels of user satisfaction and user experience. We also carried out in-depth interviews and – this is the crucial bit – asked visitors to record their experience as they walked around the museum.
There was good news and bad news. Our analysis found that most visitors loved the concept and the content of the app. But we also unearthed technical and operational issues that left many people feeling frustrated. Some visitors couldn’t reach the content while others had serious battery issues that soured a positive experience.
For the SFMOMA team, the evaluation process delivered valuable and sometimes surprising insights. We were able to show them the data and through our interviews and video footage, the museum team could see first hand the delight and moments of pain that visitors were experiencing.
The outcome.
Following our work, SFMOMA made a range of improvements that have had a measurable impact on the museum. Concrete changes. Real value.
We gave them evidence that the app changed perceptions of the museum – and modern art – for the better. The team has been able to build on the success of its content strategy and continue offering unique audio narratives.
–Stephanie Pau, Former Senior Content Producer, SFMOMA
“The true magic of a location-aware storytelling experience lies not only in the content, but in the content’s synergy with all aspects of the mobile service.
Audio, location, and WiFi need to work together seamlessly—an operational challenge in a museum that rarely stays still.
We are still learning, but we’ve also been proactive in making changes that will improve the overall mobile service.
We’ve worked with colleagues across the organization to adjust exhibition planning, installation, and operational workflows, in order to improve a mobile service with many moving parts.
You can learn more about the app and the evaluation by reading the 2017 Museums and the Web paper written by Stephanie Pau: Audio That Moves You: Experiments With Location-Aware Storytelling In The SFMOMA App.
Share this case study.
