Scaling up: Engagement platforms and large-scale collaboration

Robert Stein, American Alliance of Musuems, USA, Emerald Cassidy, The Grace Museum, USA, Jonathan Finkelstein, LearningTimes, USA, Andrea Fulton, Denver Art Museum, USA, Douglas Hegley, The Metropolitan Museum of Art, USA, Amy Heibel, Los Angeles County Museum of Art, USA, Shyam Oberoi, Royal Ontario Museum (ROM), Canada, Kate Tinworth, ExposeYourMuseum LLC, USA

Abstract

Keywords: Collaboration, Engagement Platforms, Museum Metrics, Evaluation, Software Frameworks

1. Piloting a national platform for engagement

In late 2013, the Institute for Museum and Library Service awarded a National Leadership Grant to a collaboration of U.S. art museums including the Dallas Museum of Art (DMA), Denver Art Museum (DAM), Los Angeles County Museum of Art (LACMA), and Minneapolis Institute of Art (MIA). The team also quickly invited the Grace Museum of Art from Abilene, Texas, to join and represent the needs of smaller art museums.

The project team is working together to pilot a shared engagement platform in each partner museum that is unique to the individual museum, but shares data from each museum on a common platform. In doing so, the team hopes to better understand how its mutual efforts can drive engagement with art. Each pilot will use a common software platform based on the DMA’s Friends program, but will be customized in content and display to fit the unique local needs of each partner.

The DMA Friends program was launched in January of 2012 and has recently passed ninety-thousand members. More information about DMA Friends can be found in the proceedings of past Museums and the Web publications (Stein & Wyman 2013, 2014).

2. The planning process

To begin work together and set forth plans for the pilot projects, the team met in person in Dallas in December 2013. In addition to project team members, these meetings also involved other interested participants including Zannie Voss, director of the National Center for Arts Research, Marc Vogl from Harvard’s Houser Institute for Civil Society, and Susan Chun from the Museum of Contemporary Art Chicago. The group discussed what a shared engagement data platform might look like and what might be accomplished through data sharing that would otherwise be impossible.

Recognizing that each museum partner has a variety of different local conditions, the project team conducted a system-compatibility survey to better understand the technology differences at play. Partners provided detailed information about their own software and technical environment to project leaders, who consolidated that information and looked for overlap. As a result, it became clear that while certain aspects of the technical infrastructure were similar among museum partners, many of their detailed implementations of those tools needed to support a platform for engagement were very different.

Furthermore, it should not be surprising that each museum-partner has distinctly different goals that they sought to achieve through this project. The system-compatibility survey demonstrated to the team that a shared platform of engagement like Friends would need to be able to stand alone and apart from the existing technical infrastructure of the museum while being able to integrate with an unknown variety of tools at some point in the future.

Online workshops

In addition to the technical implementation of such a project, the project team recognized that staff training and content design was likely the most challenging aspect of the project. As a result, the project team planned and led three online workshops for the staff at each partner museum. Jonathan Finkelstein from LearningTimes led those workshops. Participation in each session was robust, with members of education departments and other areas from each partner museum participating.

The first online workshop gave participants an overview of game and engagement mechanics in relationship to online badging and learning systems like the DMA Friends platform. The second workshop focused on how each partner museum could begin thinking about designing a “constellation of engagement” for its particular project. The approach asked each partner museum to think about its engagement goals for the project connected to what kinds of behaviors it might ask participants to partake in as a result. The last workshop was more practical in nature and served as an introduction to the administrative tools underpinning the existing DMA Friends engagement platform. This session amounted to a sort of training for staff to understand what the day-to-day management tasks for data entry and content management look like in this kind of system.

3. A description of each pilot

As an outcome of the design training accomplished through the three online workshops, the team asked partner museums to author a series of simple user stories that illustrated their own unique approach to piloting the Friends program in their own museums. Interestingly, the proposed pilot projects for each museum evolved naturally to highlight very diverse needs for this type of engagement in art museums.

Denver Art Museum

The Denver Art Museum’s (DAM) pilot project will use the platform to create deeper and more personal relationships with self-selecting visitors to the museum. The goal of the museum’s pilot is to foster cultural citizenship by aligning community members with relevant programs at the DAM, and potentially throughout the city, by creating an active feedback loop that enables increasing personalization of recommended experiences.

Similar to the Dallas design, visitors opt in to the program on site at the Denver Art Museum by providing basic contact information at a mobile kiosk or through their personal mobile devices. Users then log museum experiences through their visit as “check-ins” via a mobile website interface. Departing from the Dallas experience, users are invited to give feedback on those experiences at each check-in and through e-mail following their visit. When visitors first arrive, a short survey is offered with the results used to create a “playlist” or “itinerary” for that visit based on a visitor preferences including time frame, companions, interests, and motivations. With each additional visit, information from previous visits and post-visit surveys build to continually enhance a visitor’s profile, ultimately leading to the discovery of meaningful experiences that may not be obvious through traditional communications.

Goals for the DAM pilot include:

• Gaining insight into visitor needs and values during a visit
• Encouraging repeat visits
• Driving visitors to unexpected and less-obvious programs and offerings
• Fostering more personal relationships with visitors
• Nurturing individual interests and meaningful experiences with visitors
• Exploring visitor behavior and inputs across partner institutions

The Denver Art Museum pilot launch is planned for late Spring 2015.

The Grace Museum

Last spring, the Grace Museum in Abilene, Texas, began researching new and innovative ways to increase visitor attendance. During this time, the museum’s executive director, Laura Moore, was introduced to Rob Stein, deputy director for the Dallas Museum of Art. From those discussions, a collaborative partnership was formed, with the Grace becoming the first Friends membership pilot program modeled on the DMA’s free membership platform, DMA Friends. The DMA was interested to see how the Friends program would operate within a smaller museum.

Since the launch of Grace Friends in May 2014, the program has enrolled over four-hundred friends. The Grace Museum uses the software and reporting tools that have been created by the DMA to monitor and cultivate the engagement of its visitors. These tools easily track, in real time, how many new friends are registering on the iPad kiosks, in what galleries they are spending their time, and much more. The Grace Museum is able to track these data points each time a Grace Friend enters an activity code by texting via their mobile device, or at the iPad kiosk.

With this new tool, the Grace is learning more about how to treat this data and what is of most interest to their visitors. The Museum’s vision is to increase participation from Grace Friends, and then be able to see that participation transfer as data. This data can serve as a guide in Museum programming and event planning. For example, already the Grace has seen a 27 percent increase in attendance since the launch of Grace Friends on the evenings when special programming is offered. The Museum anticipates seeing this trend continue into next year.

LACMA

The Los Angeles County Museum of Art (LACMA) is piloting its new model for tracking and rewarding museum engagement with its award-winning free youth-membership program, known as NexGen. With more than 190,000 active members, NexGen offers free museum admission to children seventeen and under, with the added benefit of bringing one accompanying adult for free. The new badging and engagement program encourages NexGen members to deepen their involvement with LACMA’s collection and education programs by checking in to activities and challenges at the museum, for which they earn points. As NexGen members earn more points, they improve their ranking and receive badges—both physical and digital—to be worn on their museum lanyard and displayed on the software interface. The museum is also leveraging its campus-wide digital signage system to display a leaderboard recognizing top participants by username.

Because NexGen members visit the museum in family groups, LACMA is contributing a scheme for registering and rewarding groups of participants to the software architecture with a goal of incentivizing family members to explore and learn together. LACMA is also exploring the integration of NexGen enrollment with the badging and engagement software to streamline the enrollment process.

Key goals for the pilot project at LACMA include:

• Measuring and recognizing participation in learning activities
• Providing incentives and a unifying experience to drive participation in the hundreds of annual programs and activities offered to NexGen members on campus
• Promoting and measuring repeat visitation
• Evaluating shared data across museum partners to ascertain the effectiveness and popularity of different learning activities with different target audiences

The pilot project is scheduled to launch at LACMA in Spring 2015.

Minneapolis Institute of Arts

The Minneapolis Institute of Arts (MIA) enthusiastically joined the other museum partners in the IMLS grant supporting loyalty systems because the MIA’s current strategic plan is heavily focused on audience engagement. The Museum has had free general admission for decades and is committed to continuing that policy. Unfortunately, this has meant that the Museum has little or no information about more than half of its annual visitors (averaging over six hundred thousand per year in recent years). Without that information, the ability for the MIA to engage and build relationships with its public audiences—outside of the traditional membership model—has been severely hampered. A loyalty approach such as this one offers the promise to help the MIA identify these audiences and interact with them directly.

The approach is seen as a vital cog in a larger effort to streamline an omni-channel approach to driving audience interaction with the Museum. To that end, the MIA is in the midst of a complete overhaul of its core visitor-specific systems, including new ticketing, customer relationship management software (CRM), donor and member management, and e-mail campaign software.

As part of the MIA’s strategic approach, an entirely new membership program was launched in October 2014, including an option for free membership. The Museum is working hard to uncouple the previously conjoined elements of membership and philanthropy. In doing so, the Museum plans to offer deeply engaging experiences to members along with a new and exciting loyalty component, while at the same time shifting donation strategy to meaningful, personalized asks instead of basing requests on a general sense of commitment to the institution.

Loyalty, therefore, becomes an important component of membership, drives participation, and enables the Museum to incentivize specific behaviors, and results in the collection of vital information about each and every participant. The information is then reflected back to each person via individualized Web dashboards currently under construction. Each participant will be able to log in and find current member status, lifetime giving, current giving opportunities, the impact of their giving, loyalty point status, and the latest rewards available for redemption (based on preferences and interests, both expressed and discovered in data analysis). The planned rewards will primarily be experience based, offering exclusive events and behind-the-scenes access to the Museum and its staff.

Key goals for the MIA pilot project include:

• Reverse the recent downward trends in annual membership; increase member households by over 100 percent (from around twenty thousand to around fifty thousand) in year one of full operation under a free-membership option and loyalty program; and plan for continued growth in years two and three.
• Collect deep and rich data about visitors, which had previously been impossible because of free, unticketed general admission.
• Use the data collected to drive decisions about communications, marketing, programming, and engagement.
• Build individualized relationships with all members through clear communication, personalized dashboards, unprecedented capacity for preference-setting, and data/trend analysis.

The targeted launch date for the new loyalty program at the MIA is the end of April 2015.

4. Accommodating unique partner needs

Using the stories created by the partners mentioned above, the project team distilled the narrative expressions of the pilot projects into a discrete set of software features that would need to be supported by the project. The team chose to use Trello, a free list-making online tool, to break each use case down into smaller requirements (figure 1). The team rated each requirement by how difficult it would be to implement, whether it needed more clarification in order to be built, and whether a particular feature should instead be the responsibility of the partner museum. In general, features that would be likely to benefit the whole collaboration were the responsibility of the grant, while features that were idiosyncratic to one museum (i.e., systems integration) were left to the partners to implement. Trello was a very useful way of giving visibility to the team about the process of reducing the scope of the project towards a realistic set of software deliverables. In many collaborative software projects, scope creep and unrealistic targets are often cited as points of project failure.

Figure 1: the team used Trello to transform use cases into a list of functional requirements that were then rated by difficulty

Once the collaboration agreed upon the functional requirements that would be implemented for each project, the software development team proceeded to transform these functional requirements into software features that could begin to be coded. These features were used to populate a project management tool for the software team based on a system called Jira (https://www.atlassian.com/software/jira). Jira enhances the transparency of the requirements process and allows the entire team to see the progress of the software development team and to participate in testing and quality assurance by submitting bugs and additional feature requests.

Figure 2: the team used Jira to turn functional requirements into software development tasks and submit bug reports and feature requests

5. Process evaluation methods and findings

In addition to evaluating outcomes, this project has utilized process evaluation to chronicle and document the inner workings of how the project progresses, adapts, and changes. This is an incredibly important step when embarking on new or innovative ventures like DMA Friends and its adaptation in other art museums. Process evaluation also tracks the evolution of a project and can help enhance its long-term sustainability.

ExposeYourMuseum LLC designed an online check-in survey and has been administering it to participants in the collaboration two to three times a year. Importantly, the survey does not discriminate between the roles of partners on the project, whether large or small. The process evaluation includes project leadership, but also those working on all aspects of project rollout, from back-end technology to program implementation. Every voice is equally valued and heard, which is often challenging within hierarchical project teams and across diverse museums. Primarily though open-ended questions related to project observation and reflection, respondents are asked to take stock, identify barriers and lessons learned, and share insights into how the project works when it’s at its best and it encounters challenges. Anonymized findings are shared with the project team within a month of each survey, providing timely team insights that can lead to key course corrections. Process evaluation has allowed project leadership to respond to key criticisms quickly, has led to improved teamwork and communication, and has ensured roadblocks and sticking points were identified and remedied early.

6. Changing course: Allowing collaboration to shape software

During the course of the project, the DMA Friends program running in Dallas continued to experience dramatic growth. The team had originally anticipated that the program might grow to approximately twenty-thousand individuals during its first year. In fact, the program exceeded fifty-thousand individual members in one year and now consists of more than ninety-thousand members, while still growing at approximately seven-hundred new members each week.

The software used for the initial launch of DMA Friends leverages WordPress at the core with a series of open-source plugins—some generally available and others developed specifically for DMA Friends—which provide much of the platform functionality. The system proved to be flexible and easy to use for staff at the DMA while being relatively quick to customize and deploy. In evaluating the best approach to meet the fundamental goals of the new collaboration—the aggregation of a large set of engagement data from multiple organizations and the accommodation of the diverse needs of each participating institution—the team considered whether to expand and enhance the current underlying platform or take the opportunity to develop a new foundation specific to the project’s unique requirements.

Considering the scope of the changes needed to arrive at a suitable foundation for a cross-institutional undertaking of this scale, the project team collaboratively decided to rebuild the underlying software of the project—a decision supported by the software development team, leadership at each partner institution, and consultants on the project. The goal of the new software was to maintain all of the core concepts, relationships, engagement algorithms, and front-end user experience that has made the DMA Friends program so successful to date, while also improving the scalability and flexibility of the system. Partner museums and project consultants continue to contribute actively to the design and development of the new framework.

The new software launched first in Dallas, where it replicates the entire end-user experience of the original tools, while introducing a new staff-administration toolset. The platform’s content management system continues to support the creation of customized front-end experiences, and integration with external third-party systems—such as Salesforce and MailChimp—is being developed to match the specific needs of the project partners. New developments include the ability to recommend activities based on users’ activities, badges, and rewards and allowing individuals to engage and earn achievements as a group.

Figure 3: the new Friends’ platform software

Figure 4: authoring a badge in the new Friends’ platform software

This decision and resulting software effort delayed the originally proposed rollout of the partner pilot installations from late 2014 to spring 2015. All the project partners have revised their development timelines and planning, fully in support of the new software approach. The project team anticipates the ability to accomplish all the original year one goals of the project and the subsequent analysis in year two of the project.

7. Next steps

Through a detailed process of requirements gathering, community design, and software specification, the project team is pleased to release a revamped software framework for visitor engagement that addresses a number of key issues, namely:

• The scalability of the system to hundreds of thousands of members at each site
• The integration of real-time reporting tools that can provide an up-to-the-minute window for staff into the behaviors of visitors
• An aggregation and transformation of data from multiple discrete partner sites into a single engagement platform for analysis that crosses institutions
• The ability to support offline analysis and data mining when data scale and/or processing methods are too time intensive
• A well-described Web services API that can facilitate software integration for each partner and also facilitate client applications built on top of the same basic data framework
• Better support for two-way communication with community members to accelerate digital engagement strategies both inside and outside the museum

The team eagerly looks forward to the launch of pilot projects in each of the partner museums in the first half of 2015 and will continue adding features to the software platform in an iterative fashion. Data aggregation and analysis will be a key focus of the second year of the grant and will begin in earnest upon launch of each pilot project.

New features that are currently under development include:

• Deep integration with e-mail marketing tools like MailChimp and transactional e-mail systems like Mandrill
• The ability for users to rate, vote, and integrate social-media activity with the platform
• The integration of Friends’ engagement activities with online collection APIs
• Leveraging the Web-services API to support client-side mobile apps

In addition, the team will devote time and effort to documenting the process by which other museums can join this collaboration and launch engagement platforms of their own.

References

Stein, R., & B. Wyman. (2013). “Nurturing Engagement: How Technology and Business Model Alignment can Transform Visitor Participation in the Museum.” In N. Proctor & R. Cherry (eds.). Museums and the Web 2013, Silver Spring, MD: Museums and the Web. Published January 31, 2013. Consulted January 30, 2015. Available
http://mw2013.museumsandtheweb.com/paper/nurturing-engagement/

Stein, R., & B. Wyman. (2014). “Seeing the Forest and the Trees: How Engagement Analytics Can Help Museums Connect to Audiences at Scale.” In N. Proctor & R. Cherry (eds.). Museums and the Web 2014, Silver Spring, MD: Museums and the Web. Published February 1, 2014. Consulted January 30, 2015. Available
http://mw2014.museumsandtheweb.com/paper/seeing-the-forest-and-the-trees-how-engagement-analytics-can-help-museums-connect-to-audiences-at-scale/