Harnessing motion-sensing technologies to engage visitors with digital data

Jessica Roberts, University of Illinois at Chicago, USA, Leilah Lyons, University of Illinois at Chicago; New York Hall of Science, USA, Francesco Cafaro, University of Illinois at Chicago, USA, Rebecca Eydt, New York Hall of Science, USA


Data is an increasingly important part of how we understand the world around us, but making abstract and complex data sets accessible to museum visitors can pose a significant challenge. The CoCensus exhibit, installed at the New York Hall of Science, seeks to engage visitors in exploring mapped census data and encourage reasoning about the spatial, temporal, and quantitative properties of the data set through a customizable, interactive multiuser exhibit. This paper discusses two major design challenges in creating this novel exhibit, which utilizes embodied interaction to let visitors use their body movements to “play” with digital representations of data in a collaborative learning space. The first challenge was providing an "entry point" to engage visitors with the abstract data set, which was accomplished by allowing visitors to select personally relevant subsets of the census data and dynamically control how those subsets were displayed. Using motion-sensing technologies to situate this control in multiple users’ bodies through movements and gestures addressed the second challenge of engaging multiple participants, who were now able to coordinate their actions to jointly explore their data. This paper will discuss design iterations of CoCensus to describe how motion-sensing technologies have been harnessed to transform the floor space of an exhibit into an interactive timeline, demonstrating that museums can play a role in helping visitors develop data-reasoning skills, playfully and collaboratively.

Keywords: data; collaboration; data reasoning; engagement; interactive; personalization; data visualization

1. Introduction

Understanding data is increasingly important to understanding the modern world. Engaging learners with a complex and unfamiliar data set, however, is still difficult. In formal classroom environments where teachers are available to guide extended learning interactions, there are still a variety of challenges, including learner motivation to explore the data, how accessible learners find the tools supporting data investigation, learners having enough background knowledge to probe the data effectively, and practical constraints such as time or equipment limitations (Edelson, Gordin, & Pea, 1999). These challenges are magnified in informal, “free-choice” (Falk & Dierking, 2000) learning environments like museums, where there is often no person available to facilitate the exploration of data (some institutions have docents or interpreters, but such personnel are seldom tied exclusively to one exhibit), where the interactions learners have with the data are brief, and where visitors may have varied degrees of background knowledge and motivation for exploring a given data set. While classrooms may take advantage of the compulsory nature of the learning setting to, say, provide remedial background knowledge to learners, or require a certain amount of time on task, museum visitors will exercise their “free choice” by passing up an interactive data exhibit if it fails to engage them.

In this paper, we describe the design of CoCensus: an interactive museum exhibit that allows visitors to collaboratively explore data from the U.S. Census (Figure 1). CoCensus attempts to make big, complex sets of data (such as the U.S. Census) engaging to the masses. Data from the U.S. Census are freely available online, via the census.gov portal, but the traditional data format (spreadsheet tables) are not accessible (or engaging) to everybody.


Figure 1: visitors interact with the CoCensus exhibit at the Jane Addams Hull House Museum (left) and the New York Hall of Science (right).

Data visualizations, which visually arrange data in order to illuminate relationships among data points, have been used for decades by scientists to understand their data sets and show great promise for helping novice (non-expert) learners make sense of data in informal environments (Pousman, Stasko, & Mateas, 2007). In recent years, there has been an increased interest in how census data can be displayed on interactive data maps (Figure 2).


Figure 2: a Web-based data visualization of census data showing the percentage of male population (source: http://tigerweb.geo.census.gov/datamapper/map.html).

The number of people who visit museums every day make them appealing venues for public education and outreach. Complex sets of data, which would be otherwise confined to research laboratories or government agencies, can become new museum artifacts, where visitors are free to problematize and discuss them. Two major challenges emerge from this scenario of using museums as sites for improving public data literacy, however. First, in order to effectively leverage complex sets of data, we need to provide an “entry point” to get a visitor engaged—an aspect of the data set that learners can connect to their personal interests or experiences. Even familiar representational forms, like data maps, are not easily interpreted without some “entry point” to help learners identify with and become interested in exploring the data. We need, thus, to establish a personal connection between each visitor and the data set.

Second, museums are by definition social spaces. Multiple people will be always present in the interactive space that we design. Engaging meaningfully with an interactive exhibit is dependent on whether visitors can link the effects seen in the shared exhibit to the actions taken by visitors to manipulate the exhibit (Allen, 2004). Even if this linkage is clear (and we should stress that many multiuser exhibits fail to make the action-effect connections apparent), visitors are unlikely to find interpreting data visualizations to be a highly social experience without the presence of an activity to promote shared interpretation (Allen, 2004). Ideally, this activity should be playful and enjoyable.

This paper discusses how we addressed these two challenges within our exhibit: first, how to make the abstract and complex data set of the census concrete and accessible for visitors; and second, how to design an interaction that can support and playfully engage multiple simultaneous users. In the following sections, we will describe CoCensus, and we will then illustrate how we faced and addressed these two major challenges during the design of our exhibit.

2. Collaborative exploration of census data with CoCensus

The CoCensus exhibit, which has been displayed at the Jane Addams Hull House Museum in Chicago and at the New York Hall of Science in Corona, New York, presents U.S. Census data through an interactive and customizable data map display. U.S. Census data was chosen as a content area because there is a long history of providing interactive U.S. Census data visualizations via online portals and via classrooms, and the exhibit is one branch of a National Science Foundation (NSF)-funded grant to study reasoning about data maps in three contexts: classrooms, museums, and online (NSF INSPIRE grant no. 1248052). The CoCensus exhibit seeks to promote visitors’ inquiry and collaborative meaning making about a complex data set—the U.S. Census—by letting them manipulate a scaled centroid or “bubble” map of census data in the local geography (Figure 3).

Figure 3. Screen shot of the CoCensus display at the New York Hall of Science. Visitors' individual data "slices" are shown as scaled centroids ("bubbles") in colors chosen by the visitors. Here we see the distributions of single-person households (purple) and 5-person households (orange) in 2010.

Figure 3: screen shot of CoCensus display at the New York Hall of Science. Visitors’ individual data “slices” are shown as scaled centroids (“bubbles”) in colors chosen by the visitors. Here we see the distributions of single-person (purple) and five-person households (orange) in 2010.

To use the exhibit, visitors first select a census variable that represents them (e.g., single-person households). They then enter the interaction area where they see a large digital bubble map display showing the distribution of other people in that geographic region (Chicago or New York City) who responded on the census that they, too, can be represented as a single-person household (see Figure 3). A timeline is mapped onto the floor of the exhibit space to allow visitors to view data from three different decades—1990, 2000, and 2010—in order to allow for comparisons over time (Figure 4). If another visitor in the space has selected a different data set, for example five-person households, the two visitors together can compare the distributions and quantities of their respective data sets over time.

Figure 4. Two competing designs for timeline control were tested in situ in December 2013. The horizontal "externally consistent" configuration (left) allowed one user to change the decade of everyone's data by stepping on a floor button. The vertical "ego-moving" condition (right) allowed every person to select her own decade individually.

Figure 4: two competing designs for timeline control were tested in situ in December 2013. The horizontal “externally consistent” configuration (left) allowed one user to change the decade of everyone’s data by stepping on a floor button. The vertical “ego-moving” condition (right) allowed each person to select her own decade individually.

3. Challenge 1: Making an abstract data set concrete and accessible for visitors

Census data, though it reflects personal aspects of people’s lives (such as race, occupation, household size, and education level), can still be somewhat difficult to make meaning of without significant guidance. The entire corpus of data is very large, consisting of thousands of variables from questions that have been asked and tallied differently over time. The intricacy of the available data can be very beneficial to experts such as social scientists and epidemiologists trying to solve specific problems, but is difficult for novices to make sense of, even in classroom settings with formal instruction and guiding curriculum (Radinsky et al., 2014). Free-choice museum environments need to curate the content to a manageable scope appropriate for a brief exhibit interaction. Therefore, CoCensus was designed to display not the entire body of census data, but only small “slices” that would be personally relevant to a museum’s visitors.

Slicing down the census: Choosing appropriate data for exploration

For the initial installation at the Jane Addams Hull House Museum, CoCensus tied into the existing Mapping the Neighborhood exhibition of the Hull House Maps and Papers (http://www.uic.edu/jaddams/hull/_museum/exhibition.html). This exhibition displaying choropleth (shaded region) maps of the Chicago neighborhood around the original Hull House settlement, depicting the country-of-origin of the neighborhood’s residents in order to advocate for immigrants’ rights. CoCensus used geographic information system (GIS) software to create a modern-day version of the maps using ancestry and race data from the census (see Figure 1, left) for the entire county in order to help visitors—many of whom are local residents—explore broader patterns of settlement still visible in Chicago today.

This early iteration of the exhibit used only current census data; the ability to view data from multiple decades was incorporated later. This initial data set provided over two hundred individual ancestry and race designations, which we deemed would be overwhelming for visitors to make sense of as well as challenging to depict on a map (see Roberts, Radinsky, Lyons, & Cafaro, 2012; and Roberts, Lyons, Radinsky, & Cafaro, 2012, for detailed descriptions of map design considerations). We pared down the data set by keeping any group with at least forty-five-thousand members (roughly 1 percent of the 2010 population) as its own category and combining smaller groups into regional categories (Figure 5), which gave us twenty-three unique race and ancestry groups from which visitors could choose.

Figure 5. In the initial visualization design, displayed at the Hull House Museum, we needed to pare down the census's ancestry and race categories by grouping some smaller ancestries by region (from Roberts, Lyons, Radinsky, & Cafaro, 2012).

Figure 5: in the initial visualization design, displayed at the Hull House Museum, we needed to pare down the census’s ancestry and race categories by grouping some smaller ancestries by region (from Roberts, Lyons, Radinsky, & Cafaro, 2012).

This design had some success with participants, particularly those who strongly identified with a single ancestry or race. However, many “melting pot” Americans do not strongly associate with a single ancestry group. Several visitors commented that they were a mix of many ancestries, and some couldn’t name specific places, only identifying loosely as “miscellaneous European.” Ann, a Hull House visitor, commented that she usually abstained from answering race and ethnicity questions in surveys because, “I just think it’s stinky way to have to identify yourself” (all names are pseudonyms). While we view this tension about self-representation to be productive for sparking conversations about the census, its purpose, and how we are represented through it, we wanted to provide an opportunity for more people to feel personally connected to the data, particularly as we transferred the exhibit to the New York Hall of Science.

Expanding the Data: From one category to four

The New York Hall of Science (NYSCI) is a hands-on science center. Unlike the Jane Addams Hull House Museum, which attracts mostly adult visitors, NYSCI hosts many families and student groups. As we adapted the exhibit to the new context, we explored other data sets that might be engaging for both children and adults. We also wanted categories of data that related in some way to different aspects of a person’s lifestyle and were relatively self-explanatory. We retained the race and ancestry category and added household size, housing type (e.g., single-family home, building with five to seven units, building with fifty or more units), and industry (for which visitors are prompted to select an industry in which they work or want to work, so even children can answer with what they want to be when they grow up). Visitors make selections for all four of these categories at a kiosk outside of the interaction space and can cycle through each category of data during their interaction (Figure 6). We recognized that some visitors may not want to share their own data, and others might just want a quicker entry into the exhibit. For those visitors, we offered six preconfigured profiles of public figures: Mayor Bill DeBlasio, President Barack Obama, scientists Albert Einstein and Marie Curie, celebrity Jennifer Lopez, and fictional character Dr. Sheldon Cooper. We anticipated that even though these preconfigured profiles would not represent the visitor directly, the act of choosing that figure would suffice to foster a connection to the data represented in the profile, because the data would still be humanized, not abstract.

Figure 6. Screen captures of the kiosk used by visitors to configure their profiles for use in the exhibits. All users had the choice of choosing a pre-configured public figure profile (top right) or creating their own custom profile.

Figure 6: screen captures of the kiosk used by visitors to configure their profiles for use in the exhibits. All users had the choice of a preconfigured public-figure profile (top right) or creating their own custom profile.

This process of slicing down the large and unwieldy data set of the U.S. Census into manageable and personally relevant categories has helped concretize the abstract data. We have found that by letting visitors view data that represents them, they feel agency to talk about the populations represented. For example, some have used the data map as springboards to tell personal stories about lived experiences in places, and some have used apparent spatial patterns to ask questions about segregation and immigration (Roberts, Radinsky, Lyons, & Cafaro, 2012).

Establishing a personal connection with the data via full-body interaction

As important as determining precisely what data to present is the decision of how visitors will be able to interact with it. In the early stages of our design, we decided to create a museum exhibit that visitors can control with their own gestures and body movements, as a way of making the experience both social and playful. A shared, large (65-inch) output display was used to provide output, in order to minimize occlusions and prevent a single user from monopolizing the interaction—a situation that frequently happens when visitors interact with smaller displays (Heath & vom Lehn, 2008; Hornecker, 2010). A single, off-the-shelf motion camera (a Microsoft Kinect) proved to be good enough to recognize gestures and body movements: it provides hundreds of tracking updates every second, with an accuracy in the order of millimeters.

Visitors could develop a sense of which data set on the map represented “them” by their body movements: moving closer to the screen would cause their data set to become more opaque and prominent, while backing away would cause the data set to fade into transparency. We experimented with allowing visitors to further explore properties of their data set (like how their data is distributed across different census tracts) by creating a false physics system that would animate the scaled centroids on the map in response to visitor movements (Cafaro et al., 2014). So, for example, if a visitor jumped up, their data would similarly rise vertically away from the tracts, and then fall back in place like “snow,” where more massive data points would fall faster than smaller data points, emphasizing where the largest number of similar census respondents live. In experimenting with the timing of these animations, we found that users preferred it when the animation was either tightly coordinated with their movements, or when it followed after a considerable lag, but that in-between animation speeds were disconcerting (Cafaro et al., 2014), perhaps because the connection between their action and the animation’s response was unclear. It seems that visitors can form personal connection to something they are controlling via embodied interaction when they are either using a mental model of a “mirror” (the controlled visualization echoes their motions), or when they are using the mental model of a “trigger” (the controlled visualization is responding to some kind of force administered by the user).

Discussion: Creating entry points to data via “customization” and control

We found that customization of the experience for individual users helped to address the first challenge of providing an “entry point” into a complex data set. Providing a small range of variables for visitors to customize (rather than asking visitors to define themselves via a single variable) meant that regardless of their individual backgrounds, each visitor can view data that represents them to some degree. Even visitors who viewed data that didn’t represent their own self-identity (such as those who chose preconfigured public figure profiles) were curious to jump in and engage with the data, suggesting that personalization does not have to connect to an individual person specifically, but rather to personhood as a means of concretizing otherwise abstract and impersonal data sets.

Providing individualized control over visitors’ data sets did allow visitors to build a playful connection to their data sets, although we were surprised to find that allowing visitors to trigger animations was just as acceptable to them as having animations mirror their body actions. It seems that as long as visitors can tell that they are the ones producing the animated effect, the means of control is not critical for giving them an entry point for manipulating the data. The next challenge was to create an experience that could engage multiple people simultaneously in order to support dialogue among visitors.

4. Challenge 2: Engaging multiple visitors simultaneously

While some museum visitors prefer to explore exhibits alone, for many visitors the social component of the museum visit is a crucial part of the experience. Maintaining this social element while incorporating technology can be a challenge, however, as some technology-based “interactives” engage and immerse a single user while isolating co-visitors (Heath & vom Lehn, 2008; Hornecker, 2010). In contrast, CoCensus is a dynamic data visualization—shown on a big, shared display—that multiple visitors may control directly with their gestures and body movements. This section will discuss how CoCensus implemented full-body interactive control and how two facets of that control—visitors’ negotiation of the shared space and discoverability of interactive features—were addressed in the design.

Full-body interactive control: Supporting multiple users in playful engagement

Supporting such interactive control via gestural and bodily interaction required a system that accurately tracked visitor movements in space so the exhibit could provide more nuanced feedback on actions, thus promoting deep engagement and learning. To address the challenge of helping visitors accurately attribute actions to individual visitors (so that visitors can understand who is producing the observed feedback), it was also important for the tracking system to preserve the unique identity of each user (Cafaro et al., 2013). A motion-tracking camera, like the Microsoft Kinect, was not sufficient for both of these purposes. When we performed a small in situ formative study using only a Microsoft Kinect camera, we found that five out of nine solo visitors lost their identity at some point during their interaction with the system, from various causes. For instance, when a guided-tour group walked through the exhibition space behind the user, the current user’s identity was “given” to one of the passers-by. We have also observed that the user identity is lost and a user gets assigned a new identity when he or she inadvertently steps into the “blind corners” of the exhibition space. The identity assignment challenge is magnified when multiple users are considered. In a controlled lab study of dyads, we found that: (1) if user A enters a blind spot, followed by user B, and they both reenter the camera’s view, the system may swap their identities (Figure 7); (2) standing close or walking in front of one another may cause identity swaps, at a frequency far too often to support an interaction design relying on unique <person:data> subset pairings.

Figure 7. A Kinect camera alone is not able to preserve the identity of museum visitors when they leave and re-enter the field of view of the camera.

Figure 7: a Kinect camera alone is not able to preserve the identity of museum visitors when they leave and reenter the field of view of the camera.

In order to let each visitor control his/her own set of data, we combined identity information from passive, cheap radio frequency identification (RFID) tags and motion-sensing data from an off-the-shelf Microsoft Kinect camera (Cafaro et al., 2013). Using a combination of an Xbox Kinect camera and RFID allows each visitor to see, control, and interact with his/her own data. With the increased confidence we have in the system being able to uniquely identify and track visitors, we are able to allow the system to reconfigure data and visualizations in response to the unique collection of people within interaction range. In other words, CoCensus is able to issue personalized “invitations” to visitors to “step right up,” and gives the potential to provide automated guidance to visitors on the basis of the particular combination of visitor profiles currently “on exhibit.” This contributes to breaking down barriers and gets visitors to speak to each other (Cafaro et al., 2013).

Several open questions remain about how to engage multiple users, however. The remainder of this paper will discuss issues of negotiation of shared control, discoverability of control actions, and configuring interactions for a wide variety of users that CoCensus has faced in ongoing design iterations.

Negotiation of shared control

An important facet of any multiuser exhibit is the means of distributing control among multiple users. In the case of a data visualization exhibit, interactions with the exhibit change the representation of data, which is useful for seeing different phenomena, such as toggling between two decades to investigate a drastic change in a particular region or scrolling through several decades in order to illuminate a population’s growth over time. Therefore, the interactions of one visitor can interfere with or augment the investigations of simultaneous visitors, requiring a degree of negotiation of the exhibit control. As the number of available interactions has increased, the design of control actions has had to attend closely to how and whether visitors could share control.

Prior to the implementation of the timeline (i.e., when only modern data was available for exploration), interactivity was limited to manipulations of the transparency of data bubbles and the stacking order of the layers (i.e., whose bubbles were drawn “on top”). Moving closer to the display made one’s bubbles more opaque, obscuring the basemap on which they were drawn; if a visitor wanted to be able to orient herself to the underlying geography, she could move further from the display, making the data bubbles more transparent and the basemap more visible. Whichever visitor stood closer to the display would see his bubbles drawn on top of those of other visitors, enabling groups to toggle the layers so smaller populations were not entirely obscured by larger populations. This small amount of interactivity was unproblematic in terms of negotiation, but was limited in its ability to engage visitors: some moved back and forth in the space, but visitors did not seem to use it to aid their investigations. One visitor, Ben, even reported that he was just trying to find an “optimal distance” from the display at which he could view most easily both the data bubbles and the underlying map.

Negotiation of control became more important with the incorporation of the floor timeline for viewing multiple decades of data and the ability to switch among the four categories of data (ancestry, household size, housing type, and industry). The initial control action for a category change was to clap twice. This action could be completed by anyone in the vicinity of the exhibit, whether they were participants or observers, and could be carried out at any time. This allowed any person to disrupt the investigation of another person by changing the category, but it also allowed some pairs of visitors to expressly coordinate their actions, making statements like “Okay, next one,” and “Stop clapping!”

We hoped to augment this explicit coordination through a mutually exclusive timeline control, in which small marked spaces on the floor horizontal to the display acted as buttons to select a decade for all data sets (see Figure 4, left). This design is externally consistent, meaning it aligns with prior interfaces of timelines with which users might be familiar, specifically the “ticks” of a horizontal timeline, where left is the past and right is the future. If one visitor was standing on a button in this iteration, another user could not “steal” control; rather the first visitor would retain control of the timeline (thus manipulating everyone’s data) until he stepped forward or backward off the timeline. Again, this decision maintains external consistency; one would not expect a single map to display data from two different years. We hypothesized that this would prompt visitors to talk to each other to negotiate how to explore the data. However, we found in practice that this interaction design seemed to actually be stifling conversation. Visitors did not jointly engage; rather, one user would move on the timeline and another would either wait until the first was done, or would step onto unoccupied timeline buttons only to have no effect register on the screen, causing confusion. Thus we decided to test the alternative interaction design, that of individual control.

In this design, interactions were metaphorically consistent, drawing upon an “ego-moving” metaphor in which an individual progresses along a timeline “forward” toward the future or “back” in time. Therefore, the timeline was configured perpendicular to the display, with 1990 at the back of the interaction area and 2010 closest to the display (see Figure 4, right). Each individual was able to control his own data set, independent of his co-visitors’ movements. We predicted that this individual control would improve each individual’s interactions with the exhibit but that it may lead to confusion when two visitors tried to compare data from different decades, and the lack of forced negotiation would lead to less coordination of movements and visitor-to-visitor dialogue.

Contrary to our hypotheses, a study comparing visitor interactions and dialogue in the two conditions revealed that the ego-moving condition far surpassed the externally consistent condition in visitor interactions and data talk (Roberts et al., 2014). Instead of negotiating less because they weren’t forced to, visitors were more likely to actively coordinate movements in order to examine a particular phenomenon. We saw instances of pairs of visitors coordinating both implicitly—for example engaging in a “follow the leader” pattern where each time one visitor moved, the co-visitor followed—and explicitly, making statements like, “Step up. Step up,” and “Wait, hold up, come to 1990.” In some cases, visitors (primarily ten to thirteen year olds) physically pushed and pulled each other around and off the timeline. The ability to control one’s own data seemed to be immediately more engaging to the visitors, and that engagement prompted joint exploration and negotiation.


Interactivity in an exhibit is only beneficial if visitors know how to use it. In tangible user interfaces, physical objects can be labeled with an unambiguous “Press Me!” to indicate their functionality. Whole-body interactive systems do not have this affordance, so they need to attend carefully to the discoverability of interactive features. For this reason, CoCensus’s timeline control was mapped to the floor in order to make sure it was easily discoverable to visitors. Both configurations described above featured marked “buttons” in the interaction area. The mutually exclusive left-to-right horizontal timeline featured 8.5- by 11-inch buttons for each decade to emphasize that only one person could use a button at a time, while the ego moving front-to-back configuration employed larger rectangles the width of the interaction area, large enough for multiple users to stand in simultaneously (Figure 4). In both conditions, visitors had little trouble noticing the timeline, but in the mutually exclusive condition the large number of instances of visitors repeatedly stepping on an unoccupied button even though it had no effect suggests that the exclusivity of control was not easily discoverable. The ego-moving condition mitigated this problem as the display responded individually to participants.

Other issues of discoverability have been addressed with smaller design solutions. For example, data representations can be read more easily if viewers do not have to look back and forth between the visualization and a key (Shah & Hoeffner, 2002). In CoCensus, each user’s data sets is represented by a color. Rather than arbitrarily assigning a color to the user, she is allowed to select her own color as part of the mock mini-census survey kiosk. A key is still drawn on the display as a reminder of the selections and for the benefit of co-visitors, but the initial task of deciphering the key is made easier.

A known issue related to whole-body interactives is the lack of haptic feedback (Han & Black, 2011), meaning a physical cue providing information to a user. A physical controller such as a button or knob provides feedback when it is activated, such as clicking or depressing. Movements and gestures mid-air have no means of such physical feedback, so we instituted a slight animated “jiggle” to a person’s data bubbles when visitors move in the exhibit. The faster the visitor’s movements, the larger the perturbation of the jiggle. This slight visual cue reinforces the connection between a visitor’s physical movements and their digital data.

Discussion: Highly personalized interaction to engage multiple visitors

Through these iterations, we have learned that the version of the system that takes the most advantage of personalization (i.e., where each user controls his/her own timeline independently) proved to be the best at fostering engagement among multiple people. Initial hesitations about the potential for misinterpretations by visitors accidentally making cross-decade comparison between their data sets proved to be unfounded. Instead, visitors responded to the interactivity of the system by interacting and negotiating with each other in productive discussion. In our prior investigation into how to create data exploration entry points, we found that visitors identified with their data when they had a clear sense of control over it, but we knew that in a multiuser setting, requiring visitors to take control actions in order to build that personal connection to their data set would make joint data exploration chaotic. We found that incorporating a small amount of mirroring (the “jiggle” animations in response to the ambient movements of visitors) seemed to be an effective way to communicate the connection between the visitor and the data set they can control, even in the absence of them taking those control actions.

6. Conclusion

Learner-centered inquiry experiences are known to be a valuable educational tool (Dewey, 1985), but are difficult to support even in extended classroom interventions (Edelson et al., 1999) and especially in short museum-learning experiences (Gutwill & Allen, 2012). This ongoing iterative design process has sought to utilize embodied interaction (Hornecker, 2011; Cafaro et al., 2013) in a multiuser GIS-based data map display in order to support visitors’ collaborative data exploration. We identified two key challenges in supporting collaborative data exploration—helping visitors find an “entry point” into what might otherwise be an inaccessible mass of data, and supporting multiple simultaneous users—and we have described our attempts to address them through design iterations of an interactive data visualization exhibit. We have found that personalization can provide an entry point to data, and that this personalization might be even more effective if it is multifaceted. We also found that personalization need not be rooted in self-identity: it can be accomplished just by asking visitors to identify with a profile of their choice. We also found that in a multiuser scenario, it is just as important to communicate to visitors which aspects of an exhibit they can control as it is to make clear the effects of their control actions, and that using ambient mirroring animations can be effective at communicating this potential for control. To our surprise, visitors proved very able to coordinate among themselves when jointly exploring data, and this coordination seemed to fuel their continued probing of the data sets, suggesting that designers need not be overcautious in giving visitors power to manipulate data representations. While we used motion-sensing technologies in our exhibit, hypothesizing that they would offer special benefits for personalization and multiuser coordination, it is likely that some of the design findings we present here could be adapted to other interaction technologies. As long as key features like personalization, control attribution, communicating potential for control, and control negotiation are supported, an exhibit stands a good chance of supporting multiple visitors as they explore rich data sets.


Allen, S. (2004). “Designs for learning: Studying science museum exhibits that do more than entertain.” Science Education 88, S1, S17–S33.

Cafaro, F., A. Panella, L. Lyons, J. Roberts, & J. Radinsky. (2013). “I see you there! developing identity-preserving embodied interaction for museum exhibits.” In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1911–1920. ACM.

Cafaro, F., L. Lyons, J. Roberts, & J. Radinsky. (2014). “The uncanny valley of embodied interaction design.” In Proceedings of the 2014 Conference on Designing Interactive Systems, ACM, 1075–1078.

Dewey, J. (1985). “Democracy and education, 1916.” In J. A. Boydston & P. Baysinger (eds.). Carbondale: Southern Illinois University Press.

Edelson, D. C., D. N. Gordin, & R. D. Pea. (1999). “Addressing the challenges of inquiry-based learning through technology and curriculum design.” Journal of the Learning Sciences 8(3–4), 391–450.

Falk, J. H., & L. D. Dierking. (2000). Learning from museums: Visitor experiences and the making of meaning. Altamira Press.

Han, I., & J. B. Black. (2011). “Incorporating haptic feedback in simulation for learning physics.” Computers & Education 57(4), 2281–2290.

Gutwill, J. P., & S. Allen. (2012). “Deepening students’ scientific inquiry skills during a science museum field trip.” Journal of the Learning Sciences 21(1), 130–181.

Heath, C., & D. Vom Lehn. (2008). “Configuring ‘Interactivity’ Enhancing Engagement in Science Centres and Museums.” Social Studies of Science 38(1), 63–91.

Hornecker, E. (2010, January). “Interactions around a contextually embedded system.” In Proceedings of the Fourth International Conference on Tangible, Embedded, and Embodied Interaction, ACM, 169–176.

Hornecker, E. (2011). “The role of physicality in tangible and embodied interactions.” interactions 18(2), 19–23.

Pousman, Z., J. T.  Stasko, & M. Mateas. (2007). “Casual information visualization: Depictions of data in everyday life.” Visualization and Computer Graphics, IEEE Transactions 13(6), 1145–1152.

Radinsky, J., E. Hospelhorn, J. W. Melendez, J. Riel, & S. Washington. (2014). “Teaching American migrations with GIS census webmaps: A modified ‘backwards design’ approach in middle-school and college classrooms.” The Journal of Social Studies Research 38(3), 143–158.

Roberts, J., L. Lyons, J. Radinsky, & F. Cafaro. (2012). Proceedings from ICLS 2012. Connecting Visitors to Exhibits through Design: Exploring United States census data with CoCensus. Sydney, Australia.

Roberts, J., J. Radinsky, L. Lyons, & F. Cafaro. (2012, April). “Co-Census: Designing an Interactive Museum Space to Prompt Negotiated Narratives of Ethnicity, Community, and Identity.” In Josh Radinsky (chair), Tools for Constructing Historical Narratives: Teaching African American and Latino Histories with GIS Census Maps. Symposium conducted at the meeting of the American Educational Research Association, Vancouver, B.C., Canada. Available http://cocensus.uic.edu/?page_id=9

Roberts, J., L. Lyons, F. Cafaro, & R. Eydt. (2014). “Interpreting Data from Within: Supporting Human-Data Interaction in Museum Exhibits Through Personalization.” In Proceedings of the 13th International Conference on Interaction Design & Children. Aarhus, Denmark, 7–16. ACM.

Shah, P., & J. Hoeffner. (2002). “Review of graph comprehension research: Implications for instruction.” Educational Psychology Review 14(1), 47–69.

Cite as:
Roberts, Jessica, Leilah Lyons, Francesco Cafaro and Rebecca Eydt. "Harnessing motion-sensing technologies to engage visitors with digital data." MW2015: Museums and the Web 2015. Published February 1, 2015. Consulted .