The X, Y, and Z of digital storytelling: Dramaturgy, directionality, and design

Peter Samis, SFMOMA, USA, Tim Svenonius, SFMOMA, USA

Abstract

Keywords: digital storytelling, responsive design, narrative, information design, dwell-time, surf and dive

1. From digital dramaturgy to the journalistic model

When interactive multimedia was first being defined in the early 1990s, interface conventions were up for grabs. There were no real rules yet, and each new project faced the dual challenge of developing content and the interface within which to present it. Standards were still a long way off.

This was especially true in the early to mid-1990s, the short-lived “golden age” of CD-ROMs. Harald Kraemer (2007) has ably chronicled the “holistic approach to dramaturgy, navigation and design” that characterized early hypermedia. Kraemer notes: “Navigation can become content in itself and create a ‘reflexive liquid information architecture.’” In the early multimedia developer’s toolbox, he identified a variety of approaches, ranging from ludic storytelling to dramatic dialogue to exploration of a virtual infoscape.

These immersive approaches, each designed as both a means and an end, encapsulated a subject; in this respect, they resembled novels or films, but without the linearity of those forms. Instead, they were designed as multidimensional exploration environments, encouraging users to take a plunge into an custom designed experiential cosmos.

“The real challenge of such a dramaturgy,” wrote Kraemer, is “to find a way to design motivating modules of content within an overall visual and acoustic concept.”

In the years since that writing, digital publishing has undergone radical transformation, while design has witnessed a radical simplification. The Web as we know it now is highly optimized for quick and direct consumption across a wide variety of devices. While we continue to embrace the notion of “motivating modules of content,” we have left the “overall visual and acoustic concept” by the wayside. Increasingly, the varieties of immersive experience design and dramaturgy Kraemer documented have given way to a set of defaults that make up the common currency of today’s Web, no matter what the subject: either a short YouTube video or vertically scrolling pages punctuated with illustrations and an occasional video clip, garnished by a row of related links at the top or bottom. As an example:

Figure 1: a standard-issue vertical column of indeterminate length.

The single-column scrolling page has become the one-size-fits-all solution for any kind of information. Yet we know that most readers’ attention will drift on descent, and the number who reach the end of any story will be a tiny fraction of those who began.

Figure 2: The New York Times, in 1969 and the present. Skimming action is represented here by horizontal arrows. The vertical arrows represent the “deep dive,” or a perusing of an article text. With either format, a small fraction of readers is likely to persist to the end of the article. The dotted line indicates reader drop-off.

Even innovations in design, such as The New York Times’ now legendary “Snow Fall,” slip into this trap. How many who began reading the story, captivated by the immersive visuals, innovative parallax scrolling, sidebar vignettes, and suspense, ever made it through to the end?

It seems we have yet to learn how to reconcile our innate tendency to graze with the pleasure of the deep think.

This increasing reliance on one sole axis of storytelling—the Y—is surprising given the robust processing power, bandwidth, and digital literacy that characterize media consumption today. But viewers have little allegiance to individual stories as they skip from one search result to another, gleaning info-plankton from the sunlit surfaces and avoiding the commitments of a deep dive. Jakob Nielsen (2010, 2011) has famously suggested that the average length of a page visit is less than a minute, and that 80.3 percent of total time is spent “above the fold.” If Google is the baleen through which today’s infovores filter their diet, then it seems there are relatively few info-squids willing to take the plunge into the timeless realms of ocean below.

What has been lost and gained in this transition from the walled gardens of early CD-ROMs and insular Flash websites to the search-engine-optimized columnar Web pages of today’s hyperconnected infoverse?

Our info-foraging behavior seeks the maximum return from the minimum investment of energy and attention (Nielsen, 2003). In long columns of Web text, however, the potential distractions multiply as we navigate downward, fatigued as we are by the forced march and distracted by sidebar links. Every extraneous paragraph represents an impetus to leave the narrative-in-progress and hop instead onto another trail to find the stimulation we seek. Presented with a long scroll, we are offered no true choice except how long we are willing to persist.

This form promotes skimming or skipping over content, and most readers grow progressively less engaged. On the other hand, in a non-linear interactive experience, the reader gets to choose more frequently where he or she wants to proceed, renewing engagement with each decision point.

How is it that even though as readers we hesitate to commit to an extended Y-axis deep-dive experience, as authors we continue to write long-form narratives that rely on it? What would an X-axis complement be? How do we get to that elusive Z-axis of psychological and emotional depth, closure of the open loops of inquiry that motivate us to follow a story in the first place?

2. Five centuries of practice

We are not the first society to confront this paradox. If we are to believe historians of reading, a parallel situation occurred—not with the invention of the printing press in 1445, but rather 250 years later in both Europe and North America. Rolf Engelsing was the first to propose that the late eighteenth century saw a “reading revolution”: a transition from intensive reading, an activity that had largely characterized the literate few during the Middle Ages and Renaissance, even after the invention of the printing press, to extensive reading. This shift in both what and how people read has remained with us to varying degrees ever since.

Intensive reading characterized a Church-driven society in which people had very few books, mostly liturgical, which they read again and again on an annual round. The Enlightenment period, on the other hand, saw an explosion of free-thinking printed material: newspapers, encyclopedias, dictionaries, pamphlets, gazettes, and almanacs. The rage for reading popular literature, both high- and low-brow, brought with it an increase in analog literacy much as the tide of the Internet has recently carried the populace into increasing digital literacy. In both cases, authority sources have been threatened as a plethora of voices and perspectives has become available. What’s more, in both cases, a culture of the mashup has appeared: presented with a surfeit of literature, eighteenth-century readers “proceeded by excerpts, displacements, making connections” (Cavallo & Chartier, 2009).

So our info-skimming, multi-source, mashup society may well have begun in the late 1700s. But a counterforce, one encouraging the deep dive, erupted at the same time: the novel. Rousseau’s La Nouvelle Eloïse in France, Goethe’s Sorrows of Young Werther in Germany—these books became Zeitgeist sensations, cultural memes returning people to an intensive reading practice: they were passed from hand to hand, read aloud, committed to memory.

Such rapturous devotion to any one text seems highly unlikely today, in our info-scattered world where memes live but for a moment, mothlike. Any proliferation of complementary or competing narratives that may have characterized the past seems trivial compared to our fragmented, info-saturated, time-starved economy. But what parallels can we draw; what lessons can we learn?

In each case, a monumental technological breakthrough—the printing press or hypertext—has laid the groundwork for an eventual paradigm shift in the way people read. However, neither shift occurred instantly or completely; Gutenberg’s press didn’t annihilate the market for illuminated manuscripts overnight, but rather eroded it over two centuries. In a highly accelerated analogy, the Web has taken twenty years to mature into a format as stable and predictable as that of printed newspapers or magazines.

These changes parallel a newfound independence from established authority: in the first case, the church; in the second case, big media—networks like CBS, Fox, and Time-Warner. The shift is at first technological, then ideological.

3. Surfing and diving

In the eighteenth century as now, two impulses are present: one to surf (which we shall call the horizontal, X-axis); the other to dive (which we shall call the vertical, the Y). The X-axis, or surf, extends from a single point—the tweet—through teasers in sidebars of news sites to Flipboard magazine layouts, and on to the rows in a Pulse, Netflix, or Apple TV thumbnail grid (figures 3a–3d):

Figures 3a–3d: forms of surfing: by single ideas, subject areas, content providers, and entertainment genres.

Browsing Twitter’s endless cascade is a superficial skimming in the extreme. Tweets act as tiny billboards, each advertising an idea, a state of mind, or an off-site experience. The dive only happens on clickthrough (i.e., for an infinitesimal fraction of tweets).

Flipboard, Zite, and other aggregators—be they automated, personalized, or a combination of the two—expand on the teaser function of the tweet by previewing more of the destination content.

Figure 4: Pulse on the iPad, showing the browsing interface and preview of a single item. Thumbnails and titles in horizontal rows form the X-axis. In this example, a fifteen-minute TED video constitutes the deep dive or Y-axis.

Flipboard elegantly proposes a variety of views, ranging from quarter-page to fullscreen to, on occasion, the original Web source. What is gained:

• Enhanced access
• Cross-indexing
• Legibility

What is lost:

• The original graphic treatment
• The world in which the article was embedded
• The overall design concept, insofar as there was one

In a case like Flipboard, we can see the trade-off for this homogeneity. In essence, the reader is offered command of the X-axis, selecting from the vast number of content sources, thus winnowing down the ocean of available content to a manageable stream.

Figure 5: Flipboard on the iPad: custom-chosen topic grid, visually optimized for surfing. Clicking on a topic leads to the magazine spread seen in figure 3b.

These examples are only possible due to the uniformity of Web content formats today and their conformity to standards. Nowadays, each story is networked, shareable, part of a feed, and taggable. One leads to another in a stream of “Info Currency.” That Info Currency, in turn, can be turned into “Social Currency” by anyone holding the hose.

Where does this all lead? To a kind of expedient journalism. Reading optimized for browsing. An infinite sequence of related topics. We might ask, with T. S. Eliot: Where is the wisdom we have lost in knowledge? Where is the knowledge we have lost in information? The popular formats are expeditious, efficient, transactional. The poetics of visual/conceptual exploration have given way to a magazine aesthetic graced with occasional cinematic moments.

Is what we’re dealing with simply the replacement of Flash by WordPress? The outcome seems to be a restriction or siloing of storytelling modalities, with multimedia Web essays now invariably presented in columns, and immersive interactivity limited to the world of games.

Before ubiquitous broadband, before high-fidelity full-screen photos and streaming video, Web producers had to create graphical environments that were themselves immersive, evocative, and seductive. Now that we can tuck cinematic eye candy in columnar text formats, the demand for smart information design has receded, and the environments seem to have been reduced to plain vanilla.

Figure 6: Off to the races: a widescreen cinematic video clip dramatically usurps the breadth of the text column of The New York Times multimedia essay “The Jockey” (http://www.nytimes.com/projects/2013/the-jockey).

Figure 7: Before and after the cinematic clip, business returns to normal: the narrow Y-axis of standard long-form storytelling, proceeding chapter by chapter.

4. Questions for today

In today’s world of multi-platform publishing, shareability and responsive design have triumphed over the custom crafting of experience. Perhaps WordPress has seen such widespread adoption largely because it’s so discoverable, responsive, and plays well with others: it aggregates easily, flowing its contents into every device—and every feed. Unlike in the early days of Web and multimedia, the writers no longer sit in the same offices with designers or developers. The separation of form from content makes today’s publishing platforms possible, and each party plays his or her separate role, if only for efficiency’s sake.

Is there still room for custom experiences of the caliber of the ambitious early multimedia projects, with visual and acoustic treatments evocative of their subjects? Does any step in the direction of non-linear, feature-rich storytelling necessitate sacrificing the rich cross-linked potential of the modern Web? Is there still a way to make content wed a custom form so they work/play together to shape the reader’s experience—and yet have them travel?

Or—and this is a question we pose to the community—are the artfully crafted user experiences of old antithetical to responsive design and our multi-platform present?

Acknowledgments

Many thanks to Harald Kraemer, Fred Ritchin, and the SFMOMA team for conversations that informed this paper.

References

Cavallo, G., & R. Chartier. (2009). “A l’époque moderne, les trois revolutions de la lecture.” In Choses lues, choses vues, exhibition microsite. Paris: Bibliothèque Nationale de France. October 23. Last consulted January 25, 2015. Available http://expositions.bnf.fr/lecture/arret/01_5.htm

Kraemer, H. (2007). “Simplicity, Slowness and Good Old Stories as Strategies and Perspectives of Design in Hypermedia and Media.” In J. Trant and D. Bearman (eds.). International Cultural Heritage Informatics Meeting (ICHIM07): Proceedings, Toronto: Archives & Museum Informatics. October 24. Last consulted January 25, 2015. Available http://www.archimuse.com/ichim07/papers/ kraemer/kraemer.html

Nielsen, J. (2003). “Information Foraging: Why Google Makes People Leave Your Site Faster.”
June 30. Last consulted January 25, 2015. Available http://www.nngroup.com/articles/information-scent/

Nielsen, J. (2010). “Scrolling and Attention.” March 22. Last consulted January 25, 2015. Available http://www.nngroup.com/articles/scrolling-and-attention/

Nielsen, J. (2011). “How Long Do Users Stay on Web Pages?” September 12. Last consulted January 25, 2015. Available http://www.nngroup.com/articles/how-long-do-users-stay-on-web-pages/

Samis, P., & S. Wise. (2000). “Making the Punishment Fit the Crime: Content-driven Multimedia Development.” In D. Bearman & J. Trant (eds.). Museums and the Web 2000: Proceedings. Minneapolis, MN: Archives & Museums Informatics. Last consulted January 25, 2015. Available http://www.archimuse.com/mw2000/papers/samis/samis.html

Sanabria, J. Reinier, & P. Samis. (2014). “Particle or Wave? Linear and Non-linear Storytelling in Museums.” In N. Proctor & R. Cherry (eds.). Museums and the Web 2014. Silver Spring, MD: Museums and the Web. February 4. Consulted February 9–10, 2015. Available http://mw2014.museumsandtheweb.com/paper/mw-2014-paper-particle-or-wave-linear-and-non-linear-storytelling-in-museums/

Sherwin, K. (2014). “Beware Horizontal Scrolling and Mimicking Swipe on Desktop.” April 27. Last consulted January 25, 2015. Available http://www.nngroup.com/articles/horizontal-scrolling/

Shirky, C. (2008). “It’s Not Information Overload. It’s Filter Failure.” September 19. Last accessed January 25, 2015. Available http://blip.tv/web2expo/web-2-0-expo-ny-clay-shirky-shirky-com-it-s-not-information-overload-it-s-filter-failure-1283699

Zeldes, N. (2010). “Yes it IS Information Overload, Clay Shirky, not only Filter Failure.” May 18. Last accessed January 30, 2015. Available http://www.nathanzeldes.com/blog/2010/05/yes-it-is-information-overload-clay-shirky-not-only-filter-failure/