Simulating ideology in storytelling

The Story Extension Process, from Mei Yii Lim and Ruth Aylett (2007) Narrative Construction in a Mobile Tour Guide

Another great piece from Ruth Aylett, this time from 2007. Here, she and collaborator Mei Yii Lim are getting closer to what I’m aiming for, if taking a different approach. They kick off by describing Terminal Time, a system that improvises documentaries according to the user’s ideological preference, and an intelligent guide for virtual environments which take into account the distance between locations, the already told story, and the affinity between the the story element and the guide’s profile when selecting the next story element and location combination to take users to. They note that this approach could bring mobile guides “a step nearer to the creation of an ‘intelligent guide with personality'” but that it “omits user [visitor] interests”. (I can think of many of a human tour guide that does the same). They also touch on a conversation agent that deals with the same issues they are exploring.

This being a 2007 conference paper, they are of course using a PDA as their medium. Equipped with GPS and text to speech software, a server does all the heavy lifting.

“After [an ice-breaking session where the guide extracts information about the user’s name
and interests], the guide chooses attractions that match the user’s interests, and plans the shortest possible route to the destinations. The guide navigates the user to the chosen locations via directional instructions as well as via an animated directional arrow. Upon arrival, it notifies the user and starts the storytelling process. The system links electronic data to actual physical locations so that stories are relevant to what is in sight. During the interaction, the user continuously expresses his/her interest in the guide’s stories and agreement to the guide’s argument through a rating bar on the graphical user interface. The user’s inputs affect the guide’s emotional state and determine the extensiveness of stories. The system’s outputs are in the form of speech, text and an animated talking head.”

So, in contrast to my own approach, this guide is still story lead, rather than directly user led, but it decides where to take the user based on their interests. But they are striving for an emotional connection with the visitor. So their story elements (SE) are composed of “semantic memories [-] facts, including location-related information” and “emotional memories […] generated through simulation of past experiences”. Each story element has a number of properties, sematic memories for example incude: name ( a coded identifier); type; subjects; objects; effects (this is interesting its lists the story elements that are caused by this story element, with variable weight); event; concepts (this that might need a further definition when fist mentioned); personnel (who was involved); division; attributes (relationship to interest areas in the ontology); location; and, text. Emotional story elements don’t include “effects and subjects attributes because the [emotional story element] itself is the effect of a SE and the guide itself is the subject.” These emotional memories are tagged with “arousal” and “valence” tags. The arousal tags are based on Emotional Tagging, while the valence tag “denotes how favourable or unfavourable an event was to the guide. When interacting with the user, the guide is engaged in meaningful reconstruction of its own past,” hmmmmm.

So their prototype, a guide to the Los Alamos site of the Manhatten project, the guide could be either “a scientist who is interested in topics related to Science and Politics, and a member of the military who is interested in topics related to Military and Politics. Both guides also have General knowledge about the attractions.” I’m not convinced by the artifice of layering onto the interpretation two different points of view – as both such are being authored by a team who in their creation of the two points of view will, even if striving to be objective, will make editorial decisions that reveal a third, authentic PoV.

When selecting which SE to tell next, the guide filters out the ones that are not connected to the current location. Then “three scores corresponding to: previously told stories; the guide’s interests; and the user’s interests are calculated. A SE with the highest overall
score will become the starting spot for extension.” The authors present a pleasingly simple (for a non-coder like me) algorithm for working out which SE goes next. But the semantic elements are not the only story elements that get told. The guide also measures the Emotional, Ideological story elements against the user’s initial questionnaire answers and reactions to previous story elements and decides whether or not to add the guide’s “own” ideological experience on to the interpretation, a bit like a human guide might. So you might be told:

Estimates place the number of deaths caused by Little Boy in Hiroshima up to the end of 1945 at one hundred and forty thousands where the dying continued, five-year deaths related to the bombing reached two hundred thousands.

Or, if the guide’s algorithms think you’ll appreciate it’s ideological perspective, you could hear:

Estimates place the number of deaths caused by Little Boy in Hiroshima up to the end of 1945 at one hundred and forty thousands where the dying continued, five-year deaths related to the bombing reached two hundred thousands. The experience of Hiroshima and Nagasaki bombing was the opening chapter to the possible annihilation of mankind. For men to choose to kill the innocent as a means to their ends, is always murder, and murder is one of the worst of human action. In the bombing of Japanese cities it was certainly decided to kill the innocent as a means to an end.

I guess that’s the scientist personality talking, perhaps the military personality would  instead add a different ideological interpretation of the means to an end. As I mentioned before, I’m not convinced that two (or more) faux points of view are required when the whole project and every story element that the guide gets to choose from are already authored with a true point of view. But in many other aspects this paper is really useful and will get a good deal of referencing in my thesis.

Forgive me, this will make little sense

I feel I need to record this here, but I fear it will be nonsense to most of my readers. Looking back at it it feels like the first step into incoherent PhD madness. So skip this one if you are looking for sense and inspiration.

I’m struggling with the Chawton project, tying myself up into narrative knots. Meanwhile my collaborator Ed is powering on with his part. Today I’ve been listen to his first pass at a number of audio mixes. Which sound great by the way, and make me a bit depressed at the lack of progress my part has been making.

The problem is all to do with the paradox of scripting an emergent story. When I brief Ed part of wants to say, “we’ll tell this bit of the story in this room, and so you need to use this sound,” but that defeats my object of trying to create stories with some sense of order whatever route visitors take through a place. So I wrote Ed a short but still rambling email, that I think captures what I’m getting at. Though it might be bollocks.

Anyhow I don’t want to lose it. And so since I started this blog as a notebook, rather than a finished demonstration of my finely honed wisdom, you  are about to get an actual note to self, the slightly edited text of that email. We can all work out what it means, if anything at all, later:

I’ve had a little epiphany thinking about your question. But I’m finding it difficult to put into words.  You asked whether the six beats are connected to the recorded quotes. I said no, and I still think that but I also said that a couple might be relevant, and so they might, BUT (I think) not so relevant that you should mix them into the finished work. The ones I was thinking of were: Fanny Knight, Mr Knightly and Jane Austen. But none of those enhance any one particular beat, do you get me? So don’t need to be missed in so they are heard every time. The beats set the mood, or rather, illustrate a particular mood. So… I think we’ve looking for sound mixes that accompany the beats, eg Loneliness. In the fancy system that doesn’t exist yet, the system might choose to interweave the “lonely” track with the Fanny Knight quote. But we don’t need to do that, or rather to fix it as that, in our rough and ready version.

But that lead me to another thought. (The Epiphany.) Which is that we (I) can’t afford to put speakers everywhere. So we need to select rooms that we are putting speakers in, and for each room WITH A SPEAKER, create a choice of soundmixes, that match the whatever beat the operator (the Unguide in our case, a fancy system in the future) chooses in that Room.

So, day we have a ball in the Great Hall – at the very basic level we might have a choice of a Jolly mix (for Up beats), or a sad mix (for setbacks), do you get me? We could have six different mixes for the six beats I’ve identified ALL for the Great Hall, so that whichever beat is selected has its own music appropriate for the great hall, and ANOTHER six choices for, say, The Oak Room. Am I making sense? But of course we’ve actually got three stories, and even if they all share the first beat, that would mean a possible five mixes for the Library story, and ANOTHER five for the Montague story, for EACH room? Crazy huh?

But all of that could be a LOT of work for you, so we need to keep our ambitions in check.

SO:

Tell me how many mixes you want to create, and together we can decide on a limited number of rooms where we’ll put speakers (we did day nine, but I can live with just one), and a limited number of mixes for each speaker (I think we want at least two for this experiment). How does that sound?

Building the Revolution 

I finally got to the V&A today, for their exhibition You Say You Want a Revolution. I got turned away at the end of Cromwell Road last time, as the museum was being evacuated after a bomb-scare. 

I’m writing this review on my way home, using my phone (so please forgive my typos) partly because I want to recommend you go, and there is not long left to see it. 

The exhibition charts the western cultural revolution of 1966-1970, though John Peel’s record collection, plysbof course fashion and design from the V&As own collection and other items, such as an Apollo mission space suit borrowed from other institutions. 

One of the gimmicks of the show is the audio, an iteration of the same technology used at the Bowie exhibition a couple of years ago. I didn’t get to go to that one, but I had a demonstration of that tech from the makers Sennheisser, at a Museums and Heritage show. 

I wasn’t very impressed. Though these headphones, which play music or soundtrack to match whatever object or video you are looking at, were well  received by the media back then, in my experience the technology was clunky. Other friends who’d been confirmed that they changes between sound “zones” could be jarring, and that it was possible to stand in some places where music from two zones would alternate, vying for your attention. 

The experience this time was an improvement. It was by no means perfect: I found the music would stutter and pause annoyingly, especially if I enjoyed the track enough to find myself gently nodding my head. Occasionally the broadcast to everyone’s headphones would pause so everyone in a room could share a multimedia experience (of the Vietnam war for example) across all the gallery’s speakers, screens and projectors. These immersive over-rides were effective, in much the same way as those at IWM North, but when a track you were enjoying or a video that you found interesting was rudely interrupted, one couldn’t help but feel annoyed. I found myself forgiving the designers however, for this and even the stuttering sound of the headphones, because it all felt resonant with that late sixties “cut-up” technique. 

Where the technology really worked however was on two videos that topped and tailed the exhibition. In the first various icons and movers of the period were filmed in silent moving portraits of their current wrinkled and grey selves. Their reminiscences of the time appeared as typography overlaying their silent closed-mouth gaze, a little like Barbera Kruger’s work, while  over the headphones you heard their voice. The same characters appeared at the end, that s time as a mosaic of more conventional talking heads. And for the first time, the interpretation was didactic as each in turned challenged the current generation to build on their legacy. 

For me, one of the highlights was the section on festivals, which invited visitors to take off their headphones, lie back in the (astro)turf and let (another cut-up of) the famous Woodstock documentary wash over them on five giant screens. 

The other things I loved were, dotted around among the exhibits, tarot cards that, at first glance, looked like they might have been designed in the sixties. But then you notice references to things like Tim Berners Lee and the World Wide Web. You realise these are a subtle form of interpretation, telling a future of the sixties that apparently came true and for those of us from that future, creating correspondences and taxonomies that connect the events of 1966-70 with today. The V&A commissioned British artist Suzanne Treister to create the cards, based on her 2013 work, Hexen 2.0. And the very best thing about them is you can buy them (pictured above) in the shop which must be the first time copies of museum interpretation panels have been made available for purchase. 

Of course, the aren’t the only form of interpretation. About from the soundtrack, there are more traditional text panels, labels and booklets around the exhibition. But the cards show how cleverly the layering of meaning and interpretation has been created. Many visitors will have passed them by unnoticed, given them a cursory glance or chosen to ignore them, and will have had an entirely satisfactory experience. But for those that paused to study them in more detail a whole new layer of meaning opened up. 

I visited with a sense of duty, to try out a responsive digital technology. But I found so much more to enjoy. This is a brilliantly curatored exhibition. So much better than the didactic, even dumbed down permanent gallery of the new Design Museum which I visited before Christmas. I urge you to go, if you haven’t seen it yet. It’s only on for another month. 

A colleague who had visited the exhibition before told me how depressed it had made him: the optimism of that period seems to have been dashed upon the reactionary rocks of 2016, Brexit and Trump. But I came out with a very different mood. 

One of the early messages of the exhibition is the period as a search for utopia. The final tracks you hear as you walk out (after the video challenge issued by the old heads of the sixties) are Lennon’s 1971 single Imagine and then, brilliantly, Jerusalem

No, of course they didn’t find the Utopia they were looking for in the sixties, but we could build it…

Mind Control and responsive narrative

Among the mince pies and over-cooked turkey over Christmas, I managed to find a little time to read an interesting paper. #Scanners: Exploring the Control of Adaptive Films using Brain-Computer Interaction shows once again, that the cool people are all at the University of Nottingham. What these particular four cool guys did was put a mini cinema in an old caravan. But this particular cinema wasn’t showing an ordinary film. Rather, the “film was created with four parallel channels of footage, where blinking and levels of attention and meditation, as recorded by a commercially available EEG device, affected which footage participants saw.”

Building on research in Brain Computer Interface (BCI) the team worked with an artist to create a filmed narrative that “ran for 16 minutes, progressing through 18 scenes. However, each scene was filmed as four distinct layers, two showing different views of the central protagonist’s external Reality and the other two showing different views of their internal dream-world.” Which layers each viewer saw was selected by the EEG device, for rather by the viewers’ blinks and states of “attention” or “meditation” as recorded by the device. The authors admit to some skepticism from the research community about the accuracy of the device in question, but that was not what as being tested here. Rather, they were interested in the viewers’ awareness of the ability to control the narrative, and their reaction to that awareness.

I was interested in the paper for two reasons. First of all, their conclusions touch upon an observation I made very early in in my own research, looking at Ghosts in the Garden, I got a small number (therefore not a very robust sample) of users of that interactive narrative to fill out a short questionnaire, and I was surprised by the number of respondents who were not aware that they could control (were controlling) the story through the choices they made. The #Scanners team noticed a similar variation in awareness, but more than that, they found that “while the BCI based adaptation made the experience more immersive for many viewers, thinking about control often brought them out of the experience.”

They conclude that “a traditional belief in HCI is that Direct Manipulation (being able to control exactly what you want to control) sits at the top of both these dimensions. We examined, however, how users deviate from line, and enjoyed the experience more by either not knowing exactly how it worked, or by giving up control and becoming re-immersed in the experience. […] these deviations from the line between knowledge and conscious control over interaction are most interesting design opportunities to explore within future BCI adaptive multimedia experiences.”

With which, I think I agree.

The other reason the paper interests me is that they described their research as “Performance-Led Research in the Wild” and pointed me towards another paper to read.

Chawton Untours and more

It’s a funny feeling time. The calendar pages seem to flicker by as the year rushes towards its end, the the deadlines for various aspects of the Chawton project loom ominously. On one level I worry I have achieved so little and yet, on an other so much has gone on. So it seems inevitable that this post will consist of a number of short catch-ups on various aspects.

Untours

First of all, I’ve got a name for what we offer the public next year. I’d been struggling to think of how I’d present the project to Chawton’s visitors in a way that meant something. I’ve been calling it “the project”, “my experiment” or a “responsive environment”, none of which would sell the concept to potential participants. But a few weeks back I met a colleague who told me about an experimental opening of the Roundhouse in Birmingham. Working with a couple of performance poets, they opened the building for sneek previews that they called “Un-Tours“.

The National Trust’s Un-Tours are not quite the same as what I’m planning of course. But I thought it was a perfect name: visitors will explore the house with a volunteer, but the volunteer won’t be a guide leading them from room to room. They choose where they go, and what they look at, and the volunteer responds to their interests with the relevant natoms. So my volunteers are Unguides, and the tours, Untours (I decided we didn’t need the hyphen). I told my colleague there and then that I was nicking the name.

A collaborator!

The next exciting thing that happened was meeting Ed Holland. Ed is studying Music at Southampton, and was looking for a studio project. He has agreed to help me with the sound natoms. I met him for a second time yesterday, with the always brilliant Jeanice Brooks, and we started to break the musical narrative, focused on domestic life at the turn on the eighteeth/nineteenth centuries, which will reference the Jane Austen connections that Chawton has, without being about her (given there’s a museum dedicated to her just down the road).

Talking about sound

Of course between those two meetings with Ed, I’ve been thinking a lot about sound. As long time readers may be aware, I’m keen to put as few barriers/filters as possible between the visitor and the space they are in. So my preference is always for speakers, but Ed suggested that headphones may offer a more immersive soundscape for less money.

However, one of the key investigations of this project is to investigate a set of “contention rules”, for when more than one visitor/visiting group enter the same space with different story needs. Of course, if everyone were wearing headphones, that soundscape contention wouldn’t be an issue. Which may be a good thing (for visitor experience) as well as a bad one (for my investigation). I’ve also been thinking about other ways my paltry budget might limit what we can achieve. I hope to store all the assets on the web (in Scalar currently) so that a volunteer Unguide can use any smart device to participate (BYD). But of course, that will (I’m thinking – you may know differently) limit each Unguide to delivering just one channel of sound to his/her visitor group. Of course that limits Ed’s ambitions for a multi-channel directional soundscape, but he is making contact with some of the sound guys in our School of Engineering to see if there’s any cool stuff (or speakers) we can use at Chawton.

Assuming we don’t get to borrow anything cool though, I’ve suggested that Ed:

  • Works on a creating a music/sound library based on the lowest spec – single channel a cheap Bluetooth speaker in each room.
  • Specifies the hardware requirements for a system that might deliver his ideal soundscape, either using a multichannel directional speaker system or headphones (Imagine 20 headphone users in the house at the same time). I can guarantee I won’t be able to afford it, but it would be useful research anyway. And we could test a limited version of the concept, with borrowed equipment, during the pilot stage (currently scheduled from the beginning of December in my project plan).

My budget, though tiny, is flexible (it’s my own money) so, I could maybe stretch to something in between the two extremes, if it was something that offered some of the functionality Ed would really want, and maybe had some domestic life afterwards.

Story troubles

The thing that I’ve had most trouble with these last few weeks is the story. I wanted to have at least three narratives – one of the history of the building (and I thought an early 20th century owner, Montague Knight, would be the easiest focus for that); one on Women’s Literature, and the Austen one, mentioned above.

I’n my innocence I thought that I would quickly knock-out an emotionally compelling Montague Knight narrative, but after weeks of reading, arranging and re-arranging, I’ve realised that (duh!) real life stories don’t comply with literary “rules”. Or rather, I’ve realised that maybe my standards, my expectations, for this were too high. I’ve wasted time trying down a rabbit hole, trying to craft a story that I was going to muck up anyway by letting visitors make their choices. I was crafting a traditional guided tour, not an Untour! So, I’ve decided on a different tack. Instead, I’m going to spend some time analysing the natoms I already have, and attributing a story beats to each one. The story should (after all) be procedural.

The outcome of this experiment isn’t (wasn’t ever) meant to be the best interpretive experience. all it is is a step towards the understanding how procedural narratives might work in historic spaces.

Mulholland on Museum Narratives

Working on the narratives for the Chawton Project, I’m taking a break and catching up on reading. Paul Mulholland (with Annika Wolff, Eoin Kilfeather, Mark Maguire and Danielle o’Donovan) recently contributed a relevant first chapter to Artificial Intelligence for Cultural Heritage (ed Bordoni, Mele and Sorgente).

Mulholland et al’s chapter is titled Modelling Museum Narratives to Support Visitor Interpretation. It kicks off with the structuralist distinction between story and narrative,  and points to a work I’ve not read and should dig out (Polkinghorne, D. 1988 Narrative Knowing and the human sciences) as particularly relevant to interpreting the past. From this, the authors draw the “narrative inquiry” process which “comprises four main stages. First, relevant events are identified from the historical period of interest and organised into chronological order. This is termed a chronicle. Second, the chronicle is divided into separate strands of interest. These strands could be concerned with particular themes, characters, or types of event. Third, plot relations are imposed between the events. These express inferred causal relations between the events of the chronicle. Finally, a narrative is produced communicating a viewpoint on that period of history. Narrative inquiry is therefore not just a factual telling of events, but also makes commitments in terms of how the events are organised and related to each other.” Which is as good and concise a summary of the process of curatorial writing as I am likely to find.

There’s another useful summary paragraph later in the document. “When experiencing a museum exhibition, the visitor draws relationships between the exhibits, reconstructing for themselves the exhibition story (Peponis 2003), whether those relationships are, for example, thematic or chronological. The physical structure of the museum can affect how
visitors perceive the exhibition narrative. Tzortzi (2011) argues that the physical structure of the museum can serve to either present (i.e. give access to the exhibition in a way that is independent from its underlying logic) or re-present (i.e. have a physical structure that reinforces the conceptual structure of the exhibition).” Tzortzi there is another reference I’ve not yet discovered and may check out.

What the paper does not do however, is make any reference to emotion in storytelling. the authors seem to leave any emotional context the the visitors’ own meaning making. The chapter include a survey of current uses of technology in museums, and academic experiments including virtual tour guides and opportunities to add the own interpretations and reminiscences, as well as web-based timelines etc.

But, digital technology gives us the opportunity (or need) to break down cultural heritage narratives even more, and an earlier (2012) paper by (mostly) the same authors, Curate and Storyspace: An Ontology and Web-Based Environment for Describing Curatorial Narratives describes a system for deeper analysis. (Storyspace turns out to be a crowded name in the world of writing tools and hypertext, so eventually the ontology and Storyspace API became Storyscope). The first thing that the ontology brings to the table is that

a curatorial narrative should have the generic properties found in other types of narrative such as a novel or a film

So the authors add another structuralist tool, plot, to the story/narrative mix. “The plot imposes a network of relationships on the events of the story signifying their roles and importance in the overall story and how they are interrelated (e.g. a causal relationship between two events). The plot therefore turns a chronology of events into a subjective interpretation of those events.” But using the narrative inquiry process “the plot can be thought of as essentially a hypothesis that is tested against the story, being the data of the experiment.”

I like this idea. But its worth distinguishing between the two uses of the word “interpretation” in cultural heritage. The first use, familiar to my archaeologist colleagues, describes the process of building an understanding of of aspect of the past from the available evidence. The second, more familiar to my museum and heritage site colleagues describes the process of explaining the evidence to non-professional visitors. At its very best, the museum/heritage site form of interpretation will resemble and guide visitors though the process of inquiry that builds an understanding of the evidence on display. But most of the time the second form of interpretation more closely resembles storytelling. That’s not a fault or failure of my museum/heritage site colleagues, most visitors are time poor in story rich environments. But digital technology has the potential to allow museum and heritage site interpretation to more closely resemble the first use of the word.

What digital technology offers, is the opportunity for brave curators to offer alternative plots, or theses, and test them in a public arena, rather than just through a peer review process. Or even to create plots procedurally by following the visitors’ path of attention between objects, maybe discovering plots the curator had not imagined.

The two experiments that the authors describe go someway towards this, by their dry ontology misses an emotional component. The event ontology could surely include an authorial opinion on whether the narrative element suggests a simple emotional reponse (even as simple as hope or fear) but instead “If the tag represents an artist, then events are used to represent, for example, artworks they have created, exhibitions of their work, where they have lived, and their education history.” Dry, dry facts… There is the tiniest nod towards, if not emotion per se, the some sort of value in their brief discussion of theme:

Theme is also related to the moral point of the story. This could be a more abstract concept, such as good winning through in the end, which serves to bind together all events of the story.

Given that they say “Narratives are employed by museums for a number of purposes,
including entertainment” they haven’t given much time to what makes narratives engaging. There is hope however. In their conclusion, they do say “Other narrative
features such as characterisation and authorial intent could potentially be
foregrounded in tools to support interpretation.”

 

Interactive story beats

In my exploration of interactive storytelling I’ve concentrated on computer games, because I’m exploring the digital delivery of story. But I’ve already decided that for my experiment at Chawton next year, I’m going to “wizard of Oz” it – use actual people instead of trying to write a computer program to deliver the interactive narrative.

I’ve been thinking about the issues around that. People are natural storytellers, though some are better than others, so I have a double edge problem. As I recruit and train people to be my “wizards of Oz”, I need to train the poor story-tellers to be better, and weirdly, I need to train the great storytellers to be worse! My reasoning is this, I want to prototype what a computer might do, there’s little or no experimental value in simply enhancing a great storyteller’s natural ability with some environmental bells and whistles. So part of what I’m trying to learn is about how to systematize (is that a word? It’ll do) story.

I’ll explain about Kernels and Satellites of course, but I need (I think) some sort of simple system of identifying how different story elements might fit into the emotional journey the visitor is going to take.

So, I’m reading Robin D. LawsHamlet’s Hit Points. Laws is a game designer but mostly of tabletop, or “pen and paper” role-playing games (though he has written for some computer games too). This book attempts to systematize (I think it is a word) story, with an audience of role-playing gamers in mind. I think it may be useful for me, because it attempts to train the Game Master of such games (the “referee” who, together with the players, makes the story) to be aware of the emotional impact of each scene or action (which he calls, using a screen-writing term, “beats”) on the players, and better choose which element to serve up next to keep everyone emotionally engaged. Tabletop Roleplaying Games must be the most interactive, responsive, stories ever created. In a way, my “wizards of Oz” will be like a Game Master, not telling a story they prepared earlier, but working with their visitors to create a story on the fly, but keep it emotionally engaging.

In a handy short opening chapter called “How To Pretend You’ve Read This Book” Laws explains “With its system of beat analysis, you can track a narrative’s moment-to-moment shifts in emotional momentum. Beat analysis builds itself around the following very basic fact:

Stories engage our attention by constantly modulating our emotional responses.”

Sadly though, I can’t get away with reading just this chapter. It’s only later that he actually shares the classification of beats that he uses in his analysis.

Hamlet’s Hit Points Icons and Arrows by Gameplaywright LLP and Craig S. Grant is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

He begins with two types that he says will make up the majority of beats in any story, Procedural and Dramatic beats. Procedural beats move the protagonist towards (forfilling the audience’s hopes) or away from (realizing the audience’s fears), his practical, external goal. Dramatic beats do the same for the protagonist’s inner goals. “We hope that the beat moves him closer to a positive inner transformation and fear that it might move him towards a negative transformation.”

Laws talks a lot about hope and fear. In fact he simplifies the audience’s emotional response to every beat (which he describes as its resolution) as being a movement towards one of these poles. I’ve got fear on my nascent emotional affect and affordances diagram, its one of Panksepp’s primal emotions, but I’m not yet sure where hope sits – I wonder, is it in care?

In both types of beat, Laws describes two parties, the petitioner, who wants the thing, and the granter, who must be negotiated with. Dramatic beats are mostly actual verbal negotiations, procedural beats might also be fights, tricks, races or other challenges.

From the way Laws describes them, I’d expect that most kernels in a story are likely to be one of these two types of beat. And the other types are more likely to be satellites. He lists:

Commentary – “in which the protagonist’s movement towards or away from his goal is momentarily suspended while the author underlines the story’s thematic elements.” Laws uses Mercutio’s Queen Mab speech in Romeo and Juliet as an example.

Anticipation – which “create[s] an expectation of coming procedural success, which we look forward to with pleasure.” The example here is “Popeye has eaten his spinach. (any given episode of Popeye)”

Gratification – “a positive emotional moment that floats free from the main narrative. They often appear as rest breaks between major sequences. A musical interlude often acts as a gratification beat (unless it also advances the story, as it frequently does in musical genre).”

Bringdown – the opposite of gratification. “Jerry Lundergaard’s car alone in a desolate parking lot, is completely iced over after his father-in-law bars him from a promising business deal. (Fargo)”

Then Laws offers us three “informational beats”:

Pipe – “A beat that surreptitiously provides us with information we’ll need later, without tipping the audience to its importance.”

Question – “introduces a point of curiosity we want to see satisfied […] a question usually resolves as a down beat.”

Reveal – “provides the information we were made to desire in a previous question beat, or surprises us with new information. In the latter case it might come out of the blue, or have been set up with one or more pipe beats laying the groundwork for the surprise.” The example he uses is the Revelation that Bruce Willis’ character in The Sixth Sense is dead. “We tend to be more engaged by exposition when it has been teased to us by a prior question, or can clearly see its impact on our hopes and fears.”

(Laws explains that literary fiction makes much use of question/reveal cycles to manipulate emotion, rather than the procedural / dramatic beats that fill genre fiction and thrillers.)

Laws goes on to analyse three scripted narratives in full, Shakespeare’s Hamlet, and the films Dr No and Casablanca, but that’s not what I’m discussing now, though having recently rewatched Casablanca as part of my children’s continuing cinema education, I was  interested to read his analysis of that. It is worth pointing out, however, that the “curve” of a story like Casablanca is inexorably downward. Laws compares the maps his analysis creates with “the classic chart you may recall from secondary school literature classes” (which I’ve touched on before) and notes that the lines his analysis creates are “flatter overall. It tends to resemble a stock tracker measuring the progress over time of a slowly deflating security […] Even stories that end happily […] tend to move downward over time.” He explains that narratives build up fear with numerous incremental steps, before sudden uplifting moments of hope. So in most stories, there are simply more down beats than up beats, given that the up beats are more intense. I think there is also a point that Laws misses, in many of those narrative curves the absolute value of emotional intensity is being measured, with no thought as to whether the emotion is hopeful or fearful.

So, is all this useful to me? Well I think at the very least I think I can get my “wizards of Oz” to think about up beats and down beats, and make sure not to pile on too many down beats in a row without the occasional up beat. Whether or not heritage interpretation lends itself to procedural and dramatic beats, there is definitely room for question/reveal beats, and it could be argued that too much interpretation goes straight for the revelations without asking the questions or laying the pipes first. So I think it is something that may prove useful.

A little bit of the history of interactive storytelling at Chawton


I spent yesterday morning at Chawton, locating and counting plug sockets so @ll know my limitations as I design whatever the experience there will be next March. The visit reminded me that I had meant to write here of a previous digital interpretation experiment at Chawton.

Back then, in 2005, the Chawton House Library was not widely open to the public. Primarily a centre for the study of Women’s Literature, one could argue that the visiting academics were also heritage visitors of a sort, and the house and gardens also welcomed some pre-booked visiting groups, such as the Jane Austen Society of America, and local garden societies. In their conference paper, a team from the universities of Southampton and Sussex describe how, looking for “curators” to work with they co-opted the trust’s Director, Estate Manager, Public Relations Officer, Librarian and Gardener. All these people may have taken on the role nut just of curator, but also guide to those visiting academics and groups. The paper attempts to describe how their tours interpret the place:

visitors’ experience of the house and its grounds is actively created in personalized tours by curators.

“House and grounds are interconnected in a variety of ways, e.g. by members of the family rebuilding the house and gardens or being buried in the churchyard. Thus artifacts or areas cannot be considered in isolation. There are many stories to be told and different perspectives from which they can be told, and these stories often overlap with others. Thus information exists in several layers. In addition, pieces of information, for example about a particular location like the ‘walled garden’, can be hard to interpret in isolation from information about other parts of the estate – there is a complex web of linked information.[…]

“Curators ‘live the house’ both in the sense that it is their life but also that they want to make it come alive for visitors. The experiences offered by Chawton House are intrinsically interpersonal – they are the result of curators interacting with visitors. Giving tours is a skilled, dynamic, situated and responsive activity: no two tours are the same, and depend on what the audience is interested in. They are forms of improvisation constructed in the moment and triggered in various ways by locations, artefacts and questions.”

Tours are a brilliant way of organising all those layers of information, and I’m sure a personal tour from any one of the curators that they identifies would have been excellent. But the problem comes as soon as you try to scale, or mass produce, the effect. As I said at a conference I presented at a couple of weeks ago (I’m reminded I should write about that too) people, even volunteers, are an expensive resource, and so only the smallest places can afford to give every visitor a guided tour experience. Even then, individuals or families have to book on to a tour, joining other people whom they don’t know, and whose interests they don’t necessarily share. The guided tour experience gets diluted, less personal, less tailored to your interests. Which is when you start getting people saying they would prefer to experience the site by themselves, rather than join a tour. Of course some tour guides are better at coping with these issues than others, but visitors are wary of taking the risk with a guide they don’t know, even if they can recount experiences of brilliant guided tour experiences.

The project written about in the paper had two sides, one was to try and produce content for schools, but the other was of particular interest to me:

“The curators are interested in being able to offer new kinds of experience to their visitors. We aim to find out what types they would like to offer, and help to create them. There is thus a need for ‘extensible infrastructure’ based on a basic persistent infrastructure that supports the creation and delivery of a variety of content.”

And four questions they ask themselves are also of particular interest:

  • “How can we enable curators to create a variety of new experiences that attract and engage different kinds of visitors, both individuals and groups?
  • “How do we engage curators in co-design of these experiences?
  • “How can curators without computer science backgrounds contribute to the authoring of content for the system?
  • “How do we create an extensible and persistent infrastructure; one that can be extended in terms of devices, content and types of experience?”

At the time of writing the paper, they had conducted a workshop with their chosen curators, using a map with 3D printed features. Although “use of a map in the first instance may have triggered somewhat different content,” they discovered that “Eliciting content from curators is most naturally and effortlessly done in-situ.” (Which is my plan – I’m in the process of fixing a date with one of Chawton’s most experienced tour guides.)

I particularly liked the observation that “Listening to them is much more lively and interesting than listening to professionally spoken, but often somehow sterile and dull audio tapes sometimes found in museums and galleries.” So enthusiastically did the team connect with the curators’ presentation, that they decided to record the tours and edit them into the narrative atoms that were delivered by their  infrastructure. That infrastructure was not the subject of the paper, but if I recall correctly, GPS based running on “Palm Pilot” style hardware.

More importantly, the most pertinent conclusion was that the curators were best placed, not just to select the narrative atoms from the recorded materials but also “sort them into
themes and topics, so that the system can cater for people with different broad interests, for example landscape, flora and fauna, or how Jane Austen’s writing reflects the environment. This necessitates a learning process, which must build on existing practices and over time develops new practices based on experience and reflection.”

Storyplaces

Last Sunday I helped out with a trial of Storyplaces, a research project exploring the poetics of location based storytelling. The exploration has two big questions behind it: How do writers change what they do to write locatative text? and, How does experiencing text “on location” affect the reader?

My job for most of the day was to follow and observe readers as they used the stories (which are available as an HTML5 web-app, when you are next in Southampton), to ask them a few qualitative questions and record their answers. But before any volunteer test subjects arrived I got to give a story a go myself.

I chose The Titanic Criminal in Southampton, which took me on a walk from the Tudor House where we were based to the area known as Chapel, where my story started on the site of a working man’s house on Chapel Street. Even before the story started, I was in “storyspace” on the way to the start point. I’m not that familiar with Southampton (apart from the docks) so as I walked I was exploring new spaces. Was it novelty or the idea that a story was about to begin that made everything seem so magical? Or was it the eerily beautiful liturgy sung through the doors of the Greek Orthodox church I passed?


That sound stopped me in my tracks, and I loitered until the verse was finished, but it set up expectations that were ultimately disappointed. I was ready to be blown away by the poetics of space and story, and when I got to the start point, just other other side of a level crossing, even the run-down post industrial scene that greeted me had a certain ephemeral quality as I read the story of the houses that used to stand on this spot.

Then my phone directed me to the next location, The Grapes, a pub on Southampton’s Oxford Street. Storyplaces does not suggest a route, it just shows you the location(s) on a map from OpenStreetMap. So I followed parallel to the railway line a little way, then crossed it over a foot bridge, feeling very much as though I was on a little adventure. The Grapes has a wrought-iron sign dating from the early twentieth century, which the text of the story pointed out. But at this point I came to realise that this particular story sat uncomfortably half-way between an imaginative narrative based on fact and a guided tour of Southampton. My professional interest began to impinge on my enjoyment of the story, and I couldn’t immerse myself any more in the narrative.

And then the story broke. The text offered a link to a video on the BBC website, which failed to play, but succeeded in emptying my browser’s cache, meaning I couldn’t get back to my place in the story. I went back to base to carry on with volunteering.

I was lucky enough to we assigned to observe the writer of one of the stories as she tried out the app for the first time. We talked a little about her process of writing, and translating her imagined experience into the the rules that  the StoryPlaces software uses to deliver the narrative (a process which, we discovered, hadn’t quite done what she had intended). The conversation made me want to give it a go, and to write a least a first draft in situ, as I explore the places that later readers will be lead to by the narrative.

I shall have to ask the David and Charlie, if I can be one of the writers for a future iteration of the project. In fact, I’ve just decided I will write them an email straight away.

 

 

What I meant to say was…

Back at the University for the second day of PGRAS, the post-graduate archaeology symposium which I spoke at yesterday. My talk didn’t go brilliantly well. Despite my preparation last weekend, producing a script as well as my slide deck, I went off-script about a third of the way through, and didn’t get back on it, so I feel a lot of what I had meant to say went unsaid. I often find this when I a script myself, it’s seems I stick more to what I plan to say when I only use bullet points and ad-lib around those. When it’s a full script something in my mind rebels and I end up saying nothing in the script.

So, here’s what I meant to say:

  1. This is a session about storytelling. So I’m going to tell you a story, and like all good stories, its going to have a beginning, middle and an end. Given the audience I feel I must warn you – I can’t promise that this will have much archaeology in it. But I have included one piece, so keep an eye out for it
  2. Last time I was speaking in front of this forum, I explained that I was researching what cultural heritage interpretation might learn from digital games. Those of you that were here may remember that I’d was interested in eight “emotional triggers” (adapted from (Sylvester, 2013)) that engage players in games. You can ask me about these four afterwards. Right now I’m interested in these four, where I think cultural heritage may have more to learn from games.
    1. Generally we don’t like people Acquiring stuff from cultural heritage sites. But actually the “Can you spot ?” type sheets that heritage sites have for decades given to bored children, are using the acquisition trigger.
    2. Challenge is an interesting one, many games are at the best when the degree of challenge matches the player’s ability and they get into “flow”, but seriously how much challenge are cultural heritage visitors looking for, on a day out? We’ll briefly return to this in a while.
    3. Here’s a tip from me, of you have any musically minded mates looking for a PhD subject, then the world of music and cultural heritage interpretation is an open field. There is nothing published. Zero, Nada. Having done my literature review, its what I’d be studying, if I could play, or … er … tell the difference between notes, or even keep a rythym.
    4. But I can’t, so storytelling is the focus of my study.
  3. Before me move on to that, I’d like to pause for a small digression. Those of you who are still listening to me – take a moment to look around the audience. No I don’t want you to point anybody out. I don’t want to shame anybody. But just put your hand up if you can see anyone who isn’t looking at me, but rather looking at their mobile device.
    That’s OK. I know I can be boring. But it’s a demonstration of the secret power of mobile devices. They are teleportation machines, which can transport you away from the place you are physically in.
    And most cultural heritage visitors don’t want that. They have come to our places (they may even have used their phones to help transport them to this place – with on-line bookings or GPS route-finding) to be in the place.
    Of course, that doesn’t stop all sorts of people using mobile devices to “gamify” cultural heritage interpretation. This game at the National Maritime museum, is an example of one that adds new technology to the classic acquisition trigger. You co round the world, collecting crew and cargo from various ports. It adds the challenge trigger to the mix, because you can only SEE the ports if you look at the giant map through the screen’s interface.
  4. There’s a lot of research currently looking at interfaces for cultural heritage (Reunanen et al., 2015) considered for example, getting visitors to make swimming motions in front of a Kinect to navigate a simulated wreck site. But the more I read, and the longer I considered it, I’m more and more of the opinion that there is an interface for cultural heritage that technologists are ignoring: (click) Walking around, looking at stuff.
  5. Now, when it comes to storytelling, “walking around looking at stuff” is not without its problems. People like to choose their own routes around cultural heritage venues, avoid crowds, and look only at some of the objects.
  6. What that means, is that sites often tell their most emotionally engaging story, the beginning, (click)middle (click) and end ( click) towards the beginning of the visit, with a multimedia experience in the visitor centre, or if they can’t afford that, an introductory talk. Then, everything else (click). Which is what game designers call a branching narrative. And what Aylett (Aylett, 2000, Louchart, 2003) calls the “Narrative Paradox … how to reconcile the needs of the user who is now potentially a participant rather than a spectator with the idea of narrative coherence — that for an experience to count as a story it must have some kind of satisfying structure.” (Aylett, 2000). We can learn from our games address with paradox.
  7. Imagine then, a site where the visitor’s movements will be tracked around the site, and the interpretation will adapt to what they have experienced already. Museum and heritage sites consist of both physical and ephemeral narrative atoms (“natoms” after (Hargood, 2011)). Persistent natoms include the objects and the collection but also the spaces themselves, either because of their historic nature, or their configuration in relation to other spaces (Hillier, 1996). Ephemeral natoms are media that can be delivered to the visitor responsively including, but not exclusive to, lighting effects, sound and music, audiovisual material, and text.
    All of these natoms comprise the “curated content” of any exhibition or presentation. The physical natoms are “always on,” but the others need not be (hence the “ephemeral” designation). The idea of the responsive environment would be to eventually replace text panels and labels with e-Ink panels which can deliver text natoms specific to needs of the visitor. Similarly, loudspeakers need not play music or sound effects on a loop, but rather deliver the most appropriate piece of music for the majority of visitors within range.
    To reduce the impact of the narrative paradox (Louchart, 2003), the natoms will be tagged as either Satellites (which can be accessed in any order) or Kernels, which must be presented in a particular order (Shires and Cohan, 1988). Defining which natoms are satellites or kernels becomes the authorial role of the curator.
    Here’s comes the gratuitous piece of archaeology – does this diagram remind you of anything? (click) But in fact it seems somehow appropriate. Because, this is the Apotheosis moment. I want to make the visitor the “God” of his or her own story. Not quite putting them in the place of the protagonist, whose choices were made years ago, but both watching and controlling the story as it develops.
  8. I’m no technologist, so my plan is to “wizard of Oz” a trail run, using people following visitor groups around, rather than a fancy computer program. My intention is to test how people respond to being followed, and how such a responsive environment would negotiate the conflicting story needs of different visitor groups sharing the same space. I have a venue, the Director Chawton House has promised me a couple of weeks worth of visitors to play with, next year. This is where I am at so far, having spent a couple of weeks breaking down the place’s stories into Natoms.
    There’s a lot more to do, but next year I hope to tell you how Chawton’s visitors were able to explore the place entirely freely, (click) and still manage to be told an engaging story from (click) beginning, though (click click) middle (click) and end.