CultureGeek 2017 and Digital Customer Experiences


Better late than never, its a month since I went to two events in one week, and I’ve been so busy since then that I haven’t had time to write them up. Those of you who were following my Twitter stream live may ave some idea what excited me at the time, but for anyone else who might be interested, and more importantly for my own reflection, let me ram my thoughts together into this one post on both events.

We’ll start with Culture Geek in Kensington, which follows on from the M&H show, which I didn’t attend this year. This was the expensive one, with speakers flown in from other countries. I was pleasantly surprised to meet my colleague Alex there, so we were able to reflect a little between sessions, and there’s one thing especially we came away wanting to do, but more on that later. The conference touched on everything digital, including in-visit technology, but of course also plenty of on-line stuff. The first speaker was from that side of the field, Kimberly Drew, social media manager from New York’s Met museum. She drew on her experience as a person of colour doing a history of art degree, and how her life has changed during an internship at Harlem’s Studio Museum when a whole side of black art was revealed to her which had not been covered in her white-centric education.

Keen to share her epiphany, she and a friend started a Tumblr blog on Black Contemporary Art. Now that blog has over 200,000 followers, and she has unintentionally become “a poster child for diversity.” The Met weren’t looking for a “diversity champion” when they advertised the role of Social Media Manager (I asked her afterwards), but you can see why they snapped up such a dynamic, self-motivated blogger, with experience of, and reputation for, reaching out and expanding audiences.

Her work for the Met isn’t all about black art either. She sees the social media as the Met’s fourth space, alongside the 5th Avenue building,  The Cloisters and the Breuer. Her role there is to share 5000 years of art; connect users with the collection; highlight the ways the museum serves art and art history, and to “humanise” the museum and create invitations to participate. This last is the objective that benefits, in theory, from her previous experience, but of course they all do. Reflecting on her talk what comes across most is authenticity. Its a challenge for cultural heritage organisations, to match that authenticity of enthusiasm for both the medium and the message, someone who lives and breathes social media and the cause.

Kimberly is a young woman who inspires, and shows us how to do it, and the organisation she works for is a springboard, not a water-slides forcing her in a corporate direction. She’s one to watch.


The most interesting presentation for my research was given by Joe McFadden of the Royal Opera House. they are trying a number of digital experiments as they redevelop one of their spaces, known as the Piazza, with the intention of increasing the number of daytime visitors. Currently only the tens of thousands annually, which for a central London space, is very few. Their work is in three broad areas: Transactional – things like ordering your interval drinks online, and paying with Applepay; Experiential  – things like AR with hololens and VR (check out the work of the VOID) and post-show video on demand: and, Informational – things like personalised wayfinding (which made my ears prick up, but sadly when I quizzed Joe afterwards, he said they were struggling with the contending needs of different visitors at the same decision point, so It might not happen). We also talked about their current testing of an Alexa skill, so that Amazon Echo users could quiz their “household assistant” about whats on and even, possibly, buy tickets.

Which tied in with a fascinating presentation I saw later in the week at the Academy of Marketing’s Digital Customer Experiences event. There Prof. Merlin Stone of St Mary’s University talked about work he is doing on Baby Boomers and the heath service. These are “the largest generation of older people the world has seen”, but also the healthiest and longest living, the richest, most educated, etc etc. though its early days in the voice first market – he sees signs that they are also likely to be enthusiastic users of Alexa and other home voice assistants, and may well expect services (he was talking about health, but it applies equally well to Opera and Heritage, where baby-boomers are currently core market) to be provided by voice-first platforms.

Back to CultureGeek, Tim Wood of the Ballet Rambert showed us some simple online stuff that had proven surprisingly popular – live streaming of rehearsals. Not fancy dress-rehearsals, but studio work, the repetitive practice of moves and blocking. This is what set Alex and me off on a reverie about making a “slow TV” livestream event of a voyage down the length of the river Wey. One day….
Apart from those presentations at CultureGeek, there was interest as well in Patricia Buffa’s discussion of e-Marketing the Fondation Louis Vitton to Chinese tourists. The Chinese market isn’t a big one for my market yet, they are mostly urban tourists, and ticking off the iconic sites. But if (when?) it becomes spreads into the countryside and independent travelling there’s stuff we can learn here: the importance of Weibo/WeChat; finding Chinese celebrity advocates; doing exhibitions in Chinese partner locations; and, interestingly, the ubiquity of the QR code – “in China your QR code is your business card”.

We also got insight from the Science Museum’s use of Kickstarter to fund the rebuilding of Eric, Britain’s very first robot. We were shown a really interesting content management system created by MIT, and heard about building digital systems for a City of Culture in Hull. There were also some lovely experiments in mixed reality from the National Theatre, including a VR Alice in wonderland that the viewer experienced sitting on a toilet, and Draw Me Close, a VR opera that puts the audience in the naively drawn world of five year old Jordan. I’m not sure how sustainable the business model of this experience might be, the cast outnumbers the audience (of one) so that as the virtual Mum hugs you, or tucks you up in bed, a physical cast member also does it to you, to make an fully sensory experience. Its the closest we’ve come yet to the Ractors of The Diamond Age.

The Digital Customer experiences event was more commercial (after all, it was hosted by the Academy of Marketing at the Direct Marketing Institute). I had been invited to give a presentation, the abstract for which I posted a few weeks back. Apart from Professor Stone, whom I spoke about above, Dr Julia Wolney introduced the day with an overview of all the points in the customer “life cycle” where AI has growing potential.

Ana Canhoto gave a very interesting presentation about the conflicting attitudes to tracking and personalisation. As one respondent told her, its:

… creepy. But, then, it is just also very useful.

Dr Wolny returned to talk about her research into wearables, and the quantification of the self. As a recent wearer of an Apple watch, which I am using to incentivise my own movement, I was very interested in what she had to say. However based on her findings I’m not sure I’m typical. Women are more likely than men to track their fitness, but men are more likely to share their latest achievements. (I am not.)

But perhaps the most intriguing presentation was from Dr Fatema Kawaf – she presented a research technique I had not heard of before, but one I think may be valuable to evaluating heritage experiences. Its called The Repertory Grid, and as the linked article shows it comes out of psychology, a technique as a method to help the individual unveil his or her constructs. As Kawaf demonstrated though, it enables participants to use their own words to construct their understanding of experiences too. Kawaf was thinking about the retail experience, but I wonder if its ever been applied to heritage?

Building a story in Star Wars Indentities

A Stormtrooper marching band? The exhibition attempts to illustrate different values with illustrations like this.

It was Fathers’ day last weekend, as a treat, my family took me to the Star Wars Identities exhibition at the O2 in Greenwich. I was interested for a number of reasons, not the least of which was, being ten in 1977, I was (am) a massive fan of Star Wars. But one of the other reasons was the idea that visitors build their Star Wars identity as they go around the exhibition. This seemed to me to be a large scale, upfront attempt to personalize a cultural heritage visit. (And yes, Star Wars is cultural heritage now, I’m sure I saw other movies when I was ten, but I can’t recall what they were.)

The RFID tag that visitors use for the interactives

The mechanics of this personalization were wristbands, or if you were latex intolerant, “credit” cards, with RFID (I’m guessing) chips, and nine (not ten as advertised) stations around the exhibit where you could make choices that defined your Star Wars identity. The content of the show were props, models, costumes, concept sketches and some original art, mostly from the first six movies (though BB8 and a couple of props were squeezed in to represent the latest phase of production), with two streams of interpretation. One stream interprets the design of characters in the movies, and the second is a sort of “science of Star Wars” strand, with basic interpretation of things like genetics. Some of all this is delivered with traditional text panels, and some is aural, delivered by an IR activated headset. You are given a “medallion” style unit to hang around your neck and hook an earpiece over one ear. Then, stand in the right place in front of a panel or AV, and the relevant sound is beamed to your unit in a choice of languages. All you need to do, is control the volume… and make sure you don’t turn away from the beam, or cross your arms over the receptor, or let anybody tall stand in front of you, cutting of the beam – all of which will cut out the sound.

It’s worth  pointing out that the text comes in English and French. And that may betray the exhibition’s 2012 origins at the Montreal Science Centre. That of course explains where the science interpretation strand comes from, and why objects and stories from The Force Awakens feel shoe-horned in. One can’t argue with most of the interpretation – seeing how the characters like Yoda developed over time was interesting, the science was a bit basic, and its connections with the Star Wars story questionable (the exhibition suggests Force sensitivity is a genetic trait). The stuff on personality felt just one of many different models of personality types that, despite five post-Doc academics advising on it, reads like its been cribbed from a dodgy self-help book. Interestingly, it was the personality test that was the only interactive station that wasn’t a simple choice – visitors had to answer a number of questions before it revealed their personality profile.

When I started the experience, I was looking forward to discovering what my Star Wars identity would be, but three or four interactives in, when I realized that most of the stations were offering choices rather than revelations, I decided to rush back to the start and remake those choices – because I’ve known since I was ten what my Star Wars identity actually was: the son of Grand Moff Tarkin! Given that the character described in the link was mostly made of my direct choices, I am of course very pleased with the result. I was curious to see what my personality test said about me (or rather, about my Star Wars identity). Click on each of the buttons below the biography on the page I linked to, and it highlights the bit text in the biography that was chosen by your answers. So to reveal the personality results, all I need to do is click that button. The highlighted text says:

People often tell me I’m a generally adventurous and curious person, I also tend to be energetic and social.

… which suggests I was really getting into character when I was answering those questions, because that doesn’t sound like me at all 🙂

Actually the interactive I enjoyed most was Events. Touch your RFID bracelet to the receptor and a random life event spins into view, with a choice of how you react to it. I won a city in a “game of chance”, and had to decide (if I recall correctly) whether I governed sensibly, gave up the job, or “reveled in the prestige and borrowed liberally from the city coffers” which is the option I went for (of course). But my boy was disappointed that the random events on offer were not in some way defined by choices you’d already made – he too got the city, and I might easily have been “freed from slavery” by an event. The son of a Grand Moff in slavery? I don’t think so. 🙂

Despite its limitations (which one is prepared to forgive more when one realizes the technology is five years old), the opportunity to create a story like this was very much enjoyed by my family. I wonder if the exhibition had a deeper emotional impact on me because of it?

Simulating ideology in storytelling

The Story Extension Process, from Mei Yii Lim and Ruth Aylett (2007) Narrative Construction in a Mobile Tour Guide

Another great piece from Ruth Aylett, this time from 2007. Here, she and collaborator Mei Yii Lim are getting closer to what I’m aiming for, if taking a different approach. They kick off by describing Terminal Time, a system that improvises documentaries according to the user’s ideological preference, and an intelligent guide for virtual environments which take into account the distance between locations, the already told story, and the affinity between the the story element and the guide’s profile when selecting the next story element and location combination to take users to. They note that this approach could bring mobile guides “a step nearer to the creation of an ‘intelligent guide with personality'” but that it “omits user [visitor] interests”. (I can think of many of a human tour guide that does the same). They also touch on a conversation agent that deals with the same issues they are exploring.

This being a 2007 conference paper, they are of course using a PDA as their medium. Equipped with GPS and text to speech software, a server does all the heavy lifting.

“After [an ice-breaking session where the guide extracts information about the user’s name
and interests], the guide chooses attractions that match the user’s interests, and plans the shortest possible route to the destinations. The guide navigates the user to the chosen locations via directional instructions as well as via an animated directional arrow. Upon arrival, it notifies the user and starts the storytelling process. The system links electronic data to actual physical locations so that stories are relevant to what is in sight. During the interaction, the user continuously expresses his/her interest in the guide’s stories and agreement to the guide’s argument through a rating bar on the graphical user interface. The user’s inputs affect the guide’s emotional state and determine the extensiveness of stories. The system’s outputs are in the form of speech, text and an animated talking head.”

So, in contrast to my own approach, this guide is still story lead, rather than directly user led, but it decides where to take the user based on their interests. But they are striving for an emotional connection with the visitor. So their story elements (SE) are composed of “semantic memories [-] facts, including location-related information” and “emotional memories […] generated through simulation of past experiences”. Each story element has a number of properties, sematic memories for example incude: name ( a coded identifier); type; subjects; objects; effects (this is interesting its lists the story elements that are caused by this story element, with variable weight); event; concepts (this that might need a further definition when fist mentioned); personnel (who was involved); division; attributes (relationship to interest areas in the ontology); location; and, text. Emotional story elements don’t include “effects and subjects attributes because the [emotional story element] itself is the effect of a SE and the guide itself is the subject.” These emotional memories are tagged with “arousal” and “valence” tags. The arousal tags are based on Emotional Tagging, while the valence tag “denotes how favourable or unfavourable an event was to the guide. When interacting with the user, the guide is engaged in meaningful reconstruction of its own past,” hmmmmm.

So their prototype, a guide to the Los Alamos site of the Manhatten project, the guide could be either “a scientist who is interested in topics related to Science and Politics, and a member of the military who is interested in topics related to Military and Politics. Both guides also have General knowledge about the attractions.” I’m not convinced by the artifice of layering onto the interpretation two different points of view – as both such are being authored by a team who in their creation of the two points of view will, even if striving to be objective, will make editorial decisions that reveal a third, authentic PoV.

When selecting which SE to tell next, the guide filters out the ones that are not connected to the current location. Then “three scores corresponding to: previously told stories; the guide’s interests; and the user’s interests are calculated. A SE with the highest overall
score will become the starting spot for extension.” The authors present a pleasingly simple (for a non-coder like me) algorithm for working out which SE goes next. But the semantic elements are not the only story elements that get told. The guide also measures the Emotional, Ideological story elements against the user’s initial questionnaire answers and reactions to previous story elements and decides whether or not to add the guide’s “own” ideological experience on to the interpretation, a bit like a human guide might. So you might be told:

Estimates place the number of deaths caused by Little Boy in Hiroshima up to the end of 1945 at one hundred and forty thousands where the dying continued, five-year deaths related to the bombing reached two hundred thousands.

Or, if the guide’s algorithms think you’ll appreciate it’s ideological perspective, you could hear:

Estimates place the number of deaths caused by Little Boy in Hiroshima up to the end of 1945 at one hundred and forty thousands where the dying continued, five-year deaths related to the bombing reached two hundred thousands. The experience of Hiroshima and Nagasaki bombing was the opening chapter to the possible annihilation of mankind. For men to choose to kill the innocent as a means to their ends, is always murder, and murder is one of the worst of human action. In the bombing of Japanese cities it was certainly decided to kill the innocent as a means to an end.

I guess that’s the scientist personality talking, perhaps the military personality would  instead add a different ideological interpretation of the means to an end. As I mentioned before, I’m not convinced that two (or more) faux points of view are required when the whole project and every story element that the guide gets to choose from are already authored with a true point of view. But in many other aspects this paper is really useful and will get a good deal of referencing in my thesis.

Forgive me, this will make little sense

I feel I need to record this here, but I fear it will be nonsense to most of my readers. Looking back at it it feels like the first step into incoherent PhD madness. So skip this one if you are looking for sense and inspiration.

I’m struggling with the Chawton project, tying myself up into narrative knots. Meanwhile my collaborator Ed is powering on with his part. Today I’ve been listen to his first pass at a number of audio mixes. Which sound great by the way, and make me a bit depressed at the lack of progress my part has been making.

The problem is all to do with the paradox of scripting an emergent story. When I brief Ed part of wants to say, “we’ll tell this bit of the story in this room, and so you need to use this sound,” but that defeats my object of trying to create stories with some sense of order whatever route visitors take through a place. So I wrote Ed a short but still rambling email, that I think captures what I’m getting at. Though it might be bollocks.

Anyhow I don’t want to lose it. And so since I started this blog as a notebook, rather than a finished demonstration of my finely honed wisdom, you  are about to get an actual note to self, the slightly edited text of that email. We can all work out what it means, if anything at all, later:

I’ve had a little epiphany thinking about your question. But I’m finding it difficult to put into words.  You asked whether the six beats are connected to the recorded quotes. I said no, and I still think that but I also said that a couple might be relevant, and so they might, BUT (I think) not so relevant that you should mix them into the finished work. The ones I was thinking of were: Fanny Knight, Mr Knightly and Jane Austen. But none of those enhance any one particular beat, do you get me? So don’t need to be missed in so they are heard every time. The beats set the mood, or rather, illustrate a particular mood. So… I think we’ve looking for sound mixes that accompany the beats, eg Loneliness. In the fancy system that doesn’t exist yet, the system might choose to interweave the “lonely” track with the Fanny Knight quote. But we don’t need to do that, or rather to fix it as that, in our rough and ready version.

But that lead me to another thought. (The Epiphany.) Which is that we (I) can’t afford to put speakers everywhere. So we need to select rooms that we are putting speakers in, and for each room WITH A SPEAKER, create a choice of soundmixes, that match the whatever beat the operator (the Unguide in our case, a fancy system in the future) chooses in that Room.

So, day we have a ball in the Great Hall – at the very basic level we might have a choice of a Jolly mix (for Up beats), or a sad mix (for setbacks), do you get me? We could have six different mixes for the six beats I’ve identified ALL for the Great Hall, so that whichever beat is selected has its own music appropriate for the great hall, and ANOTHER six choices for, say, The Oak Room. Am I making sense? But of course we’ve actually got three stories, and even if they all share the first beat, that would mean a possible five mixes for the Library story, and ANOTHER five for the Montague story, for EACH room? Crazy huh?

But all of that could be a LOT of work for you, so we need to keep our ambitions in check.

SO:

Tell me how many mixes you want to create, and together we can decide on a limited number of rooms where we’ll put speakers (we did day nine, but I can live with just one), and a limited number of mixes for each speaker (I think we want at least two for this experiment). How does that sound?

Building the Revolution 

I finally got to the V&A today, for their exhibition You Say You Want a Revolution. I got turned away at the end of Cromwell Road last time, as the museum was being evacuated after a bomb-scare. 

I’m writing this review on my way home, using my phone (so please forgive my typos) partly because I want to recommend you go, and there is not long left to see it. 

The exhibition charts the western cultural revolution of 1966-1970, though John Peel’s record collection, plysbof course fashion and design from the V&As own collection and other items, such as an Apollo mission space suit borrowed from other institutions. 

One of the gimmicks of the show is the audio, an iteration of the same technology used at the Bowie exhibition a couple of years ago. I didn’t get to go to that one, but I had a demonstration of that tech from the makers Sennheisser, at a Museums and Heritage show. 

I wasn’t very impressed. Though these headphones, which play music or soundtrack to match whatever object or video you are looking at, were well  received by the media back then, in my experience the technology was clunky. Other friends who’d been confirmed that they changes between sound “zones” could be jarring, and that it was possible to stand in some places where music from two zones would alternate, vying for your attention. 

The experience this time was an improvement. It was by no means perfect: I found the music would stutter and pause annoyingly, especially if I enjoyed the track enough to find myself gently nodding my head. Occasionally the broadcast to everyone’s headphones would pause so everyone in a room could share a multimedia experience (of the Vietnam war for example) across all the gallery’s speakers, screens and projectors. These immersive over-rides were effective, in much the same way as those at IWM North, but when a track you were enjoying or a video that you found interesting was rudely interrupted, one couldn’t help but feel annoyed. I found myself forgiving the designers however, for this and even the stuttering sound of the headphones, because it all felt resonant with that late sixties “cut-up” technique. 

Where the technology really worked however was on two videos that topped and tailed the exhibition. In the first various icons and movers of the period were filmed in silent moving portraits of their current wrinkled and grey selves. Their reminiscences of the time appeared as typography overlaying their silent closed-mouth gaze, a little like Barbera Kruger’s work, while  over the headphones you heard their voice. The same characters appeared at the end, that s time as a mosaic of more conventional talking heads. And for the first time, the interpretation was didactic as each in turned challenged the current generation to build on their legacy. 

For me, one of the highlights was the section on festivals, which invited visitors to take off their headphones, lie back in the (astro)turf and let (another cut-up of) the famous Woodstock documentary wash over them on five giant screens. 

The other things I loved were, dotted around among the exhibits, tarot cards that, at first glance, looked like they might have been designed in the sixties. But then you notice references to things like Tim Berners Lee and the World Wide Web. You realise these are a subtle form of interpretation, telling a future of the sixties that apparently came true and for those of us from that future, creating correspondences and taxonomies that connect the events of 1966-70 with today. The V&A commissioned British artist Suzanne Treister to create the cards, based on her 2013 work, Hexen 2.0. And the very best thing about them is you can buy them (pictured above) in the shop which must be the first time copies of museum interpretation panels have been made available for purchase. 

Of course, the aren’t the only form of interpretation. About from the soundtrack, there are more traditional text panels, labels and booklets around the exhibition. But the cards show how cleverly the layering of meaning and interpretation has been created. Many visitors will have passed them by unnoticed, given them a cursory glance or chosen to ignore them, and will have had an entirely satisfactory experience. But for those that paused to study them in more detail a whole new layer of meaning opened up. 

I visited with a sense of duty, to try out a responsive digital technology. But I found so much more to enjoy. This is a brilliantly curatored exhibition. So much better than the didactic, even dumbed down permanent gallery of the new Design Museum which I visited before Christmas. I urge you to go, if you haven’t seen it yet. It’s only on for another month. 

A colleague who had visited the exhibition before told me how depressed it had made him: the optimism of that period seems to have been dashed upon the reactionary rocks of 2016, Brexit and Trump. But I came out with a very different mood. 

One of the early messages of the exhibition is the period as a search for utopia. The final tracks you hear as you walk out (after the video challenge issued by the old heads of the sixties) are Lennon’s 1971 single Imagine and then, brilliantly, Jerusalem

No, of course they didn’t find the Utopia they were looking for in the sixties, but we could build it…

Mind Control and responsive narrative

Among the mince pies and over-cooked turkey over Christmas, I managed to find a little time to read an interesting paper. #Scanners: Exploring the Control of Adaptive Films using Brain-Computer Interaction shows once again, that the cool people are all at the University of Nottingham. What these particular four cool guys did was put a mini cinema in an old caravan. But this particular cinema wasn’t showing an ordinary film. Rather, the “film was created with four parallel channels of footage, where blinking and levels of attention and meditation, as recorded by a commercially available EEG device, affected which footage participants saw.”

Building on research in Brain Computer Interface (BCI) the team worked with an artist to create a filmed narrative that “ran for 16 minutes, progressing through 18 scenes. However, each scene was filmed as four distinct layers, two showing different views of the central protagonist’s external Reality and the other two showing different views of their internal dream-world.” Which layers each viewer saw was selected by the EEG device, for rather by the viewers’ blinks and states of “attention” or “meditation” as recorded by the device. The authors admit to some skepticism from the research community about the accuracy of the device in question, but that was not what as being tested here. Rather, they were interested in the viewers’ awareness of the ability to control the narrative, and their reaction to that awareness.

I was interested in the paper for two reasons. First of all, their conclusions touch upon an observation I made very early in in my own research, looking at Ghosts in the Garden, I got a small number (therefore not a very robust sample) of users of that interactive narrative to fill out a short questionnaire, and I was surprised by the number of respondents who were not aware that they could control (were controlling) the story through the choices they made. The #Scanners team noticed a similar variation in awareness, but more than that, they found that “while the BCI based adaptation made the experience more immersive for many viewers, thinking about control often brought them out of the experience.”

They conclude that “a traditional belief in HCI is that Direct Manipulation (being able to control exactly what you want to control) sits at the top of both these dimensions. We examined, however, how users deviate from line, and enjoyed the experience more by either not knowing exactly how it worked, or by giving up control and becoming re-immersed in the experience. […] these deviations from the line between knowledge and conscious control over interaction are most interesting design opportunities to explore within future BCI adaptive multimedia experiences.”

With which, I think I agree.

The other reason the paper interests me is that they described their research as “Performance-Led Research in the Wild” and pointed me towards another paper to read.

Chawton Untours and more

It’s a funny feeling time. The calendar pages seem to flicker by as the year rushes towards its end, the the deadlines for various aspects of the Chawton project loom ominously. On one level I worry I have achieved so little and yet, on an other so much has gone on. So it seems inevitable that this post will consist of a number of short catch-ups on various aspects.

Untours

First of all, I’ve got a name for what we offer the public next year. I’d been struggling to think of how I’d present the project to Chawton’s visitors in a way that meant something. I’ve been calling it “the project”, “my experiment” or a “responsive environment”, none of which would sell the concept to potential participants. But a few weeks back I met a colleague who told me about an experimental opening of the Roundhouse in Birmingham. Working with a couple of performance poets, they opened the building for sneek previews that they called “Un-Tours“.

The National Trust’s Un-Tours are not quite the same as what I’m planning of course. But I thought it was a perfect name: visitors will explore the house with a volunteer, but the volunteer won’t be a guide leading them from room to room. They choose where they go, and what they look at, and the volunteer responds to their interests with the relevant natoms. So my volunteers are Unguides, and the tours, Untours (I decided we didn’t need the hyphen). I told my colleague there and then that I was nicking the name.

A collaborator!

The next exciting thing that happened was meeting Ed Holland. Ed is studying Music at Southampton, and was looking for a studio project. He has agreed to help me with the sound natoms. I met him for a second time yesterday, with the always brilliant Jeanice Brooks, and we started to break the musical narrative, focused on domestic life at the turn on the eighteeth/nineteenth centuries, which will reference the Jane Austen connections that Chawton has, without being about her (given there’s a museum dedicated to her just down the road).

Talking about sound

Of course between those two meetings with Ed, I’ve been thinking a lot about sound. As long time readers may be aware, I’m keen to put as few barriers/filters as possible between the visitor and the space they are in. So my preference is always for speakers, but Ed suggested that headphones may offer a more immersive soundscape for less money.

However, one of the key investigations of this project is to investigate a set of “contention rules”, for when more than one visitor/visiting group enter the same space with different story needs. Of course, if everyone were wearing headphones, that soundscape contention wouldn’t be an issue. Which may be a good thing (for visitor experience) as well as a bad one (for my investigation). I’ve also been thinking about other ways my paltry budget might limit what we can achieve. I hope to store all the assets on the web (in Scalar currently) so that a volunteer Unguide can use any smart device to participate (BYD). But of course, that will (I’m thinking – you may know differently) limit each Unguide to delivering just one channel of sound to his/her visitor group. Of course that limits Ed’s ambitions for a multi-channel directional soundscape, but he is making contact with some of the sound guys in our School of Engineering to see if there’s any cool stuff (or speakers) we can use at Chawton.

Assuming we don’t get to borrow anything cool though, I’ve suggested that Ed:

  • Works on a creating a music/sound library based on the lowest spec – single channel a cheap Bluetooth speaker in each room.
  • Specifies the hardware requirements for a system that might deliver his ideal soundscape, either using a multichannel directional speaker system or headphones (Imagine 20 headphone users in the house at the same time). I can guarantee I won’t be able to afford it, but it would be useful research anyway. And we could test a limited version of the concept, with borrowed equipment, during the pilot stage (currently scheduled from the beginning of December in my project plan).

My budget, though tiny, is flexible (it’s my own money) so, I could maybe stretch to something in between the two extremes, if it was something that offered some of the functionality Ed would really want, and maybe had some domestic life afterwards.

Story troubles

The thing that I’ve had most trouble with these last few weeks is the story. I wanted to have at least three narratives – one of the history of the building (and I thought an early 20th century owner, Montague Knight, would be the easiest focus for that); one on Women’s Literature, and the Austen one, mentioned above.

I’n my innocence I thought that I would quickly knock-out an emotionally compelling Montague Knight narrative, but after weeks of reading, arranging and re-arranging, I’ve realised that (duh!) real life stories don’t comply with literary “rules”. Or rather, I’ve realised that maybe my standards, my expectations, for this were too high. I’ve wasted time trying down a rabbit hole, trying to craft a story that I was going to muck up anyway by letting visitors make their choices. I was crafting a traditional guided tour, not an Untour! So, I’ve decided on a different tack. Instead, I’m going to spend some time analysing the natoms I already have, and attributing a story beats to each one. The story should (after all) be procedural.

The outcome of this experiment isn’t (wasn’t ever) meant to be the best interpretive experience. all it is is a step towards the understanding how procedural narratives might work in historic spaces.

Mulholland on Museum Narratives

Working on the narratives for the Chawton Project, I’m taking a break and catching up on reading. Paul Mulholland (with Annika Wolff, Eoin Kilfeather, Mark Maguire and Danielle o’Donovan) recently contributed a relevant first chapter to Artificial Intelligence for Cultural Heritage (ed Bordoni, Mele and Sorgente).

Mulholland et al’s chapter is titled Modelling Museum Narratives to Support Visitor Interpretation. It kicks off with the structuralist distinction between story and narrative,  and points to a work I’ve not read and should dig out (Polkinghorne, D. 1988 Narrative Knowing and the human sciences) as particularly relevant to interpreting the past. From this, the authors draw the “narrative inquiry” process which “comprises four main stages. First, relevant events are identified from the historical period of interest and organised into chronological order. This is termed a chronicle. Second, the chronicle is divided into separate strands of interest. These strands could be concerned with particular themes, characters, or types of event. Third, plot relations are imposed between the events. These express inferred causal relations between the events of the chronicle. Finally, a narrative is produced communicating a viewpoint on that period of history. Narrative inquiry is therefore not just a factual telling of events, but also makes commitments in terms of how the events are organised and related to each other.” Which is as good and concise a summary of the process of curatorial writing as I am likely to find.

There’s another useful summary paragraph later in the document. “When experiencing a museum exhibition, the visitor draws relationships between the exhibits, reconstructing for themselves the exhibition story (Peponis 2003), whether those relationships are, for example, thematic or chronological. The physical structure of the museum can affect how
visitors perceive the exhibition narrative. Tzortzi (2011) argues that the physical structure of the museum can serve to either present (i.e. give access to the exhibition in a way that is independent from its underlying logic) or re-present (i.e. have a physical structure that reinforces the conceptual structure of the exhibition).” Tzortzi there is another reference I’ve not yet discovered and may check out.

What the paper does not do however, is make any reference to emotion in storytelling. the authors seem to leave any emotional context the the visitors’ own meaning making. The chapter include a survey of current uses of technology in museums, and academic experiments including virtual tour guides and opportunities to add the own interpretations and reminiscences, as well as web-based timelines etc.

But, digital technology gives us the opportunity (or need) to break down cultural heritage narratives even more, and an earlier (2012) paper by (mostly) the same authors, Curate and Storyspace: An Ontology and Web-Based Environment for Describing Curatorial Narratives describes a system for deeper analysis. (Storyspace turns out to be a crowded name in the world of writing tools and hypertext, so eventually the ontology and Storyspace API became Storyscope). The first thing that the ontology brings to the table is that

a curatorial narrative should have the generic properties found in other types of narrative such as a novel or a film

So the authors add another structuralist tool, plot, to the story/narrative mix. “The plot imposes a network of relationships on the events of the story signifying their roles and importance in the overall story and how they are interrelated (e.g. a causal relationship between two events). The plot therefore turns a chronology of events into a subjective interpretation of those events.” But using the narrative inquiry process “the plot can be thought of as essentially a hypothesis that is tested against the story, being the data of the experiment.”

I like this idea. But its worth distinguishing between the two uses of the word “interpretation” in cultural heritage. The first use, familiar to my archaeologist colleagues, describes the process of building an understanding of of aspect of the past from the available evidence. The second, more familiar to my museum and heritage site colleagues describes the process of explaining the evidence to non-professional visitors. At its very best, the museum/heritage site form of interpretation will resemble and guide visitors though the process of inquiry that builds an understanding of the evidence on display. But most of the time the second form of interpretation more closely resembles storytelling. That’s not a fault or failure of my museum/heritage site colleagues, most visitors are time poor in story rich environments. But digital technology has the potential to allow museum and heritage site interpretation to more closely resemble the first use of the word.

What digital technology offers, is the opportunity for brave curators to offer alternative plots, or theses, and test them in a public arena, rather than just through a peer review process. Or even to create plots procedurally by following the visitors’ path of attention between objects, maybe discovering plots the curator had not imagined.

The two experiments that the authors describe go someway towards this, by their dry ontology misses an emotional component. The event ontology could surely include an authorial opinion on whether the narrative element suggests a simple emotional reponse (even as simple as hope or fear) but instead “If the tag represents an artist, then events are used to represent, for example, artworks they have created, exhibitions of their work, where they have lived, and their education history.” Dry, dry facts… There is the tiniest nod towards, if not emotion per se, the some sort of value in their brief discussion of theme:

Theme is also related to the moral point of the story. This could be a more abstract concept, such as good winning through in the end, which serves to bind together all events of the story.

Given that they say “Narratives are employed by museums for a number of purposes,
including entertainment” they haven’t given much time to what makes narratives engaging. There is hope however. In their conclusion, they do say “Other narrative
features such as characterisation and authorial intent could potentially be
foregrounded in tools to support interpretation.”

 

Interactive story beats

In my exploration of interactive storytelling I’ve concentrated on computer games, because I’m exploring the digital delivery of story. But I’ve already decided that for my experiment at Chawton next year, I’m going to “wizard of Oz” it – use actual people instead of trying to write a computer program to deliver the interactive narrative.

I’ve been thinking about the issues around that. People are natural storytellers, though some are better than others, so I have a double edge problem. As I recruit and train people to be my “wizards of Oz”, I need to train the poor story-tellers to be better, and weirdly, I need to train the great storytellers to be worse! My reasoning is this, I want to prototype what a computer might do, there’s little or no experimental value in simply enhancing a great storyteller’s natural ability with some environmental bells and whistles. So part of what I’m trying to learn is about how to systematize (is that a word? It’ll do) story.

I’ll explain about Kernels and Satellites of course, but I need (I think) some sort of simple system of identifying how different story elements might fit into the emotional journey the visitor is going to take.

So, I’m reading Robin D. LawsHamlet’s Hit Points. Laws is a game designer but mostly of tabletop, or “pen and paper” role-playing games (though he has written for some computer games too). This book attempts to systematize (I think it is a word) story, with an audience of role-playing gamers in mind. I think it may be useful for me, because it attempts to train the Game Master of such games (the “referee” who, together with the players, makes the story) to be aware of the emotional impact of each scene or action (which he calls, using a screen-writing term, “beats”) on the players, and better choose which element to serve up next to keep everyone emotionally engaged. Tabletop Roleplaying Games must be the most interactive, responsive, stories ever created. In a way, my “wizards of Oz” will be like a Game Master, not telling a story they prepared earlier, but working with their visitors to create a story on the fly, but keep it emotionally engaging.

In a handy short opening chapter called “How To Pretend You’ve Read This Book” Laws explains “With its system of beat analysis, you can track a narrative’s moment-to-moment shifts in emotional momentum. Beat analysis builds itself around the following very basic fact:

Stories engage our attention by constantly modulating our emotional responses.”

Sadly though, I can’t get away with reading just this chapter. It’s only later that he actually shares the classification of beats that he uses in his analysis.

Hamlet’s Hit Points Icons and Arrows by Gameplaywright LLP and Craig S. Grant is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

He begins with two types that he says will make up the majority of beats in any story, Procedural and Dramatic beats. Procedural beats move the protagonist towards (forfilling the audience’s hopes) or away from (realizing the audience’s fears), his practical, external goal. Dramatic beats do the same for the protagonist’s inner goals. “We hope that the beat moves him closer to a positive inner transformation and fear that it might move him towards a negative transformation.”

Laws talks a lot about hope and fear. In fact he simplifies the audience’s emotional response to every beat (which he describes as its resolution) as being a movement towards one of these poles. I’ve got fear on my nascent emotional affect and affordances diagram, its one of Panksepp’s primal emotions, but I’m not yet sure where hope sits – I wonder, is it in care?

In both types of beat, Laws describes two parties, the petitioner, who wants the thing, and the granter, who must be negotiated with. Dramatic beats are mostly actual verbal negotiations, procedural beats might also be fights, tricks, races or other challenges.

From the way Laws describes them, I’d expect that most kernels in a story are likely to be one of these two types of beat. And the other types are more likely to be satellites. He lists:

Commentary – “in which the protagonist’s movement towards or away from his goal is momentarily suspended while the author underlines the story’s thematic elements.” Laws uses Mercutio’s Queen Mab speech in Romeo and Juliet as an example.

Anticipation – which “create[s] an expectation of coming procedural success, which we look forward to with pleasure.” The example here is “Popeye has eaten his spinach. (any given episode of Popeye)”

Gratification – “a positive emotional moment that floats free from the main narrative. They often appear as rest breaks between major sequences. A musical interlude often acts as a gratification beat (unless it also advances the story, as it frequently does in musical genre).”

Bringdown – the opposite of gratification. “Jerry Lundergaard’s car alone in a desolate parking lot, is completely iced over after his father-in-law bars him from a promising business deal. (Fargo)”

Then Laws offers us three “informational beats”:

Pipe – “A beat that surreptitiously provides us with information we’ll need later, without tipping the audience to its importance.”

Question – “introduces a point of curiosity we want to see satisfied […] a question usually resolves as a down beat.”

Reveal – “provides the information we were made to desire in a previous question beat, or surprises us with new information. In the latter case it might come out of the blue, or have been set up with one or more pipe beats laying the groundwork for the surprise.” The example he uses is the Revelation that Bruce Willis’ character in The Sixth Sense is dead. “We tend to be more engaged by exposition when it has been teased to us by a prior question, or can clearly see its impact on our hopes and fears.”

(Laws explains that literary fiction makes much use of question/reveal cycles to manipulate emotion, rather than the procedural / dramatic beats that fill genre fiction and thrillers.)

Laws goes on to analyse three scripted narratives in full, Shakespeare’s Hamlet, and the films Dr No and Casablanca, but that’s not what I’m discussing now, though having recently rewatched Casablanca as part of my children’s continuing cinema education, I was  interested to read his analysis of that. It is worth pointing out, however, that the “curve” of a story like Casablanca is inexorably downward. Laws compares the maps his analysis creates with “the classic chart you may recall from secondary school literature classes” (which I’ve touched on before) and notes that the lines his analysis creates are “flatter overall. It tends to resemble a stock tracker measuring the progress over time of a slowly deflating security […] Even stories that end happily […] tend to move downward over time.” He explains that narratives build up fear with numerous incremental steps, before sudden uplifting moments of hope. So in most stories, there are simply more down beats than up beats, given that the up beats are more intense. I think there is also a point that Laws misses, in many of those narrative curves the absolute value of emotional intensity is being measured, with no thought as to whether the emotion is hopeful or fearful.

So, is all this useful to me? Well I think at the very least I think I can get my “wizards of Oz” to think about up beats and down beats, and make sure not to pile on too many down beats in a row without the occasional up beat. Whether or not heritage interpretation lends itself to procedural and dramatic beats, there is definitely room for question/reveal beats, and it could be argued that too much interpretation goes straight for the revelations without asking the questions or laying the pipes first. So I think it is something that may prove useful.

A little bit of the history of interactive storytelling at Chawton


I spent yesterday morning at Chawton, locating and counting plug sockets so @ll know my limitations as I design whatever the experience there will be next March. The visit reminded me that I had meant to write here of a previous digital interpretation experiment at Chawton.

Back then, in 2005, the Chawton House Library was not widely open to the public. Primarily a centre for the study of Women’s Literature, one could argue that the visiting academics were also heritage visitors of a sort, and the house and gardens also welcomed some pre-booked visiting groups, such as the Jane Austen Society of America, and local garden societies. In their conference paper, a team from the universities of Southampton and Sussex describe how, looking for “curators” to work with they co-opted the trust’s Director, Estate Manager, Public Relations Officer, Librarian and Gardener. All these people may have taken on the role nut just of curator, but also guide to those visiting academics and groups. The paper attempts to describe how their tours interpret the place:

visitors’ experience of the house and its grounds is actively created in personalized tours by curators.

“House and grounds are interconnected in a variety of ways, e.g. by members of the family rebuilding the house and gardens or being buried in the churchyard. Thus artifacts or areas cannot be considered in isolation. There are many stories to be told and different perspectives from which they can be told, and these stories often overlap with others. Thus information exists in several layers. In addition, pieces of information, for example about a particular location like the ‘walled garden’, can be hard to interpret in isolation from information about other parts of the estate – there is a complex web of linked information.[…]

“Curators ‘live the house’ both in the sense that it is their life but also that they want to make it come alive for visitors. The experiences offered by Chawton House are intrinsically interpersonal – they are the result of curators interacting with visitors. Giving tours is a skilled, dynamic, situated and responsive activity: no two tours are the same, and depend on what the audience is interested in. They are forms of improvisation constructed in the moment and triggered in various ways by locations, artefacts and questions.”

Tours are a brilliant way of organising all those layers of information, and I’m sure a personal tour from any one of the curators that they identifies would have been excellent. But the problem comes as soon as you try to scale, or mass produce, the effect. As I said at a conference I presented at a couple of weeks ago (I’m reminded I should write about that too) people, even volunteers, are an expensive resource, and so only the smallest places can afford to give every visitor a guided tour experience. Even then, individuals or families have to book on to a tour, joining other people whom they don’t know, and whose interests they don’t necessarily share. The guided tour experience gets diluted, less personal, less tailored to your interests. Which is when you start getting people saying they would prefer to experience the site by themselves, rather than join a tour. Of course some tour guides are better at coping with these issues than others, but visitors are wary of taking the risk with a guide they don’t know, even if they can recount experiences of brilliant guided tour experiences.

The project written about in the paper had two sides, one was to try and produce content for schools, but the other was of particular interest to me:

“The curators are interested in being able to offer new kinds of experience to their visitors. We aim to find out what types they would like to offer, and help to create them. There is thus a need for ‘extensible infrastructure’ based on a basic persistent infrastructure that supports the creation and delivery of a variety of content.”

And four questions they ask themselves are also of particular interest:

  • “How can we enable curators to create a variety of new experiences that attract and engage different kinds of visitors, both individuals and groups?
  • “How do we engage curators in co-design of these experiences?
  • “How can curators without computer science backgrounds contribute to the authoring of content for the system?
  • “How do we create an extensible and persistent infrastructure; one that can be extended in terms of devices, content and types of experience?”

At the time of writing the paper, they had conducted a workshop with their chosen curators, using a map with 3D printed features. Although “use of a map in the first instance may have triggered somewhat different content,” they discovered that “Eliciting content from curators is most naturally and effortlessly done in-situ.” (Which is my plan – I’m in the process of fixing a date with one of Chawton’s most experienced tour guides.)

I particularly liked the observation that “Listening to them is much more lively and interesting than listening to professionally spoken, but often somehow sterile and dull audio tapes sometimes found in museums and galleries.” So enthusiastically did the team connect with the curators’ presentation, that they decided to record the tours and edit them into the narrative atoms that were delivered by their  infrastructure. That infrastructure was not the subject of the paper, but if I recall correctly, GPS based running on “Palm Pilot” style hardware.

More importantly, the most pertinent conclusion was that the curators were best placed, not just to select the narrative atoms from the recorded materials but also “sort them into
themes and topics, so that the system can cater for people with different broad interests, for example landscape, flora and fauna, or how Jane Austen’s writing reflects the environment. This necessitates a learning process, which must build on existing practices and over time develops new practices based on experience and reflection.”