The Navigation of Feeling

“What are Emotions?” is a question asked by William Reddy (2001) in his book The Navigation of  Feeling: A Framework for the History of History of Emotions. The first part of that books looks at the answers, from Cognitive Psychology in the first chapter, and Anthropology in the second. He points out early, however that:

Western specialists who study emotion cannot even agree on what the term emotion means.

He references Paul Ekman (et al)’s attempts to codify emotions by the few facial expressions that all cultures seem to attach a shared meaning to. That this is in the Cognitive Psychology chapter is interesting – I would have suggested it’s a more anthropological approach, but as Reddy points out it did encourage a lot of study looking at biological indicators of emotion, such as heart rates, hormone levels, skin conductivity etc. Reddy also points out two “nagging problems: (1) What happened to emotions when arousals subsided and the face returned to normal? (2) How were emotions such as love. shame or nostalgia to be fit into the scheme, when they had no obvious single facial expressions to go with them?” Not only that, he argues that “twenty years of work by many researchers” has shown only that “In the absence of forced choice and pretest training, agreement on other than happy faces was weak. If photographs of spontaneous facial expressions were used (i.e., naturally occurring ones, rather than the carefully posed ones of the initial tests), agreement sagged further.”

He also highlights the problematic relationship between emotion and cognition: “If
a sudden sense of fear redirects attention to a dark corner of the room, why not conclude that this sense of fear is the cognition of the potential danger of that corner? No experimental or test procedure has been offered so far that would allow one to rule out this possibility; it is resisted solely on the grounds that it counters the commonsense belief that emotion is something separate from thought, something opposed to ‘reason.'” That “reason” he argues is questioned more and more. Try as they might, philosophers and researchers are finding it harder, not easier to distinguish between “what counts as voluntary or controlled, and what counts as involuntary or automatic.” This is not helped by subjects’ mis-attribution of arousal.

Reddy also explores the difficulties measuring emotion. Most psychologists use valence, a measure of how pleasant or unpleasant an emotion is, and intensity, a scale of how difficult it is to override the emotion. Of course its not so black and white, fear is an “unpleasant” emotion, and yet horror movie audiences, theme park riders and gamers actively seek it out. Similarly Reddy argues that am emotion’s intensity may be altered by the triggering event’s relevance to a person’s goals – here he uses health anxiety as an example, “we generally pursue health for its own sake; but it is obvious that health is a means or condition for the pursuit of many, many other goals. As a result, pursuit of health as a means and pursuit of health as an end in its own right are likely to be indistinguishable (both to observers and to the person involved). Like-wise, loss of health is widely regarded with fear or anxiety. Such fear or anxiety is a “badge” of the deep goal relevance of health.”

When he goes on to explore mental control, I fear he strays more into the study of cognition that emotion, but he is striving to support an argument that “emotions can be regarded as overlearned cognitive habits.” I think Panksepp and Bevan might disagree, but I’m not seeking to argue the point – for my studies, whether emotions exist beyond cognition, or are tightly intertwined with cognitive thought is hardly relevant. When it comes to a cultural, or anthropological, approach I think my conclusions will be the same – my model already separates out social emotions from the more visceral ones identified by Panksepp and Bevan.

Reddy begins his examination of an anthropological approach by outlining the idenitity crisis of the discipline itself, especially in the area of the “production of knowledge”. He spills little ink on the psychocultural model of emotions. This approach, as with cognitive psychology, is built on the idea that there is “a broad commonality in human emotions”. He tackles the contructionist approach first, with the work of Michelle Z Rosaldo. Her study of of the Ilongot people of a mountainous region of the Philippines led her to conclude that emotion was, at least in part, a product of language.  However a roughly contemporaneous study in Tahiti by Robert Levy concluded that people suffered the symptoms of grief even if they didn’t have a word for it. Its interesting to note that the two in these studies examples given are opposed on my model. Grief is one of the mammalian emotions identified by Panksepp and Bevan, while Rosaldo’s word liget, while not having a direct analogue in English seems to fit well with Fiero in my model.

However, Reddy also relates a constructionist argument that is also a more cutting critique of the psychological view of emotions, from Catherine Lutz. “In Lutz’s view, the notion that emotions are biologically based is not simply erroneous, it is part of a larger, insidious, gender-biased Western view of the self that privileges alleged male rationality over the supposedly natural emotionalism of women. Expert and lay assumptions coincided, Lutz charged, in regarding emotions as internal, involuntary, irrational, potentially dangerous or sublime, and female. Men were rational and therefore better suited to action in the public sphere. Ethnographic research showed, however, that emotions were a product of social interactions and showed, as well, that outside the West, emotions were generally not distinguished from thinking in the peculiarly sharp way Westerners distinguished them, and were generally regarded as an outcome of social interaction, rather than as rising up, ineffably, from within.” This argument requires a change in philosophy. If knowledge is constructed by culture, it hard to criticize constructed knowledge, be that headhunting in the Philippines, or Western cognitive science. So, according to Reddy, Lutz’s approach was less about culture and more about discourse as defined by Foucault. But that is an argument well beyond the scope of my work.

Reddy gives some attention to other anthropological models too, but not much. What clearly interests him about those me mentions are steers towards a concept of emotion as performance. Indeed his long conclusion to the chapter, which starts out as an attempt to reconcile cognitive psychology and anthropology, is mostly about control of emotion through performance – oversimplified as putting on an happy face, or to use the example he prefers a “‘bright face’ (mue cedang)” in Balinese.

 

Affect and affordancies: my final (?) model

I have spent far too long on rewriting this section of my thesis these last few days, constantly going back to the model itself and tweaking it, then readjusting it, them moving things around and finally tweaking and correcting it once more. And the texts has been through so many edits I’m not even sure it makes sense to anyone anymore. If you have the time and inclination, I’d appreciate it if you could give this the once over and tell me if its in English…

(Its about how games trigger emotions, by the way. Oh and I’m not doing links, or proper references at this stage – you’ll have to make do with my EndNote metatext.)

Let’s start with some neurobiological research. Panksepp and Biven’s{, 2012 #96} thesis is that all mammals share seven instinctive emotions, even if different species’ higher brain functions can be very different. They label the seven core emotions: SEEKING; RAGE; FEAR; LUST; CARE; PANIC/GRIEF; and, PLAY

The core mammalian emotions defined by Panksepp and Biven

Everything else, they say, are sensory affects such as hunger, or socially constructed. It’s interesting to note that play is here identified as an emotion, not just a behavior. What evolutionary benefit does a play emotion give us, and other mammals? Panksepp and Biven suggest “to learn nonsocial physical skills like hunting, foraging and so on. It is also surely important for acquiring many social capacities, especially nascent aggressive, courting, sexual and in some species competitive and perhaps even parenting skills. It may be an essential force for the construction of the many higher functions of our social brains. Playful activities may help young animals learn to identify individuals with whom they can develop cooperative relationships and know who to avoid […] In short, the brain’s PLAY networks may help stitch individuals into the stratified social fabric that will be the staging ground for their lives.”

Nicole Lazarro{, 2009 #69} draws on the work of Paul Ekman in her study of emotions experienced by gamers. She explains how his research identified six emotions which appear to have universal facial expressions and a number of other emotions which while not quite meeting his criteria for being universal, come very close: emotions like Naches (pride in another’s achievements), Fiero. These two, along with schadenfreude and embarrassment add handy descriptors to the social emotions that Panksepp and Biven lump together and ignore in their study. Similarly, disgust puts a name to one of the sensory affects. I’ve also added her “ wonder” to that sensory category. And, while descriptors for the six core emotions sit well with her research, some of the emotions she lists in her study, for example Amusement, Curiosity, and Anger/Frustration, add nuance to some of Panksepp and Biven’s descriptors, so I’ve included them in the diagram below. All these emotions, she says, can frequently be recorded and recognised when watching players of video games.

Lazarro’s observed emotions in gamers

So, now we have a model of the core emotions, as described by Pankspepp and Bevin, overlaid with emotions attributed to gamers in the work of Lazarro. To that I have below added a third layer. The are affective triggers, drawn from Lazarro and Sylvester’s recommendations for game mechanics, or affordancies, and that trigger emotion in gamers, and Hamari{i, 2014 #148} et al’s motivation mechanics from their study of gamification. Given that each is a conflation of descriptions from up to three sources, they deserve some further explanation.

The final (?) model

Environment and spectacle – While there is an enduring fashion for games that retain the “eight bit aesthetic”{Stuart, 2012 #165} the general trend of graphic development has been towards cinematic realism and “presence” {Riva, 2014 #4; Pinchbeck, 2005 #30; Pinchbeck, 2009 #33}. “Presence originates from the term ‘telepresence’, made famous by the computer scientist Marvin Minksy in a 1980 paper of the same name. From around 1991 (the date of the first issue of the MIT journal Presence: Teleoperators and Virtual Environments), presence has been typically defined as the capacity of the technology to make the user feel transported into a remote place and be able to efficiently interact with it.” {Pujol, 2012 #56}

Pujol and Champion attempt to unpick the definitions of presence. Starting with the idea that the ideal is a sense of being there, or blanking out the digital mediation of screen and controller, they touch upon immersion as a product of field of view and optical resolution. They also briefly summarize the idea that the human component of the system is likely to respond to the affordancies offered by the VR according to their interests, if the virtual component can in turn respond in a realistic way. They touch upon co-presence (sharing the VR with other users) before arguing that social presence (interacting with other users and virtual agents) is vitally important idea in the “potential [their emphasis] convergence between the presence and cultural heritage fields.”

“The conventional notion of presence as the sensation of ‘being there’ is a highly simplified way of expressing an internal perception of the environment and of ourselves in relation to it. A more comprehensive explanation would be that the sense of presence results from the interaction of various factors. These factors depend both on the system (immersivity, visual accuracy, real-time physical and social interactivity, invisibility of devices, consistency of the content) and on the participant (perception, attention, empathy, engagement, meaningfulness or relevance of the content, control, suspension of disbelief).” {Pujol, 2012 #56} In one game, set in the tunnels of a post-apocalyptic Moscow underground, Sylvester describes how play moved to the ruined, frozen, surface and the powerful effect it had on him as a player: “Though most would call Metro 2033 a shooter or RPG, I wouldn’t, because I don’t think its about shooting or roleplaying. I think its about discovering how a place like that makes you feel.”

Whether or not the gamers Lazarro studied felt they had been transported to the virtual world, she did see evidence of Wonder in their expressions, often from beautifully rendered or animated environmental effects. Sylvester separates out the beauty of “a sunset over the ocean” from the spectacle of “a slow-motion dive to dodge an incoming rocket.” He also offers a warning not to over use either.

Acquisition, points and leaderboards are reasonably self explanatory. At the crudest level, they are sharable social evidence of success, skill or commitment. Their affective and motivational power comes from sharing, but acquisition also has an impact on the individual. As Sylvester acknowledges, gambling games are all about acquisition, and computer games often simulate the acquisition of wealth (or simply points). Of course gambling works in two ways, and the bitter emotions of loss shouldn’t be disregarded.

Insight – Learning rewards and encourages curiosity and seeking. Not just any old learning though. “If a lesson is obvious,” says Sylvester “there’s not much buzz in finally getting it because it was always fairly clear.” Instead, he advocates a moment of insight, where everything that has come before “clicks into place and reveals the shape of the whole.”

Challenge is the emotional trigger that we most readily associate with video games, testing the the player’s dexterity and pattern learning before rewarding him or her not just with a sense of accomplishment, but progression within the game. Sylvester argues that it is not as essential to games as it might appear, and cites Dear Esther as a game that can “create powerful emotions without players struggling.” However, Lazarro says that the opportunity for challenge and subsequent mastery can be an important motivation for some, if not all gamers. She quotes the anonymous wife of one “hardcore” gamer “I always know how my husband feels about a game. If he screams ‘I hate it! I hate it! Ihate it!’ then I know two things. A) He’s going to finish it. B) He’s going to buy version two. If he doesn’t say these things he will put it down after a couple of hours.”

Threat – Sylvester conflates revulsion (disgust) and fear, but here I’m using Threat as an affordance of the core Fear affect. Some things (Sylvester cites spiders and snakes) scare enough people (whether instinctively or through cultural learning) that they are often used in games to provoke fear. Of course the fear response on a gamer sitting in an armchair in front of a monitor is going to be different from the fear of the same person lost in a dark forest, but some games work hard to create a close approximation. Sometimes it’s what you can’t see that scares you, and so game designers have learned to use cinematic effects such as jump-cuts and music to to increase tension and invoke a fear response.

Sex is a hard thing to get right in video games, though many have tried. “Let’s be honest, sex in games is rarely super sexy sex.” Say the video game journalists Nielsen and Grey {Nielsen, 2016 #166} “… the key to effective video game sex: good writing, good characters.”
Which brings us to the narrative arc, this is what film can do so well, engaging the audience’s empathy with one or more characters, as they face internal conflicts, grow and change. Change is a common factor for a lot of the affordances described above, and indeed, Sylvester declares that “the bedrock principle behind all emotional triggers is change”. I will address narrative in more detail in chapter 3. (XX)

Feedback, or rather the absence of it, is a trigger for panic/grief which, as Panksepp and Bevin explain has a lot to do with isolation in mammals. When our character stops responding to your controls, is when a player feels a fleeting moment of grief. Feedback is offered to gamers through what happens on screen, haptic engines in controllers, and by sounds and music.

There’s a lot to say about music, which I will address more fully in the next section. Before we move on, note how, in the model I’ve described, affordancies are placed in boxes radiating away from the core, social or sensory affects that they are most likely to trigger. In my text though I’ve already mentioned how some affordancies might contribute to the triggering of other affects: revulsion might help trigger fear for example. Sound and music can be used to trigger, or contribute to triggering, all the affects. Imagine the soundtrack to Jaws or Psycho, and contrast that to the music of Love Story for example. Music is playful, yes, so I’ve placed the label there, but its influence is more diffuse, an I’ve represented it in the diagram with a fuzzy halo around all the affects, to indicate how it can trigger our emotions without us even knowing:
“And music is wonderfully subtle – even more than most emotional triggers. Nobody ever gives it the credit it deserves because nobody consciously pays attention to it during play. But even though the conscious mind is oblivious, the unconscious is still processing the music into a continuous flow of feeling. You can tell because music is easily separable from the rest of the experience. Listen to a game soundtrack by itself, and you’ll feel much of what you felt during play. Play the game in silence, and you’ll be surprised at how hollow it feels.” {Sylvester, 2013 #89}

Simulating ideology in storytelling

The Story Extension Process, from Mei Yii Lim and Ruth Aylett (2007) Narrative Construction in a Mobile Tour Guide

Another great piece from Ruth Aylett, this time from 2007. Here, she and collaborator Mei Yii Lim are getting closer to what I’m aiming for, if taking a different approach. They kick off by describing Terminal Time, a system that improvises documentaries according to the user’s ideological preference, and an intelligent guide for virtual environments which take into account the distance between locations, the already told story, and the affinity between the the story element and the guide’s profile when selecting the next story element and location combination to take users to. They note that this approach could bring mobile guides “a step nearer to the creation of an ‘intelligent guide with personality'” but that it “omits user [visitor] interests”. (I can think of many of a human tour guide that does the same). They also touch on a conversation agent that deals with the same issues they are exploring.

This being a 2007 conference paper, they are of course using a PDA as their medium. Equipped with GPS and text to speech software, a server does all the heavy lifting.

“After [an ice-breaking session where the guide extracts information about the user’s name
and interests], the guide chooses attractions that match the user’s interests, and plans the shortest possible route to the destinations. The guide navigates the user to the chosen locations via directional instructions as well as via an animated directional arrow. Upon arrival, it notifies the user and starts the storytelling process. The system links electronic data to actual physical locations so that stories are relevant to what is in sight. During the interaction, the user continuously expresses his/her interest in the guide’s stories and agreement to the guide’s argument through a rating bar on the graphical user interface. The user’s inputs affect the guide’s emotional state and determine the extensiveness of stories. The system’s outputs are in the form of speech, text and an animated talking head.”

So, in contrast to my own approach, this guide is still story lead, rather than directly user led, but it decides where to take the user based on their interests. But they are striving for an emotional connection with the visitor. So their story elements (SE) are composed of “semantic memories [-] facts, including location-related information” and “emotional memories […] generated through simulation of past experiences”. Each story element has a number of properties, sematic memories for example incude: name ( a coded identifier); type; subjects; objects; effects (this is interesting its lists the story elements that are caused by this story element, with variable weight); event; concepts (this that might need a further definition when fist mentioned); personnel (who was involved); division; attributes (relationship to interest areas in the ontology); location; and, text. Emotional story elements don’t include “effects and subjects attributes because the [emotional story element] itself is the effect of a SE and the guide itself is the subject.” These emotional memories are tagged with “arousal” and “valence” tags. The arousal tags are based on Emotional Tagging, while the valence tag “denotes how favourable or unfavourable an event was to the guide. When interacting with the user, the guide is engaged in meaningful reconstruction of its own past,” hmmmmm.

So their prototype, a guide to the Los Alamos site of the Manhatten project, the guide could be either “a scientist who is interested in topics related to Science and Politics, and a member of the military who is interested in topics related to Military and Politics. Both guides also have General knowledge about the attractions.” I’m not convinced by the artifice of layering onto the interpretation two different points of view – as both such are being authored by a team who in their creation of the two points of view will, even if striving to be objective, will make editorial decisions that reveal a third, authentic PoV.

When selecting which SE to tell next, the guide filters out the ones that are not connected to the current location. Then “three scores corresponding to: previously told stories; the guide’s interests; and the user’s interests are calculated. A SE with the highest overall
score will become the starting spot for extension.” The authors present a pleasingly simple (for a non-coder like me) algorithm for working out which SE goes next. But the semantic elements are not the only story elements that get told. The guide also measures the Emotional, Ideological story elements against the user’s initial questionnaire answers and reactions to previous story elements and decides whether or not to add the guide’s “own” ideological experience on to the interpretation, a bit like a human guide might. So you might be told:

Estimates place the number of deaths caused by Little Boy in Hiroshima up to the end of 1945 at one hundred and forty thousands where the dying continued, five-year deaths related to the bombing reached two hundred thousands.

Or, if the guide’s algorithms think you’ll appreciate it’s ideological perspective, you could hear:

Estimates place the number of deaths caused by Little Boy in Hiroshima up to the end of 1945 at one hundred and forty thousands where the dying continued, five-year deaths related to the bombing reached two hundred thousands. The experience of Hiroshima and Nagasaki bombing was the opening chapter to the possible annihilation of mankind. For men to choose to kill the innocent as a means to their ends, is always murder, and murder is one of the worst of human action. In the bombing of Japanese cities it was certainly decided to kill the innocent as a means to an end.

I guess that’s the scientist personality talking, perhaps the military personality would  instead add a different ideological interpretation of the means to an end. As I mentioned before, I’m not convinced that two (or more) faux points of view are required when the whole project and every story element that the guide gets to choose from are already authored with a true point of view. But in many other aspects this paper is really useful and will get a good deal of referencing in my thesis.

Mind Control and responsive narrative

Among the mince pies and over-cooked turkey over Christmas, I managed to find a little time to read an interesting paper. #Scanners: Exploring the Control of Adaptive Films using Brain-Computer Interaction shows once again, that the cool people are all at the University of Nottingham. What these particular four cool guys did was put a mini cinema in an old caravan. But this particular cinema wasn’t showing an ordinary film. Rather, the “film was created with four parallel channels of footage, where blinking and levels of attention and meditation, as recorded by a commercially available EEG device, affected which footage participants saw.”

Building on research in Brain Computer Interface (BCI) the team worked with an artist to create a filmed narrative that “ran for 16 minutes, progressing through 18 scenes. However, each scene was filmed as four distinct layers, two showing different views of the central protagonist’s external Reality and the other two showing different views of their internal dream-world.” Which layers each viewer saw was selected by the EEG device, for rather by the viewers’ blinks and states of “attention” or “meditation” as recorded by the device. The authors admit to some skepticism from the research community about the accuracy of the device in question, but that was not what as being tested here. Rather, they were interested in the viewers’ awareness of the ability to control the narrative, and their reaction to that awareness.

I was interested in the paper for two reasons. First of all, their conclusions touch upon an observation I made very early in in my own research, looking at Ghosts in the Garden, I got a small number (therefore not a very robust sample) of users of that interactive narrative to fill out a short questionnaire, and I was surprised by the number of respondents who were not aware that they could control (were controlling) the story through the choices they made. The #Scanners team noticed a similar variation in awareness, but more than that, they found that “while the BCI based adaptation made the experience more immersive for many viewers, thinking about control often brought them out of the experience.”

They conclude that “a traditional belief in HCI is that Direct Manipulation (being able to control exactly what you want to control) sits at the top of both these dimensions. We examined, however, how users deviate from line, and enjoyed the experience more by either not knowing exactly how it worked, or by giving up control and becoming re-immersed in the experience. […] these deviations from the line between knowledge and conscious control over interaction are most interesting design opportunities to explore within future BCI adaptive multimedia experiences.”

With which, I think I agree.

The other reason the paper interests me is that they described their research as “Performance-Led Research in the Wild” and pointed me towards another paper to read.

Resonance: Sound, music and emotion in historic house interpretation

Just drafted an abstract for my Sound Heritage presentation:

This presentation explores what computer games can teach us about emotional engagement in cultural heritage interpretation. Beginning with a model of emotional affect drawn from the work of Panksepp and Biven (Panksepp, 2012), Lazarro (Lazarro, 2009), Sylvester (Sylvester, 2013)and Hamari et al (Hamari et al., 2014), it reveals how music especially has become a versatile emotional trigger in game design.

Drawing on the work of Cohen (Cohen, 1998)and Collins (Collins, 2008)eight functions that music has in games:

Masking – Just as music was played in the first movie theaters, partly to mask the sound of the projector, so music in new media can be used to mask the whir of the console’s or PC’s fan.

Provision of continuity – A break in the music can signal a change in the narrative, or continuous music signals the continuation of the current theme.”

Direction of attention – patterns in the music can correlate to patterns in the visuals, directing the attention of the user.

Mood induction; and,
Communication of Meaning- the nice distinction here is between music that makes the user sad, and music that tells the user “this is a sad event” without necessarily changing the user’s mood.

A cue for memory – The power of the music to invoke memories or prepare the mind for a type of cognitive activity is well recognized in advertising and sonic brands such as those created for Intel and Nokia.

Arousal and focal attention – With the user’s brain stimulated by music s/he is more able to concentrate on the diagesis of the presentation.

Aesthetics – The presentation argues that all too often music is used for aesthetic value only in museums and heritage sites, even if the pieces of music used are connected historically with the site or collection.

As an example, the presentation describes a project to improve the way music is used in the chapel at the Vyne, near Basingstoke. Currently, a portable CD player is used to fill the silence with a recording of a cathedral choir, pretty, but inappropriate for the space and for it’s story. A new recording is being made to recreate about half an hour of a pre-reformation Lady Mass, with choisters, organ and officers of the church, to be delivered via multiple speakers, which will be even more pretty but also a better tool for telling the place’s story.

With a proposed experiment at Chawton House as an example, we briefly explore narrative structure, extending the concept of story  Kernels and Satellites described by Shires and Cohan (Shires and Cohan, 1988)to imagine the cultural heritage site as a collection of narrative atoms, or Natoms (Hargood, 2012), both physical (spaces, collection) and ephemeral (text, video, music etc.). Music, the presentation concludes is often considered as a “mere” satellite, but with thought and careful design there is no reason why music can not also become the narrative kernals of interpretation.

 

COHEN, A. J. 1998. The Functions of Music in Multimedia: A Cognitive Approach. Fifth International Conference on Music Perception and Cognition. Seoul, Korea: Western Music Research Institute, Seoul National University.

COLLINS, K. 2008. An Introduction to the Participatory and Non-Linear Aspects of Video Games Audio. In: RICHARDSON, J. A. H., S. (ed.) Essays on Sound and Vision. Helsinki: Helsinki University Press.

HAMARI, J., KOIVISTO, J. & SARSA, H. Does Gamification Work? — A Literature Review of Empirical Studies on Gamification.  System Sciences (HICSS), 2014 47th Hawaii International Conference on, 6-9 Jan. 2014 2014. 3025-3034.

HARGOOD, C., JEWELL, M.O. AND MILLARD, D.E. 2012. The Narrative Braid: A Model for Tackling The Narrative Paradox in Adaptive Documentaries. NHT12@HT12. Milwaukee.

LAZARRO, N. 2009. Understand Emotions. In: BATEMAN, C. (ed.) Beyond Game Design: Nine Steps Towards Creating Better Videogames. Boston MA: Course Technology / Cangage Learning.

PANKSEPP, J. A. B., L. 2012. The Archaeology of Mind: Neuroevolutionary origins of human emotions, New York, W. W. Norton & Company.

SHIRES, L. M. & COHAN, S. 1988. Telling Stories : A Theoretical Analysis of Narrative Fiction, Florence, KY, USA, Routledge.

SYLVESTER, T. 2013. Designing Games – A Guide to Engineering Experiences, Sebastolpol, CA, O’Reilly Media.

Shine On: part two

In the afternoon Graham Festenstein, lighting consultant, kicked off a discussion about using lighting as a tool for interpretation. New technology, he said,  especially LED, presents new opportunities, “new revolution” in lighting. It’s smaller, with better optics, and control. And also more affordable! He used cave paintings as an example. Lighting designers could take one of three approaches to lighting such a subject:  they might try and recreate the historical lighting which, for a cave painting, would have been primitive indeed,  a tallow bowl light, revealing small parts of the painting at time and with an orange light; it’s more likely, given the needs of the visitor, that they might go for more wide angle lighting, revealing the whole of the painting at once; or, they might light for close up inspection of the work, to show the mark making techniques. Traditionally, a lighting designer would have had to chose just one of these approaches. But with the flexibility and versatile control of modern lighting technology, we can do all three things – caveman lighting, wide angle panorama, and close up technical lighting.

Graham’s presentation was not the strongest. heexplained that he was sceptical about LED lights at first pilot as a sceptic. He recalls a visit to a pilot project at the National Portrait Gallery. His first impressions were disappointing, but then he realised that what heith missed about the tungsten lighting to the way it it the gilded frames, and the the LED lighting was better serving the pictures. Then he went on to talk about colour, how the warm lights of the Tower of London’s torture expedition undermined the theme, but the presentation overall was somewhat woolly.

Zerlina Hughes, of  studio ZNA, came next, with a very visual presentation which I found myself watching rather than taking notes.It explained her “toolkit” of interpretive lighting techniques, but I didn’t manage to list all the tools. A copy of the presentation is coming my way though, so I might return with more detail on that toolkit in a later post. One of her most recent jobs looks great however, and I’m keen to go. Say You Want a Revolution, at the V&A follows on from the Bowie show a year or so ago, but with (she promises) less clunky audio technology. I want to go.

Jonathan Howard, of  DHA design, explained that like Zerlina, “most of us started as Theatre designers.” I (foolishly, I think, in retrospect) passed up an invitation to do theatre design at Central St Martins, and I think I would have been fascinated by lighting design, if I had gone, so I might have ended up at the same event, if on the other side of the podium. Museums audiences today are expecting more drama in museums, having experienced theatrical presentations like Les Miserables, and theme parks etc. I was interested to learn that in theatre, cooler colours throw objects into the background, and warmer colours push them into the foreground. This is apparently because we find the blue end of the spectrum more difficult to to focus on. In a museum space, he says, you can light the walls blue so that the edges of the gallery fall away completely. But he did have a caveat about using new lighting technology. Before rushing in to replace your lighting with LEDs and and the modern bells and whistles, ask youself:

Why are we using new tech?
Who will benefit?
Who will maintain it?
Who will support it?

Kevan Shaw, offered the most interesting insight into the State of the Art. He pointed out that lighting on the ceiling has line of sight to most things, because light travels in straight lines (mostly), and we tend to point it at things. So, he said, your ligthting letwork could make a useful communications network too. He wasn’t the first presenter to include and image of a yellow centered squat cylinder in their slide deck. And they spoke as though we all knew what it was. I had to ask, after the presentation, and they explained that it was one of these. These LED modules slip into many exisiting lamps or lumieres. They are not just a light source, but also a platform for sensors and a communications device. Lighting, Kevan argues could be the beachhead of the Internet of Things in museums.

He briefly discussed two competing architectures for smart lighting, Bluetooth, which we all know, and Zigbee, which you may be aware of through the Philip’s Hue range (which I was considering for the the Chawton experiment). He also mentioned Casambi and eyenut, I’m not sure why he thinks these are not part of the two horse race. He argues that we need interoperability. So I guess he’s saying that eventually the competing systems will eventually see a business case in adopting either Bluetooth or Zigbee as an industry standard.

With our lightbulbs communicating with each other, we can get rid of some of our wires, he argues, but it needs to be robust, reliable. And the secret to reliability is a mesh networking, robust networks for local areas. Lighting is a great place for that network to be. That capability already exits in Zigbee (so I think zigbee is what I should be using for Chawton), but its coming soon in Bluetooth. And I think Kevan believes that when it does, Bluetooth will become the VHS of the lighting system wars, and relegate Zigbee to the role of Betamax.

But the really exciting thing is Visible Light Communication, by which the building can communicate with any user with a mobile device that has a front facing camera (and the relevant software installed. He showed us a short video of the technology in Carrefour (mmm the own-brand soft goat cheese is delicious).

The opportunities for museums are obvious but, he warns, to be effectively used, museums will need resource to manage and get insight from all the data these lighting units could produce in resource. Though he says optimisticly to his fellow lighting consultants, “that need could be an opportunity for us!”

Finally we heard from Pavlina Akritas, of Arup, who took the workshop in the direction of civil engineering. Using LA’s Broad Museum as an example, she explained how in this new build, Arup engineered clever (North facing) light-wells which illuminated the museum with daylight, while ensuring that no direct Los Angeles sun fell directly onto any surface within the galleries. The light-wells included blackout blinds to limit overall light hours and photocells to measure the amountof light coming in and if neccessary, automatically supplement the light with LEDs. She also talked briefly about a project to simulate skylight for the Gagosian gallery, Grosvenor Hill.

All in all, it was a fascinating day.

This post is one of two, the first is here.

Pokemon Go: Why is it such an extraordinary success?

A Ratatta in a glass, Jerry mouse he ain’t. Photo: Tom Tyler-Jones

I was talking with my son about Pokemon Go today, and I thought it might be useful to run the game though my model of affect and affordances. Would it reveal why this game is so spectacularly popular, given the barriers to engagement that locatative games have had in the past?

My son pointed out that one thing the game has, especially over its Niantic stablemate, Ingress, is the Pokemon brand, which two or three generations have grown up with since the mid-nineties. Speaking as someone who didn’t grow up with Pokemon however I could not believe this was the only reason for its success.

A huge difference from Ingress is ease of entry. As my survey a couple of years ago may have indicated (though I could not disprove the null hypothesis) here-to-fore locatative games have only held any interest for Hard Fun (otherwise Hardcore) gamers. Pokemon seems to be the first truly casual locatative game (though some might give that honour to Foursquare, I don’t think it was much of a game).

So lets run it though the model:

Game Affects stripped

Leaderboards – Although Pokemon Go doesn’t have a leaderboard as such, it does have Gyms. Just today with my 11 year old son’s advice I managed to (very temporarily) take over a local gym from some quite high powered Pokemon of an opposing team.  So a few minutes after that, I was indeed at the top of a very local leaderboard.

Badges – There are a huge variety of medals you can win for achievements like, for example, collecting ten Pokemon of a particular type.

Rewards – Pokemon Go has LOADS of rewards. For a start, visit a Pokestop, and spin the dial and you will acquire a randomly generated reward of Pokeballs, Eggs, and Potions etc, all of which will be useful in the game. Take over a Gym (as I did this afternoon) and you can claim a reward of 10 Pokecoins every day that you keep the Gym under your control.  Capture a Pokemon, and not only can you add it to your collection, but you are also rewarded with Stardust and Evolution Candy. Every time you go up a level you also earn rewards such as new equipment.

Points and Levels – To level up, you need to earn experience points, which you get for pretty much everything you do, collecting Pokemon, especially new types, spinning Pokestops, hatching eggs, earning badges, evolving Pokemon, battling in gyms, etc.

Story/theme – There isn’t much of a story inherent within the Pokemon Go game, but players who have been brought up on the other computer games and TV series, will know of quite a complex backstory. However, not knowing this story does not seem to be a disadvantage to players. Story knowledge isn’t essential to play, and the lack of story within the game seems to attract (or at least not be a barrier to) players of all generations, many too old to have been captured by the original Pokemon game. My son also points out that as you play you do procedurally generate a story of your own trainer avatar, even if that is only in your head, as Sylvester describes.

Progress – As you go up in level, you do get better equipment, and are more like to catch Pokemon with higher combat power, and you are more likely to encounter rarer Pokemon.

Feedback – The game is casual enough that you don’t need to be looking at the screen all the time, but because the game does not allow you to put your device into sleep mode, you end up holding it, waiting for the tell-tale buzz of nearby Pokemon.

Spectacle and Environment – The graphics and Augmented reality are not very sophisticated, but they are fun. Two things make it so. One is that creatures that only exist in fiction now appear in our real life world. The other is that they can (with some luck and an little movement of the screen) appear in amusing places (on your knee, in your dinner or drink, on your friend’s head), and if they do, you can take and keep a photo.

Challenge – There isn’t much skill based challenge in the gameplay. Capturing rare Pokemon, is more of a feat of luck than skill. There is a real-world challenge of sorts though, and that is to walk around, which is the only way to hatch eggs. Some eggs only require two kilometer walks, but other more rewarding eggs require ten kilometres.

The game lacks (or doesn’t make the most of) a number of emotional triggers:

Music – My son likes the music, but I turned it off early in my play. The music isn’t a very sophisticated feedback generator. One track plays pretty much continuously, and the only changes are for evolution cut-scenes (my boy likes this track best) and Pokemon encounters.

Insight – There  is very little learning through play. My son teaches me most of what I need to learn, and he has leaned most of that through YouTube.

There is no Threat or Sex (even when you capture Male and Female Nidoran), and no real character arc.

So, given the affordancies listed above, we can predict which emotions players will be feeling: playful Amusement (from humorously placed AR Pokemon); the social emotions Fiero and Naches (because though the gameplay isn’t inherently social there are enough players currently on the streets for conversation; advice and insight; and even a degree of cooperation; to take place); the seeking emotions, Excitement and Curiosity (especially when find new types of Pokemon); Frustration, a rage affect (when Pokemon randomly break out of your Pokemon); and some degree of Care (from nicknaming, nuturing and powering up your stable of Pokemon).

And let us not forget the Panic/grief, when nothing makes your phone buzz -you are out of mobile reception or have a weak signal, and especially when your phone battery is running low!

Interactive story beats

In my exploration of interactive storytelling I’ve concentrated on computer games, because I’m exploring the digital delivery of story. But I’ve already decided that for my experiment at Chawton next year, I’m going to “wizard of Oz” it – use actual people instead of trying to write a computer program to deliver the interactive narrative.

I’ve been thinking about the issues around that. People are natural storytellers, though some are better than others, so I have a double edge problem. As I recruit and train people to be my “wizards of Oz”, I need to train the poor story-tellers to be better, and weirdly, I need to train the great storytellers to be worse! My reasoning is this, I want to prototype what a computer might do, there’s little or no experimental value in simply enhancing a great storyteller’s natural ability with some environmental bells and whistles. So part of what I’m trying to learn is about how to systematize (is that a word? It’ll do) story.

I’ll explain about Kernels and Satellites of course, but I need (I think) some sort of simple system of identifying how different story elements might fit into the emotional journey the visitor is going to take.

So, I’m reading Robin D. LawsHamlet’s Hit Points. Laws is a game designer but mostly of tabletop, or “pen and paper” role-playing games (though he has written for some computer games too). This book attempts to systematize (I think it is a word) story, with an audience of role-playing gamers in mind. I think it may be useful for me, because it attempts to train the Game Master of such games (the “referee” who, together with the players, makes the story) to be aware of the emotional impact of each scene or action (which he calls, using a screen-writing term, “beats”) on the players, and better choose which element to serve up next to keep everyone emotionally engaged. Tabletop Roleplaying Games must be the most interactive, responsive, stories ever created. In a way, my “wizards of Oz” will be like a Game Master, not telling a story they prepared earlier, but working with their visitors to create a story on the fly, but keep it emotionally engaging.

In a handy short opening chapter called “How To Pretend You’ve Read This Book” Laws explains “With its system of beat analysis, you can track a narrative’s moment-to-moment shifts in emotional momentum. Beat analysis builds itself around the following very basic fact:

Stories engage our attention by constantly modulating our emotional responses.”

Sadly though, I can’t get away with reading just this chapter. It’s only later that he actually shares the classification of beats that he uses in his analysis.

Hamlet’s Hit Points Icons and Arrows by Gameplaywright LLP and Craig S. Grant is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.

He begins with two types that he says will make up the majority of beats in any story, Procedural and Dramatic beats. Procedural beats move the protagonist towards (forfilling the audience’s hopes) or away from (realizing the audience’s fears), his practical, external goal. Dramatic beats do the same for the protagonist’s inner goals. “We hope that the beat moves him closer to a positive inner transformation and fear that it might move him towards a negative transformation.”

Laws talks a lot about hope and fear. In fact he simplifies the audience’s emotional response to every beat (which he describes as its resolution) as being a movement towards one of these poles. I’ve got fear on my nascent emotional affect and affordances diagram, its one of Panksepp’s primal emotions, but I’m not yet sure where hope sits – I wonder, is it in care?

In both types of beat, Laws describes two parties, the petitioner, who wants the thing, and the granter, who must be negotiated with. Dramatic beats are mostly actual verbal negotiations, procedural beats might also be fights, tricks, races or other challenges.

From the way Laws describes them, I’d expect that most kernels in a story are likely to be one of these two types of beat. And the other types are more likely to be satellites. He lists:

Commentary – “in which the protagonist’s movement towards or away from his goal is momentarily suspended while the author underlines the story’s thematic elements.” Laws uses Mercutio’s Queen Mab speech in Romeo and Juliet as an example.

Anticipation – which “create[s] an expectation of coming procedural success, which we look forward to with pleasure.” The example here is “Popeye has eaten his spinach. (any given episode of Popeye)”

Gratification – “a positive emotional moment that floats free from the main narrative. They often appear as rest breaks between major sequences. A musical interlude often acts as a gratification beat (unless it also advances the story, as it frequently does in musical genre).”

Bringdown – the opposite of gratification. “Jerry Lundergaard’s car alone in a desolate parking lot, is completely iced over after his father-in-law bars him from a promising business deal. (Fargo)”

Then Laws offers us three “informational beats”:

Pipe – “A beat that surreptitiously provides us with information we’ll need later, without tipping the audience to its importance.”

Question – “introduces a point of curiosity we want to see satisfied […] a question usually resolves as a down beat.”

Reveal – “provides the information we were made to desire in a previous question beat, or surprises us with new information. In the latter case it might come out of the blue, or have been set up with one or more pipe beats laying the groundwork for the surprise.” The example he uses is the Revelation that Bruce Willis’ character in The Sixth Sense is dead. “We tend to be more engaged by exposition when it has been teased to us by a prior question, or can clearly see its impact on our hopes and fears.”

(Laws explains that literary fiction makes much use of question/reveal cycles to manipulate emotion, rather than the procedural / dramatic beats that fill genre fiction and thrillers.)

Laws goes on to analyse three scripted narratives in full, Shakespeare’s Hamlet, and the films Dr No and Casablanca, but that’s not what I’m discussing now, though having recently rewatched Casablanca as part of my children’s continuing cinema education, I was  interested to read his analysis of that. It is worth pointing out, however, that the “curve” of a story like Casablanca is inexorably downward. Laws compares the maps his analysis creates with “the classic chart you may recall from secondary school literature classes” (which I’ve touched on before) and notes that the lines his analysis creates are “flatter overall. It tends to resemble a stock tracker measuring the progress over time of a slowly deflating security […] Even stories that end happily […] tend to move downward over time.” He explains that narratives build up fear with numerous incremental steps, before sudden uplifting moments of hope. So in most stories, there are simply more down beats than up beats, given that the up beats are more intense. I think there is also a point that Laws misses, in many of those narrative curves the absolute value of emotional intensity is being measured, with no thought as to whether the emotion is hopeful or fearful.

So, is all this useful to me? Well I think at the very least I think I can get my “wizards of Oz” to think about up beats and down beats, and make sure not to pile on too many down beats in a row without the occasional up beat. Whether or not heritage interpretation lends itself to procedural and dramatic beats, there is definitely room for question/reveal beats, and it could be argued that too much interpretation goes straight for the revelations without asking the questions or laying the pipes first. So I think it is something that may prove useful.

The powers of people

I was at Chawton again yesterday (before going to Petworth for yesterday’s mobile fun) to meet with Jane, one of the house’s most experienced volunteers. I’d challenged her to give me a 45 minute tour of her choice. She really wanted me to tell her what I was most interested to hear, but I wouldn’t. I wanted her unbiased perception of what were the most “important” bits of her her encyclopedic knowledge of Chawton and the surrounding area to share, given the 45 minute time limit.

(I always recommend that 45 minutes is the absolute maximum for a guided tour. In fact I suggest that half an hour is what people should work to. People who want more will stay behind to chat, but there is some evidence from the National Trust’s monitoring of visitors for conservation, that the average dwell time in a house, whatever it’s size is about 45 minutes.)

In the end she gave me what I’d call an architectural tour of the house, pointing out how the thick exterior walls of the the original manor had become interior walls after Richard Knight’s extensions. It was great, and reminded me about some of the things I’d forgotten about being a tour guide that make guided tours (with the right guide) so entertaining.

I’ve always said that guided tours often offer the best historic house experience. A good paid or volunteer guide can weave a compelling story as s/he escorts you around the house. He or she can reveal things you might otherwise have have missed. They can respond to your interests, and level of expertise, to give you a tailored experience. But Jane reminded me how they can transform the place, by pointing out those thick walls, or turning over a framed note hanging on the wall to reveal the ancient deeds from which the paper had been recycled. A good guide turns their audience into detectives – rather than simply telling them how Montague Knight installed a safe into what once had been an old garderobe chute, they help their audience work it out for themselves – a moment of insight, that emotional trigger where everything that has come before “clicks into place and reveals the shape of the whole” as Tynan Sylvester puts it.

Of course, Jane’s tour also demonstrated that the VERY best historic house experience would be to have the guide all to yourself. Not everyone on a larger tour (and there were a couple running yesterday that we bumped into) could have lifted the framed note from the wall to read the reverse. As I hung it back on the hook, I had conservation alarm bells ringing in my head. Every handling, every movement of this glass framed note (which Montague Knight had hidden beneath the floor for future generations to find) put it at risk. The more people given the opportunity I had, the greater the chance that it might be damaged.

Not everyone can do what I did, arrange a personal tour at a time of my convenience after an email introduction from the Director. For those other tour groups we met, the guided tour experience gets diluted, less personal, less tailored to each individual’s interests.

The technological approach I’m investigating might be able to address some of the personalisation challenges, but can it ever offer the magical moments of insight that Jane offered me?

Representing affect and affordancies

Game Affects stripped

Just a short post today, to share what I spent too much of yesterday doing. You may recall a previous post wherein I was struggling to represent all the different emotional models I’d been reading about in my literature review.

A presentation I’m writing for a conference in a couple of weeks gave me the opportunity to have another go, and (importantly for me) make it look a good deal prettier. By about 11.30 last night, I felt that at last I was getting somewhere. So this post simply shares my work in progress, in the picture above.

The central donut represents Panskepp’s work, then moving out from that Lazarro’s adaptation of Ekman, Sylvester’s “triggers”, and then floating around the outside, the motovational affordancies listed by Hamari et al.

It’s not “finished”. I want to return to Lazarro’s work and reassess that, because I found myself editing out a lot of her emotional responses when I was putting this together. I’m also painfully aware that a trigger like “Music” may elicit all sorts of emotional responses, and it sits possibly uncomfortably linked to Panskepp’s PLAY core emotion (though I justified my decision by saying to myself that surely that music IS play).

Now I better get back to my to-do list.