Mind Control and responsive narrative

Among the mince pies and over-cooked turkey over Christmas, I managed to find a little time to read an interesting paper. #Scanners: Exploring the Control of Adaptive Films using Brain-Computer Interaction shows once again, that the cool people are all at the University of Nottingham. What these particular four cool guys did was put a mini cinema in an old caravan. But this particular cinema wasn’t showing an ordinary film. Rather, the “film was created with four parallel channels of footage, where blinking and levels of attention and meditation, as recorded by a commercially available EEG device, affected which footage participants saw.”

Building on research in Brain Computer Interface (BCI) the team worked with an artist to create a filmed narrative that “ran for 16 minutes, progressing through 18 scenes. However, each scene was filmed as four distinct layers, two showing different views of the central protagonist’s external Reality and the other two showing different views of their internal dream-world.” Which layers each viewer saw was selected by the EEG device, for rather by the viewers’ blinks and states of “attention” or “meditation” as recorded by the device. The authors admit to some skepticism from the research community about the accuracy of the device in question, but that was not what as being tested here. Rather, they were interested in the viewers’ awareness of the ability to control the narrative, and their reaction to that awareness.

I was interested in the paper for two reasons. First of all, their conclusions touch upon an observation I made very early in in my own research, looking at Ghosts in the Garden, I got a small number (therefore not a very robust sample) of users of that interactive narrative to fill out a short questionnaire, and I was surprised by the number of respondents who were not aware that they could control (were controlling) the story through the choices they made. The #Scanners team noticed a similar variation in awareness, but more than that, they found that “while the BCI based adaptation made the experience more immersive for many viewers, thinking about control often brought them out of the experience.”

They conclude that “a traditional belief in HCI is that Direct Manipulation (being able to control exactly what you want to control) sits at the top of both these dimensions. We examined, however, how users deviate from line, and enjoyed the experience more by either not knowing exactly how it worked, or by giving up control and becoming re-immersed in the experience. […] these deviations from the line between knowledge and conscious control over interaction are most interesting design opportunities to explore within future BCI adaptive multimedia experiences.”

With which, I think I agree.

The other reason the paper interests me is that they described their research as “Performance-Led Research in the Wild” and pointed me towards another paper to read.

Mapping four different emotional models

I honestly don’t know where I’m going with this, or indeed if I’m going anywhere at all, but I wanted the give it a try.

Yesterday’s post, about Panksepp and the deep instinctive play emotion network in rats (and other mammals and maybe even birds), and taking my kids to see Inside Out a couple of weeks ago (if you haven’t seen it yet, I can’t recommend it – not as strong as Toy Story, Wall-E or Up) made me want to cross reference his list with Paul Ekman’s list of universal expressions. On his commercial site, Ekman currently seems to be claiming credit for Inside Out, which I guess is why Disgust features as a character. (Damn, now I want to put little pictures of each character in the table as well – which is all the evidence that I must stop mucking about with it and tidy our dining room/study for our approaching guests.)

So I’ve spent most of the day re-reading bits of Panksepp, Ekman, Lazarro and Sylvester and seeing if each of the models lines up to tell me anything interesting. Along the way I’ve made some other notes. For example  – contentment, relief and satisfaction seem like pretty deepseated emotions that should figure, or have equivalents, in Panksepp’s schema. Where should they go? Are they simply Homeostatic? or are they part of the SEEKING network? Also, a number of things that Lazarro calls “emotions” seem really out of place in this table, are the emotions at all? Or are they behaviours? That said what she calls Schadenfreude, really does seem to fit in the Panksepp’s emotional model – though he doesn’t use that word, he does describe the affect as part of “the dark side of human laughter.” Oh, and I need to see in Panksepp has anything to say about flow.

In conclusion I’m not sure if all this work does tell me anything interesting, but you can see the results below and I’m going to sleep on it (after I’ve tidied the dining room).

A table mapping out different “emotion” models (work very much in progress)

Neuroscience >> Psychology >> Games Triggers
Panksepp Ekman Lazarro Sylvester Notes
SEEKING Excitement? Excitement (S, P)

Curiosity (E)

RAGE Anger Frustration (H)
FEAR Fear PRIMAL THREATS
LUST BEAUTY (?)

SEXUAL SIGNALS

CARE Love, Generosity(P) Character arc
PANIC/GRIEF Sadness/

distress

MUSIC
PLAY Amusement Amusement (P)
Homeostatic affects Eg HUNGER THIRST
Sensory affects Disgust

Sensory pleasure

Visceral (S)? Eg DISGUST
Emotional control Mediate (S)?

Self-learning(s)?

Schadenfreude (P)? (Play)

Secondary process
Social emotions  

Contempt

Embarrassment

Guilt

Pride in achievement

Shame

Social bonding (P)

 

Embarrass (P)

 

Fiero (H), Naches (P)

Gratitude (P)

Ridicule (P)

Learn (S)?

Envy (P)

 

INSIGHT

 

 

 

Challenge

Tertiary process
Contentment

Relief

Satisfaction

Pleasure from work(S) ?(flow)?

Boredom (H)

Wonder (E)

Awe(E)

Relax (S)

Elevation (P)

Inspiration (P)

SPECTACLE

ENVIRONMENT

unassigned

 

We’ll have fun fun fun … (fun)

So, what I should be doing is analyzing the data I collected at Bodiam last year, but what I am actually doing is reading the some of the book that yesterdays’ discussion of the Bartle Test led me to. In particular I’ve been reading Nicole Lazzaro’s contribution to Beyond Game Design: Nine Steps Towards Creating Better Videogames, Understanding Emotions.

It got me on the first page, with a quote from the designer of some of my favourite games, Sid Meier: “Games are a series of interesting choices.” But Lazzaro expands on that truism and a way that I really like:

Games create engagement by how they shape attention and motivate action. To focus player attention, games simplify the world, enhance feedback, and suspend negative consequences – this maximises the effect of emotions coming from player choices. In the simplest terms, game mechanics engage the player by offering choices and providing feedback.

She goes on to separate User Experience (understanding how to play the game, manipulate thee controls etc) from Player Experience (having fun). Obviously the two go hand in hand, you can’t have fun if it isn’t easy to understand the controls, but by conflating the two designers might concentrate more on the “how to play” side and not enough on the emotional engagement. Emotions, she says, facilitate the player’s enjoyment; focus; decision-making; performance; and, learning. I wish I could think of a way to separate out visitor experience into two terms because I fear that cultural heritage interpretation can sometime focus on the the “how to visit” side (orientation, context setting etc) at the cost of making the visit emotionally engaging.

Then she discusses the challenge of measuring emotions, and draws on the work of Paul Ekman. She explains how his research identified just six emotions, which appear to have universal facial expressions (the expression of all the other emotions being culturally, and thus to a degree geographically specific): Anger; Fear; Surprise; Sadness; Happiness; and, Disgust. Handily, she says, these six emotions can frequently be recorded when watching players of video games. To those six, she adds another, which isn’t universal, but is relatively easily recognized, and again, very frequently seen on the faces of gamers: curiousity. I wonder how often, and in what circumstances, heritage sites provoke those seven emotions? Curiousity, I hope, is a given, but Anger? Fear? Disgust? (and I don’t just mean when faced with car parking or admission charges).

Of course she also mentions flow pointing out it is more of a state of being than an emotion. What’s really interesting though is that she observed “several aspects of player behaviour not predicted by Csikszentmihalyi’s model for flow.”

Truly absorbing gameplay requires more than a balance of difficulty and skill. Players leave games for other reasons than over-exertion or lack of challenge. In players’ favorite games. The degree of difficulty rises and falls, power-ups and bonuses make challenges more interesting, and the opportunity for strategy increases engagement. The progression of challenges to beat a boss monster and the drop of challenge at the start of the next level help keep players engaged.

Of course, one might argue that she’s taking Csikszentmihalyi balence of skill and difficulty too literally here. That anyone reading Csikszentmihalyi’s account of a rock-climber in flow, for example, will see similar fluctuations of challenge in the real world. But she does on:

Intense gameplay may produce frustration when the level of challenge is too high, but it can also produce different kids of emotions, such as curiosity or wonder. Futhermore, play can also emerge from decisions wholly unrelated to the game goal.

Additionally players spend a lot of time engaged in other activities, such as waving a Wiimote, wiggle their character or create a silly avatar, that require no difficulty to complete. Players respond to various things that characterize great gameplay for them, such as reward cycles, the feeling of winning, pacing, emotions from competition and cooperation.

She and her team at XEODesign researched the moments that players most enjoyed, and recorded the emotions that were expressed, and thus identified four distinct ways that people appear to play games, each of which was associated with a different set of emotions. This doesn’t mean there were four types of players, rather that people “seemed to rotate between three or four different types of choices in the games they enjoyed, and the best selling games tended to support at least three out of these four play styles… Likewise, blockbuster games containing the four play styles outsold competing similar titles that imitated only one kind of fun.”

What players liked the most about videogames can be summarized as follows:

  • The opportunity for challenge and mastery
  • The inspiration of imagination and fooling around
  • A ticket to relaxation and getting smarter (the means to change oneself)
  • An excuse to hang out with friends

Now surely cultural heritage sites offer at least three of those four?

Lazarro argues that “each play style is a collection of mechanics that unlocks a different set of player emotions.” And lists them thus:

Hard Fun

The emotion that the team observed here was fiero, an italian word borrowed by Eckman because decribes the personal feeling of triumph over adversity, an emotion for which there is no word in English. And the game mechanics that unlock that emotion (and possibly on the way, the emotions of frustration and boredom too) are: goals; challenge; obstacles; strategy; power ups; puzzles; score and points; bonuses; levels; and, monsters.

Easy Fun

Curiosity is the main emotion evident in the Easy Fun style of play, though surprise, wonder and awe were observed too. The game mechanics that define this style of play are: roleplay; exploration; experimentation; fooling around; having fun with the controls; iconic situations; ambiguity; detail; fantasy; uniqueness; “Easter Eggs”; tricks; story; and, novelty.

Serious Fun

What is the most common emotion observed with Serious Fun mechanics? Relaxation! The game mechanics that take players to that state are: rhythm; repetition; collection; completion; matching; stimulation; bright visuals; music; learning; simulation; working out; study; and real-world value. It’s this last mechanic that explains why its called “serious” fun. People playing in this mode also seem more ready to attach a value to their participation in the game outside the game itself – brain-training, physical exercise, developing skills or even a conscious effort to kill time (think of those people playing Candy Crush on the train).

People Fun

Happiness comes with People Fun, Lazzaro’s team observed “amusement, schadenfreude (pleasure in other people’s  misfortune) and naches (pleasure in the achievements of someone you have helped)” among players in this mode. Among the he long list of game mechanics that get people there are: cooperation; competition; communication; mentoring; leading; performing; characters; personalisation; open expression; jokes; secret meanings; nurturing; endorsements; chat; and gifting.

 

There’s a lot to think about here, but I’m excited by the possibilities. Here’s a challenge for cultural heritage interpretation. How many of these game mechanics are there already equivalents of in the visitor experience at heritage sites. And can we see value in creating equivalents for the mechanics that are missing?

Flow

Csikszentmihalyi (pronounced cheek-sent-mə-hy-ee)’s concept of flow has had a couple of mentions in recent posts, so I thought I ought to catch up on it. Back when I was doing my MA I read a lot about it in relation to museum learning (mostly in Hooper-Greenhill’s primer The Educational Role of the Museum), but some of that was written twenty years ago, so I thought I ought to get up to date. I’m pleased to report that according to his 2009 co-authored paper the basic concepts are still intact, so here goes:

Some years ago, I had a Mac Classic, a black and white 9 inch display all in one computer. I’d bought it with money from the very first Student Loan scheme in the UK. (I’d saved up in advance of going to University and I had a job reading Tarot Cards and doing the I Ching on a premium rate telephone service, so I didn’t need the money to live on.) It’s role was chiefly to write my dissertation on, but “all work and no-play, etc” so I also bought a copy of Sid Meier’s Civilisation. I recall one evening I looked at the clock on a “school night” and saw it was 10.45, so I said to myself “fifteen minutes on Civ, then bedtime”. So I sat down to play a couple of turns, and  I noticed something had gone wrong with the electric light – rather than its warm yellow glow, it was a colder blue. I turned around a saw daylight through the window. It was dawn.

And that, ladies and gentlemen, is flow.

Back in the sixties. studying the creative process, Csikszentmihalyi watched an artist at work, and noticed that when everything was going well, “the artist persisted single-mindedly, disregarding hunger, fatigue and discomfort.” He called this intrinsic motivation, and described the activity as autotelic (auto=self, telos=goal). We went on to investigate why people like rock-climbers do what they do, for no apparent (or rather extrinsic) reward. Many of the the people he worked with described feeling as though they were being carried along by the activity, and so the word flow was coined.

He set out two conditions for flow: “a sense that one is engaging challenges at a level appropriate to one’s capacities”; and, “clear proximal goals and immediate feedback about the progress that is being made.”

Let’s pause there. I recall very strongly from my MA reading back in 1999/2000 the first of those two conditions – the matching of challenge and ability. I don’t recall so well the thing about feedback. Is that a failure of my recollection, or does the museum literature I was reading stress one and downplay the other? I’ll have to go back and check. I do feel safe to say that while heritage interpretation might offer challenges to meet the level of visitors’ capacities, it may not be as good at offering proximal goals and immediate feedback, except sometimes with personal conversation, for example, in live interpretation.

Given those conditions, Csikszentmihalyi sets out what flow looks like:

  • Intense and focussed concentration on what one is doing at the present moment
  • Merging of action and awareness
  • Loss of awareness of oneself as a social actor
  • A sense that one can in principle deal with the situation because one knows how to respond to whatever happens next
  • Distortion of temporal experience (typically a sense that time has passed faster than normal)
  • Experience of the activity as intrinsically rewarding, such that often the end goal is just an excuse for the process.

If challenge and capability don’t balance, or if neither are high enough, Csikszentmihalyi suggest a range of other emotional states occurs, and set out in this handy diagram (courtesy of Wikipedia):

Mental state in terms of challenge level and skill level, according to Csikszentmihalyi’s flow model. Csikszentmihalyi, M., Finding Flow, 1997

Csikszentmihalyi points out that the goal and feedback structures of sports and games can make flow more likely, but any activity can  create flow, which makes me think of the Buddhist term mindfulness, which can come about through as mundane an activity as raking gravel.

I think I’ll be returning to flow as I go forward. It seems to be something that games can get very right, just as Civilisation did for me twenty years ago, but based on the evidence that I’ve never popped into a heritage site for just fifteen minutes and come out six hours later, interpretation still has a lot to learn.