I’ve been analysing the data collected for my evaluation of Ghosts in the Garden. Yesterday I sent my preliminary observations to the guys who created it, and by the end of today I hope to have completed the first draft of my full report. If everyone approves I’ll share it all here in future.
But I did want to share, and possibly sense-check, my key bit of insight. We asked participants to rate how strongly they agreed with a number of statements about the experience, using a seven point Likert scale. So here’s a sample of the sort of response we got to a simple statement, “The Ghosts in the Garden experience added to my enjoyment of the visit today”:
Which is very nice and positive. But I’m looking for emotional engagement, and the responses to the statement “The story I heard had a real emotional impact on me” were less positive:
Now, to be honest I’m not sure I’m asking the right question here. I used this wording only because we ask the question in a similar way at the National Trust where I work, and this being my first bit of research I wanted something that I could easily compare these data with. (For comparison, some of the National Trust’s most emotionally engaging places get something over 20% of visitors ticking to top (number seven) box, in this sample, only about 8% did.)
Asking people to rate their emotional response is according to many, a futile task, and there are likely better ways to measure it, but allow me to indulge myself for a moment. If I can assume that the story was indeed not as emotionally engaging as it might be, I might ask myself “why not”?
Remember, Ghosts is the Garden has been described by its creators as a “choose-your-own-adventure style story.” When you pick up the “listening device” your make your first choice – balloons or fireworks – and then, at every point you are offered a choice of two locations to explore, and the narration explains that the choices you make will affect the outcome of the story. And yet when we asked users whether they agreed that the choices they made changed the story, quite a bit of skepticism was evident:
So my next overriding question is, did confidence that they were changing the story affect users’ emotional engagement? I think I can do a cut of the data to find that out, but the sample size is too small to be really confident in what it might show. Given what I’ve been uncovering about the story structures of the video games I’ve played though, I beginning to wonder if there’s any value to this sort of interactivity. For me, Skyrim, with its wider story structure has been a lot less emotionally involving then either Red Dead Redemption or Dear Esther, both of which take the player towards one single, inevitable, ending. And then there’s the Narrative Paradox.
I wonder whether, rather than trying to construct a number of possible endings, Splash and Ripple (the creators of Ghosts in the Garden) might have better used their time, and the interactive nature of the device, to offer visitors a choice of points-of-view on one single story. And if they had done so, would that have made the narrative stronger, and more emotionally compelling?
[…] ← Is this an insight on the Narrative Paradox? […]
I gotta say both the emotional enagement questions and the use of Likert scales are questionable here. Ordinal cardinal.
Hi Erik! Thanks for the comment, but tell me more! I’m just feeling my way through how to evaluate stuff like this, and I so want to be better at it. I would welcome your suggestions on alternative methods. Or a “statistics 101” you could point me at, which you think it would help to read.
hi, sorry WordPress reformatted my response that ordinal is often confused with cardinal due to Likert scales.
And just a quick comment for now, but in Presence research (presence in virtual environments) there has been a debate over using Likert scales as if one can rate feelings or responses tied to words, over 1-5 or 1-7 scale.
(see Gardner and Martin’s paper at http://www.mitpressjournals.org/doi/abs/10.1162/pres.16.4.439?prevSearch=authorsfield%253A%2528Gardner%252C%2BHenry%2BJ.%2529&searchHistoryKey=)
See Mel Slater’s response to statisticians querying his and Maia’s work at http://www.mitpressjournals.org/doi/abs/10.1162/pres.16.4.447?journalCode=pres
In this paper with Laia Tost I briefly mention the issue for (virtual) heritage..(http://www.tandfonline.com/doi/abs/10.1080/13527258.2011.577796)
Basically when you ask if “The Ghosts in the Garden experience added to my enjoyment of the visit today” that may be but the emotional engagement may have happened after the GG component, does that make sense?
Also, emotional reaction or stimulation is not necessarily the same as emotional engagement, IMO. I also suspect emotional engagement is too vague and connotative-what exactly did you want to find out?
Erik. Thanks, that looks brilliant stuff, I’ll give it a read.
[…] starts now, and Erik handily left me some links in a comment on this post. I’ve only been able to read his own co-authored “Evaluating presence in […]
[…] the data I might collect for my PhD (or indeed the data I’ve already collected, such as about Ghosts in the Garden). My supervisor found a suitable course at Southampton, but frustratingly (given that its running […]
[…] familiar, like the bar charts I got from Excel (with some wrestling to get them on WordPress) in this post. I’ll admit I’m somewhat covering old ground here, but bare with me while I get my head […]
[…] you for free – linear structures are better at engaging people emotionally, because of the narrative paradox, not withstanding the problem that in real life, away from the exhibitor’s drawing board, […]
[…] storytelling medium. Though he doesn’t use the words, he’s obviously talking about the Narrative Paradox. Storytelling is linear he says, interactive experiences for something […]
[…] a very robust sample) of users of that interactive narrative to fill out a short questionnaire, and I was surprised by the number of respondents who were not aware that they could control (were controlling) the […]
[…] on the project, Splash and Ripple, I surveyed a small but decent sample of visitors. I recall being particularly disappointed by responses to the question about whether their choices had changed the story. I’m forcing […]