I’ve caught up with my fellow students on the Coursera stats course I’m completing, having managed to do four weeks worth in a little over one. I’ll probably need to go back and review over the coming week, especially as there is a Mid-Term exam coming up! A mid-term exam?!? It’s not like this even earns me any credits. Never mind though, I find the quizzes at the end of each week’s study very good practice at actually applying what I’ve learned. I’m beginning to “get it” too. The notation Prof Conway uses in the course is very different from what I’m used to from my “O” Level Maths, which I realize, is the last time I learned anything like this formally. But I suddenly realized it makes it easier to translate the maths into the command-line code at R uses.
It’s also, by-the-by, teaching me a little about data collection too. I finally realize why so many questionnaires have so many questions that sound a bit like one I’ve already answered. It’s an attempt to get some measure of validity, in circumstances when you can’t pull your sample in to repeat the test, as might happen in a better funded medical study. Humanities studies are rarely so well funded, so they have one chance to interact with their participants, and they need to minimize any chance for error. Then again long questionnaires often discourage full participation, and tend towards sampling error in that only those people who really like questionnaires complete the survey. So the number of questions becomes a bit of a balancing act.
So its making me think about what sort of questions I might ask to test some of the emotional drivers identified here. Lets imagine testing some sort of hypothetical interpretation app at a heritage site. One of the things the Statistics Once course has reminded me, is that humanities studies, in the absence of the resources to “test and retest” often build on review studies. So If I’m looking at an app, I should perhaps ask some of the questions from this recent study of apps for a museum space and an historic church.
- My sense of being in the [place] was stronger than my sense of being in the rest of the world
This one for example tests the idea of presence, and as it appears in a previous study, I’d have something to compare my results to. But that previous study didn’t involve the people I’ll be asking, so to minimize error within my own sample, I need to think of a question that asks the same thing in a different way, and see how well the answers for the two questions correlate. I could try:
- I forgot about the outside world;
- I felt transported to the [significant period of history]; or, for a negative correlation,
- During my visit I remained aware of task and chores I have back at home/work
The challenge of course is to try and make questions that aren’t too binary, with simple yes/no answers so I may need to work on these.
Othman also has a question that addresses my spectacle and sensation driver:
- I was overwhelmed with the aesthetic/beauty aspect of the [place]
Overwhelmed seems a strong word to use here, surely too binary? Can you be “a little bit overwhelmed” after all? But it could be worth me including, if only to compare my results with Othman. What other questions might address this driver?
- The [place] was impressive?
- The [place] was beautiful?
Binary, Binary. Blimey, this is tough. When I composed the questions for my Ghosts in the Garden research I tried to make questions like these by asking “how strongly do you agree/disagree with these statements?” I can’t decide now whether that’s a cop-out or not. What I need to do is apply some of the new statistics tools I’ve learned to the results I got. Though my first thought is what’s “Normal” when all the answers are clumped at one end of a Likert scale?
If the strongly agree/disagree scale isn’t a cop-out, I’ve got a whole bunch of questions about the social impacts on the visit, like:
- We enjoyed visiting as a group
- I prefer visiting with other people
- I enjoyed talking about [the place] with the people I visited with
- I enjoyed discussing the place with people I’d not met before
- I liked conversing with the [staff/volunteers]
- I like visiting when: I have [the place] to myself; there are only a few other visitors I don’t know; [the place] isn’t too crowded; [the place] is busy; [the place] is very crowded.
This last isn’t even binary! But there’s an interesting challenge in wording it in a way that everybody has the same understanding of the answers. And of course, its a less continuous scale than Strongly disagree/agree. Hmmm…
Othman also came up with that interesting question about acquisition:
- I wanted to own exhibits like those that I saw in the exhibition
Of course not every cultural heritage sites has an “exhibition” with “exhibits,” but it does make me think of a whole bunch of questions about a sense of “ownership,” like:
- I wish I could live here;
- I wish I lived here when [the place was at its prime]
- I wish I could stay as long as I wanted;
- I wish [this place] was mine;
- I’d like to buy a souvenir; and,
- How many photographs did you take? How much video did you shoot?
(Ohh, I like that last one as a measure of acquisitive desire/engagement/the tourist gaze. It might even correlate with the answers to the spectacle/sensation questions above. I wonder if any other studies have been done on the number of photographs visitors take. It seems so obvious now I’ve thought of it. Similar questions must have been asked before. )
There are some questions I’m thinking of that might manage to measure visitors’ perceptions of challenge and learning. (Of course there are all sorts of ways to measure learning, for example, give them a test pre- and post-visit. but I’m not actually trying to measure learning, rather the impact of learning on emotional engagement.) You could have a bit of fun (and get away from Likert scales) with a question like:
- If this were a test on the history of [the place], what do you think you you might score, out of 100?
But of course I’ve got plenty of Likert “strongly agree/disagree” questions as well:
- What I learned on the visit challenged what I thought I knew about [the place/the period]; which could negatively correlate too
- I didn’t learn very much new today; or
- What I learned re-enforced what I knew already about [the place/the period].
Or there could be Likert style questions where the middle rank is the desired response, something like:
- The family trail was: Too easy-2-3-Just Right- 5-6 – Too challenging; or,
- There was Too Little-2-3-Just the right amount- 5-6-Too Much interpretation
Plenty to think about.
I think I’ve exhausted by thoughts for today. But its good to have got them out of my mindspace and recorded in this blog. Feel free too tell me how rubbish these measures are (as long as you suggest alternatives!). These are very early thoughts, not by any means precious to me, so you won’t crush my ego if you point out flaws. I’m also aware that I have come up with any ways to measure the impact of character arcs or music yet.