Digital CHPs?

Excuse the the dreadful pun. CHP stands for Cultural Heritage Professionals, and the paper I am reading today is Maye, Laura A., Bouchard, Dominique, Avram, Gabriela & Ciolfi, Luigina. 2017. Supporting Cultural Heritage Professionals Adopting and Shaping Interactive Technologies in Museums, which starts off with the sentence “The role that cultural heritage professionals, including curators, museum directors, and education officers, are playing in creating interactive technologies in museums is evolving.”

I think it will be useful in slightly reframing my academic work as that of a Cultural Heritage Professional exploring in what ways interactive technologies can support my profession’s “goals through active experimentation in context and in depth.”

The paper looks at a partnership between the Interaction Design Centre at The University of Limerick and The Hunt Museum in Limerick, Ireland. The aper describes in more detail an I will go into here projects involving the Loupe, one of the tangible interaction ideas discussed here. Now I think I have already shared my feelings that this is a museum interface that only adds extra complication to a visit (and one that is after all only a phone disguised as a magnifying glass and so not a thing that pulls the visitors’ eyes away from a screen to the object. So I won’t spend much time on that – rather I will look at what they learned about CHPs creating content for interactive technologies.

“From the beginning of the project, it was clear that CHPs needed to know how different interactive technologies could support their intended visitor experience.”

page 9

The paper betrays a certain frustration with CHPs rushing to a conclusions about what they want Digital interventions to do “here the CHPs chose to focus on a particular kind of narrative experience: a guided tour rather than exploring what they might be capable of. ” While CHPs collectively agree that technology should serve a meaningful purpose, the findings from this project suggest that understanding the implications of adopting those tools in context requires further, hands-on experimentation and experience.” Well I am pleased to say this is what my PhD has all been about – I am a “CHP,” and I have been doing hands on experimentation. And indeed my experimentation encounter the issue that the authors move on to – having to find workarounds to bend the low cost tools I was using to the my needs. “However, these workarounds could require CHPs to build additional skills (as seen in this case) or make compromises between the narrative they intend to portray and the available behaviours provided by the functionalities of the interactive technology.”

Now what’s interesting about the case they mention is that the workaround that they mention “as seen in this case” is very much a product of the unintuitive interface enforced by the Loupe. The CHP involved explains: “Intern Deirdre: We printed it [the content] off because it’s easy to forget the screens that come and go. So the ones that say like ‘tilt right to continue’ it’s easy to forget that was there. But when you have it on the paper, you have to pretend to do that! Turn it over or something like that!” This is touted as good, lo-fi interactive experimentation, and so it might be – but the conclusion seems to be that CHPs should spend more time learning to use the technologies that they have available. Keep in mind that that heritage organisations generally spread their CHPs thin, covering a lot a projects at the same time. Indeed the paper does, eventually, admit that “due to financial costs, it may not be feasible for CHPs to experiment with the actual technologies” and talks of “bodystorming” which is a lovely word and kind of what I was doing at Chawton. Its also good that my “Untours” were one of the things that these researchers were hoping for when they say “other types of experiences: for example, games or free exploration visits.

The Internet of Things

Has the internet of things had its moment already? It seemed to be the thing everyone talked about a couple of years ago, but lately the enthusiasm seems to have died down. Personally I think its more of an infrastructure thing than a ‘killer app.’ It seems to me though that when there are winners in the IoT standards war, and the technology is cheap, and can be affixed without conservation issues IoT will be big for museums. But leaving that aside, today’s paper has lots of quotable quotes, so its the subject of this post. “Today’s paper” is PETRELLI, D. , DULAKE, N., MARSHALL, M., ROBERTS, A., MCINTOSH, F. & SAVAGE, J.. 2018. Exploring the potential of the internet of things at a heritage site through co-design practice. San Francisco: IEEE.

The authors of this paper hope that “By embedding digital technology into objects and spaces we can bring the attention of the visitors back to the heritage, as opposed to the digital devices, and create experiences that go beyond the delivery of information and engage visitors at an emotional level.” This is of course a very commendable intention because recently “museums have been keen to use digital technology to deliver large amount[s] of information (all the information that did not fit on the panels) despite the fact that only a minority of visitors consume just a fraction of what is available.”

What excites the authors is the idea of tangible interpretation, that responds to what the visitor does with it “this new technology holds much potential for heritage as it allows embedding sensing and computation into smart objects and spaces and seamlessly create experiences that cross the boundaries between the material and the digital.” They give some early project examples, some of which (their own) we have read about in previous papers, but there are some new ones too: “The Magic Cauldron (part of a touring exhibition on magic) engaged children in casting spells while throwing objects into the interactive cauldron that reacted with different sounds (e.g. burps) and lights depending on the object thrown in.”

The project in the paper however was for and English Heritage property, Chesters Roman Fort and The Clayton Museum “created in 1896 to host John Clayton’s collection of Roman objects, mostly from Hadrian’s Wall and its surroundings.” Like many 19th century museums that retain their Victorian display philosophy, the collection can look daunting to modern visitors, “most of them enter, spend only a few minutes looking around and leave missing the opportunity to appreciate the richness and relevance of the pieces on display.” The nineteenth century museum posed a number of other challenges to the project team – the opportunities fro physical interventions was limited; there was not enough power for suggested pico-projectors; there was no wifi and a poor mobile phone signal.

In the end, they came up with an idea that I really like. Indeed, the best of all the tangible interfaces I have read about so far: “in the vestibule the visitor would collect the votive lamp from the shrine; they would have a small number of offerings to take to the altars (marked by stands in the museum); visitors would then have to choose, among the many, which ones they wanted to give their offering to; on returning to the shrine, on their way out, with the now empty offering vessel the visitors would receive their personal “oracle”, a personalised reading of one’s character and needs based upon the choices of gods.” There were 13 gods from which the visitors could choose three to bring their votive lanterns to, so 289 possible combinations.

There were issues of coursewith the project, many caused by the remoteness of the location. Its hard to have a internet of things when the internet does to reach the museum. There were issues recognisable to everyone working in heritage, where the designers felt approvals took too long and heritage professionals wanted to dedicate more time to the early development of the project. But the authors concluded that if the internet can be enabled even in remote locations “pervasive computing then becomes an addition to the exhibition designers toolbox: it offers new ways of engaging visitors with digital content through tangible means.”

Restorative Environments

One of the challenges of my viva was that I had presented cultural heritage as only a learning environment. Which was definitely not my intention. the places I have worked with especially, historic houses and palaces with gardens and parks, and countryside estate have long been recognised as places of spiritual recharging. I argued the case (obviously adequately as I passed) that I had not focussed only on learning, but perhaps some references to papers like this one will restore any intended imbalance.

“This one” is Packer, J. & Bond, N.. 2010. Museums as Restorative Environments. Curator 54: 421 – 436.and it starts of with a doozy of a quote that might be all I need: “Mental fatigue, caused by the stresses and strains of everyday life, is a common complaint in today’s society, and the need to escape from the personal and interpersonal demands of life is one of the major reasons that people have for engaging in tourism and leisure experiences”

The paper goes on to explain that al lot of the theory of restorative environments is based on the work of Kaplan and his Attention Restoration Theory. “The capacity to continually focus attention on a particular activity can be reduced or lost through mental exhaustion.” Recovery from just mental exhaustion requires that your “attention is engaged involuntarily or effortlessly,” which (I think) Kaplan calls “fascination.” Fascination allows your directed attention be rested.

Well, I think cultural heritage is fascinating, and this theory echos the theory of Flow, which I write about in my thesis, so it seems we are on to a winner with this paper. But there are three other components of restorative environments: being away (from routine); extent (the environment need to have enough content to keep you occupied a while) and compatibility (of interest – being bored is not fascinating). These sorts of things point to the infinite horizons of a walk in the countryside as being restorative (if the countryside is compatible with your interests) but analysis of responses to museums, art museums, gardens and zoos shows that the countryside is not the only place you can recharge.

So this paper looks to see if there are factors that make one museum a better restorative environment than another. To do so they use a “satisfying experiences framework” which focuses on:

  • “object experiences, which focus on something outside the visitor, such as seeing rare, valuable, or beautiful objects;
  • cognitive experiences, which focus on the interpretive or intellectual aspects of the experience, such as gaining information or understanding;
  • introspective experiences, which focus on private feelings and experiences, such as imagining, reflecting, reminiscing, and connecting; and
  • social experiences, which focus on interactions with friends, family, other visitors, or museum staff.”

They measured this and four different sites in Australia: a museum; an aquarium; a garden; and, an art gallery. Immediately they spotted that visitors to each find different experiences the most satisfying. The object experience of the fish in the aquarium was by far the most satisfying experience in any place, the cognitive experience of the museum was the most satisfying part of that visit (hmmmm). The social experience most satisfying in the gardens and, tough the art gallery was the most balanced between the four experiences, the social experience mattered to only 10.7% of visitors and the introspective experience mattered most to 29%. But it “was found that local visitors placed more importance on social and introspective experiences, and tourists placed more importance on cognitive and object experiences.” Tourists of course “are more likely to be looking for a learning and discovery experience—they want to discover new things and often try to ‘‘see as much as they can.’’ These experiences may be incompatible with a restorative experience.”

The study concludes “Not only were national parks and beaches considered more restorative than urban environments, but among the research sites, those that were focused on natural heritage (especially the botanic garden) were considered more restorative, both in attributes and benefits, than those focused on cultural heritage (the museum and art gallery).” But for frequent visitors (rather than first time visitors) museums can “offer an alternative to natural settings as a restorative experience.”

And more importantly, if “greater attention were given to visitors’ comfort, first-time and infrequent visitors, who are less familiar with the site, may be more able to experience restorative benefits as a result of their visit.” The authors also suggest that museums should “explore ways in which introspective experiences
might be encouraged and supported” and this I feel supports what I concluded at Chawton – while we insist on authenticity in our storytelling, we are privileging cognitive experiences over introspective ones.

Iterative development

Today I am looking at a paper about a Augmented Reality (AR) project at the Svevo Museum in Italy. TH AR part of the project interests me less than their methodology. As the the authors themselves conclude, AR is a young technology and at the moment the tools for developing the AR experience are mostly in the hands of technologists “could prevent the successful development of experiences focused on content rather than on technology which are capable of attracting diverse categories of users.”

The paper is: Fenu, Cristina & Pittarello, Fabio. 2018. Svevo tour: The design and the experimentation of an augmented reality application for engaging visitors of a literary museum. International Journal of Human-Computer Studies 114: 20-35. doi: 10.1016/j.ijhcs.2018.01.009.

The Svevo Museum is dedicated to the works and life of writer Italo Svevo, and being a literary museum is has many aesthetic challenges similar to the Chawton library where I did my experiment. Bu aesthetic challenges I mean that the appreciation of literature is not the same as the appreciation of the visual arts. Essentially literature is immaterial, even ephemeral – books, even old books essentially being containers for the work, not the work itself. Even if they are “only” containers, they are valuable, and more fragile than many museum collections and providing the access totem that visitors might expect can be an issue. This may have influenced their idea to use AR as part fo the experience, though they point out that “The Svevo Tour is, to our knowledge, one of the first AR projects conceived for a literary museum (AR-literary-museum-issue) and one of the challenges was to use these techniques for engaging its visitors.” That challenge made more acute by the nature of most of their visitors – adults and seniors who are often educators.

There’s not much that’s particularly quotable in this paper (remember I am doing all this reading for the required “modest” corrections on my thesis, but given the nature of the site, and the iterative approach to development. I might well reference this paper as an example of the sort of rapid prototyping that museum professionals can repurposing “off the shelf” software – Wikitude in the case of this project, Scalar for mine.

One thing I do like is that they so call Microsoft Hololens “AR”, not Mixed Reality. see my rant about that here.

Four Heritage Discourses

Following on from Thursday’s post on working with volunteers at Bishops House Sheffield, a second paper on the project has lots of very quotable stuff in its preamble. For those following along at home, this paper is: Claisse, Caroline, Petrelli, Daniela, Ciolfi, Luigina, Dulake, Nick, Marshall, Mark T. & Durrant, Abigail C.. 2020. Crafting Critical Heritage Discourses into Interactive Exhibition Design.

It starts off with a lovely quote about how the nature of museums changed as they became less about privately owned collections and more about public institutions: “Traditionally, museums are concerned with material cultures as their chief role is collecting and preserving artifacts for future generations, and to communicate what is known about those objects. The early museums were places where touching, holding and smelling were an integral part of the visit, a courtesy paid by the curator or the collection owner to their visitors or guests. By the mid 19th century, however, the personal, physical relation with the objects was gone, mostly because of the widening of the audience and therefore the changing mission of museums as public institutions”

But they go on to talk sing the praises of “house museums” which “offer interesting settings to explore the value of materiality in a context where a visually-driven ‘cabinets and labels’ approach exhibition design is not deemed appropriate.” They continue “Exhibition design in house museums goes beyond the curation and display of artifacts: the whole house is a historic object, meaning that content and container are one” which is a truism, but a very concise and apposite one. I do have to take issue with one thing they say however. They are wrong, or at least, conflating two things when they state “This type of house museums is described as ‘living history museums.'” The term “living history museums” often describes places that use costumed interpreters which, I would argue, the vast majority of house museums don’t – at least not on a day-to-day basis. The term is also used for the open air museums of buildings/folk/vernacular history such at The Zuiderzee Museum in Enkhuizen, and the Singleton Open Air Museum. Indeed it came be said that live costumed interpretation in museums started in places like those (though the Singleton museum resisted it for decades).

Another bit, with which I do agree, is their summary of Heritage as an aesthetic experience “Far from being perceived as boring and tiring, museums are, for many visitors, ‘restorative environments’ that, with their unique aesthetics, capture imagination and facilitate recovery from mental fatigue [26] [35]. Such aesthetic experiences are chiefly about being there. The aesthetic experience is not the experience of beauty – for it can be pleasant or unpleasant; it is characterised by intense attention, extended cognitive engagement, and affective responses.”

Of course one of the challenges to historic house storytelling is the existence of an implicit “‘authorized heritage discourse’ approach – a long-established orientation to heritage as understood and interpreted via the expert’s perspective, and tending to privilege the prestigious, universal and grand narratives” which is of course what I discovered which my Chawton experiment. What I noticed though, was not only that such a discourse is hard-wired into interpretive thinking, but that it might also be an outcome of visitor choices. It does not have to be so, but I would challenge their assertion that “The concept of heritage as a process that is actively constructed and recombined over time is particularly significant in the context of house museums where animators often dress up and enact characters to visitors: ‘It is this sensory experience of an embodied performance of a lifestyle that constitutes the process, through which heritage is both encountered and constructed.’” While excellent live interpreters can indeed challenge the “authorised heritage discourse approach” many more simply reinforce it, however in discussing that issue the authors a have a lovely turn of phrase which I am going to steal … I mean cite: “Exhibition design in a house museum can find inspiration from such an embodied storytelling that weaves the narratives with physical objects and social history, a game of performance and fiction within a specific physical space.” And again they elegantly put into words another truism that is not often expressed: “Staff and volunteers play an important role here, as they weave
stories and place by sharing with visitors a mixture of facts, speculation and anecdotes about the lives of previous residents. The meaning of the surrounding heritage emerges from the visitor-docent interaction process: visitors are expected to take part in the dialogue, to question and deepen their interests. In comparison, digital technology is easily seen as dry, a distraction from the actual place and curators of house museums are reluctant to introduce it.”

The evaluation of the project expressed in the conclusion identified four arguments “from heritage discourses that we found relevant for designing and reflecting on digitally-augmented exhibitions. We discussed the value of materiality, visiting as an aesthetic experience, challenging the authorized voice and heritage as a process.” The first two I have no problem at all with – I like especially their use of triggered scents to enhance the multisensory nature of the interpretation. I do challenge whether they really challenged the authorized voice, as I think the power of the dominant ideology and the expectations of the visitor for authenticity will have combined to confound the challenge but in one point I will say they did well – they used the power of fiction to create the narrative, they were not bound by fact. I will also argue that the visitor perception of heritage as a process might not be as great as they suggest it is – but again I will concede on point, they got visitors visiting spaces more than once to uncover the story. which is an impressive feat.

Design synthesis: working with volunteers

In a way, my project at Chawton was an example of Research though Design (RtD), so among the papers I have been pointed to for my corrections is another example, which is interesting for the historic house museums that I have spent a good part of my life working with. The paper is: CLAISSE, C. , DULAKE, N. & PETRELLI, D.. 2019. Research Through Design. Digital Heritage International Congress. Research Through Design.

This paper explores to co-creation process that fed into an Interactive Tablaux project at Bishops House museum in Sheffield. The research was multiphase and involved volunteers at every stage. First of all, by the designer becoming a volunteer and working alongside them. Then but then gifting of a Creative Package to ten volunteers. The package featured six activities or “Design Probes” to encourage the volunteers’ input. The probes started of straightforward but got more creative in sequence. “For example, Best wishes (probe 1)
invited them to write about their experience at the House by sending us back a postcard; My dream exhibition (probe 3) featured a map and personalised sketchbook to share their favourite stories and museum
objects; Seed wish (probe 5) used the metaphor of growth to prompt their imagination about future scenarios for the museum.”

Although the output of the design was inspiring the researcher found it challenging to move to “the phase of ideation.” However the response to the first design probe moved the work on somewhat – the researcher synthesised a visual volunteer manifesto (which took the form of a sort of wordcloud) from the the volunteers responses. This shed light on the personal, emotional and social dimensions of volunteering, and proved to be a catalyst for discussion. The outputs of probe three became a three dimensional model, mapping the volunteers favourite spot and things in three dimensions. Such models are often made far later in the creative process, when a proposal for a final exhibition is being visualised.

The Creative Package was followed by a series of workshop with volunteers – the first one speculating on what a day in the life of previous inhabitants might have looked like – which produced as an output a number of imagined portmanteaux characters. Conversations were recorded, but not transcribed. Instead the researcher draw in response to the recoded conversation. “drawing made more sense than producing a written transcription as the sketches captured the complexity and non-linear aspects of participants’ conversation. Their unfinished quality presented the characters in a state of becoming, revealing the negotiations and compromises participants made during the co- creation workshop.”

From these drawings the researcher produced a storyboard consisting of three to five illustrated scenes for each character, which were used in the subsequent workshop, with a smaller group fo volunteers to refine the stories and check accuracy. This gave substance to the characters and defined their personality. Finally the personas were shared with the curator of social history to select appropriate objects from the collection to pair with each character.

Each stage of co-creation had a virtual output, the Design Synthesis which both recorded the output of the preceding stage and prompted the development of the subsequent one. The authors of the paper reflect on design synthesis as a tool for making sense of co-creation, bringing the richness together and nurturing collective creativity. They quote the volunteer organiser saying “What I found really interesting is how volunteers’ ideas were determining the format and outcome of the project. They [the volunteers] could really see the contributions they have made in the end result.” One of the volunteers said “It’s lovely the way you [the designer] have involved the volunteers, used our ideas and made us part of it.”

Personalising the heritage visit

One the things that my external examiner pointed out during my viva is that I had not put in enough about personalisation. A number of the articles that she recommended I look at for my corrections address that issue and this is one. Not, Elena & Petrelli, Daniela. 2018. Blending customisation, context-awareness and adaptivity for personalised tangible interaction in cultural heritage. International Journal of Human-Computer Studies 114: 3-19. doi: 10.1016/j.ijhcs.2018.01.001.

I think one of the reasons why I had somewhat skimmed over the subject in my thesis is, coming out of a career based on live interpretation, which is essentially a conversation, personalisation is built into the way I think about heritage. My work in the PhD might be summed up as trying to ensure that conversation takes place even when there isn’t an excellent live interpreter working there. Of course, my examiner, Daniella is coming at the same aim, but from a different direction, and so she does not take personalisation for granted as I am wont to do. This paper however is a useful resource looking at it from the HCI point of view and I think there will be a number of quote from it and a summary among my corrections. But which quotes? What is summarised? As you might already have guessed these posts are a first pass at answering those questions. A Bit of practise as it were before creating a restructured version of my thesis. So, the paper starts off asking what personalisation actually means in this context:

“‘Personalisation’ is a broad term that encompasses three types ofsystem behaviour: adaptability (also called customisation, the term we use hereafter) offers users a number of options to set up the application/system the way they like it; context-awareness is the ability of the system to sense the current state of the environment and to respond accordingly; adaptivity implies the system maintains a dynamic model of the on-going interaction and dynamically changes its own behaviour to adapt to the changing situation.”

I can see why they chose to call adaptability customisation, adaptivity sounds far too similar and might be confusing. Of course one factor we have to consider is that although people do make heritage visits on their own, the vast majority are as part of a group; self-organised like couples, families, or friendship groups; or organised groups such as school visits and coach tours. As the authors point out “Research that directly addresses the social dimension is still limited” but they point the way to studies that look at conversation around a context aware-table, and sharing tables around a group, among others. However it is important to include this social dimension in any consideration of personalisation, which is something I did at Chawton – the choice there were made by the visiting “group” even if, sometimes that ‘group’ consisted of one person.

The paper of course starts with some case studies of similar work, including the Italian trenches soundscape I looked at a couple of days back. In another project at the same museum uses a “pebble” with NFC capability that activates media when places in certain places around the museum. When the visitor leaves, the pebble’s journey is read and a personalised postcards printed for the user to take home. In the Hague, a similarly NFC enabled system has the user place replica objects in “an interactive ring” which plays media from a choice of three different viewpoints (two military and one civilian). A third project, The Loupe, uses a phone disguised as a magnifying glass to present AR media. My problem with all these is an HCI one, two these systems force the users to learn a new interface, placing the pebble or replica in a certain spot to activate media that seems unintuitive, especially in environments where conditioned behaviour often precludes touch, picking things up, or even putting things on museum surfaces. On the other hand, the authors do make a point later that “A synergy can be created with tangible and embodied interactions to increase visitors’ awareness they are building their own visit path.” And I must admit that when the personalisation is invisible, the visitors do not perceive it. However my evaluation of Ghosts in the Garden suggest that even when tangible interactions are involved, the visitors may still not be aware that their experience was personalised.

But leaving my issues aside. There is some really good overview stuff in this – including a table that summarises some of the factors to consider when personalising interpretation. This includes: “stable” visitor factors – like age and disability, interests and Learning preferences; factors related to the current visit – motivations, fatigue, visit history and available time etc.; the type of tracking – two in this table, proximity tracking and interaction with objects; the location – indoor or outdoors, layout, noise etc.; and the content – the media, the story.

The team brought together 25 participants in a co-design workshop (curators , computer scientists and engineers) and they came up with a classification of features by the type of personalisation they support. The first group includes features that depend on content and are activated by “customisation preparation”: is this about on-site visits or virtual visits? Is it indoors or outdoors? What are constraints – is there power and wifi? The next set is decided by the curator or interpretation staff, Most of these come under “customisation preparation” too: what is the heritage topic?; The media type?; the genre of the text?; The thematic threads?; the supported visitor profiles? the type of group? Then what is the structure of the narrative, for example a story or a Q&A? Finally what is the structure of the visit? For example, is it guided, free exploration, or a treasure hunt?

One curatorial decision that falls into context awareness is does the interaction involve augmented objects, an if so wha are input and output abilities of those objects? were I the curator on this project I would look for forms of content awareness which do not rely on objects, even though some I have written about elsewhere are fun. But that move me into a set of context awareness features that are modelled by the system itself (according to the authors): user location, proximity to exhibits, proximity to other users, and the current state of the exhibits. To give the experience the adaptivity it needs, the he system will also use data about the shorty of the individual interactions with the space/objects and delivered content – just as in my Chawton experiment the system selected content based on what had been shared with the user before.

Finally comes the customisation choices, chosen, or course by the visitor and based on their motivation and expectations. They might have been given the opportunity to express interest in topics and narrative threads, as I did (somewhat clumsily) with my Clandon prototype. And, as at Chawton, the expected a duration of the visit is a factor (though I suggest it is less an active choice of the visitor, and better modelled by the system). Of course another factor that is totally out of the control of anyone other than the visitor is what the visitor thinks the type of visit is – they might be coming for an emotional reason, or social, or fun or for learning.

The paper concludes that “fully automatic adaptivity, where the system takes all the decisions on what to present to which visitor, when and how, may not be the best solution” and argues that therefore what curators (or interpreters) value as most meaningful should be the driver of of the personalisation model. I agree, but with the proviso that if the intent is top emotional engage the visitor, many heritage stories don’t do the job well enough. The authors say “This requires a radical rethinking of how personalisation in cultural heritage manifests itself and the role curators and visitors play” and I think think that my thesis might contribute to that rethinking.

Studying immersion in virtual tourism

I am less enamoured of the next paper that my expert examiner recommended: Raptis, George E., Fidas, Christos & Avouris, Nikolaos. 2018. Effects of mixed-reality on players’ behaviour and immersion in a cultural tourism game: A cognitive processing perspective. International Journal of Human-Computer Studies 114: 69-79. doi: 10.1016/j.ijhcs.2018.02.003.

This paper describes an attempt to measure attention in a “mixed reality” environment, and hypothesis the impact of such an environment on players of a cultural tourism game. I was hoping that it would be a useful attempt to do the sort of big-budget work I had originally intend to do in my studies – tracking user attention in a cultural-heritage environment with both persistent physical natoms (narrative atoms), and more ephemeral natoms (sound, light and other digital interventions). But although it uses the sort of technology to teach attention that I had hoped to find budget for (in this case Tobii Pro Glasses 2 gaze sampling system) I compares the users reactions to a game that is available PC (i.e. screen based) and also on Microsoft holo-lens. Now Hololens is market by Microsoft as a “mixed reality” system but I am not convinced it is. It is a reasonably sophisticated augmented reality system, but all it does is overlay the user’s environment with an image projected onto the goggle of the headset that they wear. Yes, it models the physical environment reasonably well, so that (when I had a chance to use it) I could “put” a virtual archeological model of a ship on a table then walk around the table to look at the ship from different angles. But I could not interact with the virtual by manipulating the physical. I have seen better “mixed reality” with an x-box and a sandpit.

The game used in this study is a case in point. Holotour, described as “a playful audiovisual three-dimensional virtual tourism application [that] transforms users to travellers, allowing them to see and explore virtual reality environments and experience physical places in space and time without physically travelling there” can be used on a screen or on hololens. It does not involve physical reality at all. It’s a very simple point and click adventure game with the object of collecting hidden objects and adding them to your inventory. The only difference between the on-screen version and the hololens one (as far as I can ascertain from this paper) is whether you use a mouse and cursor to point and click, or or your finger, held up in the field of vision of your goggles. So its not as useful as I had hoped, not tracking visitors’ attention around a physical site.

(This sin’t to say its not a useful paper to somebody – after all, virtual tourism might be all we can do in these covid times.)

I did learn something new (to me) in this paper however a model of cognitive style (or preference – see previous rants about learning styles) called Field Dependence-Independence (FD-I). “FD-I style is a single-
dimension model which measures the ability of an individual to extract information in visually complex scenes.” It may not be as new to me as I think – I recall reading a book, or chapter in a Conceptual Development book, during my first degree (thirty years ago) by (I think) Susan Greenfield about how some people (generally younger and games literate) were better able to follow the story in Hill Street Blues, because that drama was one of the first to feature multiple stories happening on the screen at the same time. I don’t recall her mentioning FD-I but it kind of sounds like the same thing. anyhow “FD individuals tend to prefer a holistic way when processing visual information and have difficulties in identifying details in complex visual scenes. On the other hand, FI individuals tend to prefer an analytical information processing approach, pay attention to details, and easily separate simple structures from the surrounding visual context.” I wonder which I am (from my failure to take in all the info on a game’s screen I am guessing FD.

Heritage Soundscapes

At my viva my external examiner pointed me towards this interesting paper, which she had co-authored – partly, I think, as an example of how I should restructure the discussion of my Chawton experiment in my thesis. But it contains some real gems ( like “the museums studies literature points out the restorative value of an aesthetic experience that is clear of any information acquisition or learning objective and is centred instead on the sensorial experience of being there”) that makes me regret missing it in my literature review: Marshall, M. , PETRELLI, D., DULAKE, N., NOT, E., MARCHESONI, M., TRENTI, E. & PISETTI, A.. 2015. Audio-based narratives for the trenches of World War I : intertwining stories, places and interaction for an evocative experience. International Journal of Human-Computer Studies 27-39.

It’s a case study of a prototype “visitor­ aware personalised multi­point auditory narrative system that automatically plays sounds and stories depending on a combination of features such as physical location, visitor proximity and visitor preferences” Voices from the Trenches for a First World War exhibition at the Museo Storico Italiano della Guerra in Italy. What particularly interest me is that its part of the Mesch project which has some other outcomes which I refer to in my thesis. The paper describes their intent to move away from what they call “the information­ centric approach of cultural heritage.” I am sure a number of my professional colleagues would bridle somewhat at this accusation. After all, did not Tilden tell us in the 50’s that interpretation was more than mere information? But one od the things that my Chawton experiment uncovered was that actually too much “interpretation” turns out to be mere information after all.

The authors summarise previous experiments in responsive soundscapes, such as LISTEN, which “composes a soundscape of music and/or commentaries depending on the detected visitor’s behaviour: visitors that are not close or are moving are classified as unfocussed and for them a soundscape is created, while visitors that are standing still and close to the artwork are classified as focussed and a narrative (e.g. the curator describing the artwork) is played over the headphones.” Though many soundscapes are delivered by headphone, to avoid sound pollution for other visitors, the interesting project SottoVoce is designed around eavesdropping on what other people in our party are listening to. Half the respondents (in groups of two) heard the soundscape from each others phone speakers, while the other half had headphones. “When in loudspeaker mode visitors focussed on what was displayed on the screen of the mobile device and stayed close to the sound source while partners linked via the same audio on their headphones had a more dynamic visit driven by each other’s interest in the exhibits.”

“The ability to convey and evoke emotion is a fundamental aspect of sound” they say, and explain “The affective power of voice and audio storytelling has been recognised as creating a connection to the listener and is even amplified when spoken words are not coupled with the visual capture of the storyteller, creating a sense of intimacy and affective engagement.” An they built their soundscapes using the same sort of mix of music, speech and other sounds that I used (in a limited fashion) at Chawton. Some of the primary source material was recorded to sound more like oral history, with actors reading the words “with palpable emotion” to be more affective. The responsiveness is similar to that of LISTEN, but the “staying still” metric isn’t used, instead a simpler proximity method is used. woven into that soundscape are voice recordings for attentive listening, which is selected by the visitor choosing from a selection of cards. The sound was delivered by loudspeakers but, unlike SottoVoce, not on people’s own devices, rather places around the site. This was what I did for Chawton UNtours too.

The particular challenge with this project was that it was outdoors.The difficulties of maintaining equipment, connecting power and data etc means that most sites resort to delivering via mobile device. But on the other hand: “While engagement in a museum tends to be via prolonged observation, in an outdoor setting multiple senses are stimulated: there is the physical, full­body experience of being there, the sight and the sound of the surroundings, possibly the smell too. The multi-sensory setting places the visitor in direct connection with the heritage and enables engagement at an emotional, affective level rather than at a pure informative level.” (p6) The danger of using a mobile device to deliver interpretation is one I wrote about here, but essentially it stake them out of the where they are, it is the antithesis off presence.

With all this in mind the designers of the project set out five clear principles:

  • To engage at multiple levels, not just cognitive
  • To focus the visitors’ attention on the heritage, not the technology
  • To deal with group dynamics sensibly
  • To be provocative and surprise visitors, but design simple and straightforward interactions
  • To personalize content on the basis of clear conditions

The choice of sound over anything screen-based was an outcome of the second principle. Loudspeakers rather than headphones was also an attempt to focus attention on the heritage: “During a small experiment in a local outdoor heritage site, we observed that audio creates a wider attraction zone where passers­by become aware of the sound source, and a closer engagement zone around the emitting point where one has to stop and listen in order to understand what the voice says.”

So they designed a soundscape that featured music nd sound to attract visitor to a location and then vice recording to hold them there. The narratives are arranged thematically, with different voices (authoritative and intimate) indicating the nature of the content. Quite how the visitor chooses is not really made clear but I expect it is by approaching the voices that most attract them.

The team trialed the idea by observing the visitors behaviour using about 23 minutes of content, but I was disappointed that they did not come up with any solutions to the problems we encounter trying to evaluate the soundscape at The Vyne. It is hard to observe and distinguish between active listening and background listening. The authors seen to assume that if the active listening content is playing, then the partiocilapants are actively listening. The only evidence they have for this is a qualitative questionnaire, which I am not convinced is an accurate measure on engagment. Yes they said they enjoyed an benefitted from the experience, but if they did not know that was what was being tested, what proportion would have even mentioned the soundscape.

Of course they identified a number of challenges, not least fine-tuning the volume to be loud enough to attract attention and yet not so loud to cause discomfort. This is especially true of the different voices, with some by necessity quieter and more intimate. Of course they also predicted issues overs scalability – similar to the ones I planned fro but wasn’t able to properly test at Chawton “how well would such a system work in a busy environment with many groups interacting.”

The challenges of evaluating in-visit digitally enabled heritage interpretation

I am reading a paper which will help me better present my Chawton study. The paper is brand new: Nikolakopoulou, V. & Koutsabasis, P.. 2019. Methods and Practices for Assessing the User Experience of Interactive Systems for Cultural Heritage. In: Pavlidis, G., ed. Applying Innovative Technologies in Heritage Science. IGI Global, and it is another literature review, but more a sort of meta study of evaluations, called by the authors a “fortiori.”

The methodology of this literature search is exemplary. The authors searched (via Google scholar, ACM digital library, IEEEXplore, Science direct, and Springer link) and found 350 papers, and and in a second search another 150 that looked as though they might be evaluation studies of digital in-visit heritage content. They screened out the majority, for not actually being accessible, or not actually being scientific (I think my thesis would have fallen at this hurdle), leaving just 73. These were screened a second time and a number of duplicate were found and then a number which did not actually write up the evaluation. Leaving just 29 to study in depth.

In summary these papers discussed applications to”explore a virtual or physical space or place” and/or “play a game”. The authors noted the the proportion of games had gone down since a previous survey, which likely corresponds with the time of my change of focus away from a ludic system to something more to do with narrative. They were still roughly 20% of the papers they looked at. Overall, the systems used a variety of digital technologies including 3d game engines, mobile devices, mobile AR, VR, the web, multi-touch displays, location based audio, and physical and kinaesthetic (responding to body movement) interfaces.

There were a lot of studies that looked at usability of the technologies, but the paper points out only using “user satisfaction” as a metric, which is a dangerous trap to fall into. Only a few to a thorough investigation of the User Experience in that way that modern commercial companies test their websites. “A considerable number of systems are evaluated upon learning effects on users” this despite that fact that learning is often not the primary reason for days out (though it may be a validation for the museum), but again few do that in a properly scientific way. There is an interesting paper mentioned (Falvo, P. G. (2015, September). Exhibit design with multimedia and cognitive technologies impact assessment on Luca Giordano, Raphael and the Chapel of the Magi in Palazzo Medici Riccardi, Florence. In 2015 Digital Heritage (Vol. 2, pp. 525-532). IEEE.)) that I may want to take a look at.

The paper highlights the difference between empirical evaluations “in the lab” as it were and in the field. “Field evaluations are contextual, which enhanced the validity of the process and results.” But “recruiting visitors to experience or assess CH content on purpose changes their original purpose of visit, which is
something inherently connected with the visitor experience discussed in several museum and visitor studies […] which suggests a paradox to me, either test random users, and change their reason for visiting, and potentially their responses to the visit; or invite visitors especially for the test and thus make it more like a lab experiment. The reason for visiting a cultural heritage experience is actually part of the experience.

One issue they highlight is that despite being “peer-reviewed and published in important journals and
conferences, it was possible to identify several failures in the quality of reporting on empirical evaluations” in particular reporting on the number of users that participated in the evaluations. I expect that is because, due tho the limitations of evaluation in cultural heritage sites, the time taken to observe a visit, the number of participants is statistically small – small enough to put into question wether the studies are “empirical” at all I know that after a week on site I had less than ten samples. And so I tried very hard not to present what I learned as empirical evidence. Indeed maybe even too hard. That bit need sa re-write post viva.

The conclusion is a reiteration that digital interpretation evaluation should involve more cultural heritage professionals and field studies. It point out that there is more work in newer methodologies “like physical computing and tangible user interfaces” (objects that are the interfaceIt also highlights ongoing issues of a lack of systematic approaches to evaluation, organised “in distinct identifiable catagories” which I imagine would make doing meta-studies like this more meaningful. Also, “Aspects of cultural value, awareness, and presence could only be recognized in very few empirical evaluations. Evaluation studies that consider more qualitative dimensions and more related to general purposes of CH like learning effectiveness have been increased, and they are usually accompanied by comparative evaluations. Comparative evaluations represent a small number compared to the overall number of studies reviewed.”