Iterative development

Today I am looking at a paper about a Augmented Reality (AR) project at the Svevo Museum in Italy. TH AR part of the project interests me less than their methodology. As the the authors themselves conclude, AR is a young technology and at the moment the tools for developing the AR experience are mostly in the hands of technologists “could prevent the successful development of experiences focused on content rather than on technology which are capable of attracting diverse categories of users.”

The paper is: Fenu, Cristina & Pittarello, Fabio. 2018. Svevo tour: The design and the experimentation of an augmented reality application for engaging visitors of a literary museum. International Journal of Human-Computer Studies 114: 20-35. doi: 10.1016/j.ijhcs.2018.01.009.

The Svevo Museum is dedicated to the works and life of writer Italo Svevo, and being a literary museum is has many aesthetic challenges similar to the Chawton library where I did my experiment. Bu aesthetic challenges I mean that the appreciation of literature is not the same as the appreciation of the visual arts. Essentially literature is immaterial, even ephemeral – books, even old books essentially being containers for the work, not the work itself. Even if they are “only” containers, they are valuable, and more fragile than many museum collections and providing the access totem that visitors might expect can be an issue. This may have influenced their idea to use AR as part fo the experience, though they point out that “The Svevo Tour is, to our knowledge, one of the first AR projects conceived for a literary museum (AR-literary-museum-issue) and one of the challenges was to use these techniques for engaging its visitors.” That challenge made more acute by the nature of most of their visitors – adults and seniors who are often educators.

There’s not much that’s particularly quotable in this paper (remember I am doing all this reading for the required “modest” corrections on my thesis, but given the nature of the site, and the iterative approach to development. I might well reference this paper as an example of the sort of rapid prototyping that museum professionals can repurposing “off the shelf” software – Wikitude in the case of this project, Scalar for mine.

One thing I do like is that they so call Microsoft Hololens “AR”, not Mixed Reality. see my rant about that here.

Four Heritage Discourses

Following on from Thursday’s post on working with volunteers at Bishops House Sheffield, a second paper on the project has lots of very quotable stuff in its preamble. For those following along at home, this paper is: Claisse, Caroline, Petrelli, Daniela, Ciolfi, Luigina, Dulake, Nick, Marshall, Mark T. & Durrant, Abigail C.. 2020. Crafting Critical Heritage Discourses into Interactive Exhibition Design.

It starts off with a lovely quote about how the nature of museums changed as they became less about privately owned collections and more about public institutions: “Traditionally, museums are concerned with material cultures as their chief role is collecting and preserving artifacts for future generations, and to communicate what is known about those objects. The early museums were places where touching, holding and smelling were an integral part of the visit, a courtesy paid by the curator or the collection owner to their visitors or guests. By the mid 19th century, however, the personal, physical relation with the objects was gone, mostly because of the widening of the audience and therefore the changing mission of museums as public institutions”

But they go on to talk sing the praises of “house museums” which “offer interesting settings to explore the value of materiality in a context where a visually-driven ‘cabinets and labels’ approach exhibition design is not deemed appropriate.” They continue “Exhibition design in house museums goes beyond the curation and display of artifacts: the whole house is a historic object, meaning that content and container are one” which is a truism, but a very concise and apposite one. I do have to take issue with one thing they say however. They are wrong, or at least, conflating two things when they state “This type of house museums is described as ‘living history museums.'” The term “living history museums” often describes places that use costumed interpreters which, I would argue, the vast majority of house museums don’t – at least not on a day-to-day basis. The term is also used for the open air museums of buildings/folk/vernacular history such at The Zuiderzee Museum in Enkhuizen, and the Singleton Open Air Museum. Indeed it came be said that live costumed interpretation in museums started in places like those (though the Singleton museum resisted it for decades).

Another bit, with which I do agree, is their summary of Heritage as an aesthetic experience “Far from being perceived as boring and tiring, museums are, for many visitors, ‘restorative environments’ that, with their unique aesthetics, capture imagination and facilitate recovery from mental fatigue [26] [35]. Such aesthetic experiences are chiefly about being there. The aesthetic experience is not the experience of beauty – for it can be pleasant or unpleasant; it is characterised by intense attention, extended cognitive engagement, and affective responses.”

Of course one of the challenges to historic house storytelling is the existence of an implicit “‘authorized heritage discourse’ approach – a long-established orientation to heritage as understood and interpreted via the expert’s perspective, and tending to privilege the prestigious, universal and grand narratives” which is of course what I discovered which my Chawton experiment. What I noticed though, was not only that such a discourse is hard-wired into interpretive thinking, but that it might also be an outcome of visitor choices. It does not have to be so, but I would challenge their assertion that “The concept of heritage as a process that is actively constructed and recombined over time is particularly significant in the context of house museums where animators often dress up and enact characters to visitors: ‘It is this sensory experience of an embodied performance of a lifestyle that constitutes the process, through which heritage is both encountered and constructed.’” While excellent live interpreters can indeed challenge the “authorised heritage discourse approach” many more simply reinforce it, however in discussing that issue the authors a have a lovely turn of phrase which I am going to steal … I mean cite: “Exhibition design in a house museum can find inspiration from such an embodied storytelling that weaves the narratives with physical objects and social history, a game of performance and fiction within a specific physical space.” And again they elegantly put into words another truism that is not often expressed: “Staff and volunteers play an important role here, as they weave
stories and place by sharing with visitors a mixture of facts, speculation and anecdotes about the lives of previous residents. The meaning of the surrounding heritage emerges from the visitor-docent interaction process: visitors are expected to take part in the dialogue, to question and deepen their interests. In comparison, digital technology is easily seen as dry, a distraction from the actual place and curators of house museums are reluctant to introduce it.”

The evaluation of the project expressed in the conclusion identified four arguments “from heritage discourses that we found relevant for designing and reflecting on digitally-augmented exhibitions. We discussed the value of materiality, visiting as an aesthetic experience, challenging the authorized voice and heritage as a process.” The first two I have no problem at all with – I like especially their use of triggered scents to enhance the multisensory nature of the interpretation. I do challenge whether they really challenged the authorized voice, as I think the power of the dominant ideology and the expectations of the visitor for authenticity will have combined to confound the challenge but in one point I will say they did well – they used the power of fiction to create the narrative, they were not bound by fact. I will also argue that the visitor perception of heritage as a process might not be as great as they suggest it is – but again I will concede on point, they got visitors visiting spaces more than once to uncover the story. which is an impressive feat.

Design synthesis: working with volunteers

In a way, my project at Chawton was an example of Research though Design (RtD), so among the papers I have been pointed to for my corrections is another example, which is interesting for the historic house museums that I have spent a good part of my life working with. The paper is: CLAISSE, C. , DULAKE, N. & PETRELLI, D.. 2019. Research Through Design. Digital Heritage International Congress. Research Through Design.

This paper explores to co-creation process that fed into an Interactive Tablaux project at Bishops House museum in Sheffield. The research was multiphase and involved volunteers at every stage. First of all, by the designer becoming a volunteer and working alongside them. Then but then gifting of a Creative Package to ten volunteers. The package featured six activities or “Design Probes” to encourage the volunteers’ input. The probes started of straightforward but got more creative in sequence. “For example, Best wishes (probe 1)
invited them to write about their experience at the House by sending us back a postcard; My dream exhibition (probe 3) featured a map and personalised sketchbook to share their favourite stories and museum
objects; Seed wish (probe 5) used the metaphor of growth to prompt their imagination about future scenarios for the museum.”

Although the output of the design was inspiring the researcher found it challenging to move to “the phase of ideation.” However the response to the first design probe moved the work on somewhat – the researcher synthesised a visual volunteer manifesto (which took the form of a sort of wordcloud) from the the volunteers responses. This shed light on the personal, emotional and social dimensions of volunteering, and proved to be a catalyst for discussion. The outputs of probe three became a three dimensional model, mapping the volunteers favourite spot and things in three dimensions. Such models are often made far later in the creative process, when a proposal for a final exhibition is being visualised.

The Creative Package was followed by a series of workshop with volunteers – the first one speculating on what a day in the life of previous inhabitants might have looked like – which produced as an output a number of imagined portmanteaux characters. Conversations were recorded, but not transcribed. Instead the researcher draw in response to the recoded conversation. “drawing made more sense than producing a written transcription as the sketches captured the complexity and non-linear aspects of participants’ conversation. Their unfinished quality presented the characters in a state of becoming, revealing the negotiations and compromises participants made during the co- creation workshop.”

From these drawings the researcher produced a storyboard consisting of three to five illustrated scenes for each character, which were used in the subsequent workshop, with a smaller group fo volunteers to refine the stories and check accuracy. This gave substance to the characters and defined their personality. Finally the personas were shared with the curator of social history to select appropriate objects from the collection to pair with each character.

Each stage of co-creation had a virtual output, the Design Synthesis which both recorded the output of the preceding stage and prompted the development of the subsequent one. The authors of the paper reflect on design synthesis as a tool for making sense of co-creation, bringing the richness together and nurturing collective creativity. They quote the volunteer organiser saying “What I found really interesting is how volunteers’ ideas were determining the format and outcome of the project. They [the volunteers] could really see the contributions they have made in the end result.” One of the volunteers said “It’s lovely the way you [the designer] have involved the volunteers, used our ideas and made us part of it.”

Personalising the heritage visit

One the things that my external examiner pointed out during my viva is that I had not put in enough about personalisation. A number of the articles that she recommended I look at for my corrections address that issue and this is one. Not, Elena & Petrelli, Daniela. 2018. Blending customisation, context-awareness and adaptivity for personalised tangible interaction in cultural heritage. International Journal of Human-Computer Studies 114: 3-19. doi: 10.1016/j.ijhcs.2018.01.001.

I think one of the reasons why I had somewhat skimmed over the subject in my thesis is, coming out of a career based on live interpretation, which is essentially a conversation, personalisation is built into the way I think about heritage. My work in the PhD might be summed up as trying to ensure that conversation takes place even when there isn’t an excellent live interpreter working there. Of course, my examiner, Daniella is coming at the same aim, but from a different direction, and so she does not take personalisation for granted as I am wont to do. This paper however is a useful resource looking at it from the HCI point of view and I think there will be a number of quote from it and a summary among my corrections. But which quotes? What is summarised? As you might already have guessed these posts are a first pass at answering those questions. A Bit of practise as it were before creating a restructured version of my thesis. So, the paper starts off asking what personalisation actually means in this context:

“‘Personalisation’ is a broad term that encompasses three types ofsystem behaviour: adaptability (also called customisation, the term we use hereafter) offers users a number of options to set up the application/system the way they like it; context-awareness is the ability of the system to sense the current state of the environment and to respond accordingly; adaptivity implies the system maintains a dynamic model of the on-going interaction and dynamically changes its own behaviour to adapt to the changing situation.”

I can see why they chose to call adaptability customisation, adaptivity sounds far too similar and might be confusing. Of course one factor we have to consider is that although people do make heritage visits on their own, the vast majority are as part of a group; self-organised like couples, families, or friendship groups; or organised groups such as school visits and coach tours. As the authors point out “Research that directly addresses the social dimension is still limited” but they point the way to studies that look at conversation around a context aware-table, and sharing tables around a group, among others. However it is important to include this social dimension in any consideration of personalisation, which is something I did at Chawton – the choice there were made by the visiting “group” even if, sometimes that ‘group’ consisted of one person.

The paper of course starts with some case studies of similar work, including the Italian trenches soundscape I looked at a couple of days back. In another project at the same museum uses a “pebble” with NFC capability that activates media when places in certain places around the museum. When the visitor leaves, the pebble’s journey is read and a personalised postcards printed for the user to take home. In the Hague, a similarly NFC enabled system has the user place replica objects in “an interactive ring” which plays media from a choice of three different viewpoints (two military and one civilian). A third project, The Loupe, uses a phone disguised as a magnifying glass to present AR media. My problem with all these is an HCI one, two these systems force the users to learn a new interface, placing the pebble or replica in a certain spot to activate media that seems unintuitive, especially in environments where conditioned behaviour often precludes touch, picking things up, or even putting things on museum surfaces. On the other hand, the authors do make a point later that “A synergy can be created with tangible and embodied interactions to increase visitors’ awareness they are building their own visit path.” And I must admit that when the personalisation is invisible, the visitors do not perceive it. However my evaluation of Ghosts in the Garden suggest that even when tangible interactions are involved, the visitors may still not be aware that their experience was personalised.

But leaving my issues aside. There is some really good overview stuff in this – including a table that summarises some of the factors to consider when personalising interpretation. This includes: “stable” visitor factors – like age and disability, interests and Learning preferences; factors related to the current visit – motivations, fatigue, visit history and available time etc.; the type of tracking – two in this table, proximity tracking and interaction with objects; the location – indoor or outdoors, layout, noise etc.; and the content – the media, the story.

The team brought together 25 participants in a co-design workshop (curators , computer scientists and engineers) and they came up with a classification of features by the type of personalisation they support. The first group includes features that depend on content and are activated by “customisation preparation”: is this about on-site visits or virtual visits? Is it indoors or outdoors? What are constraints – is there power and wifi? The next set is decided by the curator or interpretation staff, Most of these come under “customisation preparation” too: what is the heritage topic?; The media type?; the genre of the text?; The thematic threads?; the supported visitor profiles? the type of group? Then what is the structure of the narrative, for example a story or a Q&A? Finally what is the structure of the visit? For example, is it guided, free exploration, or a treasure hunt?

One curatorial decision that falls into context awareness is does the interaction involve augmented objects, an if so wha are input and output abilities of those objects? were I the curator on this project I would look for forms of content awareness which do not rely on objects, even though some I have written about elsewhere are fun. But that move me into a set of context awareness features that are modelled by the system itself (according to the authors): user location, proximity to exhibits, proximity to other users, and the current state of the exhibits. To give the experience the adaptivity it needs, the he system will also use data about the shorty of the individual interactions with the space/objects and delivered content – just as in my Chawton experiment the system selected content based on what had been shared with the user before.

Finally comes the customisation choices, chosen, or course by the visitor and based on their motivation and expectations. They might have been given the opportunity to express interest in topics and narrative threads, as I did (somewhat clumsily) with my Clandon prototype. And, as at Chawton, the expected a duration of the visit is a factor (though I suggest it is less an active choice of the visitor, and better modelled by the system). Of course another factor that is totally out of the control of anyone other than the visitor is what the visitor thinks the type of visit is – they might be coming for an emotional reason, or social, or fun or for learning.

The paper concludes that “fully automatic adaptivity, where the system takes all the decisions on what to present to which visitor, when and how, may not be the best solution” and argues that therefore what curators (or interpreters) value as most meaningful should be the driver of of the personalisation model. I agree, but with the proviso that if the intent is top emotional engage the visitor, many heritage stories don’t do the job well enough. The authors say “This requires a radical rethinking of how personalisation in cultural heritage manifests itself and the role curators and visitors play” and I think think that my thesis might contribute to that rethinking.

Studying immersion in virtual tourism

I am less enamoured of the next paper that my expert examiner recommended: Raptis, George E., Fidas, Christos & Avouris, Nikolaos. 2018. Effects of mixed-reality on players’ behaviour and immersion in a cultural tourism game: A cognitive processing perspective. International Journal of Human-Computer Studies 114: 69-79. doi: 10.1016/j.ijhcs.2018.02.003.

This paper describes an attempt to measure attention in a “mixed reality” environment, and hypothesis the impact of such an environment on players of a cultural tourism game. I was hoping that it would be a useful attempt to do the sort of big-budget work I had originally intend to do in my studies – tracking user attention in a cultural-heritage environment with both persistent physical natoms (narrative atoms), and more ephemeral natoms (sound, light and other digital interventions). But although it uses the sort of technology to teach attention that I had hoped to find budget for (in this case Tobii Pro Glasses 2 gaze sampling system) I compares the users reactions to a game that is available PC (i.e. screen based) and also on Microsoft holo-lens. Now Hololens is market by Microsoft as a “mixed reality” system but I am not convinced it is. It is a reasonably sophisticated augmented reality system, but all it does is overlay the user’s environment with an image projected onto the goggle of the headset that they wear. Yes, it models the physical environment reasonably well, so that (when I had a chance to use it) I could “put” a virtual archeological model of a ship on a table then walk around the table to look at the ship from different angles. But I could not interact with the virtual by manipulating the physical. I have seen better “mixed reality” with an x-box and a sandpit.

The game used in this study is a case in point. Holotour, described as “a playful audiovisual three-dimensional virtual tourism application [that] transforms users to travellers, allowing them to see and explore virtual reality environments and experience physical places in space and time without physically travelling there” can be used on a screen or on hololens. It does not involve physical reality at all. It’s a very simple point and click adventure game with the object of collecting hidden objects and adding them to your inventory. The only difference between the on-screen version and the hololens one (as far as I can ascertain from this paper) is whether you use a mouse and cursor to point and click, or or your finger, held up in the field of vision of your goggles. So its not as useful as I had hoped, not tracking visitors’ attention around a physical site.

(This sin’t to say its not a useful paper to somebody – after all, virtual tourism might be all we can do in these covid times.)

I did learn something new (to me) in this paper however a model of cognitive style (or preference – see previous rants about learning styles) called Field Dependence-Independence (FD-I). “FD-I style is a single-
dimension model which measures the ability of an individual to extract information in visually complex scenes.” It may not be as new to me as I think – I recall reading a book, or chapter in a Conceptual Development book, during my first degree (thirty years ago) by (I think) Susan Greenfield about how some people (generally younger and games literate) were better able to follow the story in Hill Street Blues, because that drama was one of the first to feature multiple stories happening on the screen at the same time. I don’t recall her mentioning FD-I but it kind of sounds like the same thing. anyhow “FD individuals tend to prefer a holistic way when processing visual information and have difficulties in identifying details in complex visual scenes. On the other hand, FI individuals tend to prefer an analytical information processing approach, pay attention to details, and easily separate simple structures from the surrounding visual context.” I wonder which I am (from my failure to take in all the info on a game’s screen I am guessing FD.

Heritage Soundscapes

At my viva my external examiner pointed me towards this interesting paper, which she had co-authored – partly, I think, as an example of how I should restructure the discussion of my Chawton experiment in my thesis. But it contains some real gems ( like “the museums studies literature points out the restorative value of an aesthetic experience that is clear of any information acquisition or learning objective and is centred instead on the sensorial experience of being there”) that makes me regret missing it in my literature review: Marshall, M. , PETRELLI, D., DULAKE, N., NOT, E., MARCHESONI, M., TRENTI, E. & PISETTI, A.. 2015. Audio-based narratives for the trenches of World War I : intertwining stories, places and interaction for an evocative experience. International Journal of Human-Computer Studies 27-39.

It’s a case study of a prototype “visitor­ aware personalised multi­point auditory narrative system that automatically plays sounds and stories depending on a combination of features such as physical location, visitor proximity and visitor preferences” Voices from the Trenches for a First World War exhibition at the Museo Storico Italiano della Guerra in Italy. What particularly interest me is that its part of the Mesch project which has some other outcomes which I refer to in my thesis. The paper describes their intent to move away from what they call “the information­ centric approach of cultural heritage.” I am sure a number of my professional colleagues would bridle somewhat at this accusation. After all, did not Tilden tell us in the 50’s that interpretation was more than mere information? But one od the things that my Chawton experiment uncovered was that actually too much “interpretation” turns out to be mere information after all.

The authors summarise previous experiments in responsive soundscapes, such as LISTEN, which “composes a soundscape of music and/or commentaries depending on the detected visitor’s behaviour: visitors that are not close or are moving are classified as unfocussed and for them a soundscape is created, while visitors that are standing still and close to the artwork are classified as focussed and a narrative (e.g. the curator describing the artwork) is played over the headphones.” Though many soundscapes are delivered by headphone, to avoid sound pollution for other visitors, the interesting project SottoVoce is designed around eavesdropping on what other people in our party are listening to. Half the respondents (in groups of two) heard the soundscape from each others phone speakers, while the other half had headphones. “When in loudspeaker mode visitors focussed on what was displayed on the screen of the mobile device and stayed close to the sound source while partners linked via the same audio on their headphones had a more dynamic visit driven by each other’s interest in the exhibits.”

“The ability to convey and evoke emotion is a fundamental aspect of sound” they say, and explain “The affective power of voice and audio storytelling has been recognised as creating a connection to the listener and is even amplified when spoken words are not coupled with the visual capture of the storyteller, creating a sense of intimacy and affective engagement.” An they built their soundscapes using the same sort of mix of music, speech and other sounds that I used (in a limited fashion) at Chawton. Some of the primary source material was recorded to sound more like oral history, with actors reading the words “with palpable emotion” to be more affective. The responsiveness is similar to that of LISTEN, but the “staying still” metric isn’t used, instead a simpler proximity method is used. woven into that soundscape are voice recordings for attentive listening, which is selected by the visitor choosing from a selection of cards. The sound was delivered by loudspeakers but, unlike SottoVoce, not on people’s own devices, rather places around the site. This was what I did for Chawton UNtours too.

The particular challenge with this project was that it was outdoors.The difficulties of maintaining equipment, connecting power and data etc means that most sites resort to delivering via mobile device. But on the other hand: “While engagement in a museum tends to be via prolonged observation, in an outdoor setting multiple senses are stimulated: there is the physical, full­body experience of being there, the sight and the sound of the surroundings, possibly the smell too. The multi-sensory setting places the visitor in direct connection with the heritage and enables engagement at an emotional, affective level rather than at a pure informative level.” (p6) The danger of using a mobile device to deliver interpretation is one I wrote about here, but essentially it stake them out of the where they are, it is the antithesis off presence.

With all this in mind the designers of the project set out five clear principles:

  • To engage at multiple levels, not just cognitive
  • To focus the visitors’ attention on the heritage, not the technology
  • To deal with group dynamics sensibly
  • To be provocative and surprise visitors, but design simple and straightforward interactions
  • To personalize content on the basis of clear conditions

The choice of sound over anything screen-based was an outcome of the second principle. Loudspeakers rather than headphones was also an attempt to focus attention on the heritage: “During a small experiment in a local outdoor heritage site, we observed that audio creates a wider attraction zone where passers­by become aware of the sound source, and a closer engagement zone around the emitting point where one has to stop and listen in order to understand what the voice says.”

So they designed a soundscape that featured music nd sound to attract visitor to a location and then vice recording to hold them there. The narratives are arranged thematically, with different voices (authoritative and intimate) indicating the nature of the content. Quite how the visitor chooses is not really made clear but I expect it is by approaching the voices that most attract them.

The team trialed the idea by observing the visitors behaviour using about 23 minutes of content, but I was disappointed that they did not come up with any solutions to the problems we encounter trying to evaluate the soundscape at The Vyne. It is hard to observe and distinguish between active listening and background listening. The authors seen to assume that if the active listening content is playing, then the partiocilapants are actively listening. The only evidence they have for this is a qualitative questionnaire, which I am not convinced is an accurate measure on engagment. Yes they said they enjoyed an benefitted from the experience, but if they did not know that was what was being tested, what proportion would have even mentioned the soundscape.

Of course they identified a number of challenges, not least fine-tuning the volume to be loud enough to attract attention and yet not so loud to cause discomfort. This is especially true of the different voices, with some by necessity quieter and more intimate. Of course they also predicted issues overs scalability – similar to the ones I planned fro but wasn’t able to properly test at Chawton “how well would such a system work in a busy environment with many groups interacting.”

Pandemic and Materiality

Another new paper, Galani, A. & Kidd, J.. 2020. Hybrid Material Encounters – Expanding the Continuum of Museum Materialities in the Wake of a Pandemic. Museum & Society 18: 298 – 301, comments on the “pivot to digital” that cultural heritage has been forced to make, with varying degrees of success, by the current pandemic. We know that the initial rush to apps that heritage made in the decade after the introduction the iPhone ran out of steam as use of devices on site, and downloads were low, and the projects were often made with capital investment with no understanding of how maintenance and updates would be financed or managed.

But museums have turn to the web, if only to retain mindshare during the pandemic, despite “Luís Raposo, president of ICOM Europe, passionately declar[ing] that ‘Humans are analog [sic] creatures, not digital’,
[and] urging museums to continue to be ‘spaces where each one [of us] can be confronted with real original materialities’ post-pandemic. (Luís Raposo, ‘Museums in Face of the “Perfect Storm”’, ICOM Europe Facebook, 15 June, accessed 29 June 2020.)

The authors of this paper suggest that “As museums and their audiences turned to digital forms of engagement in the absence of physical encounters, […] new hybrid materialities were made possible within and through digital spaces.” We saw a number of fun attempts to connect locked out humans with digital representations of cultural heritage, including the J. Paul Getty museum capturing the moment with a challenge yo recreate your favourite artwork with household objects. Galani and Kidd point to this and to Robot Tours provided by the Hastings Contemporary, “developed as partof a pre-existing research initiative to allow individuals at risk of isolation, including due to disability, to experience the gallery through a remotely controlled robot’s camera.” The popularity of these during lockdown threw the limitations of physical museum visits for a significant part of the population into sharp contrast.

They argue that Hastings Robot Tour “bridges physical encounters with art and remote visiting, animating the materialities of the respective experiences. The tours urge us to challenge the proliferating use of digital as a mere tool for capturing and representing literal forms of materiality and to revisit the ways we design for, value and make sense of material sensory encounters more broadly.” However they use the pandemic and digital responses to it to argue a point that seems well rehearsed, “As people assemble the materiality of their heritage encounters through a range of digital, analogue, tangible and intangible resources, their visiting experiences transcend traditional articulations of the physical-digital divide and operate on a continuum of materialities.” Which may be true but I would offer the counterpoint that most visitors don’t come seeking to transcend traditional articulations and operate on a continuum of materialities, but rather simply to be in a place, and/or surrounded by things.

Thats said, their call for hybridity is a strong one. I will not argue with their assertion that cultural heritage should “not only […] initiate but also to nurture relationships with content creators and to accommodate resulting outputs, such as the Getty re-creations.” Neither will I deny that it is an opportunity to unpick the “perceived neutrality of digital infrastructures, such as content management systems and metadata, which shape inclusionary/exclusionary practices and interpretations of original artefacts.” The argue that Covid has given museums the confidence to use the power of hybrid (digital/material) spaces to explore on-line engagement and, importantly, to make “online audiences […] key agents in the production of digitally-mediated material encounters.”

Their last paragraph is the best bit though as it support one of the final paragraphs in my own thesis,

In museums in the wake of the pandemic – together with the sweeping call for change of the Black Lives Matter protests and debates about the decolonization of our physical surroundings – materiality is likely to remain in predicament. Understanding material encounters as part of a continuum inherently embraces reflexivity, ‘flux’, ‘in-betweenness’ and ‘liminality’ and is, therefore, fitting for these times.

The challenges of evaluating in-visit digitally enabled heritage interpretation

I am reading a paper which will help me better present my Chawton study. The paper is brand new: Nikolakopoulou, V. & Koutsabasis, P.. 2019. Methods and Practices for Assessing the User Experience of Interactive Systems for Cultural Heritage. In: Pavlidis, G., ed. Applying Innovative Technologies in Heritage Science. IGI Global, and it is another literature review, but more a sort of meta study of evaluations, called by the authors a “fortiori.”

The methodology of this literature search is exemplary. The authors searched (via Google scholar, ACM digital library, IEEEXplore, Science direct, and Springer link) and found 350 papers, and and in a second search another 150 that looked as though they might be evaluation studies of digital in-visit heritage content. They screened out the majority, for not actually being accessible, or not actually being scientific (I think my thesis would have fallen at this hurdle), leaving just 73. These were screened a second time and a number of duplicate were found and then a number which did not actually write up the evaluation. Leaving just 29 to study in depth.

In summary these papers discussed applications to”explore a virtual or physical space or place” and/or “play a game”. The authors noted the the proportion of games had gone down since a previous survey, which likely corresponds with the time of my change of focus away from a ludic system to something more to do with narrative. They were still roughly 20% of the papers they looked at. Overall, the systems used a variety of digital technologies including 3d game engines, mobile devices, mobile AR, VR, the web, multi-touch displays, location based audio, and physical and kinaesthetic (responding to body movement) interfaces.

There were a lot of studies that looked at usability of the technologies, but the paper points out only using “user satisfaction” as a metric, which is a dangerous trap to fall into. Only a few to a thorough investigation of the User Experience in that way that modern commercial companies test their websites. “A considerable number of systems are evaluated upon learning effects on users” this despite that fact that learning is often not the primary reason for days out (though it may be a validation for the museum), but again few do that in a properly scientific way. There is an interesting paper mentioned (Falvo, P. G. (2015, September). Exhibit design with multimedia and cognitive technologies impact assessment on Luca Giordano, Raphael and the Chapel of the Magi in Palazzo Medici Riccardi, Florence. In 2015 Digital Heritage (Vol. 2, pp. 525-532). IEEE.)) that I may want to take a look at.

The paper highlights the difference between empirical evaluations “in the lab” as it were and in the field. “Field evaluations are contextual, which enhanced the validity of the process and results.” But “recruiting visitors to experience or assess CH content on purpose changes their original purpose of visit, which is
something inherently connected with the visitor experience discussed in several museum and visitor studies […] which suggests a paradox to me, either test random users, and change their reason for visiting, and potentially their responses to the visit; or invite visitors especially for the test and thus make it more like a lab experiment. The reason for visiting a cultural heritage experience is actually part of the experience.

One issue they highlight is that despite being “peer-reviewed and published in important journals and
conferences, it was possible to identify several failures in the quality of reporting on empirical evaluations” in particular reporting on the number of users that participated in the evaluations. I expect that is because, due tho the limitations of evaluation in cultural heritage sites, the time taken to observe a visit, the number of participants is statistically small – small enough to put into question wether the studies are “empirical” at all I know that after a week on site I had less than ten samples. And so I tried very hard not to present what I learned as empirical evidence. Indeed maybe even too hard. That bit need sa re-write post viva.

The conclusion is a reiteration that digital interpretation evaluation should involve more cultural heritage professionals and field studies. It point out that there is more work in newer methodologies “like physical computing and tangible user interfaces” (objects that are the interfaceIt also highlights ongoing issues of a lack of systematic approaches to evaluation, organised “in distinct identifiable catagories” which I imagine would make doing meta-studies like this more meaningful. Also, “Aspects of cultural value, awareness, and presence could only be recognized in very few empirical evaluations. Evaluation studies that consider more qualitative dimensions and more related to general purposes of CH like learning effectiveness have been increased, and they are usually accompanied by comparative evaluations. Comparative evaluations represent a small number compared to the overall number of studies reviewed.”

Personalisation redux

My external examiner at my viva was Daniella Petrelli, an academic in the field of HCI (Human Computer Interfaces) who I had referenced a few time in my thesis particularly after discovering she was behind a platform to help curators to write the sort of content I had created for Chawton. I found that work too late, after completing the Chawton experiment. Among the “modest” changes that Daniella recommended in my viva is a considerable amount of further reading, including this paper, which to my shame I had not discovered in my literature search, and which would have saved me doing a whole lot of reading and improved my PhD! (Which is of course what a viva is for 🙂)

The paper (ARDISSONO, L., KUFIK, T. & PETRELLI, D.. 2012. Personalization in cultural heritage: the road travelled and the one ahead. User Modeling and UserAdapted Interaction 73 – 99.) Is an incredible useful survey and summary of personalisation techniques employed in cultural heritage up to 2012. I am pretty sure it came out of somebody else’s own PhD literature search. It is very biased of course towards computer enabled personalisation (because it comes out of that discipline) but it looks at 37 separate projects and charts a history of personalisation since the early 90’s. ” when some of the adaptive hypermedia systems looked at museum content (AlFresco (Stock et al., 1993), ILEX (Oberlander et al., 1998)) and tourism (AVANTI (Fink et al., 1998)) as possible application domains” (p7) These early experiments included “a human-machine dialogue on Italian art and combined natural language processing with a hypermedia system connected to a videodisc”, and “automatically generated hypertext pages with text and images taken from material harvested from existing catalogues and transcriptions of conversations with the curator”.

The authors chart the development of web–based interfaces that don’t rely on kiosks or laserdiscs, though WAP (Wireless Application Protocol – which delivered a very basic web service to “dumb” mobile phones) to multi-platform technologies that worked on computers and Personal Digital Assistants. They note two parallel streams of research – “Hypermedia and Virtual Reality threads” adapting the content to the user, and presenting overlays on maps etc. The appearance of PDA’s so personalisation becoming more content aware, with plug in GPS units, but the difficulty of tracking people indoors led to experiments in motion sensing, Petrelli herself was involved in Hyperaudio, wherein “standing for a long time in front of an exhibit indicated interest while leaving before the audio presentation was over was considered as displaying the opposite attitude” (I might need to dig that 2005 paper out, and 1999 paper on HIPS).

There is also an excellent section on the methodologies used for representing information, modelling the user, and matching users and content. When it talks about information for example, it suggests different Hypermedia methodologies, including:

  • “A simple list of objects representing the exhibition as “visit paths” (Kubadji (Bohnert et al., 2008));
  • Text descriptions and “bag of words” representations of the exhibits on display9 (Kubadji and PIL);
  • “Bag of concepts” representations generated by natural language processing techniques to support a concept-based item classification (CHAT (de Gemmis et al., 2008)); and
  • Collection-specific ontologies for the multi-classification of artworks, such as location and culture, and multi-faceted search (Delphi toolkit (Schmitz and Black, 2008))”

The paper also articulates the challenges to heritage institutions wanting to personalise their user experience, including a plethora of technologies and not standards yet reaching critical mass. Tracking users outside (before and after) their heritage experience is another challenge – membership organisations like the National Trust have a distinct advantage in this regard, but have spend most of the decade since this paper was written getting there. Of course heritage visits are made as part fo a group, more than by individuals, and personalisation by definition is about individuals – yet in most of the projects in this survey, the social aspect was not considered. The paper also acknowledges that most of these projects have involved screen based hypermedia while augmented reality and and physical interaction technologies have developed alongside.

Evaluation is a challenge too. In a section on evaluation which I only wish I had read before my project, the paper foreshadows the all the difficulties I encountered. But also says “a good personalization should go unnoticed by the user who becomes aware of it only when something goes wrong.” (p 25) It is reassuring too that the paper concludes “the real issue is to support realistic scenarios – real visitors and users, as individuals and groups in daily interactions with cultural heritage. It is time to collaborate more closely with cultural heritage researchers and institutions” (p27) which is (kind of) what I did. I had better quote that in my corrections and make it look as though I was inspired by this paper all along 🙂.

All our Fairytales: part 24

The original, and the best

Now its Christmas Eve, its time for the recording that started it all. In honour of the occasion I sit here typing this wearing an officially licensed Fairytale of New York tee-shirt, with an image grabbed from the video of Shane and Kirsty dancing in fake snow, while my original seven inch, 45RPM single spins on the turntable.

Yes I bought the single then. I am sure I didn’t time the purchase well for the actual Christmas run because I was desperate for new music from the band and still awaiting their album If I Should Fall From Grace With God. But I remember watching its climb up the charts with glee.

That year Rick Astley had released When I Fall in Love and in early December many thought he was going to get the top spot. But then Nat ‘King’ Cole’s version of the song was released, and I recall watching an interview n which he said he would be happy to come second if that beat him to the top spot. But, in the end he came third or fourth with the Pogues taking second place after a late entry number one from the Petshop Boys and Always on my Mind. Should it have been number one? It wasn’t and that’s all there is to say on the matter. It has never taken the top spot on any reissue following, but it is according to some pollsters Britain’s favourite Christmas song.

Is it my favourite Pogues song? It might not even be in my top ten, and I slightly resent that its the world’s favourite Pogues song. If you check Apple Music now, it occupies six of the top ten Pogues downloads/streams. If its the only one you know, let me offer you some you should listen to. Songs which I think are possibly even better than Fairytale.

  1. My personal number one is not to everyone’s taste, but its the lullaby I sung to my daughter when she was a baby. Its The Old Main Drag the life of a rent boy on the streets of London
  2. From the EP, Poguetry in Motion, I have a couple of favourites, and A Rainy Night in Soho is equal to or better than Fairytale.
  3. The Pogues did duets before Fairytale too. Haunted was one of my most played singles at the time, and I was gutted when I lost it after taking it to college to digitise for an art project. It’s hard to find because it was recorded for the soundtrack of Sid and Nancy: Love Kills and is “owned” by MCA. Shane re-recorded it with Sinead O’Connor, but I have a soft spot for Cait O’Riordan.
  4. There were lyrics controversies before Fairytale too. The Boys from the County Hell was the song that encapsulated the spirit of the early Pogues and I treasured a tee-shirt with the lyric “Lend me ten pounds and I’ll buy you a drink.” Not that that was controversial. But the line in the recorded versions “My brother earned his medals at Mei Lei in Vietnam” replaced the live version “My brother earned his medals fucking gooks in Vietnam.”
  5. Of course only one Pogues song has actually been banned, and that is Streets of Sorrow/Birmingham Six. This medley was banned under the strange law that banned recordings of IRA members, or suspected members like Gerry Adams. It was designed to stifle Adams’ voice on news programmes in particular.
  6. I realise that a lot of my favourites are from their first album Red Roses for Me. I love Transmetropolitan, for example, which sounds like a tube train rattling though the tunnels.
  7. Sea Shanty, from the same album, like Transmetropolitan, The Old Main Drag, and A Rainy Night in Soho gets to the heart of why I love The Pogues, here creating a new folk tradition of ordinary life, love and loss in London.
  8. Steams of Whiskey, again from Red Roses for Me shows us Shane’s future, and has an ace original video, made on a budget of a fiver, it seems.
  9. The Body of an American, is iconic partly because it featured semi-regularly in seminal US Police and Politics show, The Wire, every time they held a wake for a dead copper.
  10. Sally MacLennan edges The Sickbed of Cuchulainn out of my personal top ten, but that just proves my point. There are at least eleven Pogues songs I like more than Fairytale, even though I think Fairytale is very very good indeed.

Merry Christmas, everyone!

PS. A quick mention for Chris Thile, who obviously heard about this blog and having worked out that I wasn’t going to include the version from his PBS show Live From Here, bashed out another version yesterday with properly Irish sounding (but actually American) Aoife O’Donovan. I enjoyed his work with Nickel Creek, indeed theirs was one of the last CDs I bought before iTunes and then Apple Music killed that habit, and his mandolin playing is a wonder, but he avoids the controversial lyrics question by simply cutting the whole verse, which I think is a worse sin than changing or fluffing the words.