Category: Personalisation

Personalising the heritage visit

Matthew Tyler-Jones1 Comment

One the things that my external examiner pointed out during my viva is that I had not put in enough about personalisation. A number of the articles that she recommended I look at for my corrections address that issue and this is one. Not, Elena & Petrelli, Daniela. 2018. Blending customisation, context-awareness and adaptivity for personalised tangible interaction in cultural heritage. International Journal of Human-Computer Studies 114: 3-19. doi: 10.1016/j.ijhcs.2018.01.001.

I think one of the reasons why I had somewhat skimmed over the subject in my thesis is, coming out of a career based on live interpretation, which is essentially a conversation, personalisation is built into the way I think about heritage. My work in the PhD might be summed up as trying to ensure that conversation takes place even when there isn’t an excellent live interpreter working there. Of course, my examiner, Daniella is coming at the same aim, but from a different direction, and so she does not take personalisation for granted as I am wont to do. This paper however is a useful resource looking at it from the HCI point of view and I think there will be a number of quote from it and a summary among my corrections. But which quotes? What is summarised? As you might already have guessed these posts are a first pass at answering those questions. A Bit of practise as it were before creating a restructured version of my thesis. So, the paper starts off asking what personalisation actually means in this context:

“‘Personalisation’ is a broad term that encompasses three types ofsystem behaviour: adaptability (also called customisation, the term we use hereafter) offers users a number of options to set up the application/system the way they like it; context-awareness is the ability of the system to sense the current state of the environment and to respond accordingly; adaptivity implies the system maintains a dynamic model of the on-going interaction and dynamically changes its own behaviour to adapt to the changing situation.”

I can see why they chose to call adaptability customisation, adaptivity sounds far too similar and might be confusing. Of course one factor we have to consider is that although people do make heritage visits on their own, the vast majority are as part of a group; self-organised like couples, families, or friendship groups; or organised groups such as school visits and coach tours. As the authors point out “Research that directly addresses the social dimension is still limited” but they point the way to studies that look at conversation around a context aware-table, and sharing tables around a group, among others. However it is important to include this social dimension in any consideration of personalisation, which is something I did at Chawton – the choice there were made by the visiting “group” even if, sometimes that ‘group’ consisted of one person.

The paper of course starts with some case studies of similar work, including the Italian trenches soundscape I looked at a couple of days back. In another project at the same museum uses a “pebble” with NFC capability that activates media when places in certain places around the museum. When the visitor leaves, the pebble’s journey is read and a personalised postcards printed for the user to take home. In the Hague, a similarly NFC enabled system has the user place replica objects in “an interactive ring” which plays media from a choice of three different viewpoints (two military and one civilian). A third project, The Loupe, uses a phone disguised as a magnifying glass to present AR media. My problem with all these is an HCI one, two these systems force the users to learn a new interface, placing the pebble or replica in a certain spot to activate media that seems unintuitive, especially in environments where conditioned behaviour often precludes touch, picking things up, or even putting things on museum surfaces. On the other hand, the authors do make a point later that “A synergy can be created with tangible and embodied interactions to increase visitors’ awareness they are building their own visit path.” And I must admit that when the personalisation is invisible, the visitors do not perceive it. However my evaluation of Ghosts in the Garden suggest that even when tangible interactions are involved, the visitors may still not be aware that their experience was personalised.

But leaving my issues aside. There is some really good overview stuff in this – including a table that summarises some of the factors to consider when personalising interpretation. This includes: “stable” visitor factors – like age and disability, interests and Learning preferences; factors related to the current visit – motivations, fatigue, visit history and available time etc.; the type of tracking – two in this table, proximity tracking and interaction with objects; the location – indoor or outdoors, layout, noise etc.; and the content – the media, the story.

The team brought together 25 participants in a co-design workshop (curators , computer scientists and engineers) and they came up with a classification of features by the type of personalisation they support. The first group includes features that depend on content and are activated by “customisation preparation”: is this about on-site visits or virtual visits? Is it indoors or outdoors? What are constraints – is there power and wifi? The next set is decided by the curator or interpretation staff, Most of these come under “customisation preparation” too: what is the heritage topic?; The media type?; the genre of the text?; The thematic threads?; the supported visitor profiles? the type of group? Then what is the structure of the narrative, for example a story or a Q&A? Finally what is the structure of the visit? For example, is it guided, free exploration, or a treasure hunt?

One curatorial decision that falls into context awareness is does the interaction involve augmented objects, an if so wha are input and output abilities of those objects? were I the curator on this project I would look for forms of content awareness which do not rely on objects, even though some I have written about elsewhere are fun. But that move me into a set of context awareness features that are modelled by the system itself (according to the authors): user location, proximity to exhibits, proximity to other users, and the current state of the exhibits. To give the experience the adaptivity it needs, the he system will also use data about the shorty of the individual interactions with the space/objects and delivered content – just as in my Chawton experiment the system selected content based on what had been shared with the user before.

Finally comes the customisation choices, chosen, or course by the visitor and based on their motivation and expectations. They might have been given the opportunity to express interest in topics and narrative threads, as I did (somewhat clumsily) with my Clandon prototype. And, as at Chawton, the expected a duration of the visit is a factor (though I suggest it is less an active choice of the visitor, and better modelled by the system). Of course another factor that is totally out of the control of anyone other than the visitor is what the visitor thinks the type of visit is – they might be coming for an emotional reason, or social, or fun or for learning.

The paper concludes that “fully automatic adaptivity, where the system takes all the decisions on what to present to which visitor, when and how, may not be the best solution” and argues that therefore what curators (or interpreters) value as most meaningful should be the driver of of the personalisation model. I agree, but with the proviso that if the intent is top emotional engage the visitor, many heritage stories don’t do the job well enough. The authors say “This requires a radical rethinking of how personalisation in cultural heritage manifests itself and the role curators and visitors play” and I think think that my thesis might contribute to that rethinking.

Back, for a while at least

Matthew Tyler-Jones1 Comment

Woah! It’s been over a year since I last post anything on here. I return because later today is my viva voce, wherein I “defend my thesis.” Yes, that’s why I have been silent on here, I have been hammering my thesis into a shape that I am not ashamed of, reading it, proofing it, getting others to read it and proof it (and yet, yes, there are still typos) and submitting.

For the last couple of weeks, in preparation for tomorrow, I have been re-reading the thesis, meeting (virtually) with my supervisor and recruiting an old friend and academic Jane Malcolm-Davies to read it and to give me a practice viva, last week. Reading it is hard though – after a year of writing and re-writing, the words flow through me like a purge barely touching the sides. And all I take note of are the things I might have done better. The practice viva was a very useful experience though, and so I thought I might give myself another “practice” by writing this post.

What brought me back to the blog was recognising a sentence that had survived from one of my earliest posts, though eight years of other work and re-writes and edits. This blog was a useful way of processing my reading and other research as I was going it. It wasn’t what I needed though when I was rewriting the words that might have started on this site. And as this is the last chance to read my own thesis for the viva, I thought I might process it with a blog post in a similar way, hopefully anticipating some of the questions

Who is my audience?

One question Jane asked me about the thesis was “who are you writing this for?” And it’s a good question. It gave me pause to think. But I think the answer is I writ it for myself, and for people like me, heritage professionals who want to explore what digital interventions might be like, and who don’t want to be reliant on digital suppliers to tell heritage organisations what they need.

When I started this, eight years ago, I came into it with a truly open mind. My question really was “what can I and my fellow heritage professionals learn from games about storytelling in space?” I had no agenda, no thesis I wanted to test and, importantly no real experience in computer games. I guess I saw computer games as competition for my sector. Not financially speaking really (we are no completion for that global behemoth of games production); but for share of mind or… actually share of mind. I often tell the anecdote of a colleague who came to me saying that a game had made him cry the night before. At the time our organisation were trying to improve “emotional impact”, and as someone who had spurned computer games in my teens because they didn’t engage my emotions. I was intrigued, at the time my organisation was very interested it what we called emotional impact. If computer games could make my friend and colleague cry, perhaps we could learn something from them about storytelling.

So I didn’t apply for the PhD with any sort of hypothesis in mind. This was possibly a mistake, but I genuinely wanted to work out how games (and I should say “open world games”) that let the player wonder around an environment, manipulated emotion.

How did my research change?

When I started out, if I had any defined ambition at all, it might have been to create an application that tuned a heritage visit into some sort of investigative game. And I think I was definitely imagining the mobile device as the conduit of communication between place and person. Two bits of my research changed that ambition, neither of which fit into the main narrative of the thesis but were I feel worthy of including in an appendix, if only to explain why that wasn’t a route I was following.

What was a surprise to me, was during the “reading” of the three games I played, the realisation of just how poor heritage storytelling is. Note I say storytelling here specifically. It can be argued that “heritage interpretation” isn’t “storytelling,” but such an argument is broadly incompatible with a desire to engage visitors emotionally. This became research in practice with two aims: the first to see if heritage interpretation narratives could be broken down into natoms (narrative atoms), stored and reassembled to satisfy visitor’s needs; and secondly to explore how they might be put together in an emotionally engaging way.

What would I have done differently?

With the benefit of hindsight, of course I would have concentrated on this narrative conundrum from the beginning, but even arriving at the conundrum was, for me, a damascene revelation. As a professional interpreter who has been confidently telling stories in the sector for decades, realising that I might have been telling the “wrong” stories.

But practical things I would have done differently – it was only in the latter stages that I realised what useful tool Robin Laws’s story beats and transition analysis was. I wish I had found it earlier and used it to analyse at least parts of the three computer games I read.

Speaking of things I wish I had found earlier. When I was scouting around in the early stages for a “narrative database” I could easily use, the meSch project, which is written specifically for curators and heritage professionals, was only just starting. In the end I used Scalar which is written for academics, but not for the heritage environment. But had meSch been around it would have been a perfect platform to experiment with

One other thing that might fit as an answer to this question is, had I switched to my current supervision team earlier in my research I could easily have been persuaded to drop the narrative track entirely and focus instead on music and sound – where there is simply not enough research on its use in the heritage environment.

Challenges

Focussing on the academic challenges, the subject of cultural heritage interpretation overlaps so many academic disciplines that even producing a literature review was really hard. And then, pushing the work forward, I felt all the time I was falling between two stools – science and humanities. For example, exploring affect outside of the psychology, and databases outside of computing, but at the same time wanting to write something that could be understood, and taken forward, by both museum professionals and data engineers.

Where do I take this?

In the conclusion I detail a number of potential paths for further research. The one that I am most interested in professionally if further understanding, and potentially measuring, numen. I think this concept is closer to what museums and heritage sites really mean, when they talk about emotional engagement. But defining in properly and working out ways to measure it, will take a lot of work.

I am also excited by meSch – a tool written for professionals just like me. And I would love to take it for a spin on a future project, creating adaptive, personalised stories for visitors. But not just “visitors.”

I say that because my research was based very much around making the experience of people in places better, talking about responsive environments better meeting visitors’ needs. I deliberately eschewed virtual experiences, except when I first prototyped an adaptive story for the then closed Clandon Park. But in the last few months visiting places physically has been restricted. And its made me realise that the paradigm shift in story telling that I have advised for responsive environments might just as easily be applied to virtual visits. Despite being only a prototype, the Clandon experience was more satisfying for me as a creator, for reasons I explain in chapter 4. Perhaps there is be a chance, a need even, to create new personalised adaptive event numerous stories on-line.

Well that’s all the time I have. My via starts in 30 seconds

wish me luck

Personalised

Matthew Tyler-JonesLeave a comment

I am in Amsterdam, staying in a VERY EXPENSIVE HOTEL, because all the hotels in Amsterdam are VERY EXPENSIVE. We are here on a cultural city break, to take the kids to Anne Frank’s house, and the Van Gogh museum, among others. We pre-booked for those two because we had heard it was difficult to get tickets on the day. Yesterday, as we were packing to leave, I got an email from the Van Gogh museum. It contained a link to a “welcome video”. And I would like to share a few frames from that, you’ll see why.

The first impressive thing that happens, is a member of museum staff, shuffling sideways onto the screen carrying a placard, like a taxi driver at the airport, with my name on. This has never actually happened for me at an airport, but here it is happening, through digital jiggery-pokery, on an actual video. The email had mentioned it was my personal welcome video, but I hadn’t realised how personal.

Then, was the staff take their branded coats off, and hand then in at the cloakroom (I would have thought they had their own facilities, but there you go) the screen displays the weather forecast on the day of my visit. There is also a none-too-subtle hint that I should become a friend of the museum, which purports to show an interpretive wall vinyl:

This is the first time that I have seen digital technology actually used with by a museum for true specific personalisation. Ok, so it’s only my name, and weather data for a specific date, and actually it’s not that much more complex than those picture books that were advertised twenty years ago with “Your Child’s Name Here”. But it is being used to every pre-booked visitor, which at this time of year means pretty much every visitor. How long will it be before interpretation panels like the one in the last picture are actually personalised?

Building a story in Star Wars Indentities

Matthew Tyler-JonesLeave a comment

A Stormtrooper marching band? The exhibition attempts to illustrate different values with illustrations like this.

It was Fathers’ day last weekend, as a treat, my family took me to the Star Wars Identities exhibition at the O2 in Greenwich. I was interested for a number of reasons, not the least of which was, being ten in 1977, I was (am) a massive fan of Star Wars. But one of the other reasons was the idea that visitors build their Star Wars identity as they go around the exhibition. This seemed to me to be a large scale, upfront attempt to personalize a cultural heritage visit. (And yes, Star Wars is cultural heritage now, I’m sure I saw other movies when I was ten, but I can’t recall what they were.)

The RFID tag that visitors use for the interactives

The mechanics of this personalization were wristbands, or if you were latex intolerant, “credit” cards, with RFID (I’m guessing) chips, and nine (not ten as advertised) stations around the exhibit where you could make choices that defined your Star Wars identity. The content of the show were props, models, costumes, concept sketches and some original art, mostly from the first six movies (though BB8 and a couple of props were squeezed in to represent the latest phase of production), with two streams of interpretation. One stream interprets the design of characters in the movies, and the second is a sort of “science of Star Wars” strand, with basic interpretation of things like genetics. Some of all this is delivered with traditional text panels, and some is aural, delivered by an IR activated headset. You are given a “medallion” style unit to hang around your neck and hook an earpiece over one ear. Then, stand in the right place in front of a panel or AV, and the relevant sound is beamed to your unit in a choice of languages. All you need to do, is control the volume… and make sure you don’t turn away from the beam, or cross your arms over the receptor, or let anybody tall stand in front of you, cutting of the beam – all of which will cut out the sound.

It’s worth  pointing out that the text comes in English and French. And that may betray the exhibition’s 2012 origins at the Montreal Science Centre. That of course explains where the science interpretation strand comes from, and why objects and stories from The Force Awakens feel shoe-horned in. One can’t argue with most of the interpretation – seeing how the characters like Yoda developed over time was interesting, the science was a bit basic, and its connections with the Star Wars story questionable (the exhibition suggests Force sensitivity is a genetic trait). The stuff on personality felt just one of many different models of personality types that, despite five post-Doc academics advising on it, reads like its been cribbed from a dodgy self-help book. Interestingly, it was the personality test that was the only interactive station that wasn’t a simple choice – visitors had to answer a number of questions before it revealed their personality profile.

When I started the experience, I was looking forward to discovering what my Star Wars identity would be, but three or four interactives in, when I realized that most of the stations were offering choices rather than revelations, I decided to rush back to the start and remake those choices – because I’ve known since I was ten what my Star Wars identity actually was: the son of Grand Moff Tarkin! Given that the character described in the link was mostly made of my direct choices, I am of course very pleased with the result. I was curious to see what my personality test said about me (or rather, about my Star Wars identity). Click on each of the buttons below the biography on the page I linked to, and it highlights the bit text in the biography that was chosen by your answers. So to reveal the personality results, all I need to do is click that button. The highlighted text says:

People often tell me I’m a generally adventurous and curious person, I also tend to be energetic and social.

… which suggests I was really getting into character when I was answering those questions, because that doesn’t sound like me at all 🙂

Actually the interactive I enjoyed most was Events. Touch your RFID bracelet to the receptor and a random life event spins into view, with a choice of how you react to it. I won a city in a “game of chance”, and had to decide (if I recall correctly) whether I governed sensibly, gave up the job, or “reveled in the prestige and borrowed liberally from the city coffers” which is the option I went for (of course). But my boy was disappointed that the random events on offer were not in some way defined by choices you’d already made – he too got the city, and I might easily have been “freed from slavery” by an event. The son of a Grand Moff in slavery? I don’t think so. 🙂

Despite its limitations (which one is prepared to forgive more when one realizes the technology is five years old), the opportunity to create a story like this was very much enjoyed by my family. I wonder if the exhibition had a deeper emotional impact on me because of it?

Abstract: Digital Personalisation for Heritage Consumers

Matthew Tyler-Jones2 Comments

I’m speaking at the upcoming Academy of Marketing E-Marketing SIG Symposium: ‘Exploring the digital customer experience:  Smart devices, automation and augmentation’ on May 23 2017. This is what I wrote for my abstract:

Relevance to Call: Provocation, Smart Devices. Augmentation of the Customer Experience

Objective: A work-in-progress research development project at Chawton House explores narrative structure, extending the concept of story Kernels and Satellites to imagine the cultural heritage site as a collection of narrative atoms, or Natoms, both physical (spaces, collection) and ephemeral (text, video, music etc.). Can we use story-gaming techniques and digital mobile technology to help physical and ephemeral natoms interact in a way that escapes the confines of the device’s screen?

Overview: This provocation reviews the place of mobile and location technologies in the heritage market. Digital technology and social media are in the process of transforming the way that the days out market is attracted to cultural heritage places. But on site, the transformation is yet to start. New digital interventions in the heritage product have not caught on with the majority of heritage consumers. The presentation will survey the current state of digital heritage interpretation and especially the use location-aware technologies such as Bluetooth LE, NFC, or GPS. Most such systems deliver interpretation media to the device itself, over the air or via a prior app download. We explore some of the barriers to the use of mobile devices in the heritage visit – the reluctance to download proprietary apps, mobile signal and wifi complexities and most importantly, the “presence antithesis” the danger that the screen of the device becomes a window that confines and limits the user’s sensation of being in the place and among the objects that they have come to see. Also, while attempts to harness mobile technology in the heritage visit display interpretation that is both more relevant, and in some cases more personalised to the needs of the user, they also tend towards a “narrative paradox” – the more the media is tailored to the movements of the user around the site, the less coherent and engaging the narrative becomes.

Method: Story-games can show us how to create an experience that balances interactivity and engaging story, giving the user complete freedom of movement around the site while delivering the kernels of the narrative in an emotionally engaging order. At Chawton we plan to “wizard of oz” an adaptive narrative narrative for that place’s visitors.

Findings: Work so far demonstrates that a primary challenge for an automated system will be negotiating the contended needs of different groups and individuals within the same space. The work at Chawton looks to address this.

**

This is the first time I’ve written an abstract in this format, and I found it quite a challenge. What you add in and leave out is always a difficult decision, and this format, which was limited to one side, had me opting to leave out the references which I might have made room for if I had not had to write something under each of the prescribed headings. It’s also the first time I have had formal feedback on an abstract, which I share below:

Relevance to call: Good fit Smart devices, user experience,
augmentation, culture (5)
Objective: A practical case example of augmentation in a
heritage setting (5)
Lit rev: No indication of theory used, as this is a practical
case study (n/a)
Method: A specific case of Chawton House presented. (5)
Results: Interesting findings re barriers to use of mobile
devices in heritage, and the experience evaluation (4)
Generalisations: Interesting and original context of heritage
institution using augmentation, can extend to
other heritage sector applications. (4)
Total 23/25

**

So, not a bad score, but I wonder what I would have got (out of 30?) if I had included the references. Does the bibliography count within the one page limit? Or, could I have included it on a second side?

Still, not time for those questions. I have the write the actual presentation now. 🙂

The powers of people

Matthew Tyler-JonesLeave a comment

I was at Chawton again yesterday (before going to Petworth for yesterday’s mobile fun) to meet with Jane, one of the house’s most experienced volunteers. I’d challenged her to give me a 45 minute tour of her choice. She really wanted me to tell her what I was most interested to hear, but I wouldn’t. I wanted her unbiased perception of what were the most “important” bits of her her encyclopedic knowledge of Chawton and the surrounding area to share, given the 45 minute time limit.

(I always recommend that 45 minutes is the absolute maximum for a guided tour. In fact I suggest that half an hour is what people should work to. People who want more will stay behind to chat, but there is some evidence from the National Trust’s monitoring of visitors for conservation, that the average dwell time in a house, whatever it’s size is about 45 minutes.)

In the end she gave me what I’d call an architectural tour of the house, pointing out how the thick exterior walls of the the original manor had become interior walls after Richard Knight’s extensions. It was great, and reminded me about some of the things I’d forgotten about being a tour guide that make guided tours (with the right guide) so entertaining.

I’ve always said that guided tours often offer the best historic house experience. A good paid or volunteer guide can weave a compelling story as s/he escorts you around the house. He or she can reveal things you might otherwise have have missed. They can respond to your interests, and level of expertise, to give you a tailored experience. But Jane reminded me how they can transform the place, by pointing out those thick walls, or turning over a framed note hanging on the wall to reveal the ancient deeds from which the paper had been recycled. A good guide turns their audience into detectives – rather than simply telling them how Montague Knight installed a safe into what once had been an old garderobe chute, they help their audience work it out for themselves – a moment of insight, that emotional trigger where everything that has come before “clicks into place and reveals the shape of the whole” as Tynan Sylvester puts it.

Of course, Jane’s tour also demonstrated that the VERY best historic house experience would be to have the guide all to yourself. Not everyone on a larger tour (and there were a couple running yesterday that we bumped into) could have lifted the framed note from the wall to read the reverse. As I hung it back on the hook, I had conservation alarm bells ringing in my head. Every handling, every movement of this glass framed note (which Montague Knight had hidden beneath the floor for future generations to find) put it at risk. The more people given the opportunity I had, the greater the chance that it might be damaged.

Not everyone can do what I did, arrange a personal tour at a time of my convenience after an email introduction from the Director. For those other tour groups we met, the guided tour experience gets diluted, less personal, less tailored to each individual’s interests.

The technological approach I’m investigating might be able to address some of the personalisation challenges, but can it ever offer the magical moments of insight that Jane offered me?

What PhD supervisors are for

Matthew Tyler-Jones1 Comment

I had a great chat with my supervisor on Thursday, after helping out with a Masters seminar. As regular readers may have worked out, I’ve been having a great deal of trouble trying to get a coherent testable design to test out of my half-formed ideas and lofty ideals.

The problem was trying to think of a cheap way to test some of the theory I’ve come up with. I’d got hung up on trying to think of a way to track visitors round a site and test their reactions to that. Until I solved that I was handwaving the issues of breaking the story into natoms, and balancing the conflicting needs of multiple visits in the same space. Those two problems both felt more within my comfort zone. The problem is that I’m not a technologist, that bit is so far out of my comfort zone that I’d need to enlist (or pay for) one. On top of that, the tech itself isn’t that cheap – getting a wifi network into some of the heritage places I know, with their thick stone walls and sheer scale, isn’t about buying just one wifi router.

I’d mentioned the other problems (particularly in the one of negotiating conflicting needs) in the seminar. (The students had been reading about a variety of museum interpretation experiments for their “homework” and we discussed the common issue that many of the experiments focussed on the issue of a visitor in isolation, and hadn’t thought enough about multiple users in the same space). Afterwards I spent twenty minutes with Graeme, my supervisor, in his office. I felt he’d finally got what I’d been trying to say about a “responsive” environment, and his interest was particularly focused on the two issues I’d handwaved. We talked about low-tech ways or exploring both of those, and of course THAT’S what I should be doing, not worrying about the tech. These are both things I can do (I think!) rather than something I can’t .

So by the end of our chat, when Graeme had to return to his students we’d worked out the rudiments of a simple experiment.

  • What I need is a relatively small heritage site, but the possibility of lots of choices about routes, lots of intersections between spaces. What Hiller calls a low depth configuration (that last link is to a fancy new on-line edition of the book, by the way. It’s worth a read).
  • I need to work with the experts/curators of that site to “break” the stories. Break is a script-writing term, but it feels particularly appropriate when thinking about cutting the stories up into the smallest possible narrative atoms. (Although maybe “natomise” is better!)
  • Then I need to set up the site to simulate some of responsiveness that a more complex system might offer. Concealed Bluetooth speakers for example, or  switches like these that can be controlled by Bluetooth.
  • Finally, rather than try and create the digital system that tracks visitors and serves them ephemeral natoms, I can do a limited experiment with two or more humans following visitors around and remotely throwing the switches that might light particular areas of the room, play sounds or what ever other interventions we can come up with. The humans take the place of the server, and when they come together, negotiate which of their visitors gets the priority. Graeme suggested a system of tokens that the human followers could show each other – but the beauty of this concept is that the methods of negotiating could become part of the results of the experiment! The key thing is to explain to the participants that the person following them around isn’t giving them a guided tour, they can ask questions of him/her, but s/he isn’t going to lead their experience.

So, now I have a a thing that it is possible to do, with minimal help and with a minimal budget. And its a thing that I can clearly see has aims that come of the research I’ve done, and results that inform platonic ideal responsive environment I have in my head. If it works, it will hopefully inspire someone else to think about automating it.

That’s what supervisors are for!

 

@HeritageJam 2015 diary 3 – Progress

Matthew Tyler-Jones1 Comment

Its been 14 days since Heritage Jam 2015 was launched, half-way, and we’ve seen definite progress on one of the projects I’m involved with. That progress is more about the scope of the project rather than actual technical achievement (though there has been some of that too). We have now set a modest (but still stretching, especially for my limited coding skills) goal for our on-line participation, of having a basic proof of concept prototype ready to try out at the in-person Jam at the end of the month.

Our ultimate ambition is to create an open source, relatively cheap, off-line system to track visitors’ around cultural heritage sites, as mentioned (with a brief discussion of the ethics) in my last post on the subject.

But our goal for the on-line element of the Jam is to get a Rasberry Pi set up, with a wi-fi dongle capable of being put into monitor mode, a Real Time Clock and with DumpCap installed, working as a sniffer – detecting mobile devices, and using Relative Signal Strength Indicator as a proxy for proximity. Then, after testing it in my house, creating some R code to analyse the data, pulling out the useful bits, and maybe using GGobi  to visualize them in some way.

Assuming we can achieve that I’ll take the kit to the in-person Jam at York and see if I can find a place to set it up in a real world experiment, maybe with an activity that will involve encouraging users to go in and out of the monitored space, to see if we can log the same MAC addresses approaching and withdrawing from the RPi.

What’s not in scope, but the obvious next stage, would be to have two RPis doing the same thing in the same space with synchronised clocks, and then trying to match the records for each MAC address, but lets not overstretch ourselves.

Anybody else jamming on something similar?

Right now, I’m off to University, to take a couple of library books back and (hopefully) meet up with a team-mate from my other on-line Jamming team, to discuss that project, which has not made similar progress… or even started really.

The Invisible Hand Revealed!

Matthew Tyler-Jones1 Comment

Back in the days of yore (last year) I attended a workshop run by the digital art collective, Blast Theory. This week they released their report, which fills in all the gaps from my two posts worth of notes taken at the event. Seriously, they took better notes that I did – the document even reminds me of stuff I said. (Playing devils advocate, I’d asked why profiling should be considered a bad thing, if it meant people get better service from corporations – and I get the sort of jeans that make me look good. And its all in the report, though some of the typos suggest they used some sort of on-line transcription service.)

Of course, over the last couple of weeks, as I’ve been further developing the responsive interpretation experiment I’d like to do, I’ve realized that it is in its own small way, all about visitor profiling. And surveillance of course. When the National Trust were thinking of funding the experiment, I was considering trying it out at Grey’s Court, which as the childhood home of Ian Fleming, seems particularly appropriate for exploring the the idea of spying. I hadn’t got so far then as to explore what the story might be, but as I’ve developed the idea for other funding proposals I’ve thought more and more about making the surveillance aspect of the experiment central to the story too.

Part of the reason I want to run the experiment is to test how visitors feel about having their movements around the site monitored. I’ve been thinking how the revelation of that monitoring packs quite an emotional punch, and suggests a story, or at least a “B-plot” around a mystery wherein the players are cast as spies, or spycatchers, investigating the site, and where the final kernel is the revelation that as the watchmen, they are actually the watched. This has been a necessity of trying to create a generic experiment, which can run at any site that the funding requires, but I like the idea of using it to create a James Bond theme at Greys Court, or finding the spy stories in other cultural heritage sites… like Sutton House’s Wolf Hall connections.

The infuriating thing about this search for funding, is the amount I need isn’t actually that much. Well, its more than a little of course, otherwise I’d fund it myself. But at about £14,000 it’s a bit-player in the world of funding. I can scale it up of course, to get closer the sort of money being offered by many funders, but that seems disingenuous in a way.

The Invisible Hand workshop was inspired by Blast Theory’s research for Karen, “the app that psychologically profiles you as you play.” They went to Kickstarter to bring in the last £15k they’d needed. But that was for an already prototyped concept, and they only just hot their target so, what with Kickstarter’s take, the need for rewards for backers, and the reputation which I haven’t got, I don’t think that’s a method I could use…

The Invisible Hand – part 2

Matthew Tyler-Jones2 Comments

I promised more on the Blast Theory workshop I attended a couple of weeks back, and here it is. The two days were kicked off by Matt Adams of Blast Theory explaining why they’d titled the event with a term the Adam Smith had coined a couple of hundred years ago. Smith of course wrote about the Invisible Hand as a good thing, turning selfish acts into selfless ones:

By preferring the support of domestic to that of foreign industry, he intends only his own security; and by directing that industry in such a manner as its produce may be of the greatest value, he intends only his own gain, and he is in this, as in many other cases, led by an invisible hand to promote an end which was no part of his intention. Nor is it always the worse for the society that it was not part of it. By pursuing his own interest he frequently promotes that of the society more effectually than when he really intends to promote it.

Adam Smith The Wealth of Nations Book IV, chapter II, paragraph IX

However the Invisible Hand that we examined during the workshop had the power to be good, a more sinister. We were looking an profiling and personalisation. Matt started with off with an example of profiling. Mosaic is Experian’s database of UK households and organisations use to identify markets. My own day-job at the National Trust uses Mosaic, but you don’t have to be an organisation to access at least some of the data, their ForSite app is available for free  for your iOS or Android device. At the National Trust, we know that a number of the sixty segments into which Mosaic divides the population are more likely to become members than others, so certain postcodes might, for example, respond more positively to a fundraising appeal. Of course its a pretty broad brush, and describes a population by the people they live among, not as individuals. Its all about inference, not fact.

But technology enables organisations with the inclination to collect more and more data on individuals, not just addresses, and modern manufacturing processes allows a degree of mass-customization that moves the world of profiling into one of personalization. Matt spoke of the chain Zara, the high street face of Spanish company Inditex where staff collect feedback every day from customers, and send it to HQ, where designers collate and respond to what customers are looking for within weeks. Apparently only 3 or 4 units of each design are shipped to branches, and using the daily feedback from stores and a  2-3 week production process, the entire stock of a shop will change every 11 days.

Matt’s question to us all was, what does this mean for the arts?

Of course personalized art has been around for decades. Matt cited  Allan Kaprow’s 1959 work, 18 happenings in 6 parts; the neo-futurists’ ongoing Too Much Light Makes the Baby Go Blind (which is not just personalized but also randomized – a roll of a dice defines your ticket price); and Lev Manovitch’s Soft Cinema as examples.

Blast Theory themselves are not alone in the contemporary application of technology to create art: Invisible Flock; Non Zero One; Coney; and Urban Angel all create worlds that are part theatre, part game experiences that create unique experiences for each audience member. Many of these experiences collect data on their participants, is this a good thing or an abuse of privilege?

In the discussion, concern was expressed about the power of profiling and personalization. I found myself playing devils advocate. Yes, profiling is a blunt tool, and can be used to exercise power unfairly upon minorities, but personalization, if its done right, is a dialogue. Some years ago, when I was a young man, I liked Levi’s 620 jeans. Then, with some retro rock and roll, Levis introduced 501s and 620s disappeared from the shops. I was gutted, I’m not a handsome man, but I’ve got a pretty fine set of pins, which 620s showed off at their best (or did at least, when I was slimmer), tight in all the right places (around the calf I meant!!). 501s were baggy, which might have been fashionable, but what I would have given for the manufacturing process that dangles now so temptingly in front of us now, where those few of us that were not persuaded to convert to 501s might still buy 620s, even better than they were before because they’re made to measure.

Of course the debate turned to privacy, such a vital topic in these post-Snowden times, but even here, despite my abhorrence of the state spying on its democratic masters, I found myself an apologist for Big Brother. Is “Privacy” actually an aberration, a blip in the long history of society, brought about only a few generations ago by scraping together enough coin to live in the luxury of separate bedrooms? For most of human history we’ve lived with a different concept of privacy, where communities knew pretty much everything that when on, and privilege bought some people only a reduced number of people to share the toilet with.

I think we concluded that the debate was less about privacy than power. We discussed the politics of knowledge, touching upon Marilyn Strathern’s simple hierarchy of data, information and knowledge. Data being the unprocessed stuff that we see (or sense by any other means); information being that data organised in some way so we can begin to see some sense in it; and, knowledge is information that has an effect, that makes us change our behaviors in some way. The fear is that data is being harvested  by large, wealthy organisations, in such huge quantities and the “common man” or woman can not make sense of it. So we let these organisations organize it into information, and offer us knowledge that manipulates our behaviours to the organisations’ benefit. A new Invisible Hand that isn’t working for the benefit of society as a whole.

Kelly Page offered us examples. Doubleclick was a company that monitored websurfing, following clickthroughs on banner ads especially, a form of anonymous behavioural profiling. They acquired an off-line catalogue company called Abacus with the intention of merging their anonymous data with the  personal information within Abacus’ database. (though after Microsoft called foul, they were prevented from doing so by the US FTC. Later they (and their data) were acquired by Google. Page worked for a time at DunnHumby, who run Tesco’s Clubcard, and surprised us with the revelation that Inland revenue use club card data to validate your tax return.

But why should big and sometimes anonymous corporations be the arbiters of data, information and knowledge? We explored ways in which artists might take the mechanics of big data and transform it into different, playful information and knowledge. Blast Theory shared projects that tried to do just that from early experiments with Prof Tanda’s Guess-A-Where to a more recent and on-going collaboration with National Theatre Wales. Then John McGrath and Katherine Jewkes both from National Theatre Wales came on to talk about being an organisation that started out as an on-line community, and how even they struggle to cope with the “small data” that their participants have shared.  But they also gave us a taste of a lovely game/perfomance that imagines having to smuggle yourself of the border of a newly independent Wales. Part of that game involves creating a fake passport for yourself, and they were surprised by participants willingness to give their real names and data rather than making stuff up.

Giles Lane of Proboscis told us about a fascinating co-creation with Anglia Ruskin University and the R&D section of Phillips who have a interest in telehealth. Their idea which echoed our earlier discussion around the politics of knowledge, was to take an individual’s medical data and turn it into a 3D printed model, a talisman, a lucky pebble, which they carry with them. A beautiful object used as a tangible locus of meaning, mindfulness, rather than the dull data which means not enough to change behavior.

So can art use big data/infomation/knowledge to benefit society? And what are the ethics of doing so? That was the meat of our discussion, and Blast Theory, having collected our thoughts (data), are organizing them (infomation) and will shortly share them in some useful format (knowledge).

When they do, I’ll share a link here.