Photo credit: Life Shots Photography

 

As showcased in our recent look at intrepid Zapworks users’ impact on the art world, we’re witnessing AR being leveraged by creatives in increasingly innovative ways. From galleries to the workplace to our own homes, we’re seeing AR experiences that create new digital spaces for expression, dialogue and deriving deeper meaning from the world around us.

Fundamentally, AR is a wonderful medium for storytelling, driving exceptional levels of emotional attention by adding a digital layer of content to the world.

But what makes John Morrison’s collaborative work with The Verbatim Formula so inspirational is that he’s using AR to explore a technical and artistic answer to a deeply social question - how do we give visibility to those who are under-represented in decision-making while respecting their right to privacy?

 

AR experiences, created with Zapworks Studio, were accessed via zapcodes, which were later transferred to large lightbox installations.

 

Inspiring change through AR storytelling

While serving as a Practitioner & Lecturer in the Digital Media and Interaction Design group at Edinburgh Napier University’s School of Computing, John teamed up with The Verbatim Formula.

The team, who are based at Queen Mary, University of London, are a participatory arts project for care-experienced young people - those who have spent time in the care of a local authority, such as a foster home or a children’s home. Verbatim theatre techniques, that value listening and dialogue, are used to engage with care experienced young people social workers, and universities. Their important work is funded by the Arts and Humanities Research Council.

During fun and creative workshops, testimonies of care-experienced young people are shared and then recorded (verbatim) by other participants or performers - meaning they can share their so often unheard stories both powerfully and confidentially. The goal is that through building an understanding of the hidden life experiences of these young people, it will inspire positive change in both the care and education systems, improving access to learning and the support needed to succeed and thrive.

 

“The identity of the original speaker remains anonymous, and this often means that their words can be heard with special attention. Our performances create spaces for reflection and dialogue, not defensiveness or blame.”
- The Verbatim Formula

 

By using Zapworks Studio and with the help of the Zappar Support Team, John Morrison (alongside talented Digital Media Global student, Daniel Tocik) has brought these testimonies to life with AR.

Through the addition of zapcodes to light boxes, statues and printed media, AR transformed objects into portals, transporting people to an immersive experience accessed via their smartphone device. Through the innovative use of 3D avatars, the stories of care-experienced young people take tangible shape, visually mirroring the anonymous power of their audio testimonies to create emotional and meaningful dialogue with wider audiences.

I caught up with John to hear more about how this amazing project got started, his experience with Zapworks and how his Zappar-powered work ended up in the British Department of Education.

 

The AR experience in action - utilizing 3D models to tell compelling stories while protecting anonymity. Video credit: Wilfred Magnussen

 

James Wright: Hi John! Could you give me a brief overview of how you ended up collaborating with the Verbatim Formula and the overall goals of the projects you've been working on?

John Morrison: Working with AR in Zapworks Studio emerged as a by-product of my PhD research. A key aim of the project is to broaden and deepen understanding of the barriers and enablers that care-experienced students encounter on their journeys into university.

The practical outcome of this process was concerned with exploring how lens-based digital media, such as AR, can be harnessed to facilitate dialogues with stakeholders and wider audiences around factors deemed important by the care-experienced participants.

 

“I’ve literally become a catalogue of statistics, and just irrelevant facts and info. And it’s dehumanizing to be honest. If adults don’t really view you as a human then how can you view yourself? Right now, according to the system, kids have become just another number, another statistic, and it’s not whether a child is being cared for, it’s whether they’re being dealt with.”
- Leah, 14 years old, 2015 via The Verbatim Formula

 

JW: What's your personal academic/technical focus and how did this mesh with what the Verbatim Formula are trying to achieve?

JM: One of the key challenges for the project is exploring the ways you can provide people who are not used to being listened to with more control over their own stories and representation, while also protecting confidentiality. 

My friend, who was studying Applied Theatre in London, introduced me to a group of theatre academics from a participatory action research project called The Verbatim Formula. This collaborative art initiative has similar aims to my project, focusing on the importance of play and creativity for self-representation and the challenges of participation and privacy.

But instead of digital media, they have been exploring verbatim theatre techniques for attending to the voices of care-experienced young people. Our collaboration unfolded around a series of fun workshops with some of the young care experienced members of Click Wandsworth. These activities included the creation of digital verbatim statues in AR which were presented to compliment a verbatim performance at the Department of Education in London. 

 

The Verbatim Formula AR experience has been demonstrated across Europe at various events, sharing the stories of care-experienced young people. Photo credit: Paula Siqueira

JW: Why did you see AR and the use of avatars as the best way of sharing the stories of the care experienced young people?

JM: One of the early conversations with the Verbatim Formula crew centered around the simplicity of the verbatim theatre technique and yet how powerful the emotional connections were with people attending the live performances.  However, it was challenging to share these emotional and embodied experiences with wider audiences.

We were all curious if/how the earlier projects I had done in AR could translate to the context of this challenge - beginning with a simple process that audiences could interact within a range of environments using their own mobile devices.

Inspired by image theatre techniques, where participants express essential values and relationships with their bodies as a tableau, we used an Occipital Structure Sensor to capture a digital version of these that could then be loaded into Zapworks Studio.

Early observations of the way audiences engage with the AR experiences are that their spatial qualities enable different kinds of connections, compared to passive media like traditional photographs or video. This can be observed in the ways audiences physically move their positions, taking a step closer and changing their point of view to engage with the experience - gaining different perspectives while interacting with the pieces.

 

“[Daniel Tocik] decided he wanted to use Zapworks Studio because of its intergrated workflow and excellent support. Daniel is very at home in VFX compositing and 3D software and rose to the challenge of coming up with an elegant solution in a very short space of time. Any time he did face a problem, he was very impressed with the personal and professional support from the Zappar team.”
- John Morrison

 

JW: Who's been creating these AR experiences and why did you end up choosing to use Zapworks Studio? How has that creative experience been?

JM: Digital Media Global student Daniel Tocik, explored a range of AR platforms while doing an internship with me during his final year of study at Edinburgh Napier University. He decided he wanted to use Zapworks Studio because of its intergraded workflow and excellent support.

Daniel is very at home in VFX compositing and 3D software and rose to the challenge of coming up with an elegant solution in a very short space of time. Any time he did face a problem, he was very impressed with the personal and professional support from the Zappar team.

 

Nadhim Zahwai MP, Parliamentary Under-Secretary of State for Children and Families, experiences AR storytelling.

JW: What audiences have you been sharing these AR experiences with and what have the reactions been like?

JM: The first audience was at the Department of Education in early March 2019 and we gave a copy to Nadhim Zahawi MP (Parliamentary Under-Secretary of State for Children and Families) who has one on a lightbox in his office!

I then took a large 1m x 1m version with me while presenting at a couple of conferences - first at Research Through Design 2019 in Delft, Netherlands and then at the CHI2019 as part of HCI Across Borders Symposium in Glasgow, UK.

More recently, the experience has been on tour with the Care Review, with outings in Edinburgh, Glasgow, Inverness and will soon be appearing in the Scottish border town of Dumfries. People are really curious about AR experiences and the novelty factor is great for encouraging audiences to connect to the human stories at the heart of the project.

For future iterations, we would like to experiment with moving 3D recordings captured using DepthKit or Apples TrueDepth camera - so keep an eye out for that in future!

 

You can experiment with the 3D model created for the experience yourself using Zapworks Studio's Sketchfab integration!

 

Final thoughts

If this experience has inspired you to leverage AR to explore and showcase something you’re passionate about, it’s never been easier to get creating amazing immersive content. Our powerful, versatile Zapworks toolkit has everything you need, regardless of skill level - from our browser-based, drag ‘n’ drop Designer tool, to our most versatile and powerful Studio tool. If you’re looking for inspiration or support, our friendly and knowledgeable Zapworks community are on-hand on the Forum. You can also access our wide range of handy walkthroughs and visual guides on our Documentation site. Our expert team are always interested hear about your AR ideas - so if you want to chat it through, reach out and we’ll see what we can do.