AR Experience Using Live Optical Tracking Created at Rose Bruford College

A year-long R&D project to develop live streaming AR experiences began as a way to try to take theatre to people at home, while keeping it live. Led by Copper Candle, using Innovate UK funding as part of a consortium led by The Round, and supported by Rose Bruford College, the mission required motion to be captured live and transmitted to audiences all around the world via a server

Rose Bruford College PhD Researcher and the projects Creative Director, Rory Foster, describes his role, “My whole work is looking at how theatre and performance can exist in different mediums – so not just person to person audience, or humans interacting in the same space but what would I identify as key tenets of performance, be that liveness or be that sense of dramaturgy, be that complex narrative structures. How can they be brought about without necessarily having two humans sharing the same space at the same time.”

Target3D were invited to be the mocap consultants. Within Rose Bruford College’s new Centre for Digital Production, we worked with the James and the Copper Candle team on their production workflows in Unreal Engine, and supported them on character pipelines – finding ways to build their characters in order to support facial capture and body motion capture coming from two different types of software.

“We’ve created a short VR performance that is to be explored by four people at a time – with one live performer in the space with an avatar,” continues Rory. “We’ve been looking at some short narratives that explore the stories of expedition leaders who never returned.”

A key element for the hyperreality result was helping the students to build characters as Target3D Motion Capture Technician, Petros De Donker, explains, “The students were experienced in performing for theatre and film but performing for mocap has some differences. With the team we helped them with their understanding of where markers are, how to best use them, how to position their bodies and to change their posture to fit the character they were playing.

“It was interesting incorporating these different types of technology with Copper Candle; using the optical system to track the VR headset, whilst also tracking the performer who you can see in person. For the user, it’s one thing viewing something in VR and another thing interacting with something or someone in VR. Marissa was getting close to the users, touching them, making them feel as if they were in the virtual space which really enriches the experience.”

Actor Marissa, who features in the reel, is a Masters student in Collaborative Theatre Making at Rose Bruford College and previously honed her mocap knowledge on a Motion Capture Performance Workshop at Target3D. The actors’ role performing and interacting live in the VR space enriched the user experience.

Head of Centre for Digital Production for Rose Bruford College and Director of Copper Candle, James Simpson elaborates: “For this project Copper Candle, using Innovate UK funding, have developed several pipelines that allow us to control a virtual reality experience using our new streaming DMX protocol. What this means is that we’ve got half a dozen VR headsets that can be tracked inside a space using Rose Bruford’s new mocap studio – we’ve got props that can be tracked again and again using the same system and we can see all this inside the VR headset as an audience member. We are then able to control the entire environment around us using a lighting desk. So all of the scenery, all of the visuals, the lighting, the props, everything is controlled using the lighting desk.”

So, is this the end, or just the beginning? Rory concludes,

“Virtual Reality brings so many opportunities to do things that you cannot do on a natural stage or on a basic reality stage. There’s lots to explore and there are lots of opportunities to create worlds and create realities and things that happen that are completely out of the natural world.”

With thanks to Target3D for creating this news piece, original can be found here Copper Candle Produce AR Experience Using Live Optical Tracking (target3d.co.uk)