Motion Capture
Project Type / Motion Capture, Live Performance
Role / Creative Director, Sound Designer
Year / 2023
Tags / Point Cloud Motion Capture, TouchDesigner
Overview
An interactive motion capture project that will use the audience's own movements and performance to create visually stunning and emotionally evocative shadows on the screen.
Project Description
This project focuses on the creation of motion capture technology experiences. The audience will be invited to participate in an interactive performance that will help them connect with their shadows and express their inner selves. The motion capture technology will be utilized to store the audience's performance and used to create creative shadows and generative audio matching their energy. The resulting animation will be set to a commissioned music score and enhanced with visual effects.
Project Objective
To make the audience reflect on themselves, reminding them that even when they're tired, there will always be a shadow to protect them. So, there's no need to fear wearing masks in life sometime.
The Process
The creative process for this project begins with research into the users' performance, focusing on the technology to capture it. With the help of my Professors, Sven Ortel and Tejaswi Gowda, I learned about various motion capture technologies and how to establish a network connection to transfer this information to the computer.
User Experience
User Experience
User Experience
I intend for the user to feel like they are truly underwater when they experience the project. The spatial audio will create a sense of immersion, while the interactive elements will allow them to explore and interact with their surroundings. This project will leave the user with a greater appreciation for the underwater world.
In addition to the soundscape, I also created interactive elements that allow the user to explore and interact with their surroundings. This could include things like interactive bubbles, and trigger a shark animation.
Tools and Software used
Touchdesigner,
Logic Pro X,
MS Kinect SDK,
Epson Projector,
17'x10' screen,
2 channel stereo speakers.
Final Video Render
Learning Outcome
-
Understand the different qualities of spatial audio
-
Compose spatial audio backgrounds with moving and static elements
-
Link those spatial audio scenes to head-locked audio in the VR headset
-
Use Open Sound Control (OSC) to drive the spatial audio with headset movement and POV
-
Apply a broad range of terms and concepts regarding spatial audio and XR in your critical evaluation of the required readings – be ready to discuss readings in class
