top of page
cover image - MoCap_edited.jpg

Motion Capture

Project Type / Motion Capture, Live Performance
Role / Creative Director,  Sound Designer
Year / 2023

Tags / Point Cloud Motion Capture, TouchDesigner

An interactive motion capture project that will use the audience's own movements and performance to create visually stunning and emotionally evocative shadows on the screen.

Project Description

This project focuses on the creation of motion capture technology experiences. The audience will be invited to participate in an interactive performance that will help them connect with their shadows and express their inner selves. The motion capture technology will be utilized to store the audience's performance and used to create creative shadows and generative audio matching their energy. The resulting animation will be set to a commissioned music score and enhanced with visual effects.

Project Objective

To make the audience reflect on themselves, reminding them that even when they're tired, there will always be a shadow to protect them. So, there's no need to fear wearing masks in life sometime.

The Process

The creative process for this project begins with research into the users' performance, focusing on the technology to capture it. With the help of my Professors, Sven Ortel and Tejaswi Gowda, I learned about various motion capture technologies and how to establish a network connection to transfer this information to the computer.

User Experience
User Experience
User Experience

I intend for the user to feel like they are truly underwater when they experience the project. The spatial audio will create a sense of immersion, while the interactive elements will allow them to explore and interact with their surroundings. This project will leave the user with a greater appreciation for the underwater world.

In addition to the soundscape, I also created interactive elements that allow the user to explore and interact with their surroundings. This could include things like interactive bubbles, and trigger a shark animation.

Screenshot 2024-02-19 at 1.02.38 AM.png
Tools and Software used

Logic Pro X,

MS Kinect SDK,

Epson Projector,
17'x10' screen,
2 channel stereo speakers.

Final Video Render
Learning Outcome
  • Understand the different qualities of spatial audio

  • Compose spatial audio backgrounds with moving and static elements

  • Link those spatial audio scenes to head-locked audio in the VR headset

  • Use Open Sound Control (OSC) to drive the spatial audio with headset movement and POV

  • Apply a broad range of terms and concepts regarding spatial audio and XR in your critical evaluation of the required readings – be ready to discuss readings in class

bottom of page