XR Studio Project Ideation Assignment

Assignment Deliverables

1. Submit one PDF containing at least three distinct project concepts. Each concept should propose a different experience, story, interaction, or creative approach.

   • For each idea or concept include:
     • Title and description: Write 3–5 sentences explaining the central idea, what happens, and what the audience experiences.
     • Visual support: Include one sketch, storyboard, or reference image that helps explain your proposal. Quick sketches are welcome. Credit any reference images and briefly explain their relevance.
     • Tools and workflow: List the proposed tools and briefly explain how they work together. Identify the inputs, such as movement, sound, or camera footage, and the resulting visuals, sound, or lighting. Distinguish live elements from material prepared beforehand.
     • Select your preferred concept. Explain why you want to pursue it, identify one technical question, and describe a small first test that could help answer that question.
   • Label file YYYY-MM-DD Lastname Firstname XR Studio Project Ideation (.pdf)

Assignment Overview

Develop a minimum of three distinct and different project ideas that explore the creative possibilities of the XR Studio. Projects may include live performance, recorded video, animation, interactive installation, or a combination of physical and digital elements.

Start with an experience, story, or artistic question. Consider how performers, virtual environments, sound, lighting, props, and costumes could work together. Select the tools that support your idea; a successful project can use a simple setup.

This assignment focuses on ideation and planning. Choose tools that support your concept, including tools you are still learning.

Possible Project Directions

Use these starting points, combine them, or develop your own:

Virtual Production Scene

Create a short narrative or music video with performers inside an Unreal Engine environment displayed through disguise. Coordinate camera framing, ETC lighting cues, and lights on stands to connect the physical performers with the virtual setting. Design laser-cut or 3D-printed objects that connect physical performers to a digital world. Consider wearable extensions, miniature sets, masks, or props whose significance changes with the surrounding imagery. Alternatively, experiment with staging the same short action three times, changing the virtual background, lighting, and sound. Explore how those choices transform the audience’s interpretation.

Motion Capture Animation

Record Motion Capture movements with OptiTrack and apply them to original characters. Explore how an ordinary action changes meaning when performed by an unexpected character in an unfamiliar environment.

Live Motion Capture Performance

Have a performer control a digital character during a live event. Develop a relationship between the physical body and its virtual counterpart through imitation, disagreement, transformation, or choreography. Let several performers control different parts of one virtual creature or environment. Make cooperation—and occasional disagreement—visible to the audience.

Generative Music Video

Use TouchDesigner to create visuals that respond to music, voices, or movement. Explore recorded or generated audio, rhythmic editing, and coordinated lighting changes. Potentially, combine a fabricated miniature or oversized prop with virtual scenery to make a performer appear enormous or tiny. Use framing and performance to make the scale change convincing.

Pose-responsive AI Environment

Propose a workflow using pose detection in TouchDesigner to guide images or video generated with ComfyUI, with RenderStream and disguise as potential display connections. Consider how transformations and processing delays could become part of the performance. Potentially, record a performer’s actions, then have them interact with traces of their earlier movements. Build a scene around repetition, memory, or an encounter with a past self.

Build a performance around a digital system that responds imperfectly to a person. Turn delayed, exaggerated, or unexpected responses into a deliberate comic or dramatic relationship.

Lighting as a Storytelling Tool

Use the ETC lighting board, studio lights, virtual scenery, and sound to tell a story through changing atmosphere. Plan a sequence in which lighting reveals, conceals, or transforms the meaning of an action. Record multiple cues and transitions to pair with live or recorded performance.

Grading Rubric