Project Directors: Michael Nitsche, Ph. D. and Ali Mazalek, Ph.D., LCC, Georgia Tech
Project website: http://synlab.gatech.edu/projects/tui3d/home/
The TUI3D project is a joint venture between Synlab and the Digital World & Image Group at Georgia Tech. The research addresses production and performative challenges involved in creating machinima. 3D interactive performance spaces such as machinima lack intuitive control mechanisms. Set direction and acting are limited by the technical toolset available to creators, as the tools were designed to create video games rather than cinematic works. They do a poor job of capturing the performative expression that characterizes the more mature medium of film. Tangible interfaces can help bridge gap between computer film creators and the established base of traditional film creatives and their knowledgebase.
Roles
I worked with the TUI3D group from 2006-2007 in multiple roles. On the creative front, I designed the environment and the narrative embedded in it, as well as the audio that the performers would be able to use. Our goal was not to have a traditional storyline, but to create a contextually rich environment that would inspire improvisation within that story space. I also acted as creative director, overseeing the aesthetic decisions and managing the team of SCAD (Savannah College of Art and Design) students who were creating additional character models for the project. On the interface side, I designed and constructed the soft-bodied interfaces (Cactus Jack and the Uniball) that controlled the interactive character with the performance space.
Olivia's Room
Olivia's Room is the narrative environment developed as the TUI3D performance space. Development included a writing a backstory that acted as creative guidance for the team as well as structure for performers and scripts. The scripts were inspired by old time radio programs, such as Wild Bill Hickock. The audio sampled from these shows reinforced the period and setting through atmosphere rather than more intrusive, direct means. The space was laid out in concept diagrams that communicated how lighting rigs should be set and made available for the performers, so that they could "stage manage" the scene during their performances.
Cactus Jack
The first stage of the project involved setting up a performance space using MovieSandbox and the Unreal Tournament engine, as well as software devloped in Processing to interpret the data (sent as UDP packets) from the interface for use in Unreal. The group created a virtual character to perform in that space, and a tangible user interface to control it. The interface included a gyroscope inside a 3D-printed handle inspired by the analog interfaces used in traditional pupeteering, as well as accelerometers attached to the soft-bodied mannequin. The mannequin sat on a base that housed the breadboard, and a small analog joystick on that base provided secondary controls that we assigned to facial expressions. The virtual character in the digital performance space mimiced the puppet's motions, thus creating a highly intuitive interface that encouraged expressive, naturalistic performances.
 |
 |
Uniball
The next stage of development used the same environment concept, but we expanded the performance possibilities with a new tangible interface. The reconfigurable Uniball design would support a variety of virtual characters by mapping movement to muiltiple points on the 3D character rig. It housed a gyroscope and the breadboard in its soft body, which also had eight connection points distributed around its circumference in two rings. Soft arms attached to these conections, and the core could support up to four simultaneously. Each of these limbs contained a bend sensor and accelerometer to capture data on all three axes (z, x and y).
