Dynamic Cloth is an experience that allows participants to use their body movements and positions to physically interact with digital cloth 3D models that respond to the movements and gestures through changes in shapes, depth, and lighting. Keeping virtual environments and the intersection of virtual and physical as the centers of my MFA research, I wondered what other technologies and approaches I could use to explore this realm. I wanted to create an experience that engages users more and encourages different approaches to interaction and user interface. Guided by the question about how we use shared mixed reality spaces to relate to virtual worlds, I chose to work with Kinect and explore body tracking as a tool for groups of people to simultaneously interact with virtual shapes and spaces. The goal was to create a space where physical surroundings cause dynamic changes in the virtual realm and where people can explore the effects of their abilities outside of physical reality. The subtopic I am also exploring in this project involves technology directing our actions.
I created the digital cloth 3D models in Cinema 4D, and I later imported them into Isadora where I devised a program that uses the body tracking input from Kinect to cause changes to 3D models in Isadora. The models react through continuous changes in size (Z-axis), shape, and also lighting. The final iteration was installed in the ACCAD Motion Capture Lab where the participants could see the projection of the models reacting to their movements in front of them, while the Kinect was behind them capturing their movements and depth. The biggest learning takeaway from this project was learning how to use the inputs from Kinect in order to cause the types of changes I wanted to the objects and control the number ranges in which those changes can occur.
In-progress images: Optimizing the cloth model in 3ds Max (left) and one of the scenes in the Isadora patch (Right)
Early experiments with physical brightness levels (captured by Kinect) influencing the digital lighting changes in Isadora: