Theatrical Projection with the Kinect

So I’ve just finished up my first attempt to use the Kinect in a theatrical environment (The Fleck Theatre @ Toronto’s Harbourfront Centre) and thought I’d share some of what I learned…

1. Pick your framework carefully. It’s inspiring how fast Kinect support is getting implemented in things like Processing, openFrameworks, and Cinder; combine that with low-level system driver support across platforms and this means you’ve got a major choice to make, right off the bat. I’ve used Processing for ages, but I’ve been starting to chafe a little at its performance issues, so I thought I’d take this opportunity to try something else. I had a run at openFrameworks, and while I was impressed with the versatility, the amount of work to bodge all the pieces together gave me flashbacks to the first time I tried a Linux install. Then I found Cinder. All I can say is it’s a fantastic next step from Processing, retaining enough of the structure and philosophy to smooth the transition from Java to C++. Robert Hodgin is some kind of genius and there’s great forum support. Seriously recommended!
2. Assume the worst. You’ve got a maximum range of 15′ – 20′. Shiny things cause huge problems. You’ve got a field of view of 57°. There’s a time-lag of about 30ms built-in. All of these things are described in the Kinect documentation; take them seriously. Things not mentioned in the documentation: IR interference from theatrical lighting didn’t seem to be much of an issue, but don’t assume it won’t be. Scrims cause the Kinect to go into epileptic seizure, but you can improve the situation by getting something solid close behind the scrim (i.e. a traveller curtain). The longest USB extension cable that worked from the Kinect to my MacBook Pro was 15′; you may need an amp to get much past that.
3. Get under the hood. A solid understanding of how the device works is key to developing interesting applications. It’s not just the hardware, either. There’s tons of history behind this software that’s behind the lightning pace of Kinect development: the amazing work on structured-light scanning paved the way for everyone who’s experimenting with the Kinect now; all the frameworks make use of OpenGL; OpenCV is an incredible library, created by a seriously talented community. Dig into the history and the development of this stuff.

Photos & video of the show to come!