Transcript:
Today I want to talk about some low-tech methods for achieving interactive projection media. This is BlackTrax, an amazing system of sensors that tracks very small transmitters in 3D space with lights and projections. You can track performers or scenery so that moving lights and projections respond to movement, even pure improvisation.
A few years back, we wanted to explore interactive media, but at that time, we needed something more accessible for time & money. We asked, “How do you make media respond to what a dancer or an actor is doing, without using some kind of sophisticated tracking system?” I mentioned media I had made back in 2001, for a dance concert called “Winter Fantasia.” It was sort of a journey through the phases of winter. In 2001, I had been using video projectors instead of slide projectors for a while. People were still often surprised by that choice still, in 2001. I used digital video for some stuff. The tools were cruder back then. I fell back on something older that I liked. Let’s jump back to 1998. I lived alone in a cabin in New Hampshire on 200 acres of woods. I had a basset hound and lots of time in the evenings. I would write code for fun. I wrote games. I made a program that played prime numbers as music. I also made a solar system simulator.
A little history: vector coordinate graphics vs. Bitmap
Excuse me while I jump back in time some more. We need even more background. Back to 1980. Back in the very early eighties and before, there were two schools of thought on how to generate computer graphics.
One school of thought was often referred to as bitmap graphics. In this model, the screen is composed of big pixels that can be black or white, or perhaps one of a number of colors. The entire screen is always in the memory of the computer. Every pixel is always accounted for. This method favors pictures over realistic motion. Bitmap graphics were used for games like Centipede and Pac Man. Animation was achieved by changing the image rapidly, like a flip-book.
The other school of thought was often called vector-coordinate graphics. This type of graphics is based on a dot-to-dot drawing system. The computer tracks points in space, and connects them with lines. The images were not as pretty as bit-mapped graphics, but the motion was much more fluid and sophisticated. Processing power was used for the math of the animation, rather than for rendering pictures. Vector Coordinate Graphics are the basis of AutoCAD and other types of computer-aided drafting. You can draw a planet or an atom. The computer does not care; it’s just math. DIA = .000001 or 1,000,000. it’s all the same in memory.
Bit-map graphics are the basis of Photoshop and image-editing software. As pixels increase, so does needed memory. Before camcorders, VCRs, or computers, we did not control or design any content on the screens around us. It was a different mindset. In 1981, there was a camcorder in my home, and there was an Atari 400 with BASIC in my 8th grade classroom. This was a revolutionary time. I got fluent in BASIC and PASCAL in high school in the eighties, but I did not use computers again until the mid-90s. I had a desktop PC that I took around with me for design work using AutoCAD and other software. One summer in Vermont, my alarm clock broke. I returned to BASIC programming in order to write an alarm clock program out of necessity.
Now let’s return to 1998. I play with BASIC some evenings for fun. I make some games. I also made this solar system simulator with two suns. And now forward to 2001. This is the second year I needed to make snow for dance. In 2000, we did simple, vector-based falling snowflakes For 2001, there is a group of dancers, and we want busier snow. I wanted the focus of the piece to be on the dancers, not on the cyc. I went back to my solar system simulator. I made a math formula that drew a snowflake centered on each planet. I replaced the suns with one blue snowflake that could be moved around with the mouse cursor. The other snowflakes were attracted to the blue one, just like the planets around the suns. Now I had a flurry of snowflakes that could be led around the screen. The operator had one order: Keep the blue one on the lead dancer. We did two more years with this winter-themed concert. My media evolved into AVI video files. BASIC graphics began to look too crude. I kept doing lots of projections for many shows…
Applying old solutions to new problems
So here I was in 2015: “How we can track performers on stage without using any kind of motion-tracking system.?” “You know, back in 2001, I did something with some code that I wrote myself. ” We got talking about this. If we created interesting user-controlled media, we could emulate those systems in a sort of duet between dancer and operator. I created a set of effects named swarm, ripples, lightning, spotlight, whirlpool, and grid. I made screen-capture videos of each of these. I demonstrated them to choreographer Amanda Whitworth on a regular computer screen, and gave her the screen-capture demos. Together, we put together a workshop where we used one projector with this media. Sometimes she led the performance, and I followed her. Sometimes, I led the performance, and she followed me. For each section, we had rules about the media, how we reacted to each other, and a loose narrative that let us anticipate each other. After each section, attendees came down into the space to experiment with the media.
Using this media for a show
In 2016, a group of us collaborated on a special devised work. It was about humanity and life in a changing, unpredictable environment. We deliberately broke the traditional model of mounting a show, in order to discover new innovations on devising performance. We had a playwright, a composer, myself as the visual designer, a choreographer/performer, a dancer, and a vocalist. We workshopped for most of that summer, and everyone contributed any ideas that they had; there were no barriers based on job titles. For this show, we worked with greater resources than in the simple workshop a year before. Nothing lavish, but good tools. The core was a Dataton Watchout System. We had three computers linked together. One for show-control, one to serve media, and a third. The third computer ran third-party applications, including software that reacted to audio events, and the same sort of media that I built in BASIC. The Watchout system allowed me to layer the 3rd party media into the mix as a video input. It allowed chroma-key, blend-modes,and transparency. Combined with other media, the effects become richer, more interesting illusions.
A different kind of control
In 2016, I handed the control of the tech and the media over entirely to a team of students, who did an exceptional job of running the show. In 2019, the show was accepted for performance as a part of the Prague Quadrennial’s PQ Studio: Festival series. We remounted it with a partially new cast. It was retooled a bit to avoid having to take any physical scenery along. With no set, the visuals were projected only on the messy cement walls of the performance space in Prague, plus a pale-gray dance floor.
I got to control it myself this time.
This time, I didn’t have to hand off control of the media. I was half of the running crew, Stage manager Alice Shea ran the show-control system, with music, video cues, and the orchestration of all of the different media inputs
The mindset is much more like playing a musical instrument, or operating a puppet, than running any kind of tech. While I am operating this media, I am as much as part of the performance as any dancer on the stage.
This is what really sets this specific method of media control apart from any other kind; even the BlackTrax systems that I love. This media can do more than simply react to performers. It has the ability to surprise them a little, to push back against what they are doing.
It allows the scenographic space itself to join in on the work, as a collaborative performer.
It allows the scenographic space itself to join in on the work, as a collaborative performer. It is as fulfilling of an experience as a designer as anything I have known.
This video was made for Matt Kizer Design LLC Copyright © 2019 Matt Kizer. All rights reserved.