Canvas + Triggers: Programs
Canvas is a movement-interactive video installation run off a laptop computer. We call the underlying computer program
Painterbecause it creates video frames similar to abstract expressionist paintings. First used publicly at the February 2009 Phoenix Experimental Arts Festival in our piece Red Bites for a Rainy Day,
Painteris a Max 5 software program that uses a video camera to target either (a) a previously set color or (b) any movement. In each frame of video the target pixels are
painteda new color, which changes gradually from frame to frame in a random walk through the video color cube, and the
paintaccumulates on the screen. A separate parameter controls how quickly the
paintfades (to either black or white); if the target color is not present, or if there is no movement, the canvas will gradually go blank.
In performance, the target (either color or movement), the range of color used for painting, and the duration of fade time change on cues. In installation,
Paintertargets movement only, while color range and fade time reset periodically, thus varying and refreshing Canvas's response to motion.
Canvas, being visual, remains connected to its observers even in an environment rich with sound--for instance in an echoey lobby filled with noisy theatre goers. If the installation space is quieter we can set up Canvas + Triggers, which combines
Painterwith a few of our audio-producing programs that we've used in dances.
The earliest of these programs comes from the dances Playing with Liquid Mercury and Playing with Liquid Mercury II, and we may as well call it
Double Keyboardbecause it essentially turns each half of the stage into a monophonic MIDI keyboard.
Double Keyboardsplits video input from a camera into fifty vertical strips which are associated with pitches; the left twenty-five increase in pitch from left to right like a piano, but the right twenty-five decrease in pitch from left to right.
Double Keyboardfinds the strip on each half of every video frame that shows the most change from the previous video frame, and
playsthe notes associated with those strips at a volume proportional to how much change has occurred--i.e. the two halves of the stage are independent musical instruments that play louder when there's more movement and softer when there's less.
Double Keyboardis a purely reactive program: when the computer senses movement it immediately and directly reacts, so that the dancer(s) can potentially control the computer output in at least some of its particulars--though not all. In performance
Double Keyboardvaries the type of sound it produces (piano, percussion, strings) when cued by the computer operator; in installation it produces
swishesonly, as heard in the video to the right. The
swishysounds work with movement improvisation better than musical pitches do, probably because the "swishes" merge together more than discrete musical notes do, and so feel more like the continuous motion of a moving body.
Canvas + Triggers includes two more programs, from the dance Broken Chase, which have a less direct relationship with the movement they track. Both divide the video frame into fifty vertical strips like
Double Keyboard,but the strips are not used to generate pitches. The simpler of the two programs is
Footstepswhich outputs ... guess what!
anti-reactivein the sense that it does the opposite of what
Double Keyboarddo; when it senses movement,
Footstepsdoes nothing, but when it senses stillness,
Footstepsproduces the sound of footsteps. The speed and pitch of the steps is directly related to how much movement the program
Footstepsdetects just before the stillness begins. The sound of the footsteps pans back and forth across the stereo audio space, the speed of the panning determined by how quickly the filmed movement source traveled across the fifty vertical strips of video before stopping and triggering the sound.
Footstepsessentially do nothing in the absence of motion (or color if
Painteris targeting color), the last program has a certain life of its own.
Agentis a program that plays a somewhat wistful sounding constantly evolving melody; when linked to a piano sound it comes off a bit as an intensely contemplative jazz pianist. In Canvas (as in Broken Chase),
Agentresponds to movement and stillness, but while
Footstepssimply reacts to these triggers
Agentacts in ways that cannot be controlled by the dancer(s).
The model of interactivity used in designing the
Agentprogram's responses is that of human conversation.
Agentdetects when the dancer is
speaking(moving) and when the dancer is
silent(not moving). As seen in the excerpt from Broken Chase to the right (in which
Agentis playing over a set percussion track), generally
Agentwill reply immediately upon noticing the dancer is silent, but sometimes it will wait a few seconds,
thinkingabout what it is going to
say.If the dancer chooses to interrupt
Agentby beginning to move before it finishes playing,
Agentwill probably allow the interruption and stop playing; however, occasionally
Agentwill doggedly continue on regardless of the dancer's beginning to move. Likewise, the
talking,but usually it will respectfully
listento all the dancer has to
saybefore replying. Thus the dancer must always be ready to
Agentprogram's behavior in ways that are not necessary with the other programs operating in Canvas + Triggers.
Agentis not controllable, so the relationship between dancer and
Agentis interactive and not simply reactive.
In Canvas + Triggers the video program
Painteris always active, while the audio programs
Agentregularly loop through a sequence in which sometimes only one accompanies the video while at other times a combination of two or three may do so. Thus the audiovisual environment transforms from one in which audio and video (and movement) occur together to another where they do not. Each amalgamation has its own character and inspires distinct movement strategies in its human interlocutors, whose immersive experience is repeatedly renewed by the cycle of variations.