dynamic video

summary of my talk at e32 the other night:
(http://e32.hitart.com/)

started by showing some video footage of live mapping (action painting combined with storytelling) at bronxartspace last month:
(http://eswip.wordpress.com/2010/03/17/life-mapping/)

(http://www.bronxartspace.com/mar2010.php)

in the video stills above, the video projection symbolizes the world. the colored tape represents travel between different places the participants had been, leading to the spot of the piece. the video projection was always changing, so that we never saw quite the same composition repeated. my goal last night was to share how video can be manipulated by a computer in order to create dynamic video montages.

showed a short clip (still above) i had made while experimenting, trying to figure out what i might want the piece to look like. this is a linear (not dynamic) video – it just plays straight through and then repeats. ultimately i wanted the result to be unpredictable, but within parameters (for example, never too dark). this video clip is composed of the same layers used in the completed dynamic video.

where did the video clips to be combined come from? the first few were made at my “uncle’s” “farmhouse” in connecticut.

a camera on the balcony looking down onto the field recorded ruthie and i as we told each other our life stories, walking from place to place in the snow, creating the first maps.

here‘s what that camera saw:

i wanted to show more points in time, so i layered that video, spacing the footage five minutes apart in time, stacked on top of itself:

here‘s a still from the result of that editing:

the next video clip used was an animation i drew. it shows two moving lines connecting all the places ruthie and i had talked about earlier.

i projected that animation onto our earlier maps made with our footprints in the snow:

here‘s what the video camera recording that saw. this is the third video clip the computer combines to compose the dynamic montage.

after deciding to expand the project to include other collaborators, justin agreed to come over.

we had another “blank canvas” to use:

here‘s a snapshot i took while justin and i were making our initial life maps. you can see the video camera in the upper right:

editing that video to show different points in time by stacking layers resulted in the fourth video clip used in the piece, here’s a still:

the fifth video clip is a hand-drawn animation of justin’s map paired with the same map for me done earlier with ruthie::

the sixth video clip is footage of projecting those digitally-animated maps onto the maps we had made with our footprints in the snow.

here are all six videos the computer can use in different combinations to create compositions:

one decision the computer makes is which part of the video to show. here are two stills of the first video playing simultaneously, at different times. the location of elements in each video changes the composition of the image. in the lower-left of each example below, i appear to be entering the frame in one, but leaving in the next; ruthie appears to be following me in the second one. all of the examples i‘m showing are just ten seconds, but some of my work has twenty or fifty or more hours of video the computer can show.

there are different ways the computer can layer different video clips together to form new compositions. one way is by making video clips translucent:

another way to show multiple layers at the same time is by “blending” them. there are many ways to do this; here’s an example (“overlay”). the differences between some blending modes are subtle.

the differences between some blending modes are subtle, others less so. here are the six i decided the computer could use in life mapping:

the final piece uses three layers. the computer can decide which video to show on each later (and keep changing them).

the computer will keep choosing (and changing) blending modes for the top two layers.

the computer will also keep adjusting the translucency of the top two layers.

and that’s everything the computer is doing to make the ever-changing video used in life mapping. it will select which of six videos to show on three different layers, at random points in time, at random translucencies, in any of six blending methods.

the result of delegating those decisions to the computer means that the result is always a little bit unanticipated. within a predetermined structure, there’s room for unpredictability. i don’t like to know the ending when i‘m reading a novel or watching a film, and i don’t want to know exactly what i‘m going to see in my own art either. working with collaborators and computers is one way for me to keep finding surprises.