"ERROR 404 - object not found"
Tracking experiments with Ellen Røed
Bergen
1st - 8th January 2004

In January I spent a week in Bergen to work together with Ellen Røed (long-time colleague) to experiment with object tracking using softvns. One objective was to explore the possibilities of tracking tennis balls in relation to the Motherboard production "Game Set Match" (see the file in the timeline (to the right!) for more info:), but first, here is a short introduction to Ellen:

Visual/media artist.
After several years working in the professional art scene, Ellen studied at the Academy of Fine Art in Trondheim. Her previous education was from Bordeaux and Oslo University. She has produced several performances for independent theatre groups, and in 95 and 96 she was co-ordinator of the Oslo Film Festival. She has exhibited several works at exhibition spaces and theatres in Norway and abroad. Ellen has held an aspirant position at BEK (Bergen Centre for Electronic Art). To mark the end of her term at BEK she initiated and organised a one-week symposium/workshop called "Come on Petunia" which aimed at exploring strategies for structuring time in contemporary time-based art. She is currently employed as a lecturer at the Academy of Fine Art in Bergen.

---

Using video footage of a tennis player we initially started to track tennis balls and play around with how visual media could be modulated by the motion of the tracked object. I am currently considering tracked objects in synaesthetic terms to be related to the sense of touch, but as a camera is involved, sight also comes into the picture. As this experiment was being carried out to explore the possibilities in relation to live performance I cannot role out the player, or performer and the sensory stimulus that invokes a response:

seeing the ball

Processing: calculating/analysing/manouvering

touch: hitting the ball


in synthesized terms this would look like:

video eye monitoring

touch: recognising the object, and keeping hold of it (envisaged as a hand grabbing the ball then holding it as it moves through space and letting go of it as it dissappears from the vision sphere).

The point of interest for us was the actual tracking of the object, so the notional senses of sight and touch are merged, with touch being the dominating, or most important factor. So the sensory input would be:

or ...

However, if vision is considered as a spatial plane, and touch is considered as the main activating (I'm struggling for words) sense, then a visual representation would look like this:

Touch is in the foremost layer, sight relating to space in the background. By applying some transparency to the foremost layer I am attempting to show that these sense modes are interrelated.

#Amanda whispers: these visual models were concocted during "Come on Petunia" when I put the log down for while. Click the Petunia link in the timeline (to the right!) to find out what I mean.

The output is as yet undefined.

The patch involved a human gesture - a click on the video to define the object to be tracked as it came in to view - touch.

The ball tracking data was then mapped onto the rendering of the video creating feedback imagery.

simultaneously generating and affecting the dynamics (movement in space and time) of the visual imagery.

But what senses does the visual output appeal to. What cross mappings occur in the body of the beholder? Is it calculatable or subjective to the viewer? Without answering this question we aimed for something that appeared as being "lively".

It was pretty hard to track fast moving tennis balls in low-res quicktime movies, so we switched over to tracking the performer. (We tried the racket but the data was not stable - the object was not found).

We used the tracking data to affect both the visual output - time/space relationships, creating a feed back tail (like the tail of a long-tailed monkey) and to generate audio output by mapping the data to quicktime instruments. The objective, to make something "lively", meant that we were working with cross-sensory accord rather than discord. The scenario looks like this:

outputting +

Additionally, the sound of the player hitting the ball plus the sound of a ball machine as the opponent added to the audio output, creating a link between the synthesized and what would have been the "real" world had this patch been used in a live performance working on the performer as the source material.

We spent time adjusting or tuning the patch. One thing we did was to reduce/focus the visual imagery by cutting out most of the tennis court and leaving only the player and the ball, envisaging a stage scenario where the projection would appear as part of the scenography.

That is what we got through, and it was fun! I later tried out the tennis patch in a live/scenic setting during a worklab with students at the Norwegian Theatre Academy, but with neither Ellen present to work on the patch or a light designer to tune the lights the demands on lighting (in order to keep the tracked object in view), while projecting the visual output AND creating a light design for the stage (designed as half a tennis court) out-reached my capacity at that time.

To get to Fredrikstad form Oslo I take the train. I have a little over one hour to brush up my teaching plan and look at the Norwegian countryside rushing past ....

I am on the train to Fredrikstad. I get out my mac, attach my isight camera, open the tennis patch, point the camera out the window, click on an object to track in the landscape, and see what happens. Sporadic bursts of sound and imagery occur until the object passes out of view. The dynamics are caused by the speed of the train as the object itself is not moving - it is a fixed spot in space - apparently. Other passengers gather round, curious to see what is going on. As the object passes out of sight there is stillness and silence and a message pops up, black text in a pink background on my screen. "Object not found" (very clever of Ellen to implement this warning for me). I click to choose a new object, far away this time, so it stays in view longer. As the tracked object is given a privileged place on the visual output, cross-wired connections of time/space, seeing, hearing and touching become more apparent ....... Unfortunately I did not record this seance as video documentation, but I did manage to capture some stills. You can look at them here:

oslo-fredrikstad images