Luma live at Gallery of Modern Art, Spoleto, Italy

AV is easily understood as Audio-Visual but as a defining term is as broad as Electronic music.  Ian Andrews is a theorist and AV artist whose work as a member of Subvertigo VJ collective melded video surrealism with a playful activism present to this day throughout independent dance parties and cultural festivals like Electrofringe.   He defines contemporary audio-visual art as a live performance practice that, while culturally informed by the parallel history of experimental and expanded cinema, is structurally and conceptually derived from developments in sound and musical practice through the twentieth century.

Although the terminology “audio-visual” suggests that sound and vision might share equal importance, AV derives its “language” from music. In most cases AV work is concerned with formal compositional structures, of time and rhythm, which are closer to music than to specifically cinematic or visual art codes. (Andrews, 2009)

While AV works may address the performing body, the narrative text, the image in motion or stasis and structural / spatial definitions both virtual and actual, a focus on the consistent application and deployment of repeatable patterns and structures separate AV from Theatre, Dance and Cinema.   Media objects in AV work often consist, of looped sections of sound and vision, deployed in a structural pattern or stacked to form an audio-visual collage.   As with electronic music performance, this structure allows the composer to direct their material towards a near infinite number of stylistic choices.

Contemporary audio and visual practice also share a material status; the electronic signal in wire, or data. (Andrews, 2009)  Bill Viola concurs, stating in ‘Sound by Artists’ that the video camera “as an electronic transducer of physical energy into electrical impulses, bears a closer original relation to the microphone…” than the mechanical / chemical process of film. (Lander & Lexier, 1990, p. 49)  This notion of transducence, a transfer from one energy form to another, is central to a definition of AV as it is a modern, digital practice where analogue input, no matter the form, is converted to data.  The focus is placed on the signal, both the source and result of the data, not the performer, who engineers the real-time manipulation of aural and visual data into an output.  This projected output is not merely the by-product of a mathematical process; it suggests a third signal, a communication signal or meaning.  The source data can be pre-rendered and streamed or transformed and received in real-time and could be representative of anything at all.  The Semiconductor duo use seismic data as a source for their “Earthquake Films” and “Strata” (2007) and Steina Vasulka has developed the performance work “Violin Power” over 30 years, with the constant source being that of her violin.   The contextualisation of both works feature a visible or pre-empted demonstration of the specific transformative process.  The source signal and resultant signal can also be looped into one another, in a feedback system similar to anyone who has pointed a video camera at a live monitor of itself.  This trait has been explored and extended by Steina and Woody Vasulka in the 1970’s and in the present day with the closed circuit work of Botborg.

This focus on the transforming signal is another point of departure from related performative media; one that often problematises the nature of AV performance.  Weiß contends that the “…narrow contemporary definition [of visual music]… emerges live in public venues” and is not a product of the studio.   Where audiences identify a performance by the movement of performers, an expectation is readily shattered when the focus of performance is not a human body, but a transforming signal.  The clash between embodied and disembodied modes of performance exists also for the live “electronic” musician and some AV performers employ musical controllers as performative enhancements, in order to extend their ability to transform the signal, and as a way of physicalising their interactions for the audience.  The video itself has been used as a way of distancing the performer from traditional performance as well. Via email correspondence Tom Ellard explains: the “visuals distracted from the people on stage.  We were against people looking at us ‘performing’ which seemed a bit ‘rock’.”

Perhaps the closest cousin to AV performance is that of the Video Jockey or VJ.  Arising alongside the growth of Rave culture, they share a parallel history and many artists, like Tom Ellard, cross over between modes.  They certainly share a number of performative elements and the emphasis on improvisation in VJ form has moved the form towards sophisticated use of musical and non-musical controllers, controlling the construction of pseudo-narrative texts in an emergent form that is evolving away from its DJ origins, towards ‘Live Cinema’.

At this point the primary distinction is that AV performance has its foundation in live and composed audio practices, and in the manipulation of sound and video objects.  Ian Andrews contends that VJs are concerned with a visual foundation and the interaction their visuals have with a DJ or predefined musical track.  As a performance practice it draws less from musical practice than from live broadcast television; the original VJ movement sourced their gear from discarded remnants of broadcast Video editing equipment.  The influences stem not so much from Cage, Varese or Wagner but from the work of video artists, like Nam June Paik and the Vasulka’s, who also contributed to the development of various enhancements in broadcast television including the use of video synthesisers as a means of generating motion graphics.

Yes there were technical difficulties… but i digress…

Left of Left performance

The Setup:

A couple of months prior to this performance Tom Hall approached me with the idea for Lawrence and I to join him on the closing night of his installation for mixed AV performances.  The catch would be that we needed to use or approach his material, though it was up to our individual tastes to do this as we wished.

About one month prior to the performance I received around 20gb worth of source material from Tom to start playing with…

The Compositional Approach:

My approach was to work in a similar fashion to the way I have remixed sound for other artists; a somewhat subtractive fashion; therefore my influences were pretty close to home.  I wanted to simultaneously convert Tom’s footage to something more indicative of my own tastes, while maintaining an individual style / aesthetic that carried from image to sound and vice versa.  I made numerous passes of the source material through Isadora using Blends and Luma Key filters to reduce the visual elements to minimalist flecks of colour on black that would, in theory, mix well together.

As I was also experimenting with a Rutt Ettra simulation in Quartz Composer at the time I thought it would be appropriate experiment with the Z-Depth and zooming abilities to create depth in the compositions.

I also visited the installation during the day to obtain footage of the curtains which create an interesting visual simulation of the actual installation when blended with the other footage.

Ultimately I rendered out about 2hrs of prepared footage which would then be reduced to 3o mins of possible footage for the performance and sorted in 4 distinct pieces that would flow one to the other.

Sound was linked to the vision semantically (relating the sounds to whatever the image denoted) or symbolically (whatever I felt the image signified or connoted.)  A large percentage of the sound was generated by watching the footage and applying different values to a Waldorf Blofeld synthesiser until an appropriate correlation developed.  The sequences were then cut and connected to the prepared footage in Final Cut Express and rendered out as 640×360 (16:9) photo jpeg files to be mixed with in Isadora during the performance.

As a platform for doing strange things to video Isadora is somewhat peerless.  Modules are connected one to the next in a similar fashion to Max/MSP (but not at quite as low level) or Audiomulch work with audio.  As a performance tool there are some issues to overcome, particularly the inability for the video files to loop cleanly, something that works much better in programs like Resolume and VDMX.  In Isadora this can be worked around by using a loop patch put together by Fred Vaillant, however this takes over control of volume/opacity (something I needed control of) and requires files of more than 10secs length to work as, is.  So I hacked this patch for my performance and it didn’t quite work as the use of volume/opacity to trigger the envelopes does not work consistently for a reason i’ve yet to completely fathom (though I produce a more stable version for the N4rgh1l3 performance which will be reported on in one of the following blogs.)

The Performance:

curtains, lights from Performing Audiovisualist on Vimeo.

As you can see Tom did a great job setting up the space with curtains and three projectors.  With regards to my material, the level of light coming in from the windows was somewhat problematic for my very dark footage and the quick addition of a gamma corrector brightened the footage but also reduced detail to horrible pixellated blocks.  The main problem was not so much the video as it was the issue with seemingly random volume spikes (not that obvious in the selected footage but that is the only footage I would want to keep) and a rather jerky set of controls that made it easy for me to accidently turn audio/video layers on/off but not so easy to smoothly blend them.

So the reviewer calling the performance “graceless” is accurate.  This was not a live setting where I really mattered how live I was, except when it was obvious that I was screwing up.  Even with the bright Notebook light it was quite difficult to tell which buttons on the NanoKontrol I was hitting. In fact it is something of a miracle I got a performance as good as what is recorded above – I place my faith in drones!

The audience was very supportive but I was very disappointed as I felt like my good intentions/inventions were wasted on an unreliable control setup and a screen that made it difficult to actually see my work.  So I thought i’d recreate the pieces at home in our downstairs laboratory.  Here are two examples of what it was supposed to look like:

Magic Lanterns from Performing Audiovisualist on Vimeo.

Crush from Performing Audiovisualist on Vimeo.

The first is a great example of the Rutt Etra at work, splitting and abstracting depth in the image. The second is some Data Moshing of Tom’s material with some of my recent (similarly drifting) footage that I did not get to reproduce live due to the technical difficulties and time constraints.

Thoughts:

The audience and context for this work was pretty-much perfect (as opposed to the following performances which I will discuss in later blogs) with the only real issue being that of the rear-light projection coming from the street.  Now I do recall Tom mentioning that he would close the blinds for the final gig – and this was the assumption I was composing my parts for.  What does one do as a performer when this situation arises?  Accept and adapt or demand a more appropriate setup.  I certainly feel that I’ve been through many a gig where I left unhappy with my performance in part due to the setup being inconsistent with my perforative goals.  I’ve put curated enough performances to know that some artists pride themselves in their stubborn demand for a checklist of contextual settings.  Working closely with the Other Film group I’ve seen first hand how variable the goalposts are, particularly with expanded cinema that often exists only within a prescribed setup.  In order to further expand on the flexibility, sustainability of AV performance I guess, in the end it would’ve been better to reorganise my setlist to focus on the brighter material.  I certainly am not the type to pick hairs when i’ve been invited to perform at someone else’s installation.

While the conceptual framework was simple, the construction of my performance interface left a lot to be desired.  In an effort to devise a system that would allow direct control of all elements, I failed to consider how much control I needed and how few hands I had.  In the end it would not have hurt to automate a few more processes, especially given that the audience was staring around my arse and could not have seen any of the micro-gestures employed.  Did the gracelessness of the performance, and the failure of some material to trigger as intended, demonstrate that I did have live control?  If so, did the audience really care?  Based on the review I feel that it might have been better to provide more of an illusion of control; all the better for the smoother performance that the material required.  Given that the interface was finalised about 4 hours before the performance, i guess I can’t expect miracles.  It would be like learning a chord in order to play it in front of an audience without any form of practice.  But worse

It wasn’t all bad news though – bad gigs at this point in my research are… well…  good research – or at least points of contention for me to bust through.  So i need a more fluid interactive surface and more experience with solo manipulation of both audio and video in real time – with perhaps a small amount of automation / puppeting – enough to keep the performance smooth but interesting.  At very least I got a decent eyeful of Tom Hall’s VDMX setup, a program that seems to work more seamlessly in a live context (at least with regards to what I am trying to do here and now.) I promptly bought a copy and successfully deployed it in the next live scenario to be discussed…

Addendum:

Just read a review of Liquid Architecture in Sydney by Tom Ellard.  Particularly interested in his comments on Thomas Koner:

Last up was a German fellow, because the Goethe Institute sure seems able to pay for stuff. He was making a soundtrack for an unseen film, something a lot of sound students do because you can get a bunch of location recordings and play dramatic music underneath which hides your music inside Sound Art. Anyway his location recordings were pretty good and the film music was alright so I settled back into my snooze for a while. But on peeping I found he’d started showing video.

Now once you’re showing video you’re no longer doing the ‘unseen film’ – you’re bound by the same rules as any other soundtrack maker – relevance/resonance with the screened image. There wasn’t much. On screen we arrived at train stations in London in slow motion plus a difference layer in After FX. It looked quite nice for the first 5 minutes, after that, not so much. Sonically there was increasing layers of squoonsch – desperate really, as if squoonsch was the special sauce of Sound Art. That lost my interest.

What I learned: drones are the coward’s tool. Spurn drones.

Some interesting points:  when he played in Brisbane Andrew T and I spent most of the performance trying to catalogue his approach (we reckon multiple similar layers, some inverted and shifted, definately difference opacity.)  But spurn drones?  Come on Tom, I know Drones are kinda lazy, easy to mix with and synchronise with footage, but they are also the foundation of much of my compositional work and the work I chose to listen to.  Applied with footage have worked with material from Phil Niblock to the Hafler Trio, but some consideration should be made to how they are used in those circumstances avoiding blanket statements.  In particular i’m interested in the eyes shut trance state that drone music implies, how the inner cinema is broken by the combination of drones with vision and ways that this can be applied for good reason instead of novelty or laziness.

Over the next three years I will periodically demonstrate my findings through a number of live audiovisual performance experiments, documenting my approaches and related issues.

The phases are as follows:

1. addressing my background in composition and performance and current approaches;
2. addressing the literature and assimilating theoretical concepts into performance practice;
3. addressing the case studies / approaches and concerns of other audiovisualists;
4. exhibition of works and findings.

Documentation will be uploaded to VIMEO for each phase and will be used to elaborate on blog postings discussing the likes of:

• compositional and performative approaches;
• audience and context
• comparative works and inspiration;
• technological affordances and impact;

plus whatever else seems relevant at the time.

So the first phase matched up with:

• a performance as part of Tom Hall’s “Left of Left” exhibition at the Judith Wright Centre in Brisbane;

Left of Left performance poster c/o Tom Hall

• two performances and the presentation of my paper at the Australasian Computer Music Conference 2009 here in Brisbane.

Australasian Computer Music Conference 2009 - conference.acma.asn.au

Here is the first phase worth of  video material which I will discuss in the next few blogs.  Get acquainted and I’ll be back soon with a post about the Left of Left performance.

chrs

][oyd

“Performance” is the domain of “live art.”  As a blanket term for music, dance, theater, and experimental variations thereof, it can be understood as the non-participatory live presentation of body movements, images, and sounds.  In many cases, the notion of performance implies the presence of human actors or players on a stage, or a stagelike area.  The same term is used to indicate the quality of a technical apparatus in operation: we can speak of the “performance” of a specific computer system, or of a car.  This dual meaning is interesting in that it points to some general aspects of performance, for example, that it is an authorial execution system, an execution system that has a main actor.  Performance can be understood as the presentation, the making present (and perceivable) of the results of an execution. (Broeckmann, 2007, p. 199)

I was recently invited to put together a performance for a 12hr festival of experimental music in Brisbane, curated by Joel Stern.  His advice was succinct, “keep it simple and under 15minutes.”  In committing to this performance I wanted to incorporate some of the ideas I’ve been tossing around re: the performing audiovisualist; and whilst not wishing to reinvent the wheel, I did want to produce something separate from the live work i’ve been doing with N4rgh1l3.  Generally i’m much more enamoured with tools like Jitter and Isadora that allow you to define your interface and working methods for performance.  However given lack of time and need for something simple I decided to give Resolume Avenue a try, as it promised to provide the kind of simple and effective performance framework that Ableton Live provides for audio.  With regards to the Broeckmann quote above, it gave me an opportunity to assess the “performance” of the software in performance. It should be noted at this point that I was using a 15″ Macbook Pro, 2Ghz Dual Core with 2GB worth of 667mhz DDR2 ram.  The visual output was going from the DVI output to VGA on an LG projector and the sound was going out via my ESI Quatafire 610.

This is the third iteration of the software which has been primarily of interest for VJ’s.  The latest version integrates sound to image in a quite interesting fashion.  Audio and Video clips are selected and dropped onto one, or the other.  Changes in sound can be set to effect image and vice-versa; so a sonic filter could be set to sweep across the spectra while a wipe effects the visuals.  In theory they have constructed the perfect synaesthetic tool.  My setup included a number of short synthesised loops and similarly processed and edited bits of footage.  The opacity of each clip was connected to the volume within specific bands; therefore making a simple and direct link between, say, a kick drum type sound and an image that would flicker in time.

The construction of the live setup was quite intuitive in theory; drag the sound loops in – then drag the video over the top of them.  This was the practice recommended for best performance on the Resolume forum and the videos were pre-compressed in DKV format which does yield impressive framerates at 640×480.   In practice Resolume crashed once for every 5 bits of video I dropped into the setup, prompting me to save after doing anything.  The crashes seemed to be fairly arbitrary.  One bit of footage might make it crash again and again – then it would suddenly work.  The fact that I would receive NO ERROR MESSAGE was disturbing and enhanced my appreciation of the way Ableton and even Cycling74 catch and recover from potent errors.  After working at it for about 5 hours longer than I perhaps should have I finally had a setup that worked seemlessly – with controllers assigned to turn channels on an off – it all seemed like a perfectly simple little system.

At the gig, setup times were tight as they needed to push through about 100 different artists.  I found my space, connected everything and tested with the projector.  After a small amount of fiddling, to get the Macbook to see the projector – everything was working fine.  I turned off the projector and closed the lid on the Mac, as I was not performing for another hour. Gig time comes – I start the projector and open the Mac – what do you know – it isn’t outputting video!  Not that the projector is not seeing the video…  but Resolume in not sending anything but a black screen.  Fortunately I have sound and a KP3 Kaoss Pad, so I set up a couple of semi-interesting loops to go, while I reset the visuals.  Oh look, there’s my desktop!  Still nothing but a black screen from Resolume.

So I end up praising the audio element; seemlessly constructed AV pieces that work much like Ableton Live, looping and triggering without glitch or issue.  The video however…  it still won’t output anything but a black screen on that Macbook and i’m looking at having to reinstall the program again to see if that makes any difference.  How is it that the Resolume guys have gone from producing a decent VJ tool to a potentially excellent AV tool that fails on the Video side?

One comment from an audience member and fellow musician was that I “just try to do too much!”  But this was so little effort, in comparison to my usual setups and so easily delivered in theory.  A performance semi-destroyed by a random malfunctioning instability.  So my performance rating for Resolume Avenue is currently pretty low.  I’m sure they will fix it up eventually (though my forum post has so far been ignored) but the promise of simple and effective synaesthetic composition and performance is still unfulfilled.

“The degree of freedom offered to the performers is frequently competing, or in dialogue, with a programmed machine that imposes, or responds to, specific actions.  Many artists exploring this field are consciously playing with this relationship, and attempting to use the dialogue in exciting work that embodies the tension of the struggle between human and machine in an open, unstable system.  The “performance” of such a system is not immediately dependent on the involvement of an external actor, or on responses from and audience, though it may be dependent on externally set parameters and conditions.” (Broeckman, 2007, p.200)

… some of which are far from clear to the performing audiovisualist.  If you have bad “performance” from your car or guitar, there are places you can go to have them “tuned” and fixed to perform in a more optimal fashion.  For software, you either hit the forums and hope that someone else has had, and solved your problem, or you just use something else.  I’m happy to say that I have tried Resolume Avenue out on a mid 1998 Macbook Pro with more ram and a faster cpu and so far it has worked well…  but I haven’t had another performance and I still have no idea what the hell went wrong.

Broeckmann, Andreas (2007). Image, Process, Performance, Machine. in Oliver Grau (Ed),  Media Art Histories (pp. 193-205). Cambridge MA, MIT Press.