A brief description of the sound software for Fugue

---

Rainer Linz

Fugue's sound software is based around a series of customised audio players ("Fugue Players") that respond in real time to changes within the Artificial Immune System (AIS). The sound software is designed to run on a separate computer, which is networked with the AIS's computer, although both systems will run on the same machine and have done so in presentations to date.

The audio players are part of a larger collection of java classes making up the Fugue audio system. These include Server, Sound Protocol and Wave Data classes. I also wrote a Client class to simulate the AIS for development and testing.

The relationship between the sound System classes is shown to the left. What follows is a brief description of each.

Client

In Fugue, communication between the Artificial Immune System and the Sound System is through a standard network connection. An array of variables is passed to the sound system at regular intervals, representing a 'snapshot' of what is happening inside the AIS at that particular time.

With the distances involved in our collaboration we decided that I would use a simulation of the AIS in Melbourne, to establish a way of working independently of the Fugue hardware which was situated in London. Updates to the sound software could be emailed and tested on the system there.

In London, output data from the AIS could be captured into a text file while it was running, and emailed to me. My client software then, was designed to read this authentic AIS output and send the information to the Sound Server at regular intervals just as the AIS would.

Sound Server

The Sound Server is a workhorse class that makes a network connection with the AIS, and passes incoming data arrays to the Sound Protocol class (below). Some rudimentary error checking is also carried out here.

Sound Protocol

In this class the data array, or 'data state' of the AIS is compared to its previous stored counterpart to detect which values have changed in the mean time. If any, the new values are passed to handler routines which are designed to effect a change in the sound.

The handler routines are the composition and performance procedures for Fugue's sound.

The role of the Sound Protocol class is to parse and analyse the AIS data state and initiate a suitable change in the sounding process. The logic here is infinitely extensible and serves to bind AIS events to relevant sound processes.

An example of a data parsing procedure is to register changes in the direction of growth of particular immune system cells over time.

Wave Data class

This class contains composition data such as wavetables, useful index points, sequencing and other information. Currently only vocal samples are used in the sound design, and stored as 16 bit, 44.1kHz mono samples. This data is used by the Sound Protocol class' handler routines to control the audio players in real time.

Fugue Player

The Fugue Player class contains most of the functionality associated with the audio construction and rendering. This is because it is a customisation of a general purpose audio player used in my other work.

The functionality includes start, stop, volume and pan controls, as well as dynamically loading and sequencing wavetable information. The player has an internal sound buffer making it useful for direct sample manipulation. A granular synthesis provision is to be added in future.

While four Fugue Players are shown in the diagram, up to 16 may be used.

Remarks

The disk size of the Fugue audio software including audio samples is currently around 300kB, this small size making it ideal for distribution over the internet.

The customised audio player concept challenges the phonographic paradigm, of which audio CDs or internet streaming files are examples.

Here, instead of the mechanical playback of fixed sound data from a disk or file, the audio player combines and renders source wavetable information in real time, providing a listener with the possibility of a unique and customisable listening experience.

Additional Links

John Jenkins Rainer Linz - A theatre of interaction

Rainer Linz Banalities for the Perfect House