Four Real-Time Algorithms in Practice
Paul Hembree
INTRODUCTION
This article complements the technical manual Four Real-Time Algorithms (2015), by Roger Reynolds, Jaime E. Oliver La Rosa, and the author. The purpose has been to illuminate practical details, not discussed in that manual, on Reynolds’ use of those tools in his most recent compositions. Working with Reynolds since 2011, the author has observed several principles of operation that may be useful to other composers, and musicologists studying Reynolds’ works.
The author worked with Reynolds on several major projects as computer music assistant, including Positings (2013) with Southwest Chamber Music and Ensemble Signal, FLiGHT (2014-16) with the JACK Quartet and video artist Ross Karre, and Shifting/Drifting (2015) with Irvine Arditti. These projects involved collaborating with Reynolds on the design of new algorithmic components, the optimization of existing algorithms, recording seed files (which will be discussed below), the creation of score patches to control the algorithms during performance, and the practice of performing those scores with the instrumentalists in concert.
The author has also altered and performed the computer music parts, created initially by Reynolds’ previous computer music assistants, in works including MARKed MUSIC (2011), Dream Mirror (2010), and SEASONS Id: A Mind of Winter (2009-2012). The algorithms have been in continuous development over about a decade through the activities of several computer music assistants, including Pei Xiang, Ian Saxton, Jaime E. Oliver La Rosa, and the author.
Primarily the algorithms have been implemented in Miller Puckette’s Pure Data, with a preference for the Vanilla distribution, enhanced by a very small selection of externals, some third-party and some first-party.
SUMMARY OF THE ALGORITHMS
It is worthwhile to summarize briefly the operation of the four algorithms, should the technical manual not be available for the reader. The four algorithms were initially controlled by computer automation, without the live input of a computer music performer, aside from general management of amplitude levels. However, the algorithms migrated to become more performative, involving the computer musician more as a chamber musician than as a technician, controlling a subset of important variables linked with perceptually salient features.
As the name might suggest, PROLIF is used to proliferate numerous instantiations of a single audio sample at different transposition levels, drawing upon flocking metaphors for inspiration. SMEARZ involves both time stretching and blurring of spectral material, using phase vocoder techniques. MATRIX creates complex rhythmic patterns by temporally fragmenting an audio sample. THINNR can be used to radically hollow out the spectral content of a sample using two band-reject filters. Used in conjunction, these filters were intended to isolate a spectral peak between them in order to create the impression of vocal-like formants.
SEED SOURCES
The typical development cycle for an electro-acoustic work by Reynolds begins with the composition or compilation of a pool of acoustic instrumental excerpts that will become the seeds of his algorithmic processes. These seeds are usually motif-sized, but can be as short as single notes or chords and as long as extended melodic phrases.
Reynolds rarely composes these seeds as isolated entities, divorced of a musical context, but instead usually extracts these seeds from other works for the same instruments. In rare situations, these are much older works; for instance, the violin MATRIX seed used on page 21, measure 2 of Positings is taken from Kokoro (1992), page 6, system 4, measures 3-4.
EXAMPLE 1: Violin MATRIX seed , page 21, measure 2 of Positings
.
EXAMPLE 2: A fragment from Kokoro, with nearly the same pitches, page 6, system 4, measures 3-4.
Reynolds has also composed a number of new solo acoustic works intended to pair with, and provide seeds for, subsequent electro-acoustic works. For instance, the imagE/imAge pieces for guitar, contrabass and violin all have complementary electroacoustic works: Dream Mirror, MARKed MUSIC, and Shifting-Drifting, respectively. As would be expected, the acoustic material from these complementary acoustic pieces is transformed quite heavily in the electro-acoustic pieces.
For the composition of FLiGHT, Reynolds created a draft version of the acoustic-only string quartet music, which was used for an initial recording and seed extraction, followed by revisions of that material for the actual music presented during the premiere. As FLiGHT currently stands in Fall 2015, the string quartet music alternates with sections, called Weaves, utilizing the actors and computer. The computer draws from string quartet material, but currently the quartet does not play simultaneously with the computer. It is possible that this will change, because the piece is still under development.
Reynolds’ strategically uses pre-existing and companion pieces as seed sources. This process allows for greater performer familiarity with the material, greater compositional familiarity with motivic or harmonic implications, and occasionally a pre-existing recording for the acquisition of sound files.
NON-REAL-TIME SAMPLING
Reynolds has an aversion to real-time sampling of material for algorithmic processing. The vicissitudes of equipment quality and placement, unexpected performer movements, and recital hall acoustics have driven Reynolds to pre-record his seeds. Doing so allows Reynolds and his assistants to carefully craft the amplitude envelope and spectral content of his sounds, optimizing the response of the algorithms.
used in Shifting/Drifting, page 9, bottom system.
For instance, dynamic range compression is frequently applied to seeds used by MATRIX to increase the articulative potential of every moment of the sample. This is an easy and effective mastering technique for seeds used by this algorithm. As MATRIX passes over the seed, extracting grains nearby each attack, each grain from a dynamically compressed seed will simply be louder than those from a non-dynamically compressed seed.
Dynamic range compression is clearly not impossible to do with live sampling, and is probably vital to such a practice. However, non-real-time mastering of each seed allows Reynolds and his assistants to ensure an optimum and predictable output of each algorithm, making the instrument easier to perform with.
EXAMPLE 4: Spectra of a seed before and after SMEARZ processing.
Non-real-time sampling also allows for nested algorithmic processing that would be prohibitively expensive in CPU cycles. Particularly in Shifting/Drifting, Reynolds’ use of the THINNR algorithm has often be applied to seeds already processed by SMEARZ, and often these seeds are themselves layers of two separate SMEARZ processes. SMEARZ is a potent way to increase the spectral density of a sound by allowing the spectral content of any one moment to bleed into adjacent moments.
Furthermore, SMEARZ can create seeds that give THINNR a more robust block of sound to carve into. A typical performance with THINNR involves moving the two band-reject filters, occasionally isolating a band between them, creating the impression of a moving voice-like formant.
EXAMPLE 5: The same seed as EXAMPLE 4, now with THINNR processing.
THINNR AND ACOUSTIC INSTRUMENTS
Another use of THINNR involves masking the fundamental and lower partials of a seed with one or both THINNR band reject filters, allowing for unusual combinations with acoustic instruments. In MARKed MUSIC, for example, the final THINNR event involves decapitating the fundamental and lower partials of a seed file, while superimposing the resulting body of high partials over different fundamentals in the acoustic contrabass part.
In the author’s marked-up performance score, the pencilled-in admonition “BACK OFF OF LOWS” preserves Reynolds’ intent to attenuate low frequencies, including the fundamentals, using THINNR (EXAMPLE 6). This often results in what is perceived as an acoustic contrabass with a strangely inharmonic spectrum. Structurally, the seed file material here, labelled S31, is identical to the live material in the contrabass occurring on page 15, but played at a slower speed and starting earlier, thus effectively creating a converging tempo canon.
HARMONIC AND TEXTURAL INTERACTIONS
SMEARZ, MATRIX and THINNR do not transpose the seed files they process, although they may temporally distort or dislocate the order in which pitches are encountered, effectively randomizing the pitch content within a specific range depending on that of the original seed file.
In Dream Mirror and MARKed MUSIC, temporal dislocation is used in contexts with a great degree of acoustic performer flexibility. Indeed, these pieces were written for expert improvisers who are asked to creatively explore a wide-ranging but bounded set of musical possibilities. The computer musician is often asked to rapidly jump between positions in the seed files with MATRIX, provoking improvisational responses from the acoustic musicians. In the author’s performance score for MARKed MUSIC, the pencilled-in “POS” marking, short for position, followed by a jagged line, clarifies Reynolds’ intent for erratic playback of different portions of the original seed file.
In Shifting/Drifting, page 2, 2:48-3:17, Reynolds achieves a meld of computer and acoustic material by having the violinist imitate the unpredictable fragmentary process in the simultaneously occurring MATRIX algorithm, performed by the computer musician. Here the process is more restricted for the violinist than for the contrabassist in MARKed MUSIC, but the principle is roughly the same. The pitch material is almost identical between the processed seed and the acoustic violin.
Reynolds’ works more often involve predetermined, non-improvised harmonic trajectories in the acoustic music occurring simultaneously with the output of an algorithm. Reynolds often uses SMEARZ, MATRIX or THINNR to scan through, at a fixed rate, the pitch material in a particular seed file. SMEARZ and THINNR stretch out the material of a seed file in time using phase vocoder techniques, which MATRIX rhythmically samples grains of the seed file, potentially preserving their order.
Moving at a fixed rate with any of these algorithms allows Reynolds to predict when certain pitches will sound in the computer music part. Often this manual prediction process is reinforced by recording a provisional performance of the computer part and transcribing prominent pitches, which is particularly useful when working with SMEARZ and THINNR. Both of these algorithms can reinforce unexpected partials in a seed file, altering the predicted pitch content. An example of this transcription process is preserved in the bracketed pitches on the computer staff throughout Shifting/Drifting.
Often Reynolds uses the interaction of the acoustic parts with fixed-rate scanning of seed files to flirt with the tertian implications of his pitch material. Although he controls his harmonies with serial methods, he admits that he often arrives at the final row forms by scanning three or four adjacent pitches and adjusting to create ambiguous harmonic implications that are neither completely tonal nor atonal.
This occurs in conjunction with SMEARZ in the elegiac final section of Shifting/Drifting, from page 20, 17:47. The pitch outline of the entire SMEARZ seed complex can be seen in Reynolds’ sketches, or alternatively on page 23, bottom system of Shifting/Drifting.
This material is stretched from 20:19 to 21:29 with the SMEARZ algorithm, so that the final arrival of the glissandi on what is essentially a second inversion Bb major triad occurs at 21:21 (only the Bb-F fourth is preserved in the transcription of the computer part in the score).
The acoustic violin implies a vague sense of D minor, in part the accidental result of emphasis on open strings for dynamic power. The D-A open-string dyad, the memory of a recent F natural, the upper-neighbor-like motion of the Bb to the A, and a preceding quasi-dominant A-E dyad all work together to hint at D minor.
But Reynolds doesn’t let this tonal implication remain unquestioned. The computer part, with its Bb major quality, implies a potential and simultaneous relative-major to the violin’s D minor. Traditional voice-leading is eschewed through jagged octave displacements, and the implied harmony does not settle down on D minor, instead leaping back up to the supertonic-like E-Bb tritone dyad. It remains unsettled.
Reynolds has emphasized this important moment of tension to the author, preserved through performance notes in the score: a hairpin crescendo and the word “challenge,” meant to push the violinist to play forcefully here against a stentorian computer sound.
INDETERMINATE ALGORITHMIC PITCH
Of the four algorithms, PROLIF is the only one that produces sounds of indeterminate pitch content. The pitch range variable controls the range of potential random transposition applied to each instantiation of the seed sound. Reynolds typically composes using this variable by departing from or coalescing around moments with no transposition, when PROLIF locks together with the pitch content present in the acoustic instrumental parts.
An example occurs from the end page 12 to the end of page 13 of Positings, when the computer part departs from and returns to an E-F#-Bb-B pitch collection. Reynolds’ request for this trajectory is preserved in the author’s personal performance notes for this piece – in essence a miniature score for slider movements to be executed by the computer musician. Here the lines following the acronyms PR and P represent continuous changes in the sliders, performed by the computer musician, for the pitch range and pitch (pitch center) variables. Returning the sliders for these variables to MIDI continuous controller number 64 ensures that no transposition occurs.
Other variables marked in the performance notes include SR, spatialization rate, which controls both the speed and unpredictability of spatial movement for the three output channels of PROLIF, controlled by a flocking algorithm. Q controls the band-width of a randomized band pass filter applied to each PROLIF event, which is used to add depth to the PROLIF swarm through several layers of attenuation. D is density, which controls how many instantiations of the PROLIF seed are sounding at any one time. Reynolds more often employs very low densities in PROLIF, rather than high densities, at least partially to not mask the acoustic instruments. High densities are also reserved for just a few special moments in the piece, when hurricanes of sound are desired.
The instrumental enhancement on page 13, accompanying acoustic music for the computer, reveals a similar departure and return in the instrumental pitches when compared with PROLIF. Reynolds essentially applies the same transposition process in PROLIF to the instruments themselves, although with a fixed outcome. Triangles with indeterminate but relative pitches, along with clusters in the piano, also trace the contour of the seed file.
seed pitches.
A SUMMARY OF PRINCIPLES
As has been discussed, Reynolds composes with his algorithms using several principles guiding their operation. Seeds for algorithmic processes tend to be drawn from previous musical contexts, such as the imagE/imAge companion pieces, enhancing familiarity with their harmonic or performative proclivities. These seeds are always recorded in advance, rather than recorded in real-time, ensuring an optimal response from the algorithms, thus providing the computer musician with a predictable and reliable instrument. Pre-recording seeds also allows for algorithms to be nested, for instance, by using SMEARZ as a source for THINNR.
The algorithms are used simultaneously with acoustic instruments in a variety of ways. THINNR is an excellent tool for overlaying phantom spectra over the top of acoustic instruments, due to its ability to occlude the fundamental and lower partials of a seed sound. MATRIX is frequently used to create jagged, rhythmic instantiations of musical material that can unpredictably interlock with acoustic instruments. This is achieved through acoustic improvisation, or through indeterminate notation. SMEARZ is often used to stretch out a seed, providing a harmonic backdrop for an acoustic instrument. PROLIF, the only algorithm that transposes seeds, is frequently used to depart from or return to familiar pitch collections.
In all activities involving the algorithms, Reynolds treats every behavior as a provocation, and every artifact, whether it be sound or score, as a potential resource. Furthermore, the careful preparation in advance of all these resources allow performances to be uninhibited flights of musical fancy, even for the computer musician.
