*
*

Larry Polansky

Bregman Electro-Acoustic Music Studio

Dartmouth College

larry.polansky@dartmouth.edu

http://onyx.dartmouth.edu/~larry/polansky.html

Tom Erbe

School of Music

California Institute for the Arts

tre@music.calarts.edu

http://music.calarts.edu/~tre

**
Abstract**

*
Soundhack *is a widely used Macintosh program that performs a wide variety
of soundfile manipulations. The program, authored by Tom Erbe,* *includes
soundfile type conversion, spectral dynamics processing, a varispeed/sample
rate converter, soundfile convolution, ring modulation, a phase vocoder, a
binaural filter and an amplitude analysis and gain change module.

With versions 0.8 and above of *Soundhack*, the authors have implemented a
system of spectral mutation functions, which allow for a wide variety of
transformations between soundfiles in the spectral domain. This article,
largely drawn from the *Soundhack* manual (Erbe, 1994), describes the
program's spectral mutation features in detail.

**
Introduction**

Tom Erbe's* Soundhack* is a widely used program for soundfile manipulation
on the Macintosh. Most of *Soundhack's* functions are based on a flexible
FFT/IFFT process, with user control over the size, shape and resolution of the
FFT and windowing. The program offers high quality soundfile transformations
such as convolution, phase vocoder (allowing time stretching and pitch
shifting), binaural filters, spectral dynamics processing, and other
functions.

Larry Polansky's mutation functions (Polansky and McKinney, 1991; Polansky,
1992) are generalized morphological transformations which create
*mutations* "between" morphologies (ordered sets). The mutation functions,
originally designed for higher-level morphologies (such as melodies, durations
series or statistical profiles of a parametric interval over time), considers
intervals between elements of a morphology, and separate those intervals into
magnitude and sign (direction).

Each of the mutation functions "pastes" or interpolates, in a different way,
the sign or magnitude of one morphology into the sign or magnitude of the
other, creating a third morphology which is some measurable combination of the
two sources. In *Soundhack* Version 0.8 and above, the mutation functions
are implemented in the spectral domain, where the individual FFT frames of
soundfiles are considered to be the morphologies.

__
__

**
Introduction to the Mutation Functions**

Each spectral mutation function produces a different timbral "cross-fade."
Each function takes 2 soundfiles (*source* and *target*) and returns
a third soundfile, (*mutant*). The functions operate on the
phase/amplitude pair of each frequency band of the source and target spectra.
The output of the functions is a phase/amplitude pair for each frequency band
in the mutation soundfile.

Each output phase/amplitude pair in the mutation is some "combination" of the
phase/amplitude pairs of the source and target, for the corresponding frequency
band. The functions work on the sign (*contour*) or the *magnitude*
of an interval, or both. They change completely a selected number of bands from
the source to the target (*irregular*) or partially change all frames
(*uniform*).

The following diagram shows interval computation for three successive
FFT-frames of three spectral bands of one soundfile. Maximum amplitude for a
band is 1.0. This computation uses *adjacency interval*s.

The degree of mutation is called *½,* specified as a single value
between 0 (completely the "source") to 1 (completely the "target") or as a
function over the duration of the resultant soundfile. Intermediary values for
*½* create spectra which are "interpolations" (not necessarily
linear) between the two input spectra. The mutation is thus a soundfile
spectrally "between" the source and the target in some way (specified by the
mutation type) and to some degree (specified by *½*..

Five different mutation functions are used in *Soundhack*, along with two
concatenations of them. They are called: *Uniform Signed Interval
Mutation* (USIM), *Irregular Signed Interval Mutation* (ISIM),
*Uniform Unsigned Interval Mutation* (UUIM), *Irregular Unsigned
Interval Mutation* (IUIM), and the *Linear Contour Mutation* (LCM).
These functions are described in detail in Polansky (1992) and Polansky and
McKinney (1991).

Each mutation function deals with the *interval* between the *n[th]>* and *n+1[th ]*frames (time
slices) of the *i[th]* frequency band of a spectra.
That is, the amplitude in a given band for a given frame of the mutation is a
result of some transformation on the source and target amplitude intervals
between the current frame and the previous. This is called *adjacency
interval*.

In *Soundhack*, the user may select an alternative approach, *absolute
interval*, which sets an absolute amplitude to which all intervals are taken
(somewhat anchoring the mutation trajectory). The following diagram shows
absolute interval computation for three successive FFT-frames of three spectral
bands of one soundfile, absolute intervals computed to the value .5.

**
The Different Mutation Functions**

The *USIM* and the *ISIM* are the simplest mutations, a spectral
cross-fade and spectral band-replacement, respectively. The *USIM*
("uniform") operates on each spectral band interval for every FFT-frame,
cross-fading amplitude differences between corresponding spectral bands. The
*ISIM* replaces the amplitude interval of the source by that of the target
for each frame to create that of the mutation (simply adding the signed
interval to the amplitude of the previous spectral frame).

In irregular mutations, the decision of whether or not to mutate (replace the
source amplitude by the target) is determined by *½*, treated as
a probabilistic value. In other words, the *ISIM* (and *IUIM* and
*LCM*) is a stochastic spectral band replacement based on the value of
*½.*

The *UUIM* and the *IUIM* are two different ways of "pasting" or
interpolating the amplitude differences of the target spectra onto the sign of
the source, resulting in a mutation which, when completely mutated, is still
some combination of the source and target. The *UUIM* interpolates between
the source and target interval, but retains the sign, or direction of the
source interval. A completely mutated spectra by the *UUIM* has the
magnitude intervals of the target but the signs of the source intervals. The
*IUIM* is the irregular version of the *UUIM*, pasting magnitude
intervals stochastically, while retaining sign.

The *LCM*, perhaps the most unusual sounding and unpredictable mutation,
does the opposite of the *IUIM*, pasting the signs of the target onto the
magnitudes of the source, generally resulting in an unrecognizeable but often
sonically interesting spectral transformation.

It is important to stress that even with *½ = 1*, the *UUIM*,
*IUIM*, and *LCM* will not result in a mutation spectra that is the
same as the target. Rather, the spectra will be some sign- or
magnitude-equivalent "image" of the target spectra.

The mutation functions can be classified as follows:

**
Uniform Sign Magnitude**

**
USIM **yes yes yes

**
ISIM **no yes yes

**
UUIM **yes no yes

**
IUIM **no no yes

**
LCM **no yes no

Note that the opposite of "irregular" is "uniform." Uniform mutations do
something to every frequency band. How much they do depends on *½*
. Irregular mutations change a given frequency band completely from source to
target (sign, magnitude or both), but the number of frequency bands they
operate on depends on *½* . In irregular mutations,
*½* may be thought of as the "probability of replacement" for a
given band (the percentage of bands replaced). In uniform mutations,
*½* functions as an interpolation value (degree of
"between-ness"). Note that there is no *UCM* since it wouldn't make
sense: contour is defined in the context of these mutations as being up, down,
or equal.

**
Concatenated Mutations**

Irregular mutations may be combined, or *concatenated*, with their
complementary forms to produce more complete transformations. For example, the
*LCM*, which only changes the sign of intervals, can be concatenated with
either the *UUIM* or the *IUIM*, each of which transform interval
magnitude. In the former each interval is crossfaded, in the latter some
*number* of intervals are changed completely each time.

Concatenated mutations work like "pipes": for each frame the output of the
first mutation is sent to the input of the next. For the *LCM/IUIM*, each
part of the concatenation mutates an independent set of stochastically chosen
spectral bands. Although the end result of the concatenated mutations
*LCM/IUIM* and *LCM/UUIM* will be the same as the *USIM* or
*ISIM*, the intermediate mutation trajectory will be different. The
concatenated mutations tend to be jagged, since at least one of the mutations
is operating stochastically.

**
**

**
An example of a concatenated mutation**

The following examples are an attempt to visualize an *LCM/IUIM* mutation
of a ramp function into a more complex one, a highly randomized triangular
function. Individual values for these functions may be considered to be
spectral band amplitudes for one FFT frame. That is, the first function (which
might be thought of as one frame of a sawtooth wave) is mutated into the last
(one frame of an extremely noisy triangle wave).

The first visualization is a conventional "waterfall" plot, where the z-axis is
time (assuming the mutations are performed over time to effect the
transformation of one FFT into another). The x-axis is FFT-band, and the y-axis
is amplitude. This picture shows the result of ten intermediary mutations
between the source and target. The *LCM/IUIM* uses an absolute index
around the midpoint of the "sawtooth."

The second visualization shows only four of the intermediary mutations.

**
Theoretical Definitions of the Mutations**

The mutation functions are mathematically defined below. *S* and *T*
are the source and target soundfiles. *Si*, *Ti* are the amplitudes
for a given frequency band of the FFT for the *ith* frame of the sound.
*Sj*, *Tj* are either the amplitudes of the same band in the previous
frame (Relative Interval) or some absolute amplitude (Absolute Interval).
*Mi* is the new amplitude of the given frequency band of the current frame
of the output sound, *Mj* is the amplitude for that band in the previous
frame of the output sound (Relative Interval), or some absolute amplitude value
between 0.0 and 1.0 (Absolute Interval). *Tint*, *Sint* and
*Mint* are the signed magnitude intervals between the amplitude of the
current frame for a given band, and the amplitude of that band in the previous
frame (Relative Interval) or to some fixed amplitude (Absolute Interval).

Each equation applies to one frequency band of the source, target and mutant
soundfile spectra. For all functions, *Si*, *Ti*, and *Mi* run
from 0 to the number of frequency bands in the FFT.

**
**

The two basic interval functions (for sign and magnitude differences between the same frequency band of two FFT frames) are:

**
**

**
** and

The five fundamental mutation functions are :

* Uniform Signed Interval Magnitude (*USIM*)

*
Mi = Mj + (Sint*)* + ½ * (Tint* *- Sint)*

*
*

* Uniform Unsigned Interval Magnitude (*UUIM*)

*
Mi = Mj + Ssgn * (Smag* *+ ½ * *|*Tmag* *-
Smag*|*)*

-- where *Sint* and *Tint* are

*
(Ssgn* * *Smag) *

and * *

*
(Tsgn * Tmag)* respectively

* Linear Contour Mutation (*LCM*)

*
Mi = Mj + Tsgn * Smag*

(for mutated intervals)

*
Mi = Mj + Ssgn * Smag*

(general form for non-mutated intervals)

* Irregular Unsigned Interval Magnitude (*IUIM*)

*
Mi = Mj + Ssgn * Tmag*

(for mutated intervals; non-mutated intervals

same as *LCM* above)

*
*

* Irregular Signed Interval Magnitude (*ISIM*)

*
Mi = Mj + Tsgn * Tmag*

(for mutated intervals; non-mutated intervals

same as *LCM* above)[]

*
*

In *absolute interval *mutations, *Mj*, the absolute amplitude to
which the new interval is added, is interpolated between absolute values for
source and target, according to the value for *½*. The irregular
mutations are in two forms: one for the case when that particular frequency
band is chosen for mutation, one for when it is not.

**
Visual Examples of the Mutations**

The example below shows the *USIM* from a sawtooth to a slightly and
randomly rippled triangle.

__
__

The next seven examples show each of the mutations on the same two simple
functions, plotted over up to ten gradually increasing values for
*½* from 0 to 1. In the case of the incomplete mutations
(*LCM*, *UUIM*, *IUIM*) the final triangular function is not
shown (nor arrived at). For the sake of visual clarity in some of the more
complex mutations, fewer intermediary functions are shown. All mutations use
absolute intervals of .5, with maximum amplitude 1.0.

**
**

**
**

**
**

**
The Mutation Functions Implemented in Soundhack**

The following is a description of the main *Soundhack* spectral mutation
function screen and each of its functions. The main screen is shown below.

**
Type**

The *Type* box allows for selection between the seven quite different
mutation functions: *USIM* (Uniform Signed Interval Mutation), *ISIM*
(Irregular Signed), *IUIM* (Irregular Unsigned), *UUIM* (Uniform
Unsigned), *LCM* (Linear Contour Mutation), and the concatenations
*LCM/IUIM* and *LCM/UUIM*.

The simplest mutations are the *USIM* (a simple spectral crossfade) and
the *ISIM* (a spectral replacement). These two mutations, unlike the
*UUIM* and *LCM*, actually arrive at the source or target, depending
on the direction of the mutation. That is, the source or target will actually
be heard with *½* = 0.0 or *½* = 1.0, respectively).
The *IUIM*, *UUIM* and the *LCM* will, with *½* =
1, produce a "sign-equivalent" or "magnitude-equivalent" image of the target.
These "incomplete mutations" mutate either the sign or the magnitude of
intervals between successive spectral bands, but not both. That is, the
mutation will have the same signs or magnitudes as the target, but not both.

As explained above, the concatenations (*LCM/IUIM*, *LCM/UUIM*) apply
the second mutation to the output of the first. The concatenations are
complementary: for each frame the *LCM* mutates sign while the *IUIM*
and *UUIM*

mutate magnitude. Different frequency bands are used for each "stage" of the concatenation, so the mutation trajectory may be unpredictable.

**
Mutation Index ( ½ )**

*
½* determines the amount of spectral mix, from 0 to 1, between the
source and target resulting in the mutant. *½* = 0 results in all
source file, *½* = 1 all target. *½* may vary over
the course of the mutation. A constant index will result in a sound which is a
spectral mix of the source and target. More dynamic sounds are produced with an
index function, which changes *½* over time.

**
Absolute Interval**

There are two methods used by the mutation functions to compute intervals
between frequency bands: *Absolute* (the default) and *Relative*. The
Absolute Interval box may be checked or unchecked to get these two methods.

If Absolute intervals are used, absolute amplitude values may be specified
between 0.0 and 1.0 for the source and target (Source Abs. Value, Target Abs.
Value), from which all intervals will be taken. The choice of values can
produce interesting effects, often "centering" the frequencies in which the
mutation happens, making the mutations themselves less extreme. If two
different values are used, amplitudes will be "transposed" from the source to
the target. The use of Absolute intervals rather than Relative will be most
noticeable for the *LCM*, *IUIM* and *UUIM*, as well as the
concatenated mutations. Low values (between .1 and .2) are a good place to
start (note that .1 means 1/10th of the total amplitude of the soundfile's
spectra).

If Absolute is unchecked (Relative intervals), each mutation function uses
amplitude intervals between successive frames of the spectra, multiplied by
*½* , to create the corresponding frame of the mutant sound.
Relative interval mutations will tend to "drift," often in extreme ways, and
the *ISIM* and concatenated mutations may never "arrive." However, some
very interesting sonic results may be produced in this way.

**
Delta Emphasis**

If the mutation uses Relative Intervals (Absolute Intervals unchecked), a value
may be set for *Delta Emphasis* (DE). DE allows control over the degree to
which successive mutation intervals are emphasized in the resulting mutant.

DE values range from -1.0 to 1.0, with the default at 0.0 (no emphasis or de-emphasis). For positive DE values, the current frame's intervalic characteristics will be emphasized more than the previous mutant frames. For negative values, the current frame will be "damped," emphasizing the previous information.

One way to think about this is as a way of "slowing down" the mutation: a negative DE value will keep the more chaotic mutations from getting "out of control." A negative DE value will function as a kind of low-pass filter, smoothing out large differences between subsequent spectral frames and averaging the previous spectral frames into the current output. Positive DE values will accentuate the often high-frequency activity of the mutations. Delta Emphasis can be useful in tailoring the relative interval mutations, especially "incomplete" ones.

**
Band Persist (Irregular Mutations Only)**

*
Band Persist* pertains only to irregular mutations: *LCM*, *ISIM*,
*IUIM* and concatenations. It only appears on the screen when irregular
mutations are selected. High values for Band Persist (towards 1.0) will produce
more "stable" mutations. Low values (towards 0.0) will introduce a kind of
frequency pumping at the frame rate. By changing the number of frequency bands,
unusual sonic results are acheived.

As explained above, irregular mutations do not mutate every frequency band for
each FFT frame. *½* determines the percentage of bands that are
mutated for a given frame. If a band is mutated, it completely assumes the
particular characteristic (interval sign or magnitude) of the target interval,
and retains either the sign or the magnitude of the source interval. For
example, the *LCM* takes the sign of the target interval, and "pastes" it
onto the magnitude of the source. However, it only does that for (*½
* #-of-bands*). The selection of which bands to mutate in irregular
mutations is done stochastically, but setting Band Persist high will ensure
that once a band is mutated, it will keep being mutated as long as possible.
That is why a high value for Band Persist will stabilize these highly unusual
mutations, making them a bit more "well-behaved." A good experiment is to try
an irregular mutation (*LCM*, *IUIM*, *ISIM*, the
concatenations) with a fixed *½* , and two different values of
Band Persist, one high and one low.

**
Time Scale Target**

The form of the mutation functions used in *Soundhack* requires that each
soundfile (source, target, mutant) be of equal length. The default technique is
to truncate the longer of the two files, producing a mutant which is the length
of the shorter file. If Time Scale Target is checked, the target soundfile will
be time-stretched or -compressed to be of the same length as the source. Other
techniques are of course possible (including windowing and zero-padding), and
we encourage other software developers to investigate these.

**
Acknowledgements**

Various students at Dartmouth and Mills Colleges, including Chris Langmead, Eric Smith, Steve Berkley, Greg Higgs, Ted Apel and Martin McKinney have provided invaluable assistance over the past few years in helping to formulate both the spectral mutation ideas and accompanying software techniques. Ken Overton, Sergei Kossenko, and Chris Langmead all contributed valuable suggestions to this article. Susan Schwarz of Academic Computing at Dartmouth College helped create some of the figures, using Data Explorer on an RS-6000 computer. Larry Polansky was able to software-prototype some of the mutation functions during a 1994 National Endowment for the Arts sponsored residency at the Mills College Center for Contemporary Music. Dave Madole, Technical Director of the Mills College Center for Contemporary Music was especially helpful in these early stages of software development. Polansky expresses his deep appreciation to Chris Brown and John Bischoff of the Mills CCM for arranging and administering the residency.

**
References**

Erbe, Tom. 1994. *Soundhack Manual*. Lebanon, NH: Frog Peak Music.
(Section on spectral mutation written by L. Polansky)

Polansky, Larry, and McKinney, Martin. 1991. "Morphological Mutation Functions:
Applications to Motivic Transformation and a New Class of Cross-Synthesis
Techniques." *Proceedings of the International Computer Music Conference.
Montreal*. pp. 234-241.

Polansky, Larry. 1992. "More on Morphological Mutation Functions: Recent
Techniques and Developments." *Proceedings of the International Computer
Music Conference*. San Jose. pp. 57-60.