Pat McMaster

duedo21 (2020)

21 equal temperament electroacoustic composition for eight-channels. Sound parameters and spatialisation directed through sonification of data captured from five matches of the board game Blokus Duo played between the composer and an artificial intelligence.

duedo21 (2020)

Introduction

During my the composition of The Emperor’s Visit, I experimented with coding a simple artificial intelligence based on Markov chains which would generate new sounds based on my musical performance, and whose gestures would in turn inspire me in the manner of a live collaboration. This compositional and improvisation framework based on a non-human duo was at the heart of what I wished to explore further in this final assignment.

Blokus Duo

Blokus is a strategic, tile-laying board game for up to four players developed by French mathematician, biophysicist and artist Bernard Tavitian in 2000. Each player has a fixed inventory of uniquely shaped, 1-to-5-square polyomino pieces. All of a given player’s pieces are the same colour. The goal is to fit as many pieces as possible on the grid-like board, with alternating piece placement rounds cycling between players. After the first piece is placed, each subsequent piece a player places must touch at least one previously placed piece, but only at the corners/diagonally. Pieces of the same colour cannot touch along a flat side. Pieces from different players are allowed to contact each other without restriction. Played pieces cannot be moved or removed from the board, and if a player is unable to place any of their remaining pieces on the board, they must forfeit their turn. A winner is determined once all players have exhausted their ability to place pieces, with points awarded based on how many pieces each player has left in their inventory.

The Blokus Duo variant for two players uses a 14×14 grid as its base with 21 unique polyominoes per player. With its deceptively simple rules, the game is actually quite strategic and its responsive back-and-forth gameplay was a perfect jumping-off point for the framework I was looking to develop,

Compositional Elements

Wuxing – Five Phases

Using the online Blokus adaptation Blokee I played five games of Blokus Duo against an artificial intelligence. For each of these games I adopted a different strategy, inspired by the five phases, represented by elements, of the Chinese philosophical concept Wuxing – according to the ancient knowledge preserved in Shūjīng (Book of Documents), “water is said to soak and descend; fire is said to blaze and ascend; wood is said to curve or be straight; metal is said to obey and change, earth is said to take seeds and give crops.”

I created video recordings of each of these games using screen capture software and took note of the timing and order of the pieces played by both myself and the computer-controlled player, as well as the positions and orientations of each of the polyominoes throughout the game. This allowed me to build a robust, five-phase dataset which I could use as the basis for my parameter mapping.

  • My final dataset of captured information from each of the five phases included the following elements:
  • Game / phase
  • Round
  • Polyominoes played by me
  • X and Y coordinates of each component square(s) of every polyomino played
  • Polyominoes played by CPU
  • X and Y coordinates of the initial/primary component square of every polyomino played by the CPU
  • Time elapsed between rounds

Parameter Mapping

While mapping a dataset to sound parameters is certainly quite discretional, several elements of the game’s design were able to be used more-or-less directly. Being played on a 14×14 grid meant that the concept of coordinate graphing was intrinsic, and for parameters such as spatial position a simple 1:1 relationship between polyomino placement and sound source placement was possible. After consideration and experimentation of mapping many different parameters, I ultimately settled upon the following four broad parametric categories:

  • Sound generator and related parameters
  • Frequency and pitch relationship
  • Amplitude dynamics
  • Spatialisation

Improvisation

Although the framework, parameters and an approximate timeline were assembled following the five-phase recordings of the Blokus games, I wanted to allow myself the flexibility to shape the more dynamic elements of the composition – gestural variations in timbre and pitch, motion and evolution, etc. For sound parameters that were not directly mapped by the outcome of the games, I relied on my improvisational background to react to and sculpt the piece, further exploring the idea of an unconventional duo which has been central to my work this year.

Parameter Mapping

Polyominoes

There are 21 unique polyominoes available to each player. I began by numbering both the polyominoes themselves in addition to their individual component squares. The table below illustrates the pieces and the “order” in which their component squares were assigned parameters throughout the parameter-mapping process.

Sound generator and related parameters

There are four classes of polyomino available to each player: five-square (pentomino), four-square (tetromino), three-square (tromino), two-square (domino) and one-square (monomino). Since my data is generated based on each of a given polyomino’s component squares, it follows that pentominoes offer more possible parameters to map than tetrominoes and so on. I scaled the assignment of polyominoes to sound generators based on this complexity, and developed the following mapping:

Polyomino Sound generator Parameters (values determined by component square position on 14×14 grid)
monomino White noise Amplitude
domino Filtered noise Filter frequency, filter resonance
tromino Wavetable synthesis Frequency, wavetable scanning LFO depth and rate
tetromino Granular synthesis Frequency, grain size/shape, grain amount, inter-onset time
pentomino FM synthesis Frequency, mod. frequency, mod. waveform, mod. amount, timbre

Although each round involves two polyominoes being played on the board – one by myself and one by the computer-controlled player – I exclusively used the data collected from my own turn to select the sound generator and parameters using the above table. The data collected from the computer-controlled player’s contribution to the term was used for amplitude dynamics parameters.

Frequency and pitch relationship

For this composition I wanted to build a direct relationship between the choice of polyominoes and the pitch-relationships between sounds.  This gave me an opportunity to explore a temperament using twenty-one equal divisions of the octave (21 EDO or 21-TET), and to assign each individual polyomino an interval from this system of tuning.

The reference table below lists the intervals and their offset in cents, which was more practical than using their ratio when precisely tuning the sounds associated with each sound element.

 

Degree Interval Cents Degree Interval Cents
0 Unison 0 11 Wide Tritone 628.57
1 Subminor 2nd 57.14 12 Grave 5th 685.71
2 Minor 2nd 114.29 13 Fifth-Sixth 742.86
3 Submajor 2nd 171.43 14 Minor 6th 800
4 Supermajor 2nd 228.57 15 Neutral 6th 857.14
5 Subminor 3rd 285.71 16 Supermajor 6th 914.29
6 Neutral 3rd 342.86 17 Subminor 7th 971.43
7 Major 3rd 400 18 Supraminor 7th 1028.57
8 Third-Fourth 457.14 19 Major 7th 1085.71
9 Acute 4th 514.29 20 Supermajor 7th 1142.86
10 Narrow Tritone 571.43

Each non-noise sound element is paired with a version of itself that has been offset by the interval determined by the representative polyomino. Given the microtonal nature of the tuning, smaller intervals produce a chorus-like sound and larger intervals create harmonic relationships that are not typically heard in Western music.

Amplitude dynamics

Each of the computer-controlled player’s 21 available polyominoes was assigned a suggested AD amplitude envelope, ie. very short attack, medium decay. For each round, the location of the computer-controlled player’s primary component square on the 14×14 grid was mapped to initial and end amplitude levels.

Spatialisation

The playing area and its X/Y grid were a perfect fit for choosing the starting point for placing sound sources within a multichannel environment. The primary component square of each polyomino determined the spatial starting position of the sound source – a monomino placed in row 14, column 14 would appear at the extreme front and right of the listener, while one placed in row 7, column 7 would appear in the center of the space. Spatial movement as well as the sound source’s perceived ‘size’ within the space were part of the improvisation process.

Recording and composition

Recording

I consulted the spreadsheet containing the Blokus Duo data-set as both a checklist and score, allowing me to sort the component sounds I needed to record by generator source, duration, etc. and providing me with many of the parameter values to use when building the patches. Noise sources were created using a white noise sample and bandpass filtering where appropriate. Wavetable elements were recorded using the Waldorf XT synthesizer with several harmonic-rich wavetables and a complex modulation-matrix controlling the shape, depth and rate of the LFO controlling the wavetable indices. Granular sounds were produced by the Haken Continuumini and further processed by the Argotlunar real-time delay-line granulator developed by Michael Ourednik. All other sounds were recorded from the Buchla Music Easel with the help of some additional spectral morphing.

Composition

Component recordings were labelled according to the Phase and Round and then arranged in the timeline chronologically using the timing noted from the video recordings as a guide. As recording durations and envelopes varied, sound sources often overlapped, which produced a more pleasing effect than would a strict adherence to the one-round-at-a-time progression of the game. The component Buchla recordings were extended to provide a backbone for the composition, with the noise, wavetable and granular elements shifting between foreground and background.

The piece was spatialised octophonically using a familiar circular arrangement illustrated below, with speakers being placed every 45 degrees and organized into the following stereo groupings: Front, Front-Side, Rear-Side, Rear.

Final thoughts

After working with the Arduino-controlled looper bot in The Emperor’s Visit, it was exciting and rewarding to take a step further towards an abstraction of interface between composition and human-AI interactions. A great deal of work was involved in working out the parameters to be mapped and what to map them too, and also in recording and converting the datasets into parameter values, but after much trial and error the system I developed is certainly acceptable for a short composition. It could conceivably be reworked to generate a contemporary music score for acoustic or mixed instrumentation, which would also be of interest to me.

In keeping with the Chinese cosmological theme of my live EA performance, it was fascinating to learn about Wuxing, and to think about how to adopt a strategy that was inspired by each of the five phases. The idea that say, an aggressive or defensive playstyle, can directly influence the resulting music was rewarding to explore.

It’s been over fifty years since the real-time game sonification of Reunion, wherein a match between John Cage and Marcel Duchamp on a chess-board-as-audio-switch device built by Lowell Cross premiered in Toronto. In researching this assignment I discovered a number of (non-realtime) chess notation sonification projects which have been developed, both as personal projects and in academia, but they seem to struggle with many of the elements I did (parameter recording and data manipulation) and are fairly limited in the music they can output. With its 21 unique polyominoes and 14×14 grid, Blokus Duo offers quite a bit more possibilities for parametric mapping. I can imagine a future iteration of these ideas involving a sophisticated screen-scraping application to allow for real-time capture and sharing of data from matches. Additionally a system could be designed to ingest that data-stream and either interface with hardware or software instruments (via OSC or MIDI) or to produce sounds directly (using Supercollider or Max/MSP).

Program Note

Pat McMaster – duedo21

2020, duration: 5‘17“

21 equal temperament electroacoustic composition for eight-channels. Sound parameters and spatialisation directed through sonification of data captured from five matches of the board game Blokus Duo played between the composer and an artificial intelligence.

Keywords: artificial intelligence, sonification, collaborative, duo, computer-assisted, improvisation, multichannel, surround, octophonic, microtonal, 21-EDO, 21-TET, equal temperament, alternate tuning, board games 

To download a multichannel version of this composition, please get in touch.

duedo21 (2020)

Written by

Pat McMaster

in

21 equal temperament electroacoustic composition for eight-channels. Sound parameters and spatialisation directed through sonification of data captured from five matches of the board game Blokus Duo played between the composer and an artificial intelligence.

duedo21 (2020)

Introduction

During my the composition of The Emperor’s Visit, I experimented with coding a simple artificial intelligence based on Markov chains which would generate new sounds based on my musical performance, and whose gestures would in turn inspire me in the manner of a live collaboration. This compositional and improvisation framework based on a non-human duo was at the heart of what I wished to explore further in this final assignment.

Blokus Duo

Blokus is a strategic, tile-laying board game for up to four players developed by French mathematician, biophysicist and artist Bernard Tavitian in 2000. Each player has a fixed inventory of uniquely shaped, 1-to-5-square polyomino pieces. All of a given player’s pieces are the same colour. The goal is to fit as many pieces as possible on the grid-like board, with alternating piece placement rounds cycling between players. After the first piece is placed, each subsequent piece a player places must touch at least one previously placed piece, but only at the corners/diagonally. Pieces of the same colour cannot touch along a flat side. Pieces from different players are allowed to contact each other without restriction. Played pieces cannot be moved or removed from the board, and if a player is unable to place any of their remaining pieces on the board, they must forfeit their turn. A winner is determined once all players have exhausted their ability to place pieces, with points awarded based on how many pieces each player has left in their inventory.

The Blokus Duo variant for two players uses a 14×14 grid as its base with 21 unique polyominoes per player. With its deceptively simple rules, the game is actually quite strategic and its responsive back-and-forth gameplay was a perfect jumping-off point for the framework I was looking to develop,

Compositional Elements

Wuxing – Five Phases

Using the online Blokus adaptation Blokee I played five games of Blokus Duo against an artificial intelligence. For each of these games I adopted a different strategy, inspired by the five phases, represented by elements, of the Chinese philosophical concept Wuxing – according to the ancient knowledge preserved in Shūjīng (Book of Documents), “water is said to soak and descend; fire is said to blaze and ascend; wood is said to curve or be straight; metal is said to obey and change, earth is said to take seeds and give crops.”

I created video recordings of each of these games using screen capture software and took note of the timing and order of the pieces played by both myself and the computer-controlled player, as well as the positions and orientations of each of the polyominoes throughout the game. This allowed me to build a robust, five-phase dataset which I could use as the basis for my parameter mapping.

  • My final dataset of captured information from each of the five phases included the following elements:
  • Game / phase
  • Round
  • Polyominoes played by me
  • X and Y coordinates of each component square(s) of every polyomino played
  • Polyominoes played by CPU
  • X and Y coordinates of the initial/primary component square of every polyomino played by the CPU
  • Time elapsed between rounds

Parameter Mapping

While mapping a dataset to sound parameters is certainly quite discretional, several elements of the game’s design were able to be used more-or-less directly. Being played on a 14×14 grid meant that the concept of coordinate graphing was intrinsic, and for parameters such as spatial position a simple 1:1 relationship between polyomino placement and sound source placement was possible. After consideration and experimentation of mapping many different parameters, I ultimately settled upon the following four broad parametric categories:

  • Sound generator and related parameters
  • Frequency and pitch relationship
  • Amplitude dynamics
  • Spatialisation

Improvisation

Although the framework, parameters and an approximate timeline were assembled following the five-phase recordings of the Blokus games, I wanted to allow myself the flexibility to shape the more dynamic elements of the composition – gestural variations in timbre and pitch, motion and evolution, etc. For sound parameters that were not directly mapped by the outcome of the games, I relied on my improvisational background to react to and sculpt the piece, further exploring the idea of an unconventional duo which has been central to my work this year.

Parameter Mapping

Polyominoes

There are 21 unique polyominoes available to each player. I began by numbering both the polyominoes themselves in addition to their individual component squares. The table below illustrates the pieces and the “order” in which their component squares were assigned parameters throughout the parameter-mapping process.

Sound generator and related parameters

There are four classes of polyomino available to each player: five-square (pentomino), four-square (tetromino), three-square (tromino), two-square (domino) and one-square (monomino). Since my data is generated based on each of a given polyomino’s component squares, it follows that pentominoes offer more possible parameters to map than tetrominoes and so on. I scaled the assignment of polyominoes to sound generators based on this complexity, and developed the following mapping:

Polyomino Sound generator Parameters (values determined by component square position on 14×14 grid)
monomino White noise Amplitude
domino Filtered noise Filter frequency, filter resonance
tromino Wavetable synthesis Frequency, wavetable scanning LFO depth and rate
tetromino Granular synthesis Frequency, grain size/shape, grain amount, inter-onset time
pentomino FM synthesis Frequency, mod. frequency, mod. waveform, mod. amount, timbre

Although each round involves two polyominoes being played on the board – one by myself and one by the computer-controlled player – I exclusively used the data collected from my own turn to select the sound generator and parameters using the above table. The data collected from the computer-controlled player’s contribution to the term was used for amplitude dynamics parameters.

Frequency and pitch relationship

For this composition I wanted to build a direct relationship between the choice of polyominoes and the pitch-relationships between sounds.  This gave me an opportunity to explore a temperament using twenty-one equal divisions of the octave (21 EDO or 21-TET), and to assign each individual polyomino an interval from this system of tuning.

The reference table below lists the intervals and their offset in cents, which was more practical than using their ratio when precisely tuning the sounds associated with each sound element.

 

Degree Interval Cents Degree Interval Cents
0 Unison 0 11 Wide Tritone 628.57
1 Subminor 2nd 57.14 12 Grave 5th 685.71
2 Minor 2nd 114.29 13 Fifth-Sixth 742.86
3 Submajor 2nd 171.43 14 Minor 6th 800
4 Supermajor 2nd 228.57 15 Neutral 6th 857.14
5 Subminor 3rd 285.71 16 Supermajor 6th 914.29
6 Neutral 3rd 342.86 17 Subminor 7th 971.43
7 Major 3rd 400 18 Supraminor 7th 1028.57
8 Third-Fourth 457.14 19 Major 7th 1085.71
9 Acute 4th 514.29 20 Supermajor 7th 1142.86
10 Narrow Tritone 571.43

Each non-noise sound element is paired with a version of itself that has been offset by the interval determined by the representative polyomino. Given the microtonal nature of the tuning, smaller intervals produce a chorus-like sound and larger intervals create harmonic relationships that are not typically heard in Western music.

Amplitude dynamics

Each of the computer-controlled player’s 21 available polyominoes was assigned a suggested AD amplitude envelope, ie. very short attack, medium decay. For each round, the location of the computer-controlled player’s primary component square on the 14×14 grid was mapped to initial and end amplitude levels.

Spatialisation

The playing area and its X/Y grid were a perfect fit for choosing the starting point for placing sound sources within a multichannel environment. The primary component square of each polyomino determined the spatial starting position of the sound source – a monomino placed in row 14, column 14 would appear at the extreme front and right of the listener, while one placed in row 7, column 7 would appear in the center of the space. Spatial movement as well as the sound source’s perceived ‘size’ within the space were part of the improvisation process.

Recording and composition

Recording

I consulted the spreadsheet containing the Blokus Duo data-set as both a checklist and score, allowing me to sort the component sounds I needed to record by generator source, duration, etc. and providing me with many of the parameter values to use when building the patches. Noise sources were created using a white noise sample and bandpass filtering where appropriate. Wavetable elements were recorded using the Waldorf XT synthesizer with several harmonic-rich wavetables and a complex modulation-matrix controlling the shape, depth and rate of the LFO controlling the wavetable indices. Granular sounds were produced by the Haken Continuumini and further processed by the Argotlunar real-time delay-line granulator developed by Michael Ourednik. All other sounds were recorded from the Buchla Music Easel with the help of some additional spectral morphing.

Composition

Component recordings were labelled according to the Phase and Round and then arranged in the timeline chronologically using the timing noted from the video recordings as a guide. As recording durations and envelopes varied, sound sources often overlapped, which produced a more pleasing effect than would a strict adherence to the one-round-at-a-time progression of the game. The component Buchla recordings were extended to provide a backbone for the composition, with the noise, wavetable and granular elements shifting between foreground and background.

The piece was spatialised octophonically using a familiar circular arrangement illustrated below, with speakers being placed every 45 degrees and organized into the following stereo groupings: Front, Front-Side, Rear-Side, Rear.

Final thoughts

After working with the Arduino-controlled looper bot in The Emperor’s Visit, it was exciting and rewarding to take a step further towards an abstraction of interface between composition and human-AI interactions. A great deal of work was involved in working out the parameters to be mapped and what to map them too, and also in recording and converting the datasets into parameter values, but after much trial and error the system I developed is certainly acceptable for a short composition. It could conceivably be reworked to generate a contemporary music score for acoustic or mixed instrumentation, which would also be of interest to me.

In keeping with the Chinese cosmological theme of my live EA performance, it was fascinating to learn about Wuxing, and to think about how to adopt a strategy that was inspired by each of the five phases. The idea that say, an aggressive or defensive playstyle, can directly influence the resulting music was rewarding to explore.

It’s been over fifty years since the real-time game sonification of Reunion, wherein a match between John Cage and Marcel Duchamp on a chess-board-as-audio-switch device built by Lowell Cross premiered in Toronto. In researching this assignment I discovered a number of (non-realtime) chess notation sonification projects which have been developed, both as personal projects and in academia, but they seem to struggle with many of the elements I did (parameter recording and data manipulation) and are fairly limited in the music they can output. With its 21 unique polyominoes and 14×14 grid, Blokus Duo offers quite a bit more possibilities for parametric mapping. I can imagine a future iteration of these ideas involving a sophisticated screen-scraping application to allow for real-time capture and sharing of data from matches. Additionally a system could be designed to ingest that data-stream and either interface with hardware or software instruments (via OSC or MIDI) or to produce sounds directly (using Supercollider or Max/MSP).

Program Note

Pat McMaster – duedo21

2020, duration: 5‘17“

21 equal temperament electroacoustic composition for eight-channels. Sound parameters and spatialisation directed through sonification of data captured from five matches of the board game Blokus Duo played between the composer and an artificial intelligence.

Keywords: artificial intelligence, sonification, collaborative, duo, computer-assisted, improvisation, multichannel, surround, octophonic, microtonal, 21-EDO, 21-TET, equal temperament, alternate tuning, board games 

To download a multichannel version of this composition, please get in touch.

More posts