An Orderly Noise
Sound installation w/ Luis Lecea
An Orderly Noise is an 8-channel spatial sound installation based on field recordings taken with contact microphones of the electrical hums from the COVRA nuclear waste disposal facility in Borselle, the Netherlands. The piece considers the architecture of the space, and the back-end infrastructure keeping the operational ‘at work’.
The project was made possible through participation in Sound Ecologies Lab which examined interactions between industy, engineering and ecology as a part of FIBER’s project Reassemble, and was shown at Het Veem, Amsterdam.
A Binaural recording can be heard here.
Algorithmic Listening: Voice and Affect
What does the ‘Intelligent’ part of AI mean in the context of analyses of voice and speech data? This talk performed over Dutch Design Week 2021 explains and explores the ‘black box’ of technologies imbued with these skills. Using speech-to-text and sentiment analysis systems in real-time I question the ethical governance of commercially-driven applications, from their training datasets to their implementation as a bases for decision-taking.
Algorithmic Listening: Voice and Affect
Thesis publication; available to read
The development of AI tools with the skills of emotional intelligence—often dubbed ‘affective computing’—is no longer a distant goal. Sensors and algorithms that can recognise and respond to emotions are already being marketed by a range of businesses, from start-ups to the major tech companies. Audio recording of paralanguage—how we convey meaning and feelings through volume, pitch, speed, rhythm and pauses when we speak, as well as by non-verbal sounds like sighs, cries and gasps—has provided a relatively cheap source of data for such AI tools. However, paralanguage is often more ambiguous and culturally influenced than written language. My analysis of a dozen such applications and the companies that market them indicates a marked lack of transparency about the data and research from which they are derived, raising concerns about their robustness, especially given the ‘black box’ nature of the deep-learning techniques employed.
What does it mean to be ever surrounded by sensory devices and AI tools which have the capability to ‘listen-in’? This thesis explores some of the psychological, political and cultural implications. Notwithstanding the considerable momentum for developing and applying paralinguistic computing applications to make life easier or reduce costs and the increasing ‘everydayness’ of such devices, my findings point to the desirability of strengthening ethical and quality-assurance frameworks. As well as fears about personal privacy, there is a risk of harm to individuals from decision-making based on algorithms that are much less evidence-based than companies may often claim.
60pp; Risoprint on 90gsm, black & teal
Video and sound installation
Working with an audio data corpus of unscripted telephone speech used for teaching machine learning models, I explore the process of developing and applying an algorithm based on voice and speech data in this piece. The results point up a number of issues of interpretability, where missing information or ambiguities can result in biased, stereotypical and unreliable results. How this technology is shaping the way knowledge and decisions are produced is explored in a sound essay, alongside which a sound installation plays data extracted by algorithms that mimic those examined.
A variation of this work will be on show at Dutch Design Week 2021.
To the Mother of All Bowls
Using a Teensy 3.2 microcontroller, audio adaptor, inbuilt mic and amp mounted on a breadboard I applied a Fast Fourier transform (FFT) to the audio signals of a poetry reading by Judy Grahn. The FFT takes as input a number of samples from the signal and produces as output the intensity at corresponding frequencies. Pulling this data into P5.js I created an audio-reactive visualisation for the reading.
Cyantypes w/ Martina Eddone
Paying attention to the “empty” spaces in the urban fabric of a local neighbourhood was the starting point for this project; an exercise in the simple act of noticing. Martina Eddone and I collected fragments of flora from voids—spaces that live alongside the infrastructure that are in continual transformation and regeneration. Through the cyanotype printing technique we documented small details of these fragments. The results were on show as a part of Meeting Grounds at Onamatopee Project Space.
Video essay w/ Emma Schep + Hi-Kyung Eun
This video considers the future of genetic data storage. Recently the global market for direct-to-consumer DNA testing kits, which provide insights into ancestry and genetic health, have boomed. However the businesses selling these services often harness customer data in unscrupulous ways. In the very near future it will be possible to fully sequence the genetic information of any living organism on a mass scale, but how will all this data be handled? This work considers a design scenario of a future decentralised bank system that sees access to scientific knowledge as a right, whilst also providing individuals with increased autonomy over how their personal information is shared.
A Generative Adversarial Network (GAN) trained on an image dataset of 7782 spectrograms used to teach machine learning models to detect and classify seven emotions from interpretations of vocal cues. The GAN synthetically-generates pseudo-random spectrograms based on the original dataset.