2023 | Interactive Installation
Synaesthesia is an interactive audio/visual installation that premiered in Tokyo in November, 2023. Built in Touch Designer and VCV rack, it uses multiple channels of generative video and audio that influence each other in several ways. A heartfelt thank you to EarthQuaker Devices for the opportunity to bring this work to life.
Synesthesia refers to a phenomenon where one sense involuntarily triggers another—for example, hearing a sound might evoke an image or color, or listening to words might conjure shapes or visuals.
In this installation, your movements directly influence and transform the music and visuals. Both the music and visuals are generated in real time, ensuring that no two experiences are ever the same.
This illustration demonstrates how movements interact with sound and visuals in Synaesthesia. When movement is detected:
1. Sound: Each movement is counted as an event, triggering a note in the melody and shaping its rhythm in real-time.
2. Visual: Differences between the current frame and the previous frame are analyzed, generating particles that float and trace the path of the movement.
Users can see and hear the direct impact of their movements, creating an immersive experience where sound and visuals harmonize seamlessly.
This illustration represents the system of sound inputs and visual controls in Synaesthesia. From user movements to playing instruments, various interactions offer unique immersive experiences. For example, user movements generate melodies, while the VCV Rack in the system produces beats. These beats influence the contrast of the visuals. Furthermore, playing the guitar installed in the installation changes the colors of the visuals and the size of the particles. Additionally, manipulating the guitar pedals (effects units) alters the glitch levels of the visuals and the speed of the particles.
