AlloThresher is a multimodal instrument with audiovisual granular synthesis using the gestural interface. Granular synthesis is a sound synthesis method that creates complex tones by combining and mixing the simple micro-sonic elements called grains. 
With two smartphones in both hands, the gestural interface interpreted from the sensors enables you to precisely and intuitively decide and play the parameter of the granular synthesis in real-time. Graphically, the corresponding visuals are generated simultaneously for each granule based on the spectrogram of the sound that morphs and blends dynamically with the gesture. 
By breaking conventional interfaces like knobs and sliders, this seamless connection between modalities utilizes the profound advantage of the gestural interface. Moreover, the presence and gesture become part of the space and the performance so that the audience can observe and cohesively connect the audio, visual, and interface simultaneously.
While some modern digital media arts focus on the novelty of a specific technology in a single domain, this presentation and instrument suggest there are unique and creative opportunities when the multimodal digital instruments are designed cohesively over the different modalities.

​

Presented in ACM SIGGRAPH SPARKS, December 2022.

Performance in Media Arts and Technology, University of California Santa Barbara, 2023

Paper and Performance in the ICMA's International Computer Music Conference (ICMC 2024), Seoul, 2024

​