Human-Sound Interaction

We propose a new paradigm based on principles of direct, natural and embodied interaction with sound through cross-modal representation of the auditory feedback, here defined as Human-Sound Interaction (HSI).We created SoundSculpt, a system using holographic projection and mid-air haptic feedback where sounds are visualised and presented as a deformable container whose shape is determined by its associated spectral signature. With an interaction style analogous to moulding and sculpting, the user is able to boost and attenuate frequency components of the sound by interacting directly with the holographic visualisation of the container projected mid-air using hand gestures.