Fluid Audiovisual Counterpoint via Motion Tracking of Gesture

SPEAKER: Dr. Bret Battey




One important characteristic of traditional Western music theory is the balancing of hierarchical relationships in the instant and over time. Dr Battey is investigating the abstract applicability of this idea to composing audiovisual works -- by using optical motion tracking to capture the gestures made by a performer of North Indian classical vocal music. Considering movement relationships within the skeletal hierarchy and the temporaral hierarchy of gesture (with primary and secondary goal points), we can meaningfully map the captured motion data to a hierarchically ordered abstract visual algorithm to create a "fluid counterpoint" that fuses richly with the musical expression.