We present Shimon, an interactive improvisational
robotic marimba player, developed for research in Robotic
Musicianship. The robot listens to a human musician and
continuously adapts its improvisation and choreography, while
playing simultaneously with the human. We discuss the robot’s
mechanism and motion-control, which uses physics simulation
and animation principles to achieve both expressivity and safety.
We then present a novel interactive improvisation system based
on the notion of gestures for both musical and visual expression.
The system also uses anticipatory beat-matched action to enable
real-time synchronization with the human player.
Our system was implemented on a full-length human-robot
Jazz duet, displaying highly coordinated melodic and rhythmic
human-robot joint improvisation. We have performed with the
system in front of a live public audience.