Towards Real-Time Synchronization of mBot's Motion with Musical Songs

For more than a decade, researchers have been attempting to synchronize humanoid robot motion with musical beats, including incorporating ‘emotion’ of the music into the movement of robots. However, the cost the humanoid robots capable of executing motions using real-time recognition of musical notes is typically in the order of thousands of US$, which is extremely difficult for students to explore as ‘education robots', especially for students from the developing or underdeveloped countries around the world. We report here our work on exploring the possibility of applying the ‘robot-music synchronization’ technology to an affordable robot, i.e., ~US$50, to proliferate the educational value of this concept. We have chosen the mBot as the educational robot platform and showed that it can recognize the frequency of the musical sound signals in real-time and subsequently control the robot's LED (light-emitting diode) colors and motor functions for three different musical sound sources: 1) individual musical notes, 2) a rock-n-roll song, and 3) a classical music song. The limitations of the sound sensor used to collect real-time musical sound input and the deficiencies of the mBot's computing power are discussed in this paper.