Eulerian Video Processing and Medical Applications

Our goal is to reveal subtle yet informative signals in videos that are difficult or impossible to see with the naked eye. We can either display them in an indicative manner, or analyse them to extract important measurements, such as vital signs. Our method, which we call Eulerian Video Processing, takes a standard video sequence as input, and applies spatial decomposition, followed by temporal filtering to the frames. The resulting signals can be visually amplified to reveal hidden information, the process we called Eulerian Video Magnification. Using Eulerian Video Magnification, we are able to visualize the flow of blood as it fills the face and to amplify and reveal small motions. Our technique can be run in real time to instantly show phenomena occurring at the temporal frequencies selected by the user. Those signals can also be used to extract vital signs contactlessly. We presented a heart rate extraction system that is able to estimate heart rate of newborns from videos recorded in the real nursery environment. Our system can produce heart rate measurement that has clinical accuracy when newborns only have mild motions, and when the videos are acquired in brightly lit environments. Thesis Supervisor: Fredo Durand Title: Associate Professor of Computer Science Thesis Supervisor: William T. Freeman Title: Professor of Computer Science