CVPR 2015 Tutorial: Applied math as applied in cinema
2PM to 6PM on Sunday June 7, 2015. Room 105, Hynes Convention Center. Boston, Massachusetts.
This tutorial covers a wide range of topics showing how applied math techniques from image processing and computer vision have become ubiquitous in movie-making, from shooting to exhibition. It does not deal with visual effects or computer-generated images, but rather with all the ways in which applied math is used to enhance, restore, adapt or convert moving images, the purpose of these techniques being to make the images look as good as possible while exploiting all the capabilities of cameras, projectors and displays.
Current digital cinema cameras match or even surpass film cameras in color capabilities, dynamic range and resolution, and several of the largest camera makers have ceased production of film cameras. On the exhibition side, film is already virtually gone from American movie theaters. And while many mainstream movies are still being shot on film, they are all digitized for postproduction. Therefore, in this tutorial we will equate ``cinema'' with ``digital cinema,'' considering only digital cameras and digital movies, and not discussing algorithms for problems that are inherent to film, like the restoration of film scratches or color fading.
Despite the “applied math” category, very few of the algorithms intended for application in the cinema industry are ever actually used. While there are many reasons for this, it is often the case that researchers in academia are not fully aware of the impossibly high quality standards of cinema, also lacking a clear picture of what the actual needs of the industry are. This tutorial addresses these issues head-on, providing a detailed overview of several very relevant problems in cinema production, postproduction and exhibition, and the way computer vision methods can be used to solve them.
Topics covered and brief outline:
The tutorial follows the book “Image Processing for Cinema” by M. Bertalmío, published in 2014 by CRC Press. Some reviews:
it is a great relief to find Image
Processing for Cinema
Marcelo Bertalmío. He wrote it as a textbook for graduate
students in areas including applied mathematics, image
processing, and computer science, as a comprehensive guide to
digital cinema for industry professionals. … Bertalmío’s
explanations are in a league with Walter (The
Technique of the Film Cutting Room)—written
in a way that doesn’t cause the eyes to glaze over. It is
technical writing that we can read. It takes the reader into a
subject but gives him or her exit points at various levels of
curiosity satisfaction. … this book is a resource of in-depth
information on digital image technology."
is a comprehensive, informative, and well written book, which
covers all aspects of the production and processing of cinema.
(Cinema means digital cinema here.) I have not seen any other
text that serves this purpose. It does not deal with visual
effects or computer generated images, but with the ways
algorithms are used to make images look as good as possible. The
author attempts the Herculean task of going from light and color
through optics to how cameras work and the image processing
algorithms used in-camera. Finally, offline image processing
algorithms are discussed. The goal is to bridge the
communication gap between movie professionals and image
processing researchers. There are enough equations to satisfy
most researchers but the material remains accessible to
nonexperts. There is a lot of material on open questions (like
how humans really perceive color), which could provoke new
research. I recommend this book strongly. It is suitable for
people from the movie industry to researchers as well as
undergraduate and graduate students interested in imaging and
Marcelo Bertalmío. Associate Professor, Universitat Pompeu Fabra, Spain. http://www.dtic.upf.edu/~mbertalmio/
Marcelo Bertalmío (Montevideo, 1972) received the B.Sc. and M.Sc. degrees in electrical engineering from the Universidad de la República, Uruguay, and the Ph.D. degree in electrical and computer engineering from the University of Minnesota in 2001. Since 2006 he is an Associate Professor at Universitat Pompeu Fabra, Spain.
His publications total more than 7,000 citations. He was awarded the 2012 SIAG/IS Prize of the Society for Industrial and Applied Mathematics of the USA (SIAM) for co-authoring the most relevant image processing work published in the period 2008-2012. Has received the Femlab Prize, the Siemens Best Paper Award, the Ramón y Cajal Fellowship, and the ICREA Academia Award, among other honors. He is an Associate Editor for SIAM-SIIMS and the secretary of SIAM's activity group on imaging. Has an ERC-Starting Grant for his project “Image processing for enhanced cinematography”. Has written a book titled “Image Processing for Cinema”, published by CRC Press / Taylor & Francis.
His current research interests are in developing image processing algorithms allowing to shoot cinema with no more artificial lighting than what people present at the scene need to be able to see. The approach is to work out software methods mimicking neural processes in the human visual system, and apply them to images captured with a regular digital movie camera.