The purpose of this project is to improve the resolution of a movie.
Abstract
The purpose of this project is to improve the resolution of a movie. Given a collection of images (frames of a certain movie),
it is possible to fuse them to a one higher resolution image, called the “Super-Resolution Image”. From this “Super Image” we extract the frames of the “improved” movie. This process is a classic reconstruction problem, which can be represented as a least squares minimization problem.
An example for the usefulness of this project:
– A satellite in space, takes pictures of the earth. the pictures are taken one after the other, and form a video sequence. Because the resolution of these images is important for many things, we can use the algorithm we developed to improve the resolution of these images, so that we can see more details on them.
How can we improve the resolution of the movie frames?
The frames of the movie are showing the same object with minor differences. These differences result from the movement of the camera during the shot of the movie. We use these differences to extract data from the various movie frames, to construct the frames in higher resolution.
Describing The Algorithm
The “Super-Resolution” problem is an example of an inverse problem: Given the high resolution images we could have produced the low resolution images from them. But, we have the inverse problem here: We are given the low resolution images, and we want to create a high resolution images from them, and form a movie with good quality.
We use the following algorithm:
At first, we will describe mathematically how to produce the low resolution images of a certain object from a high resolution image of that object – This process is called “Capturing”. Notice that in order to evaluate the “Capturing” process, we don’t the high resolution images (because we don’t have them).
Using the “Capturing” process, and the low resolution images, we will try find a high resolution image. This image is one which a capture of it would produce low resolution images which are the most similar to the ones we were given in the beginning of the process.
This problem is a classic optimization problem. We used the algorithm of the “Steepest Descent” to solve it.
This figure demonstrates the process of the algorithm we developed: The images called (Y1,…,Yimnum) are the input images, and they are usually in poor quality. We try to fuse these images to create the image called X, which is a good quality image of the entire object. From this image, we extract the images called (Y’1,…,Y’imnum) which are the frames of the high resolution movie – The output of the process.
Results
This is one of the input images. We used 40 input images for this example, and they form a short movie. As you can see, this image is a low resolution one.
The following image is the output image of the process. It is a high resolution image, and we created the original movie from this image, but with one difference – it’s a higher quality movie.
User interface
Acknowledgments
We would like to thank our supervisor Guy Gilboa for his support and guidance throughout this project. Also we would like to thank the Ollendorff Minerva Center Fund which supported this project.