There are many practical uses for the super-resolution problem. For instance, a satellite in orbit can photograph pictures as it moves in space.
Abstract
There are many practical uses for the super-resolution problem. For instance, a satellite in orbit can photograph pictures as it moves in space. Its movement is known in advance, so it can be translated into mathematical equations. The resolution of the pictures taken have a huge importance for intelligence purposes and many more. The project will present a C++ approach algorithm solving the super-resolution problem.
Background
Super-resolution is a method of improving the resolution of a picture using several almost identical pictures. A picture is a sample of a moment in reality with a fixed sampling rate. Increasing the resolution is in fact an exact interpolation of the sample. Therefore, given a sequence of almost identical pictures with only limited motion of the camera between them, and under the assumption that the sampling rate (the distance between two samples) is the same in the picture sequence, but the samples fall in different places on the picture, we can perform an exact interpolation, so that the result is a picture with more samples than before. This means, the picture looks as though it was sampled at a higher rate, when if fact it wasn’t, but was enhanced using the super resolution digital processing.
In figure 1 we can see the fixed sampling rates in two pictures, and the interpolation that can be achieved using the difference between the pictures.
Figure 1
The solution
At first, we input the picture stream into the program. Then using an iterative local minima algorithm (the Nelder – Mead Simplex algorithm) we locate the relative movement between the pictures. We then construct a big picture X from the pictures. This picture is a high resolution picture, and has more pixels than the original stream of pictures. The amount of improvement is set within the program and can be changed. The output is a stream of better resolution pictures cut from the picture X exactly as it was created, but this time, unlike the construction of X, the pictures take the improved resolution.
Tools
This project is a continuation of a yearly project in the VISL lab in the Technion. The previous project was written in Matlab. In order to run the algorithm on a DSP in the future, our project was to migrate the Matlab code into C/C++, while keeping the result of the algorithm consistent with the Matlab version.
In order to use C++, we defined a series of classes to deal with the complex data structures of the system. We designed a Vector and a Matrix in order to deal with any form of vector or matrix, including a vector consisting of matrices, or a matrix of matrices or vectors and so on. We also defined a wide variety of functions to assist us in the code migration.
Special attention was given to functions unique to Matlab such as fminsearch – a Nelder-Mead Simplex algorithm to locate local minima, and various other functions described in the code.
Results
Figures 2 and 3 are two pictures from a twenty picture input stream. There is a subtle movement of the camera differing between the two.
Figures 2
Figures 3
Figure 4 is the result of the algorithm. Notice the slight improvement in the letters – the transitions between black and white are smoother.
Figure 4
The improvement is better visible in a close-up as we see in figures 5 and 6:
Figure 5
Figure 6
We also examine several variables of the algorithm process and how they effect running time
Acknowledgment
We would like to thank our supervisor Guy Gilboa for his support and guidance throughout this project, as well as the entire laboratory staff. We would also like to thank the Ollendorff Minerva Center fund which supported this project.