The purpose of this project is to implement a proposal in which the illumination distribution of a real scene is estimated from the shadow cast by a known object.
Abstract
The purpose of this project is to implement a proposal in which the illumination distribution of a real scene is estimated from the shadow cast by a known object. The algorithm was first suggested in the article Illumination from Shadows / Imari & Yoichi Sato, Katsushi Ikeuchi , March 2003, and in the follow up article, Stability issues in recovering illumination distribution from brightness in shadows, December 2001.
The main focus of the project was on the implementation of an algorithm that recovers the illumination distributions around a known single object.
Theoretical Background
Given an object with a zero reflectance coefficient placed on a lambertian surface, the total radiance of the surface is given by the following expression:
where L0(θ, φ) is the illumination arriving from a solid angle of 
corresponding to the angles (θ, φ) as defined in the following sketch. Here S(θ, φ) represent the occlusion coefficient for each light source with respect to the surface point; S=0 whenever the source is occluded by the object, and S=1 otherwise. The constant C represents the albedo of the surface (constant for a lambertian surface).
In case of a digital image, the integral may be approximated by a finite sum. Therefore in order to find the illumination distribution in the scene, the following steps are to be implemented:
- Discretization of the light sources into points on a hemisphere with an a-priori unknown intensity distribution.
- Each image pixel, which is a direct relation to the image radiance, satisfies the following equation:
where ai = S(θi,φi)cos(θi)δωi represents the shadow coefficient for each point source and N equals the number of points on the hemisphere. Taking into account a sufficient amount of pixels, which produce N equations, the value of the light sources Li may be calculated.
Implementation of the Algorithm
The implementation to be hereby described was conducted on images of a black cube placed on a lambertian white surface. Various light sources were used, including one or two artificial sources or sunlight in a range of viewing angles. Only the cube’s geometrical dimensions were used as a priori given data. The realization used MATLAB 6.5.
The basic implementation comprised of the following steps:
- Building a hemisphere of point light sources, equally distributed with unknown values
- Extraction of the objects parameters from the image – finding the position of the cube’s corners
- Extraction of the camera angle by constructing a model of the cube, then turning the model at various angles. Then, transforming it into two dimensions in order to conduct a minimization of the square error in respect to the corners extracted from the original image
- Construction of a three dimensional model of the cube and the surface, and application of the adequate transformations to match the constructed image to the original image. This in order to find the actual position of the image surface points in respect to the object
- Sampling pixels of the surface (the original image or the constructed one)
- Calculation of the shadow coefficient for each unknown light source and each sampled pixel, and thus estimating the radiance of the light sources
- Examination of the acquired results, which was done in two manners:
- Representation of the calculated value on the illumination hemisphere
- Rendering the appearance of the object using the recovered sources
Means of Improvement
The main problem that arose following the basic implementation was the instability of the algorithm, due to the difficulty in obtaining a sufficient number of independent equations (in comparison to the number of variables). Therefore, a number of methods were considered:
- Pixel sampling whilst using clustering of the shadow coefficients – choosing pixels with independent attributes
- Merging light sources with similar shadow coefficients in an attempt to lower the number of shadow coefficients used
- Adaptive sampling of light sources
- Forcing smoothness – this method was useful only in scenes containing a single light source
Conclusion
During the project, the feasibility of the proposed algorithm was proven for the case of a simple object with no reflection upon a lambertain surface. The illumination distribution complied fairly well with the actual light source distribution, though it should be noted that the results for a singular light source where better than those achieved for two separate sources. We found that the basic algorithm lacks stability, which is especially crucial for scenes having more than one light source. Pixel selection by ways of clustering, merging close sources and forcing smoothness, managed to stabilize the outcome, and visually improve the results of applying the algorithm on the three dimensional model, though a problem of separation of two adjacent shadows still exists.
All in all, the method suffers from the necessity to be singularly adapted to each image, though the same principles apply to any chosen scene. In addition, for more complex objects, such as: non-lambertian surfaces, a non zero reflectance coefficient, etc., the same principles may be used using more elaborated illumination distributions.
Reconstructed Image Original Image
Acknowledgement
We would like to thank Dr. Yoav Schechner for his continues support and helpful ideas throughout the project, and to the VISL lab personnel, especially Mr. Johanan Erez, for their assistance and guidance when ever it was needed. We are also grateful to the Ollendorff Minerva Center Fund for supporting this project.