Ester Hait

Blind Facial Image Quality Enhancement

using Non-Rigid Semantic Patches

Ester Hait, Guy Gilboa

Electrical Engineering Department,

Technion – Israel Institute of Technology, Haifa, Israel

 

Abstract:

We propose to combine semantic data and registration algorithms to solve various image enhancement problems, such as denoising, super-resolution and color-correction, in the case of facial images.

We address the difficult real-world problem of blind image quality degradation, involving multiple flaws: noise following possible nonlinear processing, resolution reduction, a certain degree of motion blur and contrast and color changes. To handle it, we rely on today’s easily available photography devices, and assume that prior high-quality data of the person to be processed is available.

We use semantically-aware patches, with adaptive size and location regions of coherent structure and context, as our building blocks. We define a new affinity measure derived from the non-rigid Demon registration [1]. We then construct data-driven measure-based affinity spaces, displaying various expression variations of facial features. The measure’s robustness to quality degradation allows to accurately match low-quality features to similar examples. Thus, we obtain a high-quality facial image, while preserving identity, pose and expression.

We demonstrate our method for the real-world problem of cellular photography enhancement of dark facial images. We show how our method significantly enhances image quality, both visually and quantitatively. Relying on only tens of personal priors, we enhance image quality and embed new image details. We compare our method to state-of-the-art denoising, deblurring and super-resolution methods.

 

 

Experimental Results:

We compare our results visually and quantitatively (using the NIQE score [2]) to the following methods:

1. State-of-the-art BM3D denoising [3].
2. State-of-the-art blind deblurring using a coupled adaptive sparse prior [4].
3. State-of-the-art blind deblurring using a dark channel prior [5].
4. State-of-the-art super-resolution using sparse representation [6], using different dictionaries.

Quality Enhancement Example (1020X768 pixels): identity #2, right pose:

Quality Enhancement Example: identity #2, right pose, close up:

Comparison to Super-resolution (1020X768 pixels): identity #2, right pose:

 

 

Quality Enhancement Example (950X640 pixels): identity #3, frontal pose:

 

Quality Enhancement Example: identity #3, frontal pose, close up:

Comparison to Super-resolution (950X640 pixels): identity #3, frontal pose: