The objective of this project is to design a drowsiness detection system using image processing methods in real time.
Abstract
The objective of this project is to design a drowsiness detection system using image processing methods in real time. The system uses a CCD camera with Infra Red illumination which is pointed to the driver’s face. Using the image acquisition toolbox, images are acquired and processed using our Matlab code. The IR illumination creates the bright pupil effect which creates a nearly perfect circle. The system searches for the eyes (circles) in each frame using edge detection and feature measurements. If the eyes are not found for 5 consecutive frames, the system draws the conclusion that the driver is drowsy and issues a warning signal. In order to receive the bright pupil effect in daylight a visible light filter was attached to the camera lens.
The bright pupil effect
The problem
As the eye closure occurrences dramatically increase during the 10-second period preceding an accident, monitoring such closures is a good way to determine drowsiness and prevent the accident. This 10 second range means that the system has to determine if the driver is drowsy in only a few seconds. The algorithm that will be used should not only be fast but effective with as less false alarms (alerting when the driver is awake) and false findings (mistaking other features in the image as eyes) as possible.
The solution
We tried three different approaches:
1. The “clean up” algorithm –
- Edge detection and determining thresholds
- Removing isolated pixels and dilation of image
- Labeling image
- Computing feature measurements for image regions
- determining search parameters and performing search
- marking eyes or alerting driver
after edge detection after clean up
2. The correlation algorithm –
We created an “eye mask”. By finding where the maximum correlation is between the frame and the mask we found the pupil. Each frame should be “mean removed” to insure proper correlation
Eye mask
after correlation with eye mask
3. The Hough transform algorithm –
The Hough transform is a mathematical technique for detecting shapes in an image by mapping points in real space to curves or surfaces in an abstract space whose points represent the desired shapes in real space. The circular Hough transform maps a two-space (x,y) to a three-space of circles (x,y,r).
Transform results
Transform results Running time(for one frame):
1. Clean up – 2.6 sec
2. Correlation – 7.4 sec
3. Hough transform – 48.4 sec
We found the “clean up” algorithm to be the most accurate and the fastest of the three. In order to alert the driver within 10 sec we needed to improve the running time for 1 frame. The algorithm uses predefined sizes and thresholds; we needed to make the algorithm more adjusting
- Sampling before processing
- Adjusting threshold
- defining the search parameters
- Search area
- based on the previous position of the eyes in the last frame we can limit the search in the next frame
Results
Running time for one frame has decreased from 2.6 sec to 0.3 sec.
marking eyes
alerting driver
Real time acquisition
The SONY PC-110E camera is connected to a computer using a FireWire connection. MATLAB interfaces with the camera using the Image Acquisition Toolbox.
(14)
Tools
This project was developed in a Matlab 6.5, on a PC platform environment, using the image processing toolbox . The movies were filmed using a Panasonic vx37 and a Sony PC-110E video camera, transferred to a computer using Adobe Premiere. The real time images were transferred to the computer using the image acquisition toolbox.
Conclusions
- Nighttime detection is more accurate, due to the fact that the pupils are bigger and clearer than in daytime. furthermore, there are almost no lighting problems
- Glasses (including sunglasses) may cause reflections that resemble pupils .though, the occurrences of that are negligible
- Combine an algorithm that checks if when the head is turned to avoid false alarms
- The matlab works slow and uses a lot of virtual memory during the real time acquisition. The functions may run faster in C/C++
Acknowledgment
We are grateful to our project supervisors Eli Appelboim and Asaf Cohen for their help and guidance throughout this work, and The VISL lab staff: Aharon, Ina, and the lab engineer Johanan, for their support and willingness to assist with every problem we encountered during this project.
We are also grateful to the Ollendorff Minerva Center Fund for supporting this project.