Smart Glasses for the Blind: Traffic Light Recognition

In this project, we developed and algorithm for recognition and identification of pedestrian traffic lights.

In this project, we developed and algorithm for recognition and identification of pedestrian traffic lights. Our method could be implemented in smart glasses for blind pedestrians, helping them to navigate safely in urban traffic.
Assumptions and Requirements

  • Day light
  • Ignore out-of-order traffic lights
  • Recognize the closest traffic light only
  • Camera min resolution: 5 M pixel

Data Collection
xx traffic light images were used (xx red and xx green)

Random pixels from red-green-black areas in traffic light were picked to check statistics of color distribution

1
Color Detection
For each color (green or red), two color masks in HSV color space were created:

Strict Mask – has tight HSV limits. It finds the distinctive Red and Green colors that we expect to see in the traffic light.

2

Red Hue < 0.05
0.95 < Red Hue < 1
0.7 < Red Sat < 1
0.7 < Red Val < 1
0.3 < Green Hue < 0.54
0.7 < Green Sat < 1
0.4 < Green Val < 1

3

Loose Mask –  uses loosen HSV limits to find wider areas with potential traffic lights to overcome problems due to shadow and camera saturation.

Loose Mask results were filtered in a way that the Strict Mask connected components will include at least a single pixel from the Strict Mask. Morphological dilation was used in order to unify close connected components into one.

4
Detect Black
Similarly, the statistical distribution of black pixels in the database was was used to find black pixels of the box surrounding the lights.

5
 Descriptor Creating and Matching
The purpose it to convert the perimeter of a shape into a format that allows comparison to a database. The comparison has to be scale and rotation invariant. The inputs to this link are image segments which each contain a connected component that passed the Detect Black filter. The outputs are grades and bounding boxes for each given image segment.

A “work section” was created by making use of the connected component data to extract a boundary box around it.

6

Split into 4 channelsMain channel, Secondary  channel, Mixed channel, Color Mask channel

If the input is a Red connected component’s image:

Main channel is the R in the RGB image.
Sec channel is the G in the RGB image.
Mix channel is the sum of G and B in the RGB image.
If the input is a Green connected component’s image:

Main channel is the G in the RGB image.
Sec channel is the B in the RGB image.
Mix channel is the sum of R and B in the RGB image.

All Main and Sec channels run through Median Filter and Contrast adjustment.
Both Mix channel run through Median Filter, Gaussian Filter and Contrast adjustment.

Color Mask Channel is a B&W image formed by taking thresholds over the channels of the work section HSV image.

7
Edge Detection
Edges were detected using Canny edge detector, edge were divided to connected components and the longest edge-let assumed to be the main part of the wanted edge. Connect edge-lets with one missing pixel between them by dilating with 3 by 3 structural element.

8
Descriptor
The descriptor is a sampling of the distance from the edge to the centroid over a predetermined set of angles.

9
Red and green descriptor
10
ROC Curve
11
Sample Results
Picture1