This project of computerized vision aims to construct a robotic model of automated driving by recognizing traffic signs in a non-artificial environment.
Abstract
This project of computerized vision aims to construct a robotic model of automated driving by recognizing traffic signs in a non-artificial environment. The core of the work is a traffic-sign-recognizing program, by means of sampling, scanning and analyzing pictures. It uses the computer program to command and control the movement of a Lego robot.
The project was devised in THE VISION AND IMAGE SCIENCE LABORATORY (VISL). It implements computer vision techniques on the LEGO-MINDSTORMS platform. It is realized using a Lego vehicle and movement controller, equipped with a wireless video camera. The video stream is transmitted to a PC, which in turn activates the analysis and recognition program on a captured frame. The traffic sign detected determines the consequent movement of the car, through a command transmitted to the car via an infrared tower.
Three main goals are accomplished
Picture sampling, picture analysis and traffic sign recognition
Detecting stationary objects – traffic signs – in a non-artificial environment, while the camera is in motion. This is done using well known, low complexity and fast computer vision techniques, such as: color detection, region growing and shape recognition. Thus, once the frame is analyzed and a traffic sign has been detected, corresponding commands are transmitted to direct the consequent movement of the Lego car. This process must be accomplished at real-time movement speed.
This process is the core of the project and has been programmed using C++ software.
Commanding movement of the robot
The vehicle’s movement is commanded by wireless remote-control devices only.
Integrating the systems
Creating one environment which integrates the use of hardware and software components of varying interfaces.
Project Environment
Lego Vehicle
1.RCX Unit – small programmable unit by Lego based on the H8 by Hitachi, provides serial I/O, A/D, 16k ROM, 32K RAM and timers. This unit can be loaded with programs up to 6k. It produces voltage to the two engines that control the vehicle according to the value of an inner variable. While running the program the value of the variable can be controlled via the infra red tower.
2.Video Camera and Transmitter – records and transmits the video stream as captured by the lens to a receiver, and then via USB Instant Video onto the computer. The wireless transmission enables free and independent movement of the vehicle.
3.Batteries – two batteries: one supplying power to RCX and video camera, value: 7.2V, the other sustaining the transmitter, value: 10.8V.
Central Control Unit
1.PC (Computer) – the majority of this project is implemented and run on a PC with 2 pairs of USB and software enabling USB Instant Video. Mainly programmed using Visual Studio 6 containing MFC and ActiveX classes.
2.USB Instant Video – links between the video receiver and the PC. A pair of USB is exclusively assigned to USB Instant Video as it demands a wide bandwidth. VideoOCX ActiveX classes were used to capture from this device.
3.Video Transmitter/Receiver – transmits and receives RGB image to be fed onto the computer.
4.Infrared Tower – transmits movement commands to the vehicle (up to 2-2.5 meters).
The Arena
The arena was set up with seven selected traffic signs.
Means of achieving solution
The algorithm
Color Coordinates
The PC receives an image in RGB coordinates, representing all colors by combinations of varying levels of Red, Green and Blue. In order to overcome the influences of changing lighting conditions on RGB coordinates, each examined pixel is converted to HSV coordinates. HSV stands for Hue (the real essence of the color) Saturation Value.
Object Detection
The entire frame is scanned at 4 pixel intervals. The HSV values for each fourth pixel are checked against the range of values for Red or Blue. In case of a match to either color, neighboring pixels are checked for matching color values in a process of Region Growing. Through this process additional parameters regarding the examined object are being accumulated, such as: extreme points (Xmax, Xmin, Ymax, Ymin), center of mass, number of pixels, average hue value. Preliminary filtering takes place using these parameters, to rule out irrelevant objects and interferences.
Shape Recognition
The parameters collected are used to determine the shape of the object. Using the extreme points the object’s height and width are ascertained to create a virtual, rectangular shape with an internal grid division. The locations of extreme points within the grid layout are matched against those of known shapes (triangle or circle).
Some signs might appear as two separate objects, such as the no-entrance sign, halved by a wide white strip. The process of shape recognition continues, as detailed above, on the two objects.
When there is more than one possibility for the identity of a traffic sign based on the shape determined (for example, circular blue left-turn-only sign or circular blue right-turn-only sign), further exploration of the central part of the sign takes place.
Conclusions
- The Lego Vehicle recognized and responded to the traffic signs in the non-artificial arena. It moved freely through the arena with no external guidance
-
In the process of shape recognition, it must be considered that the object might be located at varying distances and angles to the vehicle, which affects its shape and dimensions as captured by the camera
-
Changes of location and direction within the arena, as well as the angle of the camera, can cause lighting alterations and create reflections that make color recognition – and therefore shape determination – more difficult
-
The use of lighting invariant coordinates, such as HSV, does not fully solve the problem of shading and dazzling. Solutions to this issue involve the use of either appropriate filters or software that can process and improve the dynamic range of the picture
-
While working with a number of integrated complex systems, in addition to the appropriate functioning of each of the components, the interfaces between them must operate properly to enable the system to run smoothly
- The model constructed provides a sound basis for the recognition of many additional traffic signs
Acknowledgment
We are grateful to our project supervisor Johanan Erez for his help and guidance throughout this work.
We are also grateful to the Ollendorff Minerva Center Fund for supporting this project.