| Vision based pursuing
of moving vehicle from bird's view - PART I (LEGO Lab) |
by Adir Hacohen & Yonatan Cohen Supervised by Shie Manor |
Abstract
Our system performs pursuit of a vehicle by another ('catching' and 'target'
vehicle).
The information about the vehicles' locations is received from a ceiling
camera that oversees the entire "playground".
A controlling computer tracks the movement of both vehicles and controlls
the movement of the 'catching' vehicle.
Working environment
- 2 'LEGO Mindstorm' vehicles controlled by RCX brick + IR transmitter attached to a PC.
- Ceiling mounted dome camera (WATEC 207-CD)
- PC + video card. (Pentium 4 & ASUS Geforce V7700/64 VIVO graphics card)
- The software (self developed) is implemented in C++ and contains two
processes LegoCatch and CamCapture
that exchange information. The code is flexible for addition of
more vehicles / obstacles in the future.
CamCapture is based on a code example found on the internet which captures from cameras which support the Microsoift Video for Windows standard. (For reference search www.codeguru.com for AVICap).
LegoCatch is based on the Lego Mindstorms Visual C++ Win32 Interface by Daniel Berger (link). The interface enables to send controlls for the RCX through the IR channel using the IR transmitter of 'LEGO Mindstorms Robot Invention Kit version 1.5' (it does not support the USB based new versions).
 |
 |
| Figure 1 The Catcher (yellow) and the target (black), | Figure 2 The RCX unit |
The algorithm
The implemented algorithm has 3 phases:
1. Picture sampling
2. Recognition of the vehicles
3. Movement control
1. Sampling
CamCapture samples the pictures
in 16 bit RGB format using 320*220 resolution.
LegoCatch performs the analysis
of the captured pictures and translates visual information to controll
commands for the RCX.
On each iteration LegoCatch notifies
CamCapture and receives a captured
image.
The image is then transformed by LegoCatch
to a green image with 128 shades (the captured image is RGB but we found
that using only the green color gives good enough results).
The first capture is taken when the playground is empty. This is used
as a reference image. Each new captured image is subtracted from the reference
image and the difference image used as input to the algorithm.
This method is used in order to work with a unic background (e.g. in the
figure you can see that the light reflections are eliminated) and also
to avoid changes in the background caused by changing light conditions
etc..
|
|
|
|
|
 |
 |
|
|
|
2. Recognition of the vehicles
In order to recognize the two vehicles we have to distinguish them
from the background and from each other.
The recognition is done as follows:
I)
We scan the matrix pixel by pixel.
for each pixel if |pixel value| < Threshold we will copy it to a linked
list.
Each node will hold the pixel value & its location (x,y).
Now we have a list of the pixels that are suspected as part of an object.
II )
We move a pixel (a) from the list to another list. then We scan the first
list for neighbour pixels of (a).
If neighbors are found we will move them to the second list and look for
their neighbours.
When there are no more neighbors in the first list and the first list
still contains pixels we start another list.
We repeat this phase until there are no more pixels in the original list.
We eliminate lists that hold less than 20 pixels (we consider them as
noise).
Now we have a list for each object on the playground.
Each list header holds:
- Number of pixels
- Average shade
- Coordinates of the center of mass (by calculating the average position of the pixels)
- Diagonals ratio (supposed to be close to 1 in symmetric objects)
Knowing the properties of each vehicle we can now decide of the position
of each vehicle.
3. Movement control
The RCX block has 3 inputs and 3 outputs.
The Lego vehicle has two wheels, each wheel is connected to an engine.
Each engines is controlled by a different RCX output and has 8 speed levels.
The RCX code controls the engines speed.
We designed 3 modes of movement for the catcher:
1. Forward - the 2 engines are set to level 7 ( full speed)
2. Curve left / Curve right - one engine is set to level 1 (minimal speed)
and the other is set to level 8 (full speed)
3. Spin - one engine is set to level 7 (forward) and the other is set
to level -7 (reverse)
 |
|
|
Calculating the movement direction
 |
|
|
a : The catcher's direction is (in degrees) determined by the vector generated between the catcher's last position and its current position.
b : The target's direction is determined by the vector generated between the catcher's Current position and the target position
g : The movement direction is the difference between the two angles
g varies between -180 and 180 degrees
if 45 < |g| < 180 spin right/left
according to the angle size then move forward
if 10 < |g| < 45 curve right/left
if |g| < 10 move forward
* comment : The first movement must be forward regardless of the target position in order to get the initial direction
The target vehicle
Built on the same platform as the catcher vehicle, controlled by an RCX which is preprogrammed to move in fixed paths that have been loaded to it's memory slots. The RCX's inputs are connected to 3 light sensors. When the sensors reach a certain threshold the vehicle stops.
Catch
When the 'catching' vehicle reaches the target the two objects unite
to one big object. LegoCatch stops the Catcher and two lights located
in front of the catcher are being turned on. The target sensors are exceeding
the threshold and the target stops. LegoCatch sends a "CATCH" message.
 |
| Figure 8 snapshot of a catch |
Results
High reliability was achieved due to the fact that we used only 2 vehicles that could be easily distinguished and that the vehicles were relatively slow.
Acknowledgment
We are grateful to our project supervisor Shie Manor, the lab engineer
Johanan Erez and the lab assistant Dror Ozana. We would also like to express
our gratitude to the Ollendorf
Minerva Center Fund for supporting this project.
Related documentation
