This project is another stage in building a system for identifying and tracking targets in the marine environment.
Abstract
This project is another stage in building a system for identifying and tracking targets in the marine environment. The purpose of this project is transforming a cameras equipped vessel into an unmanned autonomous vessel. Previous projects carried out horizon alignment, and performed targets detection and tracking in the marine environment from videos taken in daylight. In this project we will present an algorithm for identifying and tracking targets in the marine environment from video taken during nighttime. The input of the algorithm will be videos from two types of cameras: starlight scope and thermal camera. The output of the algorithm will be videos with marked targets. The algorithm was tested on videos taken by both types of cameras and gave good results for static targets and for moving targets, on the horizon and perpendicular to the horizon, and in cases where there’re multiple targets appearing together.
The Algorithm
The algorithm is divided to two main phases. The first phase is identification. The algorithm at this phase is different for the starlight scope videos and the thermal videos. The second phase is classification and tracking, and it’s the same for both types of videos.
The algorithm for analyzing videos from a starlight scope camera
- Pre-processing– running averaging filter to smooth the image and blur the gridlines which are created as a side effect in the videos
- Decimation– used to reduce runtime and model the background
- Modeling background– marine environment can be characterized by a Gaussian distribution and therefore we calculate the required properties for this modeling: mean and standard deviation
- Marking suspicious pixels – we indicate a pixel as part of a target if it deviates from the Gaosein distribution we found. To filter the horizon, we’ll make a comparison for each pixel to pixel from its right and from its left environment
- Spatial noise filtering and merge to objects– filtering small objects assuming that they are noise, then combine small and close objects by morphological operations
phase of classification and tracking
- Update the number of occurrences– for each object we found in the current frame
- Searching in the previous frame– for each object we found in the current frame we check correlation in close area in previous frame
- if the correlation between the current and the previous frame isn’t high enough- we assume that this is a new object and therefore we move on
- if the correlation between the current and the previous frame is high enough, we assume that this is an old object and therefore we calculate the mass center of the object from the previous frame and the velocity of this object
- Associating and updating object properties – giving the object from the current frame the properties (such as the number of occurrences and velocity) of the object from the previous frame
- Reduce the number of occurrences – for all the objects we haven’t found
- Estimate new position – of all the objects we haven’t found in the current frame according to the avarage velocity of a few last frames
- Marking object – after we have identify an object enough times we marke it as a target and draw a red rectangle around it. We make sure that the condition for stop tracking a target is more strict then the condition for start tracking it to avoid losing unstable targets
The algorithm for videos from a thermal camera
- Pre-processing – running averaging filter to smooth the noise of the sea waves
- Filter the horizon using Hough transform – using the Hough transform we find the longest line and it will represent the horizon. we find the brightest pixels around the horizon
- Marking suspicious pixels from the current image – finding the brightest pixels in the full frame by using the frame histogram
- Marking suspicious pixels from the gradient image – finding the brightest pixels in the image gradient by using the gradient histogram. we indicate a pixel as part of a target if it’s one of the pixels that we found in parts 2,3 or 4
- Spatial noise filtering and merge to objects – filtering small objects assuming that they are noise, then we will combine small and close objects by morphological operations
Results
Figure 1 – A frame from the starlight scope movie
Figure 2 – The frame after the identification phase
Figure 3 – The frame after the classification and tracking phase
Figure 4 – A frame from the thermal movie
Figure 5 – The frame after the identification phase
Figure 6 – The frame after the classification and tracking phase
Conclusions
The algorithm gives very good results in the identification phase, but should be improve in the tracking phase.
The results proves that the background modeling and the approach taken are suitable to the problem.