Detecting and Tracking People in a Public Area

The need for smarter and better security solutions has increased dramatically in the last several years following the wave of global terrorism and rising crime.

The need for smarter and better security solutions has increased dramatically in the last several years following the wave of global terrorism and rising crime. Many of the existing solutions are based on cameras monitored by a human operator, which is prone to making mistakes and not noticing certain events (especially while monitoring several cameras at once).

The purpose of this project is to detect and track people in an environment of an average small business (a restaurant) using a low resolution standard security camera, and to do so at maximum reliability. The algorithm detects targets at predetermined entering points of the scene, and tracks them using frame subtractions, movement estimation, and adaptive noise filtering.


The problem
Our scene was an ordinary fast food restaurant on an ordinary business day. Some of the problems are related to human behaviour:

  • Targets shape and contour constantly change randomly
  • Hard to distinguish people from other objects
  • Occlusions often occur
  • Velocity changes – hard to track still targets

Other prolems were scene related:

  • Location of camera caused targets to have different area at different distance
  • The large windows cause reflections from cars and people outside the restaurant


The solution
Our approach was based on two inputs:

  • Subsequent frame subtractions: frame[n]-frame[n-1] which is sensitive to movement but cannot track still targets
  • Reference frame subtraction: frame[n]-Ref (Ref is a frame taken when the room was empty). This input was very noisy especially due to furniture movement over time, but had to be used for still targets

In order to overcome the noise using the Ref frame we came up with a method for removing noise from it. This was done by creating a noise image using logical operations and subtracting it: frame[n]-Ref-Noise

Basic algorithm block diagram:

  • Image Processing – Performs histogram equalization, frame substractions, various kinds of noise filtering and BW thresholding
  • Detect – Detects entering targets using a mask. Ensures that false tarets aren’t detected
  • Track – Tracks targets in the room using two inputs:
    • Subsequent frame subtractions – sensitive to movement
    • Subtraction of reference* frame from current frame – for still targets (very noisy input)

      • Further noise filtering especially from reference input
      • Target location estimation using velocity vector
      • Algorithms for target occlusions, tracking drift, tracking becoming unreliable

* reference frame is a frame taken when the room was was empty.

Targets being tracked (only 2):                                                             Result of subsequent frame subtraction:
3 4

Reference frame subtraction:                                                                   Noise image:
5 6


  • MATLAB 7.01 was used for implementing the algorithms
  • One static color security camera (720×570 interlaced)
  • Sony Camcorder for recording the video



Demo movies


We would like to thank our supervisor Johanan Erez for his support and guidance throughout this project.
In addition we would like to thank Mr. Aviad Hayat the owner of the
“Lunch Bar” restauraunt at the Hutzot Hamifratz shopping mall for permitting us to install the camera in his restaurant.
We are also grateful to the Ollendorf Minerva Center Fund for supporting this project.