Automatic Detection of TV Commercials

This project deals with Automatic Detection of Commercials Blocks. One of the most frustrating problems a TV viewer has to deal with is the vast number of commercials.

Abstract

This project deals with Automatic Detection of Commercials Blocks.
One of the most frustrating problems a TV viewer has to deal with is the vast number of commercials interrupting the continuity of the program. Therefore, the viewer needs a tool, which will make his TV time much more pleasant.
The goal of this project is to provide a tool which will detect those commercial blocks efficiently, and when combined with a VCR, it could stop a recording of a TV program when the commercials start, and continue the recording when they end.

 

The Problem

The problem we dealt with was how to determine if a certain video input was a TV program or a commercial block. While to the human eye, distinguishing between the TV program and the commercial is fairly easy, to the computer it is very difficult, since it can’t understand the content of the video segment in order to differ between them, like humans. Therefore, it was necessary to find a way to distinguish between TV programs and commercials, without relying on the content of the video segment.

 

The solution

Due to the fact that each type of program has its unique characteristic (e.g. Cartoons, movies, news, TV series, sports, etc.),it is very difficult, and maybe impossible, to come up with a solution that will work for every type of program. Therefore, we decided to split this project into two parts:
In the first part we dealt with the types of programs that are mostly recorded by viewers: TV series (not cartoons) and movies. In order to detect commercial blocks, we performed Hard Cuts detection and Text detection on the video segment. We assumed that in TV series and movies, Hard Cuts and Text rarely appear. Where as, in commercials they appear very often. Integration between those two tools can provided us an efficient algorithm that can detect commercial blocks in a reliable way.
In the second part of the project we dealt with Cartoons and Sports. Using the proper histogram detection for each type of program, we detected the commercial block out of the TV program.
Together with the algorithm built in the first part of the project, almost all of the types of programs exist were dealt with.

The general idea of the project is described in Figure 1:

1
Figure 1 – Block diagram of the algorithms built in the project

 

Hard Cuts Detection

Hard Cuts are scene-breaks which result from splicing two shots together without any transition. They are perceived as an instantaneous change from one shot to another. Therefore, in order to detect a Hard Cut, we’ve subtracted two sequential frames according to this formula:

2

When FD is above a certain threshold, this means we have detected a Hard Cut.

 

Text Detection

One of the main identifiers of a commercial is the appearance of a clear, sharp and noticeable text of the product’s name or the advertiser, mostly at the end of the commercial. Therefore, we have written an algorithm that detects text in a frame. The algorithm uses an Edge Detection method, using the Sobel algorithm. Then we detect each line in the text, and filtering the results.

In the second part of the project, due to the fact that there are no shared characteristics for Cartoon shows and Sports shows, we decided to split this part and write a different algorithm for each type.

 

Cartoon Algorithm

The main difference between Cartoon blocks and commercial blocks is that a Cartoon block contains illustrated image frames, while a commercial block contains mostly photographed image frames. The difference between those two types of frames is that in the illustrated images, there are many hues that appear frequently in the frame. However, in the photographed images, each frame has a wider spectrum of hues, which causes each hue to appear less frequently. In order to distinguish between the two, we checked the HSV histogram of each frame and searched for the number of frequently appeared hues. If this number is higher than a certain threshold, then the frame is considered an illustrated frame, otherwise, it is considered a photographed frame. After sorting each frame, we built a commercial vector and split it to commercial blocks.

 

Sports Algorithm

The most obvious characteristic in Sports shows is a global dominant hue for most of the Sports frames. This hue represents the field that the game takes place on (grass for soccer game, parquet for basketball game, etc.). However, in a commercial block, there is no global dominant hue for all the frames. The way we differenciated those two types of frames is by performing a colorcube histogram and searching for the dominant hue in each frame. Then, we searched for the global dominant hue and built a Sports vector containing all frames that have that hue as their dominant one. After building that vector, we split it to Sports blocks and finally transformed them to commercial blocks.

 

Tools

In this project we used the Matlab program in order to analyze each video segment, frame by frame. In order to get the digital video segment, we used a home video tape, and converted it using a video card and the Premiere program.

 

Conclusions

This project dealt with automatic detection of TV commercials. We can conclude that each algorithm we built was highly efficient in detecting the commercial block.

In each video segment that we tested the TV series and movies algorithm with, above 90% of the commercial blocks were detected successfully, and 0% of the TV program was falsely detected as a commercial block.

From the results of the Cartoon algorithm, we can clearly state that this algorithm gives a perfect solution to the Cartoon problem, since the results show above 99% successful detection without any false detection.

The results of the Sports algorithm show us that the algorithm is very efficient in dealing with the Sports problem, although there was a minimal percentage of false detection, caused by the fact that before going to / coming back from commercials, the camera moved from the field to the broadcasters, causing the algorithm to falsely detect this small part as commercials. It is necessary to state, that the most important part of the Sports video, the game itself, was not falsely detected at all.

Acknowledgment

We would like to thank our supervisor Eli Appelboim for his support and guidance throughout this project.
We are also grateful to the Ollendorff Minerva Center for supporting this project.