skip to content

Automatic Movie Content Analysis

The MoCA Project


The MoCA Project

MoCA Project: Text Segmentation and Text Recognition in Digital Videos


There is no doubt that video is an increasingly important modern information medium. Setting free its complete potential and usefulness requires efficient content-based indexing and access. One powerful high-level index for retrieval is the text contained in videos. This index can be built by detecting, extracting and recognizing such text. The index enables the user to submit sophisticated queries such as a listing of all movies featuring John Wayne or produced by Steven Spielberg. Or it can be used to jump to news stories about a specific topic, since captions in newscasts often provide a condensation of the underlying news story. For example, one can search for the term "Financial News" to get the financial news of the day. The index can also be used to record the broadcast time and date of commercials, helping the people who check for their clients whether their commercial has been broadcasted at the arranged time on the arranged television channel. Many other useful high-level applications are imaginable if text can be recognized automatically and reliably in digital video.

Unlike other systems using text for the indexing of videos we do not take advantage of close-captions that might be transmitted on some television channels; rather, our system extracts the text from the video itself. The algorithms we propose make use of typical characteristics of text in videos to enable and enhance segmentation and recognition performance. We also demonstrate their suitability for indexing and retrieval by our video retrieval application.


Video Examples



First Approach

Some video samples of our (older) text segmentation algorithms for grayscale videos (SPIE 2666):


video type
orignal movie
segmented movie
moving text,
stationary scene
example1
(1508 KB)
example1
(737 KB)
moving text,
moving scene
example1
(898 KB)
example1
(1508 KB)
stationary text,
moving scene
example1
(1036 KB)
example1
(593 KB)


Second Approach

Some video samples of our (older) segmentation algorithms for color videos:


video type
orignal movie
segmented movie
closing sequence with
title and credits
Title1
(1.9 MB)
Title1
(2.2 MB)
closing sequence with
title and credits
Title2
(3.6 MB
Title2
(2.3 MB)
commercial Philadelphia
(717 KB)
Philadelphia
(327 KB)


New Approach

Now some video samples of our new segmentation algorithms for color videos:


video type
original and segmented video
closing sequence with
title and credits
Demo Video
(1.9 MB)
closing sequence with
title and credits
newTextSegmentationDemo.jpg
(2.4 MB)
commercial newTextSegmentationDemo.jpg
(1.8 MB)



Source Code


Text Segmentation:
Text Recognition: