Automatic Movie Content Analysis
The MoCA Project 
The MoCA Project
Navigation
MoCA Project: Text Segmentation and Text Recognition in Digital Videos
There is no doubt that video is an increasingly important modern information medium. Setting free its complete potential and usefulness requires efficient content-based indexing and access. One powerful high-level index for retrieval is the text contained in videos. This index can be built by detecting, extracting and recognizing such text. The index enables the user to submit sophisticated queries such as a listing of all movies featuring John Wayne or produced by Steven Spielberg. Or it can be used to jump to news stories about a specific topic, since captions in newscasts often provide a condensation of the underlying news story. For example, one can search for the term "Financial News" to get the financial news of the day. The index can also be used to record the broadcast time and date of commercials, helping the people who check for their clients whether their commercial has been broadcasted at the arranged time on the arranged television channel. Many other useful high-level applications are imaginable if text can be recognized automatically and reliably in digital video.
Unlike other systems using text for the indexing of videos we do not take advantage of close-captions that might be transmitted on some television channels; rather, our system extracts the text from the video itself. The algorithms we propose make use of typical characteristics of text in videos to enable and enhance segmentation and recognition performance. We also demonstrate their suitability for indexing and retrieval by our video retrieval application.
Video Examples
First Approach
Some video samples of our (older) text segmentation algorithms for grayscale videos (SPIE 2666):
| video type |
|
|
|
stationary scene |
(1508 KB) |
(737 KB) |
|
moving scene |
(898 KB) |
(1508 KB) |
|
moving scene |
(1036 KB) |
(593 KB) |
Second Approach
Some video samples of our (older) segmentation algorithms for color videos:
| video type |
|
|
| closing sequence with title and credits |
 (1.9 MB) |
 (2.2 MB) |
| closing sequence with title and credits |
 (3.6 MB |
 (2.3 MB) |
| commercial |  (717 KB) |
 (327 KB) |
New Approach
Now some video samples of our new segmentation algorithms for color videos:
| video type |
|
| closing sequence with title and credits |
Demo Video (1.9 MB) |
| closing sequence with title and credits |
 (2.4 MB) |
| commercial |  (1.8 MB) |
Source Code
Text Segmentation:
- for grayscale videos (359 KB)
- for color videos (279 KB)
- for color videos (see general MoCA download page)
Text Recognition:
- for for color videos