Automatic Video Abstracting

What is a Video Abstract?

We all know what the abstract of an article is: a s hort summary of a document, often used to preselect material relevant to the use r. The medium of the abstract and the document are the same, namely text. In the age of multimedia, it would be desirable to use video abstracts in very much th e same way: as short clips containing the essence of a longer video, without a b reak in the presentation medium. However, the state of the art is to use textual abstracts for indexing and searching large video archives. This media break is harmful since it typically leads to considerable loss of information. For exampl e it is unclear at what level of abstraction the textual description should be; if we see a famous politician at a dinner table with a group of other politician s, what should the text say? Should it specify the names of the people, give the ir titles, specify the event, or just describe the scene as if it were a painting, emphasizing colors and geometry? An audio-visual abstract, to b e interpreted by a human user, is semantically much richer than a text. We defin e a video abstract to be a sequence of moving images, extracted from a longer vi deo, much shorter than the original, and preserving the essential message of the original.

Why are Video

Abstracts useful?

The power of visual abstracts can be helpful in many application contexts. Let us look at some examples. Multimedia archives. With the advent of multimedia PCs and workstations, the Wor ld Wide Web and standardized video compression techniques, more and more video m aterial is being digitized and archived worldwide. Wherever digital video materi al is stored, we can use video abstracts for indexing and retrieval. For instanc e, the on-line abstracts could support journalists when searching old video mate rial, or when producing documentaries. Another example is the Internet movie dat abase IMDb on the Web (http://uk.imdb.com/). It is indexed on the basis of "hand -made" textual information about the movies; sometimes, a short clip, selected a t random, is also included. The index could easily be extended by automatically generated video abstracts. Movie marketing. Trailers are widely used for movie advertising in cinemas and o n television. Currently the production of this type of abstract is quite costly and time-consuming. With our system we could produce trailers automatically. In order to tailor a trailer to a specific audience, we would set certain parameter s such as the desirable amount of action or of violence. Another possibility would be a digital TV magazine. Instead of reading short tex tual descriptions of upcoming programs you could view the abstracts with out even having to get up from your couch (supposing you have an int egrated TV set and Web browser). And for digital video-ondemand systems the content provider could supply video abstracts in an integrated fashion. Home entertainment. If you miss an episode of your favorite television series the abstracting could perfor m the task of telling you briefly what happened "in the meantime". Many more inn ovative applications could be built around the basic video abstracting technique .

Demo Video

Short Demo Video (MPEG-1: 31MB)

Publications

Rainer Lienhart, Silvia Pfeiffer, and Wolfgang Effelsberg. Automatic Trailer Production. In Handbook of Multimedia Computing, CRC Press, 1998. to appear.
Silvia Pfeiffer, Rainer Lienhart, Stephan Fischer and Wolfgang Effelsberg, Abstracting Digital Movies Automatically, Journal of Visual Communication and Image Representation, Vol. 7, No. 4, pp.345-353, December 1996. [Abstract] [PS : 97KB]; also Technical Report TR-96-005, April 1996.
Rainer Lienhart, Silvia Pfeiffer, and Wolfgang Effelsberg. Video Abstracting. Communications of the ACM, Vol. 40, No. 12, pp.55-62, December 1997. [Abstract] [PS : 557KB] [PDF : 132 KB] [PDF from ACM : 4313 KB]
Rainer Lienhart, Silvia Pfeiffer, Stephan Fischer. Automatic Movie Abstracting. Technical Report TR-97-003, Praktische Informatik IV, University of Mannheim, July, 1997. [Abstract] [PS : KB] [PDF : 189 KB]