Demo Video

Download

Manual

Publications

Related Projects

Introduction

The Interactive Media on Demand (IMoD) system is a generic recorder for network-based audio, video and interactive media streams. Interactive media streams are data streams produced by applications that allow user-interaction with the medium itself. Typical examples of such applications are shared whiteboard systems, distributed virtual environments and networked computer games.

In contrast to many other recording tools for interactive media, the IMoD system operates independent of a certain application. This could be achieved by defining an application level protocol framework for interactive media (the RTP/I protocol) which captures the common aspects of the transmission of interactive media streams (in analogy to the RTP protocol which captures common aspects of audio and video). The IMoD system operates only on the elements of the RTP/I protocol framework and does not try to decode the application specific payload of media streams. Thus, the IMoD system is capable of recording and playing back any interactive media stream that is based on the RTP/I protocol. The IMoD system provides random access to recordings and is also capable of recording and playing back RTP-based audio and video streams in synchronization with interactive media streams.

The following sections provide a brief overview over the basic concepts used by the IMoD system.

Interactive Media

In order to provide such a generic recording service that is reusable for a whole class of media, we define a media model (a detailed discussion of this model can be found in 3). A distributed interactive medium has a state. For example, the state of a shared whiteboard is defined by the content of all pages currently present in the shared whiteboard. An application for distributed interactive media generally maintains a local copy of the medium's state. It is therefore said to have a replicated distribution architecture. The state of a distributed interactive medium can change for two reasons, either by passage of time or by events. Generally, a state change caused by the passage of time does not require the exchange of information between application instances since each user's instance can independently calculate the required state updates. Any state change that is not a fully deterministic function of time is caused by an event. Information about events must be transmitted to all remote instances of the application in order to keep the remote copies up to date.

RTP/I Protocol

The Real Time Protocol for Distributed Interactive Media (RTP/I) 3 is a protocol framework for distributed interactive media. It consists of two parts: a data transfer protocol for the transport of events, states, and requests for state information, and a control protocol for meta-information about the medium and the participants of a session.

The data transfer protocol provides a standardized framing for so-called application data units (ADUs). This framing contains information that is common to the distributed interactive media class. Typical examples for the information contained in the RTP/I data framing include a timestamp which indicates the time at which an event happened or a state was calculated, the type of the data (e.g., event or state), and the unique ID of the ADU's sender. RTP/I can also be used to request the state of a sub-component in a standardized way.

RTP/I is closely related to the Real Time Transport Protocol (RTP) 4 which is mainly used for the transmission of audio and video. However, while RTP/I reuses many aspects of RTP, it has been thoroughly adapted to meet the needs of distributed interactive media.

IMoD System

The Interactive Media on Demand (IMoD) 1,2 system provides a generic recording and playback service for RTP/I-based interactive and RTP-based non-interactive media streams. The recording and playback of interactive media streams is based exclusively on the common aspects of distributed interactive media as they are exposed by RTP/I. For this reason, the IMoD system is capable of handling any interactive media stream using the RTP/I protocol, regardless of the specific media type or encoding. For example, a recording might contain a distributed dynamic 3D model combined with a shared whiteboard, as well as audio and video data. Figure 1 depicts the playback of a 3D model in synchronization with a video stream.

The main idea of the generic recording is to record the ADUs (states and events) as they are transmitted during a live session. At a later time, the stream of ADUs can be replayed in the proper order and with the correct timing as provided by the RTP/I header information. The properties of the replayed ADUs are re-calculated to match the conditions at replay time. The stream produced in this way can be interpreted and presented to a user by an unmodified application.

One challenging problem with the development of a generic recording service for distributed interactive media is the realization of random access to the recorded stream. The main problem here is that the listening applications must be initialized with the current state of the medium before they are able to follow the recorded stream. Since the recorder is generic, it will not be able to calculate this required state information. However, using the information contained in the common RTP/I protocol headers, it can determine a combination of recorded state and event ADUs which puts the receivers into the desired state at the access position. A detailed discussion of generic random access mechanisms for interactive media streams can be found in 1.

Figure 1: Example of a recorded session containing a video and an interactive media stream being played back by the IMoD system. The interactive media stream is decoded by the application in the upper left corner. The video stream is received by vic, depicted in the lower left corner. The video shows the display of the interactive media application captured during the recording. The IMoD client application is shown on the right.

Acknowledgments

References

1. Hilt, V., Mauve, M., Kuhmünch, C., Effelsberg, W. A Generic Scheme for the Recording of Interactive Media Streams. In Proceedings of IDMS'99 (Toulouse, France, 1999), Springer Verlag, pp. 291-304, 1999.
2. Hilt, V., Mauve, M., Vogel, J., Effelsberg, W. Interactive Media on Demand: Generic Recording and Replay of Interactive Media Streams. In Proceedings of ACM Multimedia 2001 (Technical Demonstration), Ottawa, Canada, 2001.
3. Mauve, M., Hilt, V., Kuhmünch, C., Effelsberg, W. RTP/I - Towards a Common Application Level Protocol for Distributed Interactive Media. IEEE Transactions on Multimedia, Vol. 3 No. 1 (March 2001), pp. 152 - 161.
4. Schulzrinne, H., Casner, S., Frederick, R., Jacobson, V. RTP: A Transport Protocol for Real-Time Applications. Internet-Draft, IETF, draft-ietf-avt-rtp-new-09.txt, March 2001.