How to Debug an MBone Session

Wolfgang Effelsberg

Praktische Informatik IV

University of Mannheim

effelsberg@informatik.uni-mannheim.de

Often MBone sessions suffer from high packet loss rates, leading to low audio and video quality. It is not easy for an end user to find out what the reasons for the loss are and how to remove them. Nevertheless, this document tries to give you some hands-on methods to deal with common problems.

How to identify packet loss

  • Packet loss leads to rectangular disturbances in the video transmission: moving objects either appear at multiple locations in the video frame or disappear from the video. Extreme packet loss leads to a completely distorted or frozen picture.

  • Packet loss also leads to disruptions in the audio stream. Sometimes clattering sounds can be heard additionally. Unlike the video distortions audio disturbances often come from other origins than the network (noise cancellation, radio interference, etc.) 
  • In the video conferencing tool vic the frame rate and the loss rate are indicated next to the small video window in the main panel (in percent). 

  •  

     

  • The tools vic and vat also allow you to look at RTP error statistics: click on MENU and then on GLOBAL STATS. 
  • The audio tool rat allows you to look at RTP error statistics: left click on any participant.
In case of significant packet loss you should take notes of the statistics during a session in order to be able to give feedback to the sender of the session, your local system administrator and/or the people running the MBone overlay network in your country. It might also be helpful to take screen shots during the session when something goes wrong.

How to find out where packet loss occurs

The hard problem is to find out WHERE loss occurs and WHY. Packets might even get lost on the receiving machine. This frequently happens with video, if the receiving machine does not have the processing power to decode and display the video. 

A good way to figure out where packet loss occurs is the mtrace tool. Ask you system administrator to install mtrace on your machine, and to give you the rights to run it. It would be best if s/he could also assist you with the usage of mtrace since it takes some experience to understand its output.

Even if you cannot use mtrace you may want to try to observe patterns in the packet loss rate:

  • Does the loss rate drop if you mute the video (i.e. freeze the video by checking the respective check-box)? - This indicates a performance bottleneck on your machine.
  • Use the top-program to check for other CPU-time consuming processes on your machine. - This may also indicate a performance bottleneck on your machine.
  • Does the loss rate rise if you produce additional traffic in your LAN (e.g. high volume ftp)? - This indicates a performance bottleneck within your LAN.
  • Do several receiving sites experience the same loss pattern? - This might indicate a common point of loss. If all receivers report the same loss pattern it is likely that there is a problem at the sending site.
  • Is the loss correlated with activity in the whiteboard? - This might indicate a bandwidth bottleneck that leads to frequent (reliable multicast based) retransmissions that can further escalate bandwidth problems.
Please note that in order to not disturb an ongoing session you should test your configuration before the event that is to be transmitted. 

How to protect your session from packet loss

There are several things you can do to increase the likelihood that your session will not suffer from packet loss:

  • Set up a test session. Try to arrange for a test date one or two weeks before the actual session will take place. If possible this should be the same day of the week and the same time as the real session. This way you will notice if there is some other regular transmission (e.g., a lecture or a seminar) at that time of the week which might cause packet loss for your session. Ideally all participants that plan to attend the real session should also attend the test session. Use all the tools (same tool AND version) in the test session that are going to be needed for the real session. Have every participant comment on the quality of the test session. If necessary take steps to improve the quality during the test session. This may involve contacting your national MBone administrator.
  • Start the real session ahead of time (at least one hour). Use the time to eliminate any new problems. At the time of this writing the MBone is not stable, new problems may appear between your test session and the real session. This also helps you to eliminate new problems at your site, e.g. someone fiddled with your system recently and has (unwillingly) changed your configuration, you forgot to turn off your mike and its batteries have emptied, ...
  • When it is very important that your MBone transmission works properly, and in particular when you are the sender of an important MBone session, you should contact your system's administrator and your national MBone administrator ahead of time and inform her/him of the date and time of the sessions (test and regular session). S/he can then analyze problems while the session is going on.
What to do when you experience packet loss

It is always a good idea to first run local tests with the MBone tools between machines at your site, e.g. two PCs or workstations on the same LAN. This will help you to identify local problems, such as:

  • insufficient audio or video capabilities of your machine (parameters not set properly, wrong drivers installed, incompatible software updates, missing hardware, ...)
  • weak video encoding and decoding performance on your machine
  • limited number of video streams that can be decoded in real-time by your software (or hardware)
  • other processes running on your machine that consume a considerable amount of processing power
  • performance bottlenecks within your local network
  • and many more ...
When you are sure that your local system is not the problem there is very little you can do as an end user to find out where problems might be WITHIN THE NETWORK. The reason is that you will probably not have the experience and definitely not have the passwords to take a close look at the internal statistics of IP routers, ATM switches, etc. in the backbone. Therefore we recommend that you find out the contact information of the MBone administrator of your country's research network. He/she will be able to take a close look at the backbone links and active components, in particular MBone tunnels, packet filters, bandwidth limits for multicast IP, etc.

One thing you can try in order to reduce the packet loss rate is to limit the video transmission. Since video typically consumes most of the bandwidth while not being that important, it is often worth trying not to send any video at all in order to improve the transmission quality for audio and shared workspace.

If everything else fails you might want to consider setting up a tunnel or a reflector. A tunnel establishes a new connection from your site to the MBone overly network. Setting up a tunnel must be done by someone who knows what s/he is doing (e.g., your system administrator). A reflector is a piece of software that usually runs at the site the event is taking place. Its job is to forward packets that arrive on a certain multicast group to the recipients that are connected via unicast to it. Packets from the participants that are connected to the reflector via unicast are transmitted to all other participants that are connected to the reflector via unicast and to the multicast group. In this way it is possible for those participants who suffer unacceptable packet loss to connect to the reflector via unicast instead of joining the multicast session. This often improves their situation since unicast generally experiences much less packet loss than multicast. It is important to realize that using a reflector is not a good solution (it defeats the whole purpose of using multicast) but it is an acceptable fallback strategy if the MBone is broken.

Three golden rules for MBone sessions:

  1. Run local tests with your machine and the MBone tools before blaming problems on the backbone.
  2. Never use an ongoing (serious) session for test purposes. Never disturb the speaker with comments on audio and video quality because that is a nuisance not only for the speaker (who will loose concentration) but also for the other listeners. Arrange separate test sessions ahead of time.
  3. Ask your local administrator and the MBone specialist of your MBone country backbone to be online when an important seminar takes place.
Common sources of problems and how to avoid them (collected by the COST264 partners):

On Linux Systems port numbers of 61000 and above are used for IP masquerading. You will often receive a port in use error when trying to start MBone tools on these ports. Therefore make sure that sessions are announced with ports below 61000.

Reflectors (and in fact all other MBone tools) should be started with even port numbers, since RTP requires an even port number and uses the next odd port number for its control protocol RTCP.

Don't start a reflector on the same machine as the regular MBone tools. Often this will result in either the reflector or the tools not getting the multicast packets.

Make the slides available as .ps or .pdf before the session starts. This provides a backup in case that the shared whiteboard fails. The shared whiteboard may fail more often than audio and video since it requires a working bidirectional communication for realizing reliability.

Deactivate silence suppression in the audio tool, since this tends to cut off the first syllables of each spoken word.
 

wwwadmin@pi4.informatik.uni-mannheim.de
Last modified: Thu Mar 23 08:23:43 MET 2000