How to Debug an MBone Session
Wolfgang Effelsberg
Praktische Informatik IV
University of Mannheim
effelsberg@informatik.uni-mannheim.de
Often MBone sessions suffer from high packet loss rates, leading to
low audio and video quality. It is not easy for an end user to find out
what the reasons for the loss are and how to remove them. Nevertheless,
this document tries to give you some hands-on methods to deal with common
problems.
How to identify packet loss
-
Packet loss leads to rectangular disturbances in the video transmission:
moving objects either appear at multiple locations in the video frame or
disappear from the video. Extreme packet loss leads to a completely distorted
or frozen picture.
-
Packet loss also leads to disruptions in the audio stream. Sometimes clattering
sounds can be heard additionally. Unlike the video distortions audio disturbances
often come from other origins than the network (noise cancellation, radio
interference, etc.)
-
In the video conferencing tool vic the frame rate and the loss rate
are indicated next to the small video window in the main panel (in percent).
-
The tools vic and vat also allow you to look at RTP error
statistics: click on MENU and then on GLOBAL STATS.
-
The audio tool rat allows you to look at RTP error statistics: left click
on any participant.
In case of significant packet loss you should take notes of the statistics
during a session in order to be able to give feedback to the sender of
the session, your local system administrator and/or the people running
the MBone overlay network in your country. It might also be helpful to
take screen shots during the session when something goes wrong.
How to find out where packet loss occurs
The hard problem is to find out WHERE loss occurs and WHY. Packets might
even get lost on the receiving machine. This frequently happens with video,
if the receiving machine does not have the processing power to decode and
display the video.
A good way to figure out where packet loss occurs is the mtrace
tool. Ask you system administrator to install mtrace on your machine,
and to give you the rights to run it. It would be best if s/he could also
assist you with the usage of mtrace since it takes some experience
to understand its output.
Even if you cannot use mtrace you may want to try to observe
patterns in the packet loss rate:
-
Does the loss rate drop if you mute the video (i.e. freeze the video by
checking the respective check-box)? - This indicates a performance bottleneck
on your machine.
-
Use the top-program to check for other CPU-time consuming processes
on your machine. - This may also indicate a performance bottleneck on your
machine.
-
Does the loss rate rise if you produce additional traffic in your LAN (e.g.
high volume ftp)? - This indicates a performance bottleneck within your
LAN.
-
Do several receiving sites experience the same loss pattern? - This might
indicate a common point of loss. If all receivers report the same loss
pattern it is likely that there is a problem at the sending site.
-
Is the loss correlated with activity in the whiteboard? - This might indicate
a bandwidth bottleneck that leads to frequent (reliable multicast based)
retransmissions that can further escalate bandwidth problems.
Please note that in order to not disturb an ongoing session you should
test your configuration before the event that is to be transmitted.
How to protect your session from packet loss
There are several things you can do to increase the likelihood that
your session will not suffer from packet loss:
-
Set up a test session. Try to arrange for a test date one or two weeks
before the actual session will take place. If possible this should be the
same day of the week and the same time as the real session. This way you
will notice if there is some other regular transmission (e.g., a lecture
or a seminar) at that time of the week which might cause packet loss for
your session. Ideally all participants that plan to attend the real session
should also attend the test session. Use all the tools (same tool AND version)
in the test session that are going to be needed for the real session. Have
every participant comment on the quality of the test session. If necessary
take steps to improve the quality during the test session. This may involve
contacting your national MBone administrator.
-
Start the real session ahead of time (at least one hour). Use the time
to eliminate any new problems. At the time of this writing the MBone is
not stable, new problems may appear between your test session and the real
session. This also helps you to eliminate new problems at your site, e.g.
someone fiddled with your system recently and has (unwillingly) changed
your configuration, you forgot to turn off your mike and its batteries
have emptied, ...
-
When it is very important that your MBone transmission works properly,
and in particular when you are the sender of an important MBone session,
you should contact your system's administrator and your national MBone
administrator ahead of time and inform her/him of the date and time of
the sessions (test and regular session). S/he can then analyze problems
while the session is going on.
What to do when you experience packet loss
It is always a good idea to first run local tests with
the MBone tools between machines at your site, e.g. two PCs or workstations
on the same LAN. This will help you to identify local problems, such as:
-
insufficient audio or video capabilities of your machine (parameters not
set properly, wrong drivers installed, incompatible software updates, missing
hardware, ...)
-
weak video encoding and decoding performance on your machine
-
limited number of video streams that can be decoded in real-time by your
software (or hardware)
-
other processes running on your machine that consume a considerable amount
of processing power
-
performance bottlenecks within your local network
-
and many more ...
When you are sure that your local system is not the problem there is very
little you can do as an end user to find out where problems might be WITHIN
THE NETWORK. The reason is that you will probably not have the experience
and definitely not have the passwords to take a close look at the internal
statistics of IP routers, ATM switches, etc. in the backbone. Therefore
we recommend that you find out the contact information of the MBone
administrator of your country's research network. He/she will be able
to take a close look at the backbone links and active components, in particular
MBone tunnels, packet filters, bandwidth limits for multicast IP, etc.
One thing you can try in order to reduce the packet loss rate is to
limit the video transmission. Since video typically consumes most of the
bandwidth while not being that important, it is often worth trying not
to send any video at all in order to improve the transmission quality for
audio and shared workspace.
If everything else fails you might want to consider setting up a tunnel
or a reflector. A tunnel establishes a new connection from your
site to the MBone overly network. Setting up a tunnel must be done by someone
who knows what s/he is doing (e.g., your system administrator). A reflector
is a piece of software that usually runs at the site the event is taking
place. Its job is to forward packets that arrive on a certain multicast
group to the recipients that are connected via unicast to it. Packets from
the participants that are connected to the reflector via unicast are transmitted
to all other participants that are connected to the reflector via unicast
and to the multicast group. In this way it is possible for those participants
who suffer unacceptable packet loss to connect to the reflector via unicast
instead of joining the multicast session. This often improves their situation
since unicast generally experiences much less packet loss than multicast.
It is important to realize that using a reflector is not a good solution
(it defeats the whole purpose of using multicast) but it is an acceptable
fallback strategy if the MBone is broken.
Three golden rules for MBone sessions:
-
Run local tests with your machine and the MBone tools before
blaming problems on the backbone.
-
Never use an ongoing (serious) session for test purposes. Never disturb
the speaker with comments on audio and video quality because that is a
nuisance not only for the speaker (who will loose concentration) but also
for the other listeners. Arrange separate test sessions ahead of
time.
-
Ask your local administrator and the MBone specialist of your MBone country
backbone to be online when an important seminar takes place.
Common sources of problems and how to avoid them (collected by the COST264
partners):
On Linux Systems port numbers of 61000 and above are used for IP masquerading.
You will often receive a port in use error when trying to start MBone tools
on these ports. Therefore make sure that sessions are announced with ports
below 61000.
Reflectors (and in fact all other MBone tools) should be started with
even port numbers, since RTP requires an even port number and uses the
next odd port number for its control protocol RTCP.
Don't start a reflector on the same machine as the regular MBone tools.
Often this will result in either the reflector or the tools not getting
the multicast packets.
Make the slides available as .ps or .pdf before the session starts.
This provides a backup in case that the shared whiteboard fails. The shared
whiteboard may fail more often than audio and video since it requires a
working bidirectional communication for realizing reliability.
Deactivate silence suppression in the audio tool, since this tends to
cut off the first syllables of each spoken word.
wwwadmin@pi4.informatik.uni-mannheim.de
Last modified: Thu Mar 23 08:23:43
MET 2000 |