EUMETCast Tellicast

Attempts to diagnose and reduce the number of "Missed" packets as reported by the Tellicast client.

Tips for Linux

Watch for missed packets

Download the ethernet interface watcher (Perl script for Linux only)

It will report the number of packets received and the various error counters; see if the number of Tellicast missed packets corresponds to any particular type of error.

Examples:

Sometimes missed packets are preceded by UDP errors, sometimes not.
time   Eth:pkts errs drop bytes    intr UDP:pkts port errs rxbuf TC:missed

Tellicast missed 13 packets for no obvious reason:
1405357848  917    0    0  1132978  122      902    0    0     0    13 *ERR*

Tellicast missed 65 packets but immediately prior to that were 1334 UDP errors:
1405357955 1795    0    0  2229901  417      921    0 1334     0    0
1405357958 4815    0    0  5957422  153     4358    0    0     0    65 *ERR*

Tellicast missed 159 packets but immediately prior to that were 878 UDP errors
and 242 UDP Rcvbuf errors:
1405361538 1920    0    0  2389731  553      930    0  878   242    0
1405361541 6138    0    0  7619178  153     6200    0    0     0    159 *ERR*

tc-recv.log file messages explained

Download the recv.log file analyser (Perl script for Linux, might run on Windows)

OK? Message Meaning
Received announcement for channel (subscribed) Channel announced and will be received (in recv-channels.ini)
Received announcement for channel (not subscribed) Channel announced but will not be received (not in recv-channels.ini)
Did not join channel (all files of filelist already received) No need to join channel
Connecting to announcement channel (announcement channel) Connecting
Connected to announcement channel (announcement channel) Connected
Connecting to data channel (invited) Connecting
Connected to data channel (invited) Connected
Disconnecting from data channel (finished) Finishing
Disconnect from data channel completed (finished) Finished
Disconnect from data channel completed (channel closed) Closed
Got no reannouncement for channel releasing the channel Forget the channel
Got no reinvitation for channel releasing the channel Forget the channel
Reconnecting to data channel (channel lost) Rejoin after loss???
Reconnecting to announcement channel (channel lost) Rejoin after loss???
Reconnected to announcement channel (channel was lost) Rejoin after loss???
Disconnecting from data channel (channel lost) Finishing after loss???
Disconnecting from announcement channel (channel lost) Finishing after loss???
Disconnect from data channel completed (channel lost) Finish after loss???
Disconnect from announcement channel completed (channel lost) Finish after loss???
Received filelist L on channel File listing
Received file I on channel A file (specified id)
Delivered file F id I from channel Same file (specified name and id)
Delivered all N files of filelist L from channel All files
All N files of filelist L received All files received
Internal file system full - removing information about delivered file ERROR: see troubleshooting guide D9, D12, D123; Either the file database size is too low (should be > 500 MB), or too slow (not on RAMdisk), or packets lost (see Lost Messages).
Internal file system full - removing received job ERROR: see troubleshooting guide D9, D12, D123; Either the file database size is too low (should be > 500 MB), or too slow (not on RAMdisk), or packets lost (see Lost Messages).
Deleted filelist Maybe due to "Internal file system full"???
Failed to open data channel (wrong interface address) No data was received a couple of seconds after channel was opened (i.e. multicast channel joined), either because of slow network, or because the PID is not selected and multicast channel not available at all. If due to delays then it should succeed a few seconds later and start receiving files. Frequent messages might indicate a performance problem (although no files might be lost).
File transmission I ended: filelist missing/incomplete Can be ignored
File transmission I interrupted: filelist missing/incomplete Can be ignored
File transmission I ended/interrupted: filelist missing/incomplete Can be ignored
Write error for file No space left on device ERROR: obvious - your disk has filled up
Missed parts of file ERROR: a file has been lost because it was missing some parts
Missed file ERROR: a file has been lost, see surrounding messages for the reason (eg. full database, lost messages, late opening of channels)
Channel C Lost message M (message lost) ERROR: Lost messages mean incomplete files and will cause losses in most cases
File transmission I ended: N file(s) missing/incomplete Can be ignored, related to the above filelist messages
File transmission I interrupted: N file(s) missing/incomplete Can be ignored, related to the above filelist messages
File transmission I ended\/interrupted: N file(s)* missing/incomplete Can be ignored, related to the above filelist messages
(key missing) Can be related to queuing in the EKU communication, not always an error. If persistent, the EKU communication has stopped, either by hardware (EKU heat, hangup, etc.) or by software, see Critical below.
Missing key Can be related to queuing in the EKU communication, not always an error. If persistent, the EKU communication has stopped, either by hardware (EKU heat, hangup, etc.) or by software, see Critical below.
Critical dongle error (eToken server terminated). Trying to restart child. Is mostly related to EKU communication problems and the above messages. If no response from the EKU for longer than 10 sec, this message is thrown and a restart initiated. If after restart the EKU ist still unresponsive it will be disabled.

Regarding "message lost", a slow file database is responsible for 80 % of the local losses. Other reasons are: