NetDC manual [ back ] [ forward ]

NetDC is code that runs in the background constantly. It is meant to be automated and involve little if any manual work from the data center administrators. However errors will occur, processing will cease, machines will go down, and disks will fill up. It is this reason that NetDC administrators should keep an eye on NetDC's functioning and to clean up after it when things go wrong.

As NetDC is set into full operation, some processes need to be put "on the clock". That is to say that there are a few programs that need to be put into "cron". As user "netdc" you can edit your cron table with the command "crontab -e". On some systems, such as Sun Solaris® , you might have to specify the EDITOR environment variable with the name of a text editor that you typically use, such as "vi".

In the cron table, you'll want to put in the following references. Note that this just represents examples. Each installation will have different pathnames and time settings. Refer to the man pages on "cron" and "crontab" if you do not understand these entries.

First:

0 * * * * /users/netdc/bin/shipment_handler

"shipment_handler" will pass through each NetDC request directory and check two different things. If the request is flagged as having all data products shipped to the user, then the directory is deleted. If the user has no other request directories, the directory named after the user is deleted as well. "shipment_handler" also checks for user directories where data merges are in progress. If the data merge has taken longer than the requested number of days to complete, a shipment is forced on the data present in the directory and the request is flagged as having all products shipped.

There may be occasions where shipment_handler overlooks old request directories that never completed processing. The administrator should assess whether those old directories should be deleted manually.

Finally:

30 1 * * * find /users/netdc/tmp -mtime +1 -name 'netdc.*'-exec rm -f {} \;

This represents an example cleanup command. By using "find" with the "-mtime" parameter, it is possible to remove files that are older than a certain number of days. In the above example, the filename filter "-name" is also employed, so that only files in the directory "/users/netdc/tmp" with the name pattern "netdc.*" are removed after they are one day old. Notice the execute parameter at the end of the expression that carries out the removal. For removing old directories, use "rm -rf" instead.

It is recommended that this removal technique be employed with the FTP directory where NetDC writes its shipments. That way, shipment files can remain in the FTP directory for a few days before being removed to free up disk space.

When NetDC encounters errors, it will more often than not return an ERROR message and an error code. These messages can sometimes be helpful, whereas others tend to be more technical and only useful to those who understand the source code. There are two different places that NetDC reports errors.

The first avenue is through email to the NetDC administrator. Sometimes the mail is sent intentionally by NetDC to report some warning, other times, the mail arrives from a non-zero exit code, which is then forwarded to the administrator with a message from the program describing the fatal error or a message from the operating system. In pretty much all of the fatal exit cases the error code reported can be tracked down to a specific section of source code within NetDC. Many times the error code is reported along with the name of the function that failed. A technically willing administrator can look at the source code to see the exact point of the error and troubleshoot from there.

The other place to report errors is through the file "netdc_activity.log", which is located in the "tables" directory. This file is not just for logging error reports, but also for displaying NetDC's current activity, especially as relates to incoming and outgoing messages. Administrators are encouraged to monitor this file to assure NetDC's proper functioning and also to look there for additional information in the case of a fatal error.

introduction •• overall concept •• request format •• request reception and delegation
datagrams •• local request processing •• inventory requests •• response requests
waveform requests •• product shipment •• installation and setup •• writing interface code
troubleshooting •• future implementations •• conclusion •• appendix A - summary of NetDC datagrams

About IRIS | Members | Programs | USArray | Seismic Monitor | Earthquakes | SeismoArchives
Mailing Lists | Stations | Data | Software | Publications | News | Contact | Site Map | Search