NetDC manual [ back ] [ forward ]

After request routing is finished, each data site turns to their own local processing assignment. If a local file such as "response.request" or "data.request" should be found in the user's request directory, then NetDC will call the appropriate functions for handling each type of request. Because response data tend to be processed in a different fashion than waveform data, and so on for other data types, NetDC maintains separate functions to handle each data type.

The code in NetDC organizes a series of function branches to implement this concept. Inventory data is handled by "local_inventory()" in a source file called "local_inv.c". Response data is handled by "local_response()" in a source file called "local_resp.c". This naming convention will generally be adhered to for functions that are called and remain running until data is returned. These functions are referred to as "persistent" and work well for data queries that take little time to process.

Waveform data tends to take more time to be assembled and can face delays of hours or even days before data is returned. This requires a slightly different calling structure where we have two functions to process a data request. One function is used for submission of the data request to the data center's processing system. The other is used to deliver data back to the NetDC system once the data are ready. For waveform requests, the functions are kept in the source file "local_data.c" and are named "local_data_submit()" and "local_data_receive()" respectively.

These "local" functions are generally considered "fixed" into the NetDC code and it should not be necessary for any user installing NetDC to modify the code for their own use. It may be necessary for the installer to make changes to the local functions, but this condition should be avoided as much as possible, because leaving the code as-is will make future software updates easier to implement.

Let us examine the flow of request processing through these local functions (Fig 5.1). A local function is called from the main routine and the request file for that data type is passed onto it. From there parameters are added to the request before forwarding on to the interface function.

Fig 5.1 - Local request processing flow

For example, a response request would require that "local_response()" be called with the file "response.request" listed as the file to read. "response.request" would contain just .RESP lines that the local site was to process. "local_response()" will take those lines from the file and add a header to the beginning which provides information about the user requesting the response data. All of this will be passed onto the interface routine for processing a response request by opening a pipe to it.

The interface routine is an independent executable called by the local function that is specifically designed for the user's installation site. This code needs to be developed at the time of installation in order to interface with the data center's unique data processing methodology. The convention is to name the executables in the style of "process_inv", "process_resp", and "process_data". These routines are generally called with no command line arguments but instead take their parameters through an input stream referred to as stdin. The requested data gets back to the local function that called it through a file that the interface routine creates. The filename of the returned data is initially determined by the local function and is passed onto the interface routine before processing begins.

Looking at the example of response request processing again, we would find "local_response()" running "process_resp" by opening a pipe and passing request information and the request lines through that pipe stream. "process_resp", reading the stream lines one at a time, pulls out the necessary request information and then comes to the request lines. For each request line, "process_resp" will invoke the local processing system to get the response data. "process_resp" will be coded to know exactly how to request this data and how to filter the output into a standard format. This return data will go to a filename that was designated by "local_response()". This way, when "process_resp" exits, "local_response()" will know exactly where to find the output.

Upon return to the local function, the output data file is read and the contents are transferred to a specially named file in the user's request directory. The data product is checked off in the checklist file and if that product type is ready for shipment (i.e. there is no other data to merge), the shipment routine is called and the product is sent to the user.

This description of local request processing sums up the general aspects of how NetDC handles these data requests. Each data type will have some degree of variation in terms of how the data query is carried out. The responsibility lies with the site installer to determine what is required to interface NetDC with the local site's information base. This should not require a great deal of the NetDC code modification. Instead, the calling structure has been organized such that the installer really only needs to modify or create the interface routine to access and filter the data that the user is requesting.

introduction •• overall concept •• request format •• request reception and delegation
datagrams •• local request processing •• inventory requests •• response requests
waveform requests •• product shipment •• installation and setup •• writing interface code
troubleshooting •• future implementations •• conclusion •• appendix A - summary of NetDC datagrams

About IRIS | Members | Programs | USArray | Seismic Monitor | Earthquakes | SeismoArchives
Mailing Lists | Stations | Data | Software | Publications | News | Contact | Site Map | Search