|
||||||
Project Focus |
Web Services at the DMC |
We are pleased to introduce a suite of core web-services
for the direct access of waveform and metadata held at the IRIS DMC.
The web services use standard HTTP protocol technologies and are relatively
simple to understand and use. They can be accessed programmatically with
a wide range of programming languages (e.g. Perl, Python, Java), command line
utilities such as wget and curl or
with any web browser. Additionally, by using HTTP over port 80, the services
are not restricted by network firewalls. We anticipate that the new web services
will be used to develop novel software applications including web based applications
both internally and externally to the IRIS DMC. We also anticipate that the web
services will make the data that we manage more accessible to users outside of
the traditional academic seismological community.
The following table summarizes the core web services.
| Service Name | Description |
|---|---|
| dataselect | Retrieves individual segments of waveform time-series data; one segment per query. Returns miniSEED output. |
| tracedsp | Digitally filters miniSEED data produced by the dataselect webservice. Returns miniSEED or SAC output. |
| ms2ascii | Converts miniSEED data produced by dataselect or tracedsp into ASCII output. The ASCII output can be easily read without knowledge of SAC or miniSEED. Especially useful to non-seismologist. For example, weather researchers who wish to use barometric reading from seismic stations. |
| plotter | Reads miniSEED data from dataselect or tracedsp and produces time series plots in PNG format. |
| timeseries | Combines the functionality of: dataselect, tracedsp, ms2ascii and plotter. Retrieves individual segments of waveform time-series data; one segment per query. The data delivered can be optionally gain corrected, digitally filtered and deconvolved with standard filtering algorithms. The output data can be in several formats including: miniSEED, SAC, ASCII and plot images (png). |
| bulkdataselect | Retrieves multiple segments of time-series waveform data in a single query. The data is sent to the caller in miniSEED output. |
| resp | Retrieves channel instrument response in RESP format |
| sacpz | Retrieves channel instrument response in SAC ASCII (poles and zeros) format. |
| station | Retrieves station and channel information in StationXML format. |
| availability | Retrieves waveform data availability information in two formats: XML, list. The XML format is similar to StationXML. The list format can be directly fed to the bulkdataselect service. |
| quake (coming soon) |
Retrieves earthquake event information in QuakeML format |
Each web service has a home page located at the http://www.iris.edu/ws/<SERVICENAME> where you can find documentation on how to use the web service.
The web services use simple, and easily understood, calling conventions. They do not use complex web service mechanisms such as SOAP. If you can figure out how to craft a URL, you should be able to get going quickly. All but one of the services (bulkdataselect) use URL requests of the form:
http://www.iris.edu/ws/<SERVICENAME>/query?<PARAMETERS>
As a illustrative example, consider a request to generate and retrieve a waveform plot using the timeseries web service for the station ANMO, the date February 27th, 2010 and the time range 06:30 GMT to 10:30 GMT (coincidental with an 8.8 event in Chile). ANMO belongs the IU network. For this example we choose location and channel identifiers of 00 and BHZ respectively. The following table summarizes the query parameters:
Name | Value |
|---|---|
network | IU |
station | ANMO |
location | 00 |
channel | BHZ |
starttime | 2010-02-27T06:30:00 |
endtime | 2010-02-27T10:30:00 |
output | plot |
The parameters are serialized as a set of name=value pairs separated by ampersand (&) characters. This is the standard way in which query parameters are represented for HTTP GET requests. For this example we will have:
network=IU&station=ANMO&location=00&channel=BHZ&starttime=2010-02-27T06.30.00&endtime=2010-02-27T10.30.00&output=plot
The final query becomes:
This will produce a plot looking like the following:
This plot shows the units as "COUNTS". If sensor units were desired, add scale=AUTO to the parameter list like this:
If miniSEED data is desired instead of a plot, simply change output=plot to output=miniseed like this:
To aid in the understanding of how to construct request URL's, most of the web services have a "URL Builder" form where one can enter parameter values and see the resulting URL.
Shown here is the URL Builder for the timeseries web service. This web service takes a relatively complex set of parameters, some of which are mutually exclusive. The form dynamically enables and disables fields which are mutually exclusive. This can be a time saver when trying to understand call logic.
The waveform access web services bulkdataselect, dataselect and timeseries retrieve data from both the waveform archive and the real time collection system (BUD) . The services seamlessly combine the two storage systems so that the user does not need to be aware of their distinction. When overlapping data exists in both systems, data is preferentially pulled from the archive.
The bulkdataselect service is useful for getting large collections of channel waveform segments. For its input it takes a list of channels and time ranges as shown here:
TA A25A -- BHZ 2010-084T00:00:00 2010-091T00:00:00
IU ANMO 00 BHZ 2010-084T00:00:00 2010-091T00:00:00
IU ANMO 10 HHZ 2010-084T00:00:00 2010-091T00:00:00
II KURK 00 BHN 2010-084T00:00:00 2010-091T00:00:00
The input list can be long. As the web service processes a request, it streams
the data back to the client. By streaming data back to the client, the memory
footprint on the server is greatly reduced, thus making large request processing
scaleable. The data is sent back as one concatenated bundle of miniSEED data.
The input list is submitted with an HTTP POST request. This can be easily done
with the UNIX commands wget and curl. The online
documentation at http://www.iris.edu/ws/bulkdataselect explains
this in more detail.
The dataselect and timeseries web services are useful for retrieving individual segments of waveform data. In some applications this maybe easier to use. As mentioned previously timeseries provides several output format options as well as signal processing capability. In general bulkdataselect is a faster download mechanism and its use is encouraged for users who simply want large blocks of data.
The resp, sacpz, and station web services allow the user to retrieve channel response information in three different formats: RESP, SAC Poles and Zeros and StationXML. In addition, the station service returns other meta information including geographic location.
The availability web service returns information about waveform availability. The output from this service can be directly fed to the bulkdataselect service. The following example demonstrates using the wget command to retrieve all B channel waveforms within 30 degrees of the Chilean event discussed above.
wget "http://www.iris.edu/ws/availability/query?cha=B*&start=2010-02-27T06:30:00&end=2010-02-27T10:30:00&lat=-36&lon=-73&maxradius=30.0&output=bulk" -O chile.txt wget --post-file=chile.txt -O chile.miniseed http://www.iris.edu/ws/bulkdataselect/query
The first wget command retrieves the list channel time range segments and saves the results to a file named chile.txt The second wget command sends the list to the bulkdataselect service and saves the output to chile.miniseed.
All of the metadata services take network code, station code, location code and channel code query parameters as well as time range information. The station and availability web services also allow for geographic location queries.
The web services dataselect, tracedsp, ms2ascii, plotter, resp and timeseries take part in a system called ICAB: IRIS Caching Artifact Builder. ICAB allows for the passing of data between web services through a token reference system. When a client invokes an ICAB webservice, instead of downloading the the generated data item (e.g. a miniSEED file), it can return a token string that references that data item. The data items are known as "artifacts". The artifacts persist for a period of time in a cache. Tokens, which refer to cached artifacts, can be passed from one web service to another. For example, a token generated by dataselect could be passed to tracedsp and then the token from tracedsp could be passed to the plotter web service.
The timeseries web service wraps the functionality of the ICAB services dataselect, tracedsp, ms2ascii, plotter and resp. We anticipate that more ICAB services will be developed in the future. You can read more about ICAB at the article posted at: http://www.iris.washington.edu/ws/icab.html
Four web service clients written in Perl are available for download at http://www.iris.edu/ws/wsclients: FetchBulkData, FetchMetadata, FetchRESP, FetchSACPZ. These scripts should run with any relatively recent Perl interpreter. Run the scripts with no arguments to see useful usage information. Please see the online documentation for more information.
This is the first "big" rollout of web services providing access to the IRIS DMC's main archive and database. Despite our best testing efforts, there will undoubtedly be hiccups as we release the services to a wide audience. As we move forward we will be both refining our existing services and providing new services.
Submitted by Bruce Weertman, IRIS DMC
Support: This work is funded by the NSF (Award # EAR-0552316) and the USGS (Award # G10AC00533).
