Skip to content

Retirement of the email-based BREQ_FAST service

Tags: data services

The email-based BREQ_FAST (Batch, Request, Fast) data request mechanism was developed in 1988, the same year the IRIS Data Management Center was formed, and 3 years prior to the DMC’s relocation to its long-term home in Seattle.  By far it has been our longest-lived request mechanism.  Initially BREQ_FAST returned so-called “full SEED” volumes, files containing both time series data and metadata in a single file.  Eventually a BREQ_FAST request could also return miniSEED (data only), dataless SEED (metadata only), or sync listings (availability).  In 2019 the IRIS DMC stopped delivering full SEED volumes.  By then we had a suite of well established web services, which are supported by many tools maintained by EarthScope and the research community, and offer those same or equivalent data types.  In fact, BREQ_FAST requests have been fulfilled using data from web services behind the scenes for many years.

The time has come to sunset this service; the BREQ_FAST request mechanism is being retired and will stop being supported by the EarthScope data management platform sometime between June 2024 and the end of the year.

Why retire BREQ_FAST?

There are a number of reasons for retiring BREQ_FAST, most of which are related to the resources that are required to keep it running in an environment where requests for data are expanding exponentially.  The custom download requires making a copy of data and holding it for many days for the requestor to come pick it up. This means every request requires storage to duplicate data in the archive.  The system also requires a significant amount of bookkeeping and, at times, manual intervention by staff members.  In addition, an email-based system cannot be used with authentication, which we are required to add to our services.  Finally, use of BREQ_FAST has been declining since 2016 and now accounts for less than 10% of the data delivered by the facility.

While this retirement will require changes to the workflows of those who use the service, many alternatives exist to retrieve the same information more conveniently for users and more efficiently for the facility.

What alternatives are available?

There are a number of excellent alternatives to retrieving miniSEED and StationXML (metadata).

The command-line FetchData program has been available for more than a decade and is capable of reading a BREQ_FAST-formatted request and retrieving miniSEED, StationXML, RESP, and/or SAC Poles and Zeros.  In fact, this is the program used at the facility to fulfill BREQ_FAST requests.  The FetchData program is even capable of finding and requesting data from a network of federated FDSN data centers, making it easy to get data from EarthScope and many other data centers at the same time.

To collect large volumes of miniSEED data, the ROVER program robustly retrieves data from our data center.  The program can recover from partial downloads and even add more data to existing downloads.

Beyond FetchData and ROVER there is the ever-popular Python-based ObsPy, and many other tools that can access data via web services.

Need help understanding the alternatives or have other questions? If you have questions regarding how to transition your use of BREQ_FAST to other tools or general questions about EarthScope services please send a message to data-help@earthscope.org and we’d be happy to help!