F.5 Very large data sets

FITTP and FITAB cannot write multi-volume tapes. Some spectral-line and VLBA databases (and perhaps some continuum databases) may be so large that the file cannot fit on one modern tape even with BLOCKING = 10. What can you do to backup your data? The simplest solution is to use FITAB. This task can write uv data in compressed form and can break up a data set into “pieces.” You write as many pieces as will fit on the first tape, noting what the piece number is when the end-of-tape is reached. You can then tell FITAB to begin with that piece number on the next tape (BDROP = n-1 where n is the number of the piece that encountered the end-of-tape). Unfortunately, the FITS format used by FITAB, while perfectly legitimate, is only understood by AIPS versions beginning with 15APR99. Furthermore, FITAB output is only understood by AIPS and OBIT. If the data set is so large that it will not fit on the required tape device using FITTP or the data must be taken to a system that does not understand FITAB’s format, there are several approaches that you can adopt.

First, you could SPLIT out the database into single-source databases and back each of these up individually. Alternatively, you could subdivide the large database in several smaller databases with UVCOP by specifying a different time range for each of the smaller databases and then back these up individually. Another way to solve the problem is to realize that the calibration and flagging information that you have carefully generated during the calibration is contained in the extension tables — the raw data that you loaded is not modified until you finally SPLIT out the individual sources. Consequently, you can write create a dummy uv database to which all the extension tables are attached with the task TASAV, then save this “database” on tape with FITTP. The raw visibilities can be saved in the form of copies of the archive tapes.