4.1 Copying data into AIPS multi-source disk files

There are several ways to write VLA data to AIPS multi-source uv data sets on disk. They include:

  1. For data from the historic VLA, download the files from the archive and run FILLM to load the data. See O.1 for the details.
  2. For data from the new EVLA, download the files from the archive and run BDF2AIPS to load the data. See 4.1.1 below for the details.
  3. For an AIPS multi-source data set written to a FITS tape or disk file during an earlier AIPS session, use UVLOD or FITLD to read the tape.
  4. Single-source data sets that are already on disk may be combined with multiple executions of UV2MS or, if the files are dissimilar, with multiple executions of MULTI followed by DBCON.
  5. Data from the Australia Telescope may be loaded from disk files into AIPS using the task ATLOD which is now included with AIPS.

Data from other telescopes can be read into AIPS only if they are written in AIPS-like FITS files already or if you have a special format-translation program for that telescope. The VLBA correlator produces a format which is translated by the standard AIPS task FITLD; see 4.1.2. A translation task for the Westerbork Synthesis Telescope (WSLOD) is available from the Dutch, but is not distributed by the NRAO with the normal AIPS system.

4.1.1 Reading EVLA archive files into AIPS

The NRAO Archive makes available, among other things, data from the EVLA, which began observing in January 2010. Your EVLA data are stored as an “Science Data Model” (SDM) format file in “SDMBDF” (Science Data Model Binary Data Format) in the NRAO archive. They may be read out of the archive in that format or a CASA measurement set format. Go to the web page

http://archive.cv.nrao.edu/

and select the Advanced Query Tool. Fill out enough of the form to describe your data and submit the query. If the data are not yet public, you will need the Locked Project Access Key which may be obtained from the NRAO data analysts. To avoid the need for this key, you may log in to my.nrao.edu after which it will know if you are entitled to access particular locked projects. The query will return a list of the data sets which meet your specifications. On this form, enter your e-mail address. If you have disk access at NRAO Socorro and have a world-writable directory, enter that directory as the download destination. Otherwise let the download go to /home/e2earchive and be sure to check the tar file button. Select the SDM-BDF dataset format if you wish to reduce the data fully in AIPS. If you select the CASA MS format, you will be required to use CASA either for all your work or at least to convert the MS format to a UVFITS format which AIPS can read. Sadly, such UVFITS files lack a lot of useful information and so they are not recommended. The archive tool will notify you, by e-mail, when your data set is available and will provide information on how to retrieve the data. Copy the tar file to a data directory that you control, and untar it.

Unlocked files will be downloaded to the NRAO public ftp site

ftp://ftp.aoc.nrao.edu/e2earchive/

by default and you may then use ftp to copy the file to your computer. Locked files will go to a protected ftp site and you must use ftp to download those, even within NRAO. The instructions for downloading will be e-mailed to you. Be sure to specify binary for the copy. If you are located in the AOC in Socorro, you may set an environment variable to the archive location, e.g.

export E2E=/home/ftp/pub/e2earchive  C R

for bash shells

setenv E2E /home/ftp/pub/e2earchive  C R

for C shells such as tcsh

and simply read unlocked data files directly from the public download area. Note that the file will be deleted automatically after 48 hours in both public and protected data areas.

You may now run AIPS and load the data via the verbs BDFLIST and BDF2AIPS. These verbs run programs in the OBIT software package to load your data directly into AIPS including flag (FG), index (NX), calibration (CL), over-the-top (OT), SysPower (SY), and CalDevice (CD) tables which you will not get from CASA. Note that these verbs require that OBIT be installed on your computer — as it is in Socorro — and that ObitTalk be in your $PATH. OBIT is relatively easy to install and may be obtained from www.cv.nrao.edu/~b
cotton/Obit.html

SDMBDF files may be read into AIPS using BDFLIST to learn what is in your data set and then BDF2AIPS to translate the data. Thus

> DEFAULT BDF2AIPS; INP  C R

to initialize all relevant adverbs.

> DOWAIT 2 ; DOCRT 1  C R

to wait for the verbs to finish and to display the log file on the terminal after the OBIT task finishes. DOWAIT 1 displays the messages as they are generated but is insensitive to returned error conditions from the OBIT tasks. Be sure to set DOWAIT -1 after using BDF2AIPS.

> ASDMF(1) = ’path_to_asdm_dir  C R

to set the path name to your data directory into the adverb. Note the lack of close quote so that case is preserved.

> ASDMF(2) = ’asdm_file_name  C R

to put the rest of the data file name in the second adverb since the names are almost always longer than 64 characters. Trailing blanks in ASDMF(1) will be ignored.

> BDFLIST  C R

to list the contents of the SDMBDF. Note particularly the “configuration” numbers.

> OUTNA myname  C R

to set the AIPS name.

> OUTCL ’ ’  C R

to take default (UVEVLA) class.

> OUTDI 3  C R

to write the data to disk 3 (one with enough space).

> DOUVCOMP FALSE  C R

to write visibilities in uncompressed format. There are no weights at present, so there is no loss of information in compressed format, but the conversion from compressed format costs more than reading the larger data files.

> FOR CONFIG = 0:100 ; BDF2AIPS; END  C R

to load all of the configurations in your data, terminating with error messages on the first configuration number not present in your data (when DOWAIT is 2).

There are other adverbs — NCHAN, NIF, BAND, and CALCODE — available if needed to limit which data are read. CONFIG is frequently all that is needed to select data, but these others may be needed if more complicated modes of observing were used.

If BDF2AIPS is executing correctly, your message terminal will report information about the data being loaded including the data selection, the data scans loaded, and information about some of the tables written. Once BDF2AIPS has completed, you can find the database on disk using:

> INDI 0 ; UCAT  C R

to list all cataloged uv files

This should produce a listing such as:

Catalog on disk  3  
Cat Usid Mapname      Class  Seq  Pt     Last access      Stat  
  1  103 MyName      .UVDATA.   1 UV 05-FEB-2017 12:34:16

You might then examine the header information for the disk data set by:

> INDI 3 ; GETN 1 ; IMHEAD  C R

set name adverbs, list header in detail.

This should produce a listing like:

AIPS 1: Image=MULTI     (UV)         Filename=MyName      .UVDATA.   1  
AIPS 1: Telescope=EVLA               Receiver=EVLA  
AIPS 1: Observer=Dr. Crys            User #=  200  
AIPS 1: Observ. date=29-OCT-2016     Map date=29-JAN-2018  
AIPS 1: # visibilities    876000     Sort order  TB  
AIPS 1: Rand axes: UU-L-SIN  VV-L-SIN  WW-L-SIN  BASELINE  TIME1  
AIPS 1:            SOURCE  INTTIM  WEIGHT  SCALE  
AIPS 1: ----------------------------------------------------------------  
AIPS 1: Type    Pixels   Coord value     at Pixel     Coord incr   Rotat  
AIPS 1: COMPLEX      1   1.0000000E+00       1.00  1.0000000E+00    0.00  
AIPS 1: STOKES       2  -1.0000000E+00       1.00 -1.0000000E+00    0.00  
AIPS 1: FREQ       128   4.0393400E+09      65.00  1.0000000E+06    0.00  
AIPS 1: IF          31   1.0000000E+00       1.00  1.0000000E+00    0.00  
AIPS 1: RA           1    00 00 00.000       1.00       3600.000    0.00  
AIPS 1: DEC          1    00 00 00.000       1.00       3600.000    0.00  
AIPS 1: ----------------------------------------------------------------  
AIPS 1: Coordinate equinox 2000.00  
AIPS 1: Maximum version number of extension files of type SU is   1  
AIPS 1: Maximum version number of extension files of type AN is   1  
AIPS 1: Maximum version number of extension files of type FQ is   1  
AIPS 1: Maximum version number of extension files of type NX is   1  
AIPS 1: Maximum version number of extension files of type CD is   1  
AIPS 1: Maximum version number of extension files of type SY is   1  
AIPS 1: Maximum version number of extension files of type GC is   1  
AIPS 1: Maximum version number of extension files of type CT is   1  
AIPS 1: Maximum version number of extension files of type FG is   1  
AIPS 1: Maximum version number of extension files of type WX is   1  
AIPS 1: Maximum version number of extension files of type CL is   1  
AIPS 1: Maximum version number of extension files of type HI is   1

This header identifies the file as a multi-source data set (Image=MULTI) with 876000 floating-point visibilities in time-baseline (TB) order. There are 128 spectral channels in each of 31 entries on the IF axis. These correspond to “spectral windows” in the correlator and have, at best, only a loose correspondence to the separate electronic channels. The description of the frequency (FREQ) axis shows that the first IF is at 4039 MHz and has 128 MHz total bandwidth. The parameters of the other IFs are determined from the data in the FQ table file and cannot be read directly from this header; these values are shown in the SCANlisting from LISTR. The header shown above indicates that the data are in compressed format since the number of pixels on the COMPLEX axis is 1 and the WEIGHT and SCALE random parameters are present. Uncompressed data does not use these random parameters and has 3 pixels on the COMPLEX axis.

If your experiment contains data from several bands BDF2AIPS will place the data from each band in separate data sets. Also, if you observed with several sets of frequencies or bandwidths in a given observing run these will be assigned different FQ numbers by BDF2AIPS. You can determine which frequencies correspond to which FQ numbers from the SCANlisting provided by LISTR. If you find multiple FQ numbers in your data set, we strongly advise you to run UVCOP to separate them into different files. This will greatly simplify your data reduction.

4.1.2 Reading data from FITS files with FITLD

FITLD is used to read FITS-format disk files (and tapes) into AIPS. It recognizes images, single- and multi-source uv data sets, and the special FITS uv-data tables produced by the VLBA and DiFX correlators (“FITS-IDI” format). In particular, VLA data sets that have been read into AIPS previously with FILLM and then saved to tape (or pseudo-tape disk) files with FITTP and FITAB can be recovered for further processing with task FITLD. (The older task UVLOD will also work with uv data sets in FITS format, but it cannot handle image or FITS-IDI format files.)

A multi-source data file with all of its tables can be read from a FITS FILE by:

> TASK FITLD’ ; INP  C R

to review the inputs needed.

> DATAIN filename  C R

to specify the FITS disk file (see 3.10.3).

> DOUVCOMP FALSE  C R

to write visibilities in uncompressed format.

> OUTNA ’ ’  C R

take default (previous AIPS) name.

> OUTCL ’ ’  C R

take default (previous AIPS) class.

> OUTSEQ 0  C R

take default (previous AIPS) sequence #.

> OUTDI 3  C R

to write the data to disk 3 (one with enough space).

> INP  C R

to review the inputs (several apply only to VLBA format files).

> GO  C R

to run the program when you’re satisfied with inputs.

The data-selection adverbs SOURCES, QUAL, CALCODE, and TIMERANG and the table-control adverbs CLINT and FQTOL are used for VLBA-format data only.. See Chapter 9 for more specific information.