AIPS HELP file for VLBAARCH in 31DEC24
As of Thu Apr 25 13:54:39 2024
VLBAARCH: Procedure to archive VLBA correlator data.
INPUTS
INTAPE 0.0 9.0 Tape drive #. Use 0 for disk!
DATAIN Disk file name, jobs location
-> specify the name FORMAT
FORMAT 0.0 1.0 Disk file name format:
0 -> job number + 0001.FITS
1 -> FITLD style, which is
[:][#]
NCOUNT 0.0 9999.0 Number of VLBA tapes to load
OR number of disk files to
load -> then leave APARM=0
APARM No of FILES PER TAPE to load
NFILES 0.0 9999.0 Number of files to skip, e.g.
519 to start @ 5200001.FITS
ONLY for disk or FIRST TAPE
OUTNAME Output filename, 9 characters
Use - or multiple
passes : [ pass#]
e.g., "BX999Z P1" (= 9 char)
OUTDISK 0.0 15.0 Disk drive with ample space
SUBARRAY 0.0 10.0 Set <> 0 ONLY if you do have
subarrays, otherwise leave 0
set subarray to #>=2 of subs
BANDPOL For non-standard polarization
* USE ONLY FOR ODD SETUPS *
DATAOUT Archive directory or logical:
VLBAARCH combines VLBALOAD, VLBAFIX and FITAB @01JAN04
To use these procedures type 'RUN VLBAARCH'.
For disk space requirements see 'EXPLAIN VLBAARCH'
Do NOT run simultaneous VLBAARCH's in the same userno!
HELP SECTION
VLBAARCH
VLBAARCH: Procedure to archive VLBA correlator data.
Type: Procedure
Use: VLBAARCH is a procedure that loads VLBA data with weights
above 0.1 (versus 0.7 for normal VLBALOAD) from tape or
disk and may be used for any data from the VLBA correlator
VLBAARCH will perform VLBAFIX, VLBAFPOL and use FITAB to
write the output to FITS disk files. It clears the catalog
(i.e. deletes or 'zaps' files that are not needed anymore)
Type RUN VLBAARCH (after VERSION 'MYAREA' where MYAREA is
a defined logical where both VLBAARCH.001 and VLBAARCH.HLP
reside) to make the VLBAARCH procedure available.
Note that VLBAARCH permanently flags data with weights
less than 0.7 (which is fine for almost all data sets),
does not apply any compression, attaches a CL table with a
default interval of 15 seconds (0.25 minutes), and now is
THE DEFAULT PROCEDURE TO ARCHIVE VLBA USER DATA FROM JAN04
VLBAARCH redefines ordinary VLBALOAD, VLBAFIX and VLBAFPOL
To use VLBAUTIL's VLBALOAD, VLBAFIX and VLBAFPOL you have
to restore them with 'RUN VLBAUTIL' (and vice versa) again
Documentor: Lorant Sjouwerman, NRAO - version 1: 01JAN04
Related Programs: VLBAUTIL (VLBALOAD, VLBAFIX, VLBAFPOL) FITAB
Adverbs:
INTAPE......The input tape drive number. Use 0 for disk files.
Do not mount the tape, but do let it settle for a
while after loading the tape. It will be ejected
when the procedure has ended - if it does not come
out by it self, something went wrong in the script
Multiple tapes (NCOUNT>1): archiving starts after
loading the first batch of tapes, ejecting each
after it has finished loading it and requiring you
to load the next tape until archiving starts after
loading (and ends with ejecting) the last tape.
Use DATAIN and FORMAT for disk files - also see NFILES below
DATAIN......Name of the disk file or the logical or directory
where the indicvidual correlator jobs can be found
(for jobs, DATAIN uses the standardized job names)
FORMAT......Switch to select the disk file name convention.
NCOUNT......The number of tapes to read (use APARM) or the
number of files to read from disk.
APARM.......Leave 0 for disk loads - else this is the number
of files to read in order for each of the tapes.
Read the number of files on a VLBA correlator tape
from the listing inserted in the tape case(s).
NFILES......Number of files to skip on the tape or disk before
reading data. For example, 1 => start of next file,
-1 => start of previous file, 0 => no tape movement
at all, i.e., start where the tape is now. If you
have more than one tape NFILES ONLY APPLIES to the
first tape - all remaining will use NFILES = 0 !
(continue at the start of the next tape is probably
what you want when loading VLBA distribution tapes)
NFILES is also used for disk files, in particular
for disk files named after job numbers. E.g, use
NFILES = 519 to start with job number 520, in which
case the file will be named 5200001.FITS. NCOUNT>1
will result in the next job number (521) to be read
if FORMAT is set to zero (0). If FORMAT is non-zero
then disk file names assume the FITLD default, i.e.
a literal file name to which a number is appended.
See DATAIN in EXPLAIN FITLD for more on this.
OUTNAME.....The name for the output files, usually the EXP-code
and 'EXP-code P1' etc for multiple passes. The name
must be 9 characters or less to avoid overflow. In
the past, before Aug04, longer names such as a full
'EXP-code PASS1' etc have been used, but we needed
more characters in the description to distinguish
files/frequency-id that are very close in frequency
OUTDISK.....The disk drive number for the output data. Choose
a disk with a large amount of space for processing.
If TkB is the total number of kilobytes from all
distribution tapes or disk files, then you will use
about 1.5*TkB for a single frequency file with the
correct polarization labeling (single or all polzn)
and about 3-4*TkB for multi-frequency, dual polzn.
In the latter case, VLBAARCH will use other disks,
i.e. IN/OUTDISK parameters are set to zero for the
used archive VLBAFIX/FPOL/FITS(=FITAB) procedures.
SUBARRAY....Set to number of subarrays if more than one - one
(sub)array is not a subarray -> leave 0 for one.
BANDPOL.....Usually BANDPOL is determined correctly, but if you
you know it is going to be wrong, specify it here.
These are usually very odd cases, e.g., 6IFs RLLLRR
and BANDPOL is only used in case the script cannot
determine it - so it is safe to use in a data set
with multiple determinable BANDPOLs and one odd one
DATAOUT.....To specify where the files should go (usually just
a logical, e.g., DA01: for the first data disk).
All output names should be unique if configured as:
DATAOUT=OUTNAME+_#ST#IF+BW[K/M]HZ#CH+FREQ+GHZ+ID,
where #ST stands for number of 'STOKES' parameters
for unknown, or #POL stands for total polarization
(R=RR, L=LL, H=half, ie RR+LL, F=full, V=VV, H=HH)
#IF=number of IFs times BW=the bandwidth per IF in
KHZ or MHZ, #CH is number of channels per IF, and
FREQ is the frequency at the reference pixel in GHz.
Furthermore there is a freq-id/sequence appended
to DATAOUT. When TkB is the total number of kBytes
from all distribution tapes or disk files, then you
will need about 1.5*TkB here for all archive files.
Note: if DATAOUT is on OUTDISK you need ~5*TkB max.
The total DATAOUT length should be <=48 characters,
i.e., DATAOUT given should be 4 char +":" or less.
Thus DATAOUT='HOME:' and 'DA01:' etc are OK options
WHEN TO USE VLBAARCH
VLBAARCH loads the data from one or more VLBA export tapes or disk files
uncompressed into AIPS, it splits off the different frequencies,
possibly corrects for polarization labeling and writes out the FITS
files uncompressed to disk with standard names. When it ends
successfully, no trace will be left on the AIPS disks. This is not a
calibration pipeline but a standard way to archive VLBA data in user
friendly and recognizable disk files.
Do not use VLBAARCH if you need to apply data selection criteria or
otherwise need more control over e.g. FITLD. For this redefine the
VLBAUTIL procedures with RUN VLBAUTIL (see 'EXPLAIN VLBAUTIL') or use
the individual AIPS tasks.
VLBAARCH "step-by-step" : (see also 'EXPLAIN VLBAUTIL')
> load all the raw correlator data from tape or disk uncompressed into
AIPS using VLBALOAD (only keep weights > 0.7; may create more than
Pone file)
- merge GC, TY, and PC tables for each of the files if needed
- split out dual frequency observed S/X and 90/50 data sets if needed
- split out different frequency-ID's (usually different observing bands)
- reindex all files with INDXR using a CL-interval of 0.25 (15 seconds)
> correct all data sets with VLBAFIX for sub-arrays, sort order,
polarization
- sort the data in the right TB sort order with MSORT if needed
- arrange in subarrays using USUBA
- correct for polarization labeling (VLBAFPOL)
> write each of the data sets to an uncompressed disk fits file using
FITAB
- file names are usually like =
?POL?IF?MHZ?CH_?GHZxx where ?POL = F(ull)POL, H(alf)POL (ie RR+LL),
RPOL or LPOL (or # STokes) where ?IF?MHZ = # IF's per POL, each ?MHZ
wide, devided in # CH channels and where ?GHZ is the observing
frequency of this frequency-ID (xx) file [older versions (from
jun03-dec03) have .FQ-#, .FXPOL, etc.]
> remove all the data from the AIPS disks (and cross your fingers!)
Then the files should appear in NRAO's (VLBA) data archive as one or
more multi-source, single frequency-ID FITS files. You can grab these if
you can (ie, they are public or you know your project key), and they
should be okay for you to start calibrating (eg with VLBACALA, VLBARUN
or ACCOR), presuming you have examined your data and considered editing
bad data before starting. The exact data handling one can get from the
HIstory file (using AIPS' PRTHI).
EXPLAIN SECTION