As of Wed Jun 12 9:23:39 2024

VLBAARCH: Procedure to archive VLBA correlator data.


INTAPE          0.0         9.0    Tape drive #. Use 0 for disk!
DATAIN                             Disk file name, jobs location
                                      -> specify the name FORMAT
FORMAT          0.0         1.0    Disk file name format:
                                    0 -> job number + 0001.FITS
                                    1 -> FITLD style, which is
NCOUNT          0.0      9999.0    Number of VLBA tapes to load
                                     OR number of disk files to
                                     load -> then leave APARM=0
APARM                              No of FILES PER TAPE to load
NFILES          0.0      9999.0    Number of files to skip, e.g.
                                     519 to start @ 5200001.FITS
                                     ONLY for disk or FIRST TAPE
OUTNAME                            Output filename, 9 characters
                                    Use  - or multiple
                                    passes : [ pass#]
                                    e.g., "BX999Z P1" (= 9 char)
OUTDISK         0.0        15.0    Disk drive with ample space
SUBARRAY        0.0        10.0    Set <> 0 ONLY if you do have
                                    subarrays, otherwise leave 0
                                    set subarray to #>=2 of subs
BANDPOL                            For non-standard polarization
                                    * USE ONLY FOR ODD SETUPS *
DATAOUT                            Archive directory or logical:

          VLBAARCH combines VLBALOAD, VLBAFIX and FITAB @01JAN04
          To use these procedures type 'RUN VLBAARCH'.
          For disk space requirements see 'EXPLAIN VLBAARCH'
          Do NOT run simultaneous VLBAARCH's in the same userno!


VLBAARCH:           Procedure to archive VLBA correlator data.

Type: Procedure
Use:  VLBAARCH is a procedure that loads VLBA data with weights
      above 0.1 (versus 0.7 for normal VLBALOAD) from tape or
      disk and may be used for any data from the VLBA correlator
      VLBAARCH will perform VLBAFIX, VLBAFPOL and use FITAB to
      write the output to FITS disk files. It clears the catalog
      (i.e. deletes or 'zaps' files that are not needed anymore)
      Type RUN VLBAARCH (after VERSION 'MYAREA' where MYAREA is
      a defined logical where both VLBAARCH.001 and VLBAARCH.HLP
      reside) to make the VLBAARCH procedure available.
      Note that VLBAARCH permanently flags data with weights
      less than 0.7 (which is fine for almost all data sets),
      does not apply any compression, attaches a CL table with a
      default interval of 15 seconds (0.25 minutes), and now is


      VLBAARCH redefines ordinary VLBALOAD, VLBAFIX and VLBAFPOL
      to restore them with 'RUN VLBAUTIL' (and vice versa) again

Documentor:         Lorant Sjouwerman, NRAO - version 1: 01JAN04

  INTAPE......The input tape drive number. Use 0 for disk files.
              Do not mount the tape, but do let it settle for a
              while after loading the tape. It will be ejected
              when the procedure has ended - if it does not come
              out by it self, something went wrong in the script
              Multiple tapes (NCOUNT>1): archiving starts after
              loading the first batch of tapes, ejecting each
              after it has finished loading it and requiring you
              to load the next tape until archiving starts after
              loading (and ends with ejecting) the last tape.

  Use DATAIN and FORMAT for disk files - also see NFILES below

  DATAIN......Name of the disk file or the logical or directory
              where the indicvidual correlator jobs can be found
              (for jobs, DATAIN uses the standardized job names)

  FORMAT......Switch to select the disk file name convention.

  NCOUNT......The number of tapes to read (use APARM) or the
              number of files to read from disk.

  APARM.......Leave 0 for disk loads - else this is the number
              of files to read in order for each of the tapes.
              Read the number of files on a VLBA correlator tape
              from the listing inserted in the tape case(s).

  NFILES......Number of files to skip on the tape or disk before
              reading data. For example, 1 => start of next file,
              -1 => start of previous file, 0 => no tape movement
              at all, i.e., start where the tape is now. If you
              have more than one tape NFILES ONLY APPLIES to the
              first tape - all remaining will use NFILES = 0 !
              (continue at the start of the next tape is probably
              what you want when loading VLBA distribution tapes)

              NFILES is also used for disk files, in particular
              for disk files named after job numbers. E.g, use
              NFILES = 519 to start with job number 520, in which
              case the file will be named 5200001.FITS. NCOUNT>1
              will result in the next job number (521) to be read
              if FORMAT is set to zero (0). If FORMAT is non-zero
              then disk file names assume the FITLD default, i.e.
              a literal file name to which a number is appended.
              See DATAIN in EXPLAIN FITLD for more on this.

  OUTNAME.....The name for the output files, usually the EXP-code
              and 'EXP-code P1' etc for multiple passes. The name
              must be 9 characters or less to avoid overflow. In
              the past, before Aug04, longer names such as a full
              'EXP-code PASS1' etc have been used, but we needed
              more characters in the description to distinguish
              files/frequency-id that are very close in frequency

  OUTDISK.....The disk drive number for the output data. Choose
              a disk with a large amount of space for processing.
              If TkB is the total number of kilobytes from all
              distribution tapes or disk files, then you will use
              about 1.5*TkB for a single frequency file with the
              correct polarization labeling (single or all polzn)
              and about 3-4*TkB for multi-frequency, dual polzn.
              In the latter case, VLBAARCH will use other disks,
              i.e. IN/OUTDISK parameters are set to zero for the
              used archive VLBAFIX/FPOL/FITS(=FITAB) procedures.

  SUBARRAY....Set to number of subarrays if more than one - one
              (sub)array is not a subarray -> leave 0 for one.

  BANDPOL.....Usually BANDPOL is determined correctly, but if you
              you know it is going to be wrong, specify it here.
              These are usually very odd cases, e.g., 6IFs RLLLRR
              and BANDPOL is only used in case the script cannot
              determine it - so it is safe to use in a data set
              with multiple determinable BANDPOLs and one odd one

  DATAOUT.....To specify where the files should go (usually just
              a logical, e.g., DA01: for the first data disk).
              All output names should be unique if configured as:
              where #ST stands for number of 'STOKES' parameters
              for unknown, or #POL stands for total polarization
              (R=RR, L=LL, H=half, ie RR+LL, F=full, V=VV, H=HH)
              #IF=number of IFs times BW=the bandwidth per IF in
              KHZ or MHZ, #CH is number of channels per IF, and
              FREQ is the frequency at the reference pixel in GHz.
              Furthermore there is a freq-id/sequence appended
              to DATAOUT. When TkB is the total number of kBytes
              from all distribution tapes or disk files, then you
              will need about 1.5*TkB here for all archive files.
              Note: if DATAOUT is on OUTDISK you need ~5*TkB max.
              The total DATAOUT length should be <=48 characters,
              i.e., DATAOUT given should be 4 char +":" or less.
              Thus DATAOUT='HOME:' and 'DA01:' etc are OK options


VLBAARCH loads the data from one or more VLBA export tapes or disk files
uncompressed into AIPS, it splits off the different frequencies,
possibly corrects for polarization labeling and writes out the FITS
files uncompressed to disk with standard names. When it ends
successfully, no trace will be left on the AIPS disks. This is not a
calibration pipeline but a standard way to archive VLBA data in user
friendly and recognizable disk files.

Do not use VLBAARCH if you need to apply data selection criteria or
otherwise need more control over e.g. FITLD. For this redefine the
VLBAUTIL procedures with RUN VLBAUTIL (see 'EXPLAIN VLBAUTIL') or use
the individual AIPS tasks.

VLBAARCH "step-by-step" : (see also 'EXPLAIN VLBAUTIL')
> load all the raw correlator data from tape or disk uncompressed into
  AIPS using VLBALOAD (only keep weights > 0.7; may create more than
  Pone file)
- merge GC, TY, and PC tables for each of the files if needed
- split out dual frequency observed S/X and 90/50 data sets if needed
- split out different frequency-ID's (usually different observing bands)
- reindex all files with INDXR using a CL-interval of 0.25 (15 seconds)
> correct all data sets with VLBAFIX for sub-arrays, sort order,
- sort the data in the right TB sort order with MSORT if needed
- arrange in subarrays using USUBA
- correct for polarization labeling (VLBAFPOL)
> write each of the data sets to an uncompressed disk fits file using
- file names are usually like =
  ?POL?IF?MHZ?CH_?GHZxx where ?POL = F(ull)POL, H(alf)POL (ie RR+LL),
  RPOL or LPOL (or # STokes) where ?IF?MHZ = # IF's per POL, each ?MHZ
  wide, devided in # CH channels and where ?GHZ is the observing
  frequency of this frequency-ID (xx) file [older versions (from
  jun03-dec03) have .FQ-#, .FXPOL, etc.]
> remove all the data from the AIPS disks (and cross your fingers!)

Then the files should appear in NRAO's (VLBA) data archive as one or
more multi-source, single frequency-ID FITS files. You can grab these if
you can (ie, they are public or you know your project key), and they
should be okay for you to start calibrating (eg with VLBACALA, VLBARUN
or ACCOR), presuming you have examined your data and considered editing
bad data before starting.  The exact data handling one can get from the
HIstory file (using AIPS' PRTHI).