; VLBAMPHC
;---------------------------------------------------------------
;! applies VLBARUN calibration to additional phase stopping centers
;# RUN POPS VLBI UTILITY CALIBRATION
;---------------------------------------------------------------
;;  Copyright (C) 2019
;;  Associated Universities, Inc. Washington DC, USA.
;;
;;  This program is free software; you can redistribute it and/or
;;  modify it under the terms of the GNU General Public License as
;;  published by the Free Software Foundation; either version 2 of
;;  the License, or (at your option) any later version.
;;
;;  This program is distributed in the hope that it will be useful,
;;  but WITHOUT ANY WARRANTY; without even the implied warranty of
;;  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;  GNU General Public License for more details.
;;
;;  You should have received a copy of the GNU General Public
;;  License along with this program; if not, write to the Free
;;  Software Foundation, Inc., 675 Massachusetts Ave, Cambridge,
;;  MA 02139, USA.
;;
;;  Correspondence concerning AIPS should be addressed as follows:
;;         Internet email: aipsmail@nrao.edu.
;;         Postal address: AIPS Project Office
;;                         National Radio Astronomy Observatory
;;                         520 Edgemont Road
;;                         Charlottesville, VA 22903-2475 USA
;---------------------------------------------------------------
VLBAMPHC  LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC
VLBAMPHC: Applies VLBARUN cal to another phase stopping center

          To run this procedure you MUST type RUN VLBAUTIL and
          RUN VLBARUN and RUN VLBAMPHC first to define the
          procedures in AIPS.  Do this only once.

                                   -- Load the data from disk --
DATAIN                             Disk file name
DOUVCOMP                           Compress data? Use -1, 0 or 1
OUTNAME                            File name (name)
OUTDISK                            Working disk with ample space
                                   -- OR existing file --
INNAME                             Input file name
INCLASS                            Input file class
INSEQ                              Input file sequence number
INDISK                             Disk number for input file
                                   -- data w VLBARUN files --
IN2NAME                            Input file name
IN2CLASS                           Input file class
IN2SEQ                             Input file sequence number
IN2DISK                            Disk number for input file
                                   -----------------------------
OPTYPE                             For CONT, PSEU or SPEC (LINE)
                                   -----------------------------
DOCLIP                             > 0 -> use CLIP at these clip
                                   levels
DOFLAG                             > 0 -> RFLAG on each source
IMSIZE                             Image size =-1 for no images
          Hint: when setting directory and e-mail lower-case
          letters can be retained by NOT using the close quote.
DOPLOT                             <= 0 no plots;=1 some plots
                                   >1 huge number of plots
OUTFILE                            DIRECTORY FOR HTML AND PLOT
                                     FILES: if you want an html
                                     file with plots please set
                                     the dir where the html and
                                     plot files can be put.
OUTTEXT                            E-MAIL ADDRESS: if you want
                                     an e-mail when the script
                                     is complete, set the
                                     address here.
BADDISK                            Disks to avoid for scratch
----------------------------------------------------------------
VLBAMPHC
Type: Procedure
Use:  VLBAMPHC is the procedure that uses the output of VLBARUN to
      apply to a seconday data set containing the data for a secondary
      phase stopping position.  VLBARUN is primarily used for
      relatively simple data sets, so VLBAMPHC mostly applies to simple
      observations that have been correlated with multiple phase
      centers.

      Type RUN VLBAUTIL, RUN VLBARUN, and RUN VLBAMPHC to define all
      the procedures.  Do not do this again unless you lose the procedures
      somehow.  HELP PROCS will show the names of all defined
      procedures.

      The procedure is run by typing : VLBAMPHC

      The procedure will check the inputs (as far as it can) to avoid
      later interruptions, and then proceed directly, if the data are
      already on disk.  The procedure will clear the message file when
      it starts, so print any messages you need to save.  This insures
      that the printed message file at the end contains only the
      messages from this run.

      If the secondary data set is already on disk use INNAME et al.
      adverbs and make sure that it does not have a BP table or a CL
      table numbered more than 1.  The procedure P_RESTART will take
      care of this for you.  Note that you should keep FG table 1
      usually since that contains the on-line flags.  Other FG tables
      may be made by previous runs of VLBAMPHC or other tasks.  Delete
      these if you want, or use them as a better starting place.

      It is a VERY good idea to run TASAV before you do this to keep
      the original TY, GC, BL and also FG tables number 1 in case of
      disasters.

  DATAIN......48-character name of the disk file from which to read a
              FITS file.  It must be in the form
                  <logical>:<file name>
              or
                  <node>::<logical>:<file name>
              where <node> is the remote computer name, <logical> is the
              environment variable (logical name) for the disk area in
              which the file named <file name> is stored.  <node> is
              usually omitted when the file is local to the current
              computer.  If DATAIN is not found, the task will try
              DATAIN with the character 1 appended.
              Beginning 2003-Oct-16 FITLD can read more than one disk
              file at a time.  In that case, they must all have the
              same name except that the last letter(s) are the
              sequence number 1 through NCOUNT with no leading zeros.
              DATAIN must give the base name only, omitting the
              sequence number.  Then either
                    DATAIN = 'MYDATA:file_
              or
                    DATAIN = 'MYDATA:file_1
              can access files in $MYDATA named file_1, file_2,
              file_3, etc.  Note that this is similar to the pieces of
              a FITS pieced UV-table data set, but only one UV-table
              data set may be read at a time.
  DOUVCOMP....If true (> 0.5) then output data will be compressed
              which saves disk space at the expense of losing some
              weighting information.
  OUTNAME.....The name for the output files. The output files will all
              be given the class UVDATA and assigned distinct sequence
              numbers as needed.
  OUTDISK.....The disk drive number for the output data if loading from
              disk. Choose a disk with a large amount of space.  If
              data are already loaded into AIPS (i.e. you are using
              INNAME, INCLASS etc) then OUTDISK is ignored and INDISK
              is used.
  INNAME......Input UV file name (name) that is already loaded.
              Note that you CANNOT use both DATAIN/OUTNAME/OUTDISK
              and INNAME/INCLASS/INSEQ/INDISK
  INCLASS.....Input UV file name (class) that is already loaded.
  INSEQ.......Input UV file name (seq. #) that is already loaded.
  INDISK......Disk drive # of input UV file.  NOTE: this is also
              used as OUTDISK.
  IN2NAME.....Input UV file name (name) for primary data set that is
              already loadedand has calibration (CL) and bandpass
              tables to be applied to the secondary data set
  IN2CLASS....Input UV file name (class) for primary data set
  IN2SEQ......Input UV file name (seq. #) for primary data set
  IN2DISK.....Disk drive # of input UV file for primary data set.
  OPTYPE......The type of observation and requested output files:
              ''     = 'CONT' for channels < 65; 'SPEC' otherwise
              'CONT' =  Plain continuum observations, uses pulse cals
                        unless INVER = -1
              'PSEU' = Pseudo-continuum observations, uses pulse cals
                       unless INVER = -1.  The data is NOT averaged
                       over channels, but the resulting images are
                       continuum images - used for large FOVs
              'SPEC' = Spectral line observations.  Manual pulse-cal
                       is performed.  Data is unaveraged and images
                       are spectral line cubes of the phase referenced
                       target sources ONLY.
              'LINE' = Same as 'SPEC'.
  DOCLIP......If DOCLIP(1) > 0, clip parallel hand data at DOCLIP(1)
              Jy and cross-hand data at DOCLIP(2) Jy.  If DOCLIP(2)=0
              no cross-hand clipping is done.  Parallel hand data are
              flagged when cross-hand data are and vice versa.
  DOFLAG......> 0 says to run RFLAG twice for each source, the first
              time to set the flagging levels and the second time to
              apply them.  Plot files are made in each pass but do not
              appear in the output html page (if any).  If you are
              observing a source with strong lines, the value of the
              pre-clip in Jy (FPARM(13)) is set to max (100, DOFLAG).
              <= 0 -> use any flag tables present with the data file
              but do not create any new ones.  Warning these RFLAG
              flag tables can get large.
  IMSIZE......Output target image sizes.  Minimum IMSIZE of 512 up to
              8192 pixels.  Sources listed in CALSOUR are automatically
              imaged with a size of 128 by 128 unless FACTOR is set.
              Pixel size is automatically set by frequency:
              2.25/Frequency(GHz) mas.
              IMSIZE=-1 do not make images.
  DOPLOT......Make diagnostic plots to judge quality of procedure
              results.
                 <=0 => no plots
                 1 => some plots
                 2 => lots of plots (could be hundreds)
  OUTFILE.....Directory of html and plots files.  If you want the
              diagnostic plots written to disk and an html file
              created to make the plots easier to examine please set
              directory here.  This is limited to 37 characters
              (including final /).  Environmental variables are
              allowed and must end with a ':'.  Mixed environmental
              and regular are not allowed, e.g.,
                OUTFILE='/home/computer/BM394/pipeline/
                OUTFILE='FITS:'
              are allowed.
              BUT:
                OUTFILE='FITS:BM394/
              is not.
              A subdirectory will actually be used so this directory
              may contain previous subdirectories.  It should not
              contain any PostScript files.  A text file containing
              all of the messages will be saved in the subdirectory
              with the name messages.txt.
  OUTTEXT.....E-mail address for notifications when script is done or
              failed.
  BADDISK.....A list of disk numbers to be avoided when creating scratch
              files.
---------------------------------------------------------------------------
VLBAMPHC:          Procedure that attempts to calibrate VLBI secondary
                   phase stopping data sets using VLBARUN output on primary
Documentor:        Eric Greisen
Related Programs:  VLBARUN, RFLAG, IMAGR

VLBAMPHC applies simple procedures to use the calibration produced by
VLBARUN on the primary data set (indicated by IN2NAME et al.) to
calibrate a secondary phase stopping position in a separate file from
the primary.  When DiFX correlates multiple phase stopping positions
within a single array pointing position, the primary output file
contains the data stopped at the pointing position plus all scans on
sources not having multiple phase stopping positions.  Seconday data
sets are produced, one for each secondary phase stopping position.
Use VLBARUN on the primary data set.  Then use VLBAMPHC on each
secondary data set, one at a time.

Note that the primary data set does not have to be calibrated with
VLBARUN, although that is the expectation.  Any calibration that
produces a CL table good for all times in the data set and a bandpass
table will do for the primary (IN2NAME) file.

Necessary input adverbs are:
   DATAIN or INNAME (INCLASS, INSEQ and INDISK) -> so the secondary
           dataset is defined
   IN2NAME, IN2CLASS, IN2SEQ, IN2DISK -> so the primary dataset is
           defined.

More detail about defaults
--------------------------
VLBAMPHC tries to pick sensible defaults if inputs are left blank.
Below describes how it makes these decisions.
  OPTYPE=''   -> VLBAMPHC checks the total number of channels and
                 decides from that whether dataset is continuum (64 or
                 less) or spectral line (>65).
  IMSIZE=0    -> 512

Output of plots to disk, creation of html file and e-mail message
-----------------------------------------------------------------
VLBAMPHC will create .gif files on disk in a subdirectory of the
directory defined by OUTFILE of the plots that are requested.  It then
makes a vlbamphc.html file that includes headings for the plots and at
the bottom a list of the CL tables and what calibration is added to
each.  This vlbamphc.html file is intended to make examination of the
diagnostic plots easier, so the user can quickly judge whether the
pipeline worked.  A link to the message file from the run is also
provided in the vlbamphc.html file.

It is STRONGLY recommended that the directory defined by OUTFILE be
empty of PostScript files alsthough it may contain date-stamped
subdirectories.  VLBAMPHC creates and deletes PostScript files in this
directory so it is safer if there aren't any such files for VLBAMPHC
to potentially clobber or delete.  VLBAMPHC makes a subdirectory in
OUTFILE named by the date and time and the final gif files and
vlbamphc.html will reside in the subdirectory.  VLBAMPHC checks if
there is an existing vlbamphc.html file and will not run if there is.
That should happen only if you restart VLBAMPHC within the same
minute.

It is also recommended that if OUTFILE is used then an e-mail address
be provided in OUTTEXT.  The e-mail will state whether VLBAMPHC is
done (subject: VLBAMPHC DONE) or failed (subject: VLBAMPHC FAILED).
The FAILED message only catches failure to do a problem with how
VLBAMPHC was set up, if the pipeline fails because a task fails that
is not captured and no e-mail message is sent.  If OUTFILE is set, the
DONE message includes the URI to the vlbamphc.html file assuming that
the machine you are viewing it on is the "localhost".  So if you are
on that machine, you can copy that URI ("file:///etc") into your web
browser and look at the plots.  If you are not on the machine where
the plots were created then just copying the file:///... URI will not
work.

CONSIDERATIONS FOR MAC OS X AND OTHER OS THAT MIGHT NOT INCLUDE COMMON
LINUX COMMAND-LINE TOOLS  -- the output of plots to display using .gif
and html files requires the command-line command "convert" or "magic".
If you do not have "convert" or "magic" then you cannot use the
OUTFILE input.  There are packages you can install on your machine
that will give you access to linux command-line tools like convert.

What does VLBAMPHC do?
---------------------
Below are the basic calibration steps that VLBAMPHC performs.  This
does not include all the plotting etc..
 1) checks that the inputs are correct as well as it can, sets some
    defaults.
 2) Uses TACOP to copy the highest CL and BP table versions from
    IN2NAME to INNAME.  Then sets the source number in the output CL
    table to zero so that it applies to all sources.
 3) If DOFLAG > 0, VLBAMPHC then runs RFLAG on the secondary target
     source (INNAME et al.).  It is run in the mode where individual
     baseline-based clip levels are used (NS tables created).
 4) Then SPLIT is run to apply the calibration and flag table.
 5) Finally IMAGR is run if IMSIZE >= 0.  If OPTYPE='SPEC', separate
    image cubes are made for each IF.

The html file, if requested, will contain the RFLAG plots in color for
the before and after passes of RFLAG.  It will also contain a UVPLT of
the SPLIT data and a KNTR plot of the output image(s).
