; PCFLG
;---------------------------------------------------------------
;! interactive flagging of Pulse-cal data in channel-TB using the TV
;# TASK UV TV-APPL CALIBRATION EDITING vlbi INTERACTIVE
;-----------------------------------------------------------------------
;;  Copyright (C) 2016-2017
;;  Associated Universities, Inc. Washington DC, USA.
;;
;;  This program is free software; you can redistribute it and/or
;;  modify it under the terms of the GNU General Public License as
;;  published by the Free Software Foundation; either version 2 of
;;  the License, or (at your option) any later version.
;;
;;  This program is distributed in the hope that it will be useful,
;;  but WITHOUT ANY WARRANTY; without even the implied warranty of
;;  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;  GNU General Public License for more details.
;;
;;  You should have received a copy of the GNU General Public
;;  License along with this program; if not, write to the Free
;;  Software Foundation, Inc., 675 Massachusetts Ave, Cambridge,
;;  MA 02139, USA.
;;
;;  Correspondence concerning AIPS should be addressed as follows:
;;         Internet email: aipsmail@nrao.edu.
;;         Postal address: AIPS Project Office
;;                         National Radio Astronomy Observatory
;;                         520 Edgemont Road
;;                         Charlottesville, VA 22903-2475 USA
;-----------------------------------------------------------------------
PCFLG     LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC
PCFLG:  edit pulse-cal data using the TV display and cursor
INNAME                             UV data (name).
INCLASS                            UV data (class).
INSEQ             0.0    9999.0    UV data (seq. #).  0 => high
INDISK                             Disk unit #.       0 => any
INVERS            0.0              PC table version number
DOCAT            -1.0       2.0    Catalog work file ?
IN2SEQ            0.0              Sequence number of work file
IN2DISK                            Disk number of work file
DOHIST          -10.0       1.0    Record flags in history file
SOURCES                            Source list
CALCODE                            Calibrator code '    '=>all
TIMERANG                           Time range to include
BIF               0.0      100.0   Lowest IF number 0=1
EIF               0.0      100.0   Highest IF number
ANTENNAS                           Antennas to include
SUBARRAY                           Subarray to consider
DOCENTER                           > 0 -> center the grey-scale
                                   in the TV, else avoid menu
BADDISK                            Disks to avoid for scratch
                                   and for master grid file.
----------------------------------------------------------------
PCFLG
Type:  Task
Use:   Grids pulse-cal data to form an image with pulse-cal number on
       the horizontal axis (essentially a frequency axis) and time on
       the vertical axis.  After making this "master" grid, PCFLG
       displays the grid on the TV smoothing the data in time and
       showing every n'th pixel in channel as needed to fit the
       display.  Then, PCFLG displays its interactive options: fiddle
       colors and contrast, change zoom, set windows (and reload the
       new subimage with less time smoothing), and select cells to
       flag.  PCFLG can fit the phases for delays and display either
       the input data or the residual data (selection can be made
       interactively).  After some flagging, the input data may be fit
       over again to adjust the residual values.

       **** When viewing phases, select VIEW RESID to see phases which
       can be interpreted by humans.  VIEW INPUT phases often go
       through a large part of a turn between channels. After
       significant flagging, you will probably want to REDO DELAY.

       Note that PCFLG flags the pulse-cal data in a new PC table, not
       the visibility data.  This last is, of course, the main
       function.  The grid file may be cataloged and kept for more
       than one session of interactive editing.  All commands to flag
       data are written to a special FC extension to the master grid
       file.  As a result, they are not lost when the TV hangs up, the
       power fails, the program aborts, or the user exits the program
       willingly.  Commands in the FC file may be deleted and their
       effects on the master grid file undone.  The actual flagging of
       the pulse-cal table is done at the end of all interactive
       operations at the users' command.
Adverbs:
  INNAME.....UV file name (name).          Standard defaults.
  INCLASS....UV file name (class).         Standard defaults.
  INSEQ......UV file name (seq. #).        0 => highest.
  INDISK.....Disk unit #.                  0 => any.
  DOCAT......True (>0) means to use a cataloged file for the master
             grid.  It creates a new one or uses an old one, depending
             on the value of IN2SEQ.  If DOCAT = 2 and IN2SEQ = 0, then
             the program returns to the user immediately, prepares the
             cataloged master grid file, and then exits.  The program
             must be rerun with an appropriate, non-zero value of IN2SEQ
             to flag the data.  This option allows the user to do other
             things while creating a large master grid file.  IN2DISK is
             used for cataloged master grid files with BADDISK  used for
             scratch and, if IN2DISK = 0, cataloged master grid files.
  IN2SEQ.....Sequence number of cataloged master grid file.  0 => create
             new one, else use existing one of seq number = IN2SEQ.
             NOTE: to use this option, all of the adverbs must have the
             same value as when the old grid file was created.
  IN2DISK....Disk number of cataloged work file.  0 => use any disk for
             pre-existing file and any disk not listed in BADDISK
             (independent of DOCAT) to create a new cataloged file.
             Ignored when DOCAT <= 0.
  DOHIST.....> 0 => record task execution and flagging info in the
             history file (this can be a lot!).  <= 0.0 means to omit
             the flagging info and <= -9.5 means to omit the execution
             information as well.
  SOURCES....List of sources to be gridded. '  '=> all; if any starts
             with a '-' then all except ANY source named.
  CALCODE....Sources may be selected on the basis of calibrator code:
                '    '  =>  any calibrator code selected
                '*   '  =>  any non blank code (cal. only)
                '-CAL'  =>  blank codes only (no calibrators)
                anything else = calibrator code to select.
  TIMERANG...Time range of the data to be gridded. In order:
             Start day, hour, min., sec, end day, hour, min., sec. in
             days relative to reference date.
  BIF........Lowest IF to grid.  IFs are now gridded along with
             spectral channels in one longer horizontal axis.
             If this is undesirable, run PCFLG 1 IF at a time.
             This is only a little more painful than the previous
             treatment of each IF as a separate cube in the grid.
  EIF........Highest IF to grid.    0 => highest.
  ANTENNAS...A list of the antennas to grid.  If any number is negative
             then all antennas listed are NOT desired and all others
             are.  All 0 => grid all.
  DOCENTER...> 0 => display the grey-scale approximately at the center
                    of the TV window even if this makes it overlap the
                    menu.
             <= 0 => display the grey-scale to the right of most of
                    the menu if at all possible.
  BADDISK....Disk number(s) to avoid for scratch files AND, if IN2DISK
             is zero, for the master disk file (independent of DOCAT).
----------------------------------------------------------------
     PCFLG has three main steps:
(1) Grid the data to a master grid with channels/IFs on the x axis and
    time on the y axis.  The cell size in time is set by the data and
    data are not averaged in time.  This master grid saves the real
    and imaginary parts of the pulse-cal data separately along with a
    flag number for each sample.  Each data sample is accompanied by
    the real and imaginary values of the input data corrected by the
    fit delay and center phase.  Each data sample also contains its
    frequency so that new fits may be found and revised residuals
    computed.  The master grid will not be completely regular in its
    times.  No more than 1 blank row will appear between scans and
    different sources cannot be in the same row.  Thus the y axes in
    the master grid (and in the TV grid) will be somewhat irregular.
(2) The interactive session(s) in which the user selects data to be
    flagged and alters the TV display of the data to assist in the
    process.
(3) The actual flagging of the data and writing of the history file.  A
    new PC table is written containing all input PC data but with
    certain samples magic blanked.
All steps may be done in a single execution.  However, if DOCAT is used,
then they may be done in several sessions.

Most operations on the master grid are relatively efficient and
reference only small parts of that file.  However, some read and write
the whole file (all times, Stokes, antennas, etc.) to ensure that the
operation is truly complete.  Breaking your data set up into smaller
pieces will speed such operations greatly without compromising the
editing.  This is now less inportant as computers have become more
capable.

The interactive session is driven by a menu which is displayed on the TV
graphics screen.  Move the cursor to the desired operation (noting that
the currently selected one is highlighted in a different color on many
TVs) and press any button to select the operation.  The first (left-most
column) of choices is:

--------------
| OFFZOOM    |   turn off any zoom magnification
| OFFTRANS   |   turn of any black & white enhancement
| OFFCOLOR   |   turn of any pseudo-coloring
| TVFIDDLE   |   as in AIPS, allows zoom, pseudo-color contours or black
                 and white enhancement
| TVTRANSF   |   black and white enhancement as in AIPS
| TVPSEUDO   |   many pseudo-colorings as in AIPS
| DO WEDGE ? |   switches choice of displaying a step wedge
| LOAD xxxx  |   switch TV load transfer function to xxxx
| LIST FLAGS |   list selected range of flag commands
| UNDO FLAGS |   remove flags by number from FC table and from the
                 master grid
| REDO FLAGS |   reapply all flags to master grid
| REDO DELAY |   recompute the fits for delays and center phases and
                 recompute the corrected (residual) PC values
| VIEW RESID |   to switch from viewing the input data to viewing the
                 residual data (only phase differs)
| VIEW INPUT |   to switch from viewing the residual data to viewing
                 the input data (only phase differs)
| DO LABEL ? |   turns on/off axis labeling
--------------
Note: when a flag is undone, all cells in the master grid which were
first flagged by that command are restored to use.  Flag commands done
after the one that was undone may also, however, have applied to some of
those cells.  To check this and correct any improperly unflagged pixels,
use the REDO FLAGS option. This option even redoes CLIP operations!
After an UNDO or REDO FLAGS operation, the TV is automatically reLOADed
if needed. Note that the UNDO operation is one that reads and writes the
full master grid.

The load to the TV for all non-phase displays may be done with all
standard transfer functions: LINear, LOG, SQRT, LOG2 (more extreme
log).  The menu shows the next one in the list through which you may
cycle.  The TV is reloaded immediately when a new transfer function is
selected.  Only one of the VIEW yyyyy options will appear, depending
on which form you are currently viewing.  Note that the two forms
differ only in phase but that that affects vector RMS and amplitude of
the vector difference as well as the phase display types.

Column 2 offers type-in controls of the TV display and controls of which
data are to be flagged.  In general, the master grid may be too large
to display on the TV screen in its entirety.  The program begins by
loading every n'th channel and time smoothing by m time intervals in
order to fit the full image on the screen.  However, you may select a
sub-window in order to see the data in more detail.  You may also
control the range of intensities displayed (like the adverb PIXRANGE
in TVLOD inside AIPS).  The averaging time to smooth the data for the
TV display may be chosen, as may the averaging time for the "scan
average" used in some of the displays.  Which correlators are to be
flagged by the next flagging command may be typed in.  All of the
standard STOKES values, plus any 4-bit mask may be entered (see HELP
file above).  The antenna number may be typed in.  Flagging may be
done only for the current antenna and IF and source, or it may be done
for all antennas and/or IFs and/or sources.  Note that these controls
affect the next LOADs to the TV or the flagging commands prepared
after the parameter is changed.  When the menu of options is displayed
at the top of the TV, the current selections are shown along the
bottom.  If some will change on the next load, they are shown with an
asterisk following.

----------------------
| ENTER BLC          |   Type in a BLC on the terminal
| ENTER TRC          |   Type in a TRC on the terminal
| ENTER AMP PIXRANGE |   Type in the intensity range to be used for
                         loading amplitude images to the TV
| ENTER PHS PIXRANGE |   Type in the phase range to be used for loading
                         phase images to the TV
| ENTER RMS PIXRANGE |   Type in the intensity range to be used for
                         loading images of the rms to the TV
| ENTER R/M PIXRANGE |   Type in the value range to be used for loading
                         rms/mean images to the TV
| ENTER SMOOTH TIME  |   To enter the time smoothing length in units of
                         the master grid cell size
| ENTER SCAN TIME    |   To enter the time averaging length for the
                         "scan average" in units of the master grid cell
                         size
| ENTER ANTENNA      |   To enter a desired antenna using the terminal
| ENTER STOKES FLAG  |   To type in the 4-character string which will
                         control which correlators (polarizations) are
                         flagged.  Note: this will apply only to
                         subsequent flagging commands. It should be
                         changed whenever a different Stokes is
                         displayed.
| ENTER CH SMOOTH    |   To type in the FWHM of a Gaussian smoothing
                         in spectral channels in the data type being
                         loaded.
| SWITCH SOURCE FLAG |   To switch between having all sources flagged by
                         the current flag commands and having only those
                         sources included in this execution of PCFLG
                         flagged. The former is desirable when a time
                         range encompasses all of 2  calibrator scans.
| SWITCH ALLANT FLAG |   To switch between flagging only the current
                         antenna and flagging all antennas.
| SWITCH ALL-IF FLAG |   To rotate the flag all IFs status from one IF
                         to a range of IFs, to all IFs.  Applies to
                         subsequent flag commands.
----------------------
The all-IF flag remains true if the input data set has no more than one
IF and the all-source flag remains true if the file has only one source.

An extra word should be said about the "scan average".  This is used
solely for displaying the difference of the data at time T and the
average of the data at times near T.  This average is computed with a
"rolling buffer".  Thus for a scan average time of 30 seconds and data
at 10 second intervals, the average for a set of 7 points is as
follows:
     time:        average of times:
       00           00   10   20
       10           00   10   20
       20           10   20   30
       30           20   30   40
       40           30   40   50
       50           40   50   60
       60           40   50   60

The third column of options is used to control which data are displayed
and to cause the TV display to be updated.  The master grid must be
converted from complex to amplitude or phase for display.  Using
either scalar or vector averaging, it may be converted to the rms of
the amplitude or the rms divided by the mean of the amplitude.  It may
also be converted to the amplitude of the vector difference between
the current observation and the "scan average" as defined above or the
absolute value of the difference in amplitude with the scalar-average
amplitude or the absolute value of the difference in phase with the
vector scan average.  This column has the options:

----------------------
| DISPLAY AMPLITUDE  |   To display amplitudes on the TV
| DISPLAY PHASE      |   To display phases on the TV
| DISPLAY RMS        |   To display scalar amplitude rms on the TV
| DISPLAY RMS/MEAN   |   To display scalar amplitude rms/mean on the TV
| DISPLAY VECT RMS   |   To display vector amplitude rms on the TV
| DISPLAY VRMS/VAVG  |   To display vector amplitude rms/mean on the TV
| DISPLAY AMP V DIFF |   To display the amplitude of the difference
                         between the data and a running (vector) "scan
                         average"
| DISPLAY AMPL DIFF  |   To display the abs(difference) of the amplitude
                         of the data and a running scalar average of the
                         amplitudes in the "scan"
| DISPLAY PHASE DIFF |   To display the abs(difference) of the phase of
                         the data and the phase of a running (vector)
                         "scan average"
| DISPLAY STOKES xx  |   To switch to Stokes type xx (where xx can be
                         RR, LL, XX, YY) - cross-hands do not occur in
                         PC table data.
| OFF WINDOW + LOAD  |   Reset the window to the full image and reload
                         the TV
| SET WINDOW + LOAD  |   Interactive window setting (like TVWINDOW)
                         followed by reloading the TV.
| LOAD LAST ANTENNA  |   Reload TV with the current parameters and the
                         previous antenna in sequence.
| LOAD NEXT ANTENNA  |   Reload TV with the current parameters and the
                         next antenna in sequence.
| LOAD LAST PIECE    |   Load the previous overlapping piece of the
                         data
| LOAD NEXT PIECE    |   Load the next overlapping piece of the data
| LOAD               |   Reload TV with the current parameters.
----------------------
SET WINDOW + LOAD is "smarter" than TVWINDOW and will not let you set a
window larger than the basic image.  Therefore, if you wish to include
all pixels on some axis, move the TV cursor outside the image in that
direction.  The selected window will be shown.  LOAD LAST PIECE and
LOAD NEXT PIECE appear only when there are too many times to fit on
the TV screen at the current smoothing parameter.  It lets you load
one piece at a time in sequence.  The pieces will overlap somewhat.

The fourth column is used to select the type of flagging to be done.
During flagging, a TV graphics plane is used to display the current
pixel much like CURVALUE in AIPS.  Buttons A and B do the flagging
(except A switches corners for the area and time range modes).  Button C
also does the flagging, but the program then returns to the main menu
rather than prompting for more flagging selections.  Button D exits back
to the menu without doing any additional flagging.  Another graphics
plane is used to show the current area/time/antenna being flagged.  All
flagging commands can create zero, one, two, or more entries in the
flagging list; hit button D at any time.  There are also two clipping
modes, an interactive one and one in which the user enters the clip
limits from the terminal.  In both, the current image computed for the
TV (with user-set windows and data type, but not any other windows or
alternate pixels etc. required to fit the image on the TV) is examined
for pixels which fall outside the allowed intensity range.  Flagging
commands are prepared and the master file blanked for all such pixels.
In the interactive mode, buttons A and B switch between setting the
lower and upper clip limits, button C causes the clipping to occur
followed by a return to the main menu, and button D exits to the menu
with no flagging.  The options are

-------------------
| FLAG PIXEL      |   To flag single pixels
| FLAG/CONFIRM    |   To flag single pixels, but request a yes or no on
                      the terminal before proceding
| FLAG AREA       |   To flag a rectangular area in Channel-T
| FLAG TIME RANGE |   To flag all channels for a range of times
| FLAG CHANNEL-DT |   To flag a channel for a range of times
| FLAG TIME       |   To flag all channels for a specific time
| FLAG A CHANNEL  |   To flag all times for a specific channel
| CLIP BY SET #S  |   To enter from the terminal a clipping range for
                      the current mode and then clip
| CLIP INTERACTIV |   To enter with the cursor and LUTs a clipping range
                      for the current mode and then clip the data.
| CLIP BY FORM    |   To clip selected antennas using the "method"
                      and clipping range of some previous clip
                      operation.
-------------------
The last operation allows you to apply a clipping method already used on
one antenna to other antennas and/or Stokes.  CLIP BY FORM asks for a
command number (use LIST FLAGS) and applies its display type (amp,
phase, rms, rms/mean), averaging interval and clip levels to a range of
antennas and Stokes (as entered from the terminal).  To terminate the
operation, doing nothing, enter a letter instead of one of the
requested antenna numbers.  To omit a Stokes, reply, if requested for
a flag pattern, with a blank line.  You may watch the operation being
carried out on the TV as it proceeds.

The right-most column has only the option:

--------
| EXIT |    Go resume AIPS and enter the flags in the data
--------

Before the flags are entered in the data, PCFLG asks the user whether or
not he actually wishes to do this.  You must respond yes or no.

When the menu is displayed, status line(s) are also displayed on the TV
to show the averaging time, corners, antenna/IF, Stokes, etc. of the
currently displayed data.  If these will change on the next load, an
asterisk is show after the values.

The building of the master file can be a lengthy process for large uv
data sets.  If you are going to run PCFLG more than once - WITH THE SAME
PARAMETERS - then use the DOCAT and IN2SEQ options to catalog the master
file during the first execution and to restart with that file in later
executions.  Note that DOCAT = 2 with IN2SEQ = 0, allows immediate
resumption of AIPS since no interactive flagging will be done.

PCFLG keeps track of the source name associated with each row of data.
When averaging to build the master grid and to build the displayed
grids, PCFLG will not average data from different sources and will
inform you that it has omitted data if it has had to do so for this
reason.  For multi-source files, the source name is displayed during the
CURVALUE-like sections.
