As of Mon Jul 15 12:39:15 2024

SNFLG: writes flags based on discrepant solutions in SN files


INNAME                             UV data (name).
INCLASS                            UV data (class).
INSEQ             0.0    9999.0    UV data (seq. #).  0 => high
INDISK            0.0       9.0    Disk unit #.  0 => any
INEXT                              Input'SN' or 'CL' table
INVERS                             Input table file version no.
SOURCES                            Source list
QUAL            -10.0              Source qualifier -1=>all
TIMERANG                           Time range to consider
STOKES                             Stokes type to consider
SELBAND                            Bandwidth to include (kHz)
SELFREQ                            Frequency to include (MHz)
FREQID                             Freq. ID to include, -1=>all
BIF               0.0      100.0   First IF to consider, 0=>1.
EIF               0.0      100.0   Last IF to consider
DOSTOKES         -1.0        1.0   > 0 -> flag all Stokes if
                                   one is bad
DOIFS            -1.0        1.0   > 0 -> flag all IFs if one
                                   is bad
SUBARRAY          0.0              Subarray to consider 0=>all
ANTENNAS                           Antennas to consider 0=>all
CUTOFF                             Table weight below which data
                                   will not be considered, 0 =>
                                   consider all.
FLAGVER         -1.0               FG table to be copied and
                                   updated.  -1 -> none.
OPTYPE                             What is examined: JUMP, AMP
                                   A&P, DELA
DPARM                              Control parameters
                                   (1) JUMP: max allowed phase
                                             jump (deg)
                                   (1) AMP : max allowed
                                             deviation from mean
                                             in rms's
                                   (2) both: time interval (sec)
                                       Usually a bit <- SOLINT
                                   (1-6) A&P: min,max amp,
                                       min,max phase, min,max wt
                                   (1-4) DELA: min, max delay,
                                   (7) A&P, DELA: > 0 -> flag
                                       all antennas where one
                                       is bad
                                   (8) A&P, DELA> 0 -> generate
                                       flags at failed (blanked)
BADDISK                            Disks to avoid for scratch


Type:  Task
Use:   Writes flagging info based on discrepencies in the SN table
       solutions.  OPTYPE='JUMP' flags on phase differences between
       adjacent entries in SN files on a baseline-by-baseline basis.
       It also writes flags to flag data before the first SN/CL table
       record and after the last SN/CL record, but only if TIMERANGE
       is not set.  Furthermore, whenever a solution for a particular
       antenna is marked bad or its weight is too low, a flag is
       written to extend 70 percent of the way (in time) to the next good
       solution (or all the way if both solutions are bad).

       OPTYPE='AMP' finds the mean gain amplitude and rms separately
       for each source, antenna, IF, and polarization.  It then
       produces flags for all solutions more than DPARM(1) * rms away
       from the mean.

       OPTYPE='A&P' examines every solution and generates flags for
       those amplitudes < DPARM(1) or > DPARM(2), and phases <
       DPARM(3) or > DPARM(4) in degrees (where phase runs from -180
       to +180 degrees.

       OPTYPE='DELA' examines every solution and generates flags for
       those delays < DPARM(1) or > DPARM(2), and rates <
       DPARM(3) or > DPARM(4) in ns and Milli-Hz.

       This task (OPTYPE='JUMP') was initially contributed by Lincoln
       Greenhill and is based on a task initially designed by Mark
       Reid, both of CfA.
  INNAME.....UV file name (name).       Standard defaults.
  INCLASS....UV file name (class).      Standard defaults.
  INSEQ......UV file name (seq. #).     0 => highest.
  INDISK.....Disk unit #.               0 => any.
  INEXT......'SN' or 'CL' table to be consider
  INVERS.....Version number of table to consider, 0=>highest no.
  SOURCES....Source list.  '*' = all; a "-" before a source name means
             all except ANY source named.  'ALL' means to ignore the
             source identifier in the SN/CL table and combine all of
             them into one "source".
  QUAL.......Only sources with a source qualifier number in the SU
             table matching QUAL will be used if QUAL is not -1.
  TIMERANG...Time range of the data to be considered. In order:
             Start day, hour, min. sec, end day, hour, min. sec. Days
             relative to reference date.
  STOKES.....The desired Stokes type to be considered:
             STOKES = 'V' or 'DIFF' gets you the phase difference of R
             and L treated as the data for JUMP or the ratio of R and
             L gains treated as the data for AMP.  For A&P both
             polarizations are considered with these STOKES values.
             'R' = RCP only, 'L' = LCP only, else both.
  SELBAND....Bandwidth of data to be selected. If more than one IF is
             present SELBAND is the width of the first IF required.
             Units = kHz.
  SELFREQ....Frequency of data to be selected. If more than one IF is
             present SELFREQ is the frequency of the first IF
             required. Units = MHz.
  FREQID.....Frequency identifier to select (you may determine which
             is applicable from the OPTYPE='SCAN' listing produced by
             LISTR). If either SELBAND or SELFREQ are set, their
             values overide that of FREQID.  However, setting SELBAND
             and SELFREQ may result in an ambiguity.  In that case,
             the task will request that you use FREQID.  -1 => all,
             0 => first
  BIF........Lowest IF to consider.   0 -> 1
  EIF........Highest IF to consider.  0 -> highest
  DOSTOKES...> 0 => flag all Stokes if one is bad
  DOIFS......> 0 => flag all IFs if one is bad
  SUBARRAY...Limit operation to specified subarray.  0 -> all
  ANTENNAS...A list of the antennas to be considered. All 0 => all.
             If any number is negative then all antennas listed are
             NOT to be considered and all others are.
  CUTOFF.....If >= 0, the task will examine the weight column of the
             table and if the weight < CUTOFF, the task will consider
             that sample to be bad.  All samples with weight <= 0 are
             considered bad.  An FG table entry will be generated for
             the time surrounding a bad solution.
             CUTOFF is ignored for 'A&P' except as a default for
  FLAGVER....Input FG version number.  This FLAGVER will be copied to
             a new version and the new flags appended to it.  0 ->
             highest, -1 => none.
  OPTYPE.....'JUMP' to flag based on jumps in the SN phases on a
                    baseline basis.
             'AMP'  to flag on gain amplitudes more than DPARM(1) * rms
                    away from the robust mean.
             'A&P'  to flag on solutions outside specified ranges in
                    amplitude, phase, and weight (S/N ratio).
                (1) Flag solution times > DPARM(1) * rms of that
                    antenna, source, IF, and polarization.  0 -> 6
                    Note that these mean and rms values are robust,
                    ignoring outliers.  Deleting at 3 sigma is
                    excessive therefore.
                (2) Time range around solution time to be flagged.
                    Usually the SOLINT as in JUMP.  The time range is
                    the central time minus DPARM(2)/2 to the central
                    time plus DPARM(2)/2.
                (1) Lowest allowed gain: 0 -> 0.
                (2) Highest allowed gain: 0 -> 9999
                (3) Lowest allowed phase: 0 -> -200
                (4) Highest allowed phase: 0 -> +200
                (5) Lowest allowed weight: 0 -> CUTOFF
                (6) Highest allowed weight: 0 -> 999999
                (7) > 0 => flag all antennas in the IF and
                    polarization when one antenna is bad.
                (8) > 0 => also make flags for all failed solutions
                    (where Real and Imag are blanked)
                (1) Lowest allowed delay (ns): 0 -> -100000.
                (2) Highest allowed delay (ns): 0 -> 100000.
                (3) Lowest allowed rate (mHz): 0 -> -10000.
                (4) Highest allowed rate (mHz): 0 -> +10000,
                (7) > 0 => flag all antennas in the IF and
                    polarization when one antenna is bad.
                (8) > 0 => also make flags for all failed solutions
                    (where Real and Imag are blanked)
             'JUMP' or any other OPTYPE
                (1) Maximum allowed phase jump between adjacent SN
                    entries (deg)
                (2) Minimum time interval over which to judge jumps
                   (sec).  Times within DPARM(2) of each other are
                   regarded as the same.  It is important that
                   DPARM(2) be less than any time difference between
                   two independent solutions in the SN table.
  BADDISK....The disk numbers to avoid for scratch files (sorting
             tables mostly).


Notes from Lincoln Greenhill re JUMP:

The idea of flagging based on baseline-dependent jumps in phase does
seem counter to the AIPS philosophy. However, we think of it as

UV data is available in baseline-dependent terms.  At time T0 and T1,
two contributing stations could have atmospheric solutions that jump
by reasonable amount, say +/-35 deg. However, this could correspond to
70 deg of jump on the corresponding baseline.  The UV data between T0
and T1 must be calibrated in the context of this baseline-dependent
phase jump (since the SN entries are differenced).  If that jump is
more than say 1 rad, then at times between T0 and T1, a linear
interpolation could be sufficiently discrepant from the true
atmospheric phase contribution as to contribute a fractional

Conversely, two stations could jump by +70 deg. This would be grounds
for flagging (if 1 rad is the threshold).  But the difference is 0 and
there is no jump on the corresponding baseline.  This data should be
OK. I could introduce a case in which the atmosphere is well
correlated over two antennas but otherwise highly variable.  The
baseline-dependent phase jump from T0 to T1 would be 0 and the data
good. The antenna dependent phase jump from T0 to T1 would be
unacceptably large.