As of Tue Jan 16 4:41:14 2018

REFLG: Compresses flag tables


INNAME                             Input UV file name (name)
INCLASS                            Input UV file name (class)
INSEQ             0.0     9999.0   Input UV file name (seq. #)
INDISK            0.0        9.0   Input UV file disk unit #
SOURCES                            Restrict to listed sources
CALCODE                            Calibrator code ' ' all
FLAGVER                            Flag table version
CPARM                              (1) Time interval in sec
                                        IMPORTANT - see HELP
                                   (2) flag C(2) channels
                                      between flag groups
                                   (3) Flag full spectrum if
                                       fraction flagged > C(3)
                                   (4) Flag all times if
                                       fraction flagged > C(4)
                                   (5) Flag all baselines if
                                       fraction flagged > C(5)
                                   (6) Flag antenna if
                                       fraction flagged > C(6)
                                   (7) =1 -> flag cross hands
                                             if parallel flagged
                                       =2 -> flag all polariz.
                                             if any flagged
                                   (8) =1,3 -> omit combination
                                             of baselines
                                       >= 2 -> omit combination
                                             of IFs


Task:  Many flagging tasks examine large amounts of data but generate
       flag table entries over small ranges of time and/or spectral
       channel.  It may be possible to compress such tables by
       combining these ranges.  REFLG attempts to do this.  It also
       has options to delete full things if enough channels,
       baselines, etc. are already flagged.  After the time-frequency
       operation, the task optionally takes another pass through the
       flags looking for ways to combine IFs and baselines.  That may
       produce some gains, but is not likely to be so profitable and
       can actually fail badly in some cases.  You should always use
       FGDIF to check that the input and output FG tables flag the
       same data - then rerun REFLG if desired using CPARM(2) -

       A warning about sources - if the input table has flags that
       apply to a source other than those requested via SOURCES and
       CALCODE, the flags may be lost if they do not specify a
       specific source number > 0.  Flags specifying a specific
       source will be copied intact at the beginiing if they are not
       in the list specified by SOURCES and CALCODE.  Again FGDIF is a
       good idea with a very general source list.
  INNAME.....Input UV file name (name).      Standard defaults.
  INCLASS....Input UV file name (class).     Standard defaults.
  INSEQ......Input UV file name (seq. #).    0 => highest.
  INDISK.....Disk drive # of input UV file.  0 => any.
  SOURCES....List of sources to for which flags are reviewed.  The
             task has to list all times for a source and check them
             against the flags.  Preparing the list of times can be
             expensive so this adverb lets you avoid sources which are
             not relevant to the flag table.  '*' or ' ' = all; a "-"
             before a source name means all except ANY source named.
  CALCODE....Also used to limit sources selected: '*' => any
             non-blank, '-CAL' => only blank, ' ' => all, otherwise
             must match.
  FLAGVER....Specifies the version of the flagging table to be
             compressed.    <= 0 => highest numbered table.
             A new table is always written.
  CPARM......(1) Normal interval between samples (IMPORTANT) in sec.
                 Use a little bit less than the normal interval.
                 0 -> 10.  This parameter can have interesting and
                 confusing consequences when data have been time
                 averaged by say UVAVG.  Let us assume you have
                 averaged 1 second data with no flags to 5 seconds and
                 that the data started at time 0.  Thenthe average
                 times will be 2, 7, 12, 17, etc seconds.  But at the
                 end of the scan (e.g. time 20) there could be onl;y
                 one 1-sec record so its averaged time would be 20.
                 Thus to separate the 17 and the 20 one must set
                 CPARM(1) = 2.9 say.  However, if the data input to
                 UVAVG already have extensive flagging, the times may
                 be disturbed from this nice pattern (depnding on
                 OPCODE in UVAVG).  In this case, the possibilities
                 are endless and there is no simple answer to a
                 correct value for CPARM(1).  (One might consider TIME
                 or even GRID in UVAVG to alleviate this issue.)  The
                 simplest thing to do in the heavy editing case might
                 be to set CPARM(1) = 0.95.
              The following parameters are used on the flag
              information compiled by this task.  That information
              includes only those flags that apply to specific ranges
              in time, channel, and IF and to only a single baseline.
              More general flags are not included.  If CPARM(i) <= 0,
              the test is not performed.
              (2) Flag up to CPARM(6) "good" channels located between
                 groups of flagged channels.  Thus if we are to flag,
                 for example, channels 12-14, 16-20, 23-24, and 28-30
                 and FPARM(7)=2, then we will flag 12-24 and 28-30.
              (3) If the fraction of spectral channels flagged at a
                 particular time and baseline exceeds CPARM(3), flag
                 all channels.
              (4) If the fraction of times flagged at a particular
                 channel and baseline exceeds CPARM(4), flag all
              (5) If the fraction of baselines flagged at a particular
                 channel and time exceeds CPARM(5), flag all
                 baselines.  Note that autocorrelations are allowed,
                 so the maximum number of "baselines" is (Mant *
                 (Mant+1)) / 2 where Mant is the maximum antenna
                 number.  Thus the EVLA with Mant = 28 has 406
                 "baselines" rather than 351 when antenna 28 is in the
              (6) If the fraction of baselines to a particular antenna
                 flagged at a particular channel and time exceeds
                 CPARM(6), flag all baselines to that antenna.  Note
                 that autocorrelations are allowed, so the maximum
                 number of "baselines" to an antenna is the maximum
                 antenna number.
              (7) = 1 -> if parallel hands flagged, flag the cross
                  = 2 -> if any polarization flagged, flag all
              (8) The final pass through the data may be unreliable
                  with odd data sets (e.g. a single baseline).  To
                  turn off the baseline combination, set CPARM(8) to 1
                  or 3.  To turn off the IF combination, set CPARM(8)
                  to 2 or more.