; SNFLG ;--------------------------------------------------------------- ;! Writes flagging info based on the contents of SN files ;# TASK UV CALIBRATION EDITING ;----------------------------------------------------------------------- ;; Copyright (C) 2001, 2003, 2009, 2012 ;; Associated Universities, Inc. Washington DC, USA. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License as ;; published by the Free Software Foundation; either version 2 of ;; the License, or (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public ;; License along with this program; if not, write to the Free ;; Software Foundation, Inc., 675 Massachusetts Ave, Cambridge, ;; MA 02139, USA. ;; ;; Correspondence concerning AIPS should be addressed as follows: ;; Internet email: aipsmail@nrao.edu. ;; Postal address: AIPS Project Office ;; National Radio Astronomy Observatory ;; 520 Edgemont Road ;; Charlottesville, VA 22903-2475 USA ;----------------------------------------------------------------------- SNFLG LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC SNFLG: writes flags based on discrepant solutions in SN files INNAME UV data (name). INCLASS UV data (class). INSEQ 0.0 9999.0 UV data (seq. #). 0 => high INDISK 0.0 9.0 Disk unit #. 0 => any INEXT Input'SN' or 'CL' table INVERS Input table file version no. SOURCES Source list QUAL -10.0 Source qualifier -1=>all TIMERANG Time range to consider STOKES Stokes type to consider SELBAND Bandwidth to include (kHz) SELFREQ Frequency to include (MHz) FREQID Freq. ID to include, -1=>all BIF 0.0 100.0 First IF to consider, 0=>1. EIF 0.0 100.0 Last IF to consider DOSTOKES -1.0 1.0 > 0 -> flag all Stokes if one is bad DOIFS -1.0 1.0 > 0 -> flag all IFs if one is bad SUBARRAY 0.0 Subarray to consider: 0 ->1 ANTENNAS Antennas to consider 0=>all CUTOFF Table weight below which data will not be considered, 0 => consider all. FLAGVER -1.0 FG table to be copied and updated. -1 -> none. OPTYPE What is examined: JUMP, AMP A&P DPARM Control parameters (1) JUMP: max allowed phase jump (deg) (1) AMP : max allowed deviation from mean in rms's (2) both: time interval (sec) Usually a bit <- SOLINT (1-6) A&P: min,max amp, min,max phase, min,max wt (7) A&P: > 0 -> flag all antennas where one is bad ---------------------------------------------------------------- SNFLG Type: Task Use: Writes flagging info based on discrepencies in the SN table solutions. OPTYPE='JUMP' flags on phase differences between adjacent entries in SN files on a baseline-by-baseline basis. It also writes flags to flag data before the first SN/CL table record and after the last SN/CL record, but only if TIMERANGE is not set. Furthermore, whenever a solution for a particular antenna is marked bad or its weight is too low, a flag is written to extend 70% of the way (in time) to the next good solution (or all the way if both solutions are bad). OPTYPE='AMP' finds the mean gain amplitude and rms separately for each source, antenna, IF, and polarization. It then produces flags for all solutions more than DPARM(1) * rms away from the mean. OPTYPE='A&P' examines every solution and generates flags for those amplitudes < DPARM(1) or > DPARM(2), and phases < DPARM(3) or > DPARM(4) in degrees (where phase runs from -180 to +180 degrees. This task (OPTYPE='JUMP') was initially contributed by Lincoln Greenhill and is based on a task initially designed by Mark Reid, both of CfA. Adverbs: INNAME.....UV file name (name). Standard defaults. INCLASS....UV file name (class). Standard defaults. INSEQ......UV file name (seq. #). 0 => highest. INDISK.....Disk unit #. 0 => any. INEXT......'SN' or 'CL' table to be consider INVERS.....Version number of table to consider, 0=>highest no. SOURCES....Source list. '*' = all; a "-" before a source name means all except ANY source named. 'ALL' means to ignore the source identifier in the SN/CL table and combine all of them into one "source". QUAL.......Only sources with a source qualifier number in the SU table matching QUAL will be used if QUAL is not -1. TIMERANG...Time range of the data to be considered. In order: Start day, hour, min. sec, end day, hour, min. sec. Days relative to reference date. STOKES.....The desired Stokes type to be considered: STOKES = 'V' or 'DIFF' gets you the phase difference of R and L treated as the data for JUMP or the ratio of R and L gains treated as the data for AMP. For A&P both polarizations are considered with these STOKES values. 'R' = RCP only, 'L' = LCP only, else both. SELBAND....Bandwidth of data to be selected. If more than one IF is present SELBAND is the width of the first IF required. Units = kHz. SELFREQ....Frequency of data to be selected. If more than one IF is present SELFREQ is the frequency of the first IF required. Units = MHz. FREQID.....Frequency identifier to select (you may determine which is applicable from the OPTYPE='SCAN' listing produced by LISTR). If either SELBAND or SELFREQ are set, their values overide that of FREQID. However, setting SELBAND and SELFREQ may result in an ambiguity. In that case, the task will request that you use FREQID. -1 => all, 0 => first BIF........Lowest IF to consider. 0 -> 1 EIF........Highest IF to consider. 0 -> highest DOSTOKES...> 0 => flag all Stokes if one is bad DOIFS......> 0 => flag all IFs if one is bad ANTENNAS...A list of the antennas to be considered. All 0 => all. If any number is negative then all antennas listed are NOT to be considered and all others are. CUTOFF.....If >= 0, the task will examine the weight column of the table and if the weight < CUTOFF, the task will consider that sample to be bad. All samples with weight <= 0 are considered bad. An FG table entry will be generated for the time surrounding a bad solution. CUTOFF is ignored for 'A&P' except as a default for DPARM(5). FLAGVER....Input FG version number. This FLAGVER will be copied to a new version and the new flags appended to it. 0 -> highest, -1 => none. OPTYPE.....'JUMP' to flag based on jumps in the SN phases on a baseline basis. 'AMP' to flag on gain amplitudes more than DPARM(1) * rms away from the robust mean. 'A&P' to flag on solutions outside specified ranges in amplitude, phase, and weight. DPARM......'AMP': (1) Flag solution times > DPARM(1) * rms of that antenna, source, IF, and polarization. 0 -> 6 Note that these mean and rms values are robust, ignoring outliers. Deleting at 3 sigma is excessive therefore. (2) Time range around solution time to be flagged. Usually the SOLINT as in JUMP. The time range is the central time minus DPARM(2)/2 to the central time plus DPARM(2)/2. 'A&P': (1) Lowest allowed gain: 0 -> 0. (2) Highest allowed gain: 0 -> 9999 (3) Lowest allowed phase: 0 -> -200 (4) Highest allowed phase: 0 -> +200 (5) Lowest allowed weight: 0 -> CUTOFF (6) Highest allowed weight: 0 -> 999999 (7) > 0 => flag all antennas in the IF and polarization when one antenna is bad. 'JUMP' or any other OPTYPE (1) Maximum allowed phase jump between adjacent SN entries (deg) (2) Minimum time interval over which to judge jumps (sec). Times within DPARM(2) of each other are regarded as the same. It is important that DPARM(2) be less than any time difference between two independent solutions in the SN table. ---------------------------------------------------------------- Notes from Lincoln Greenhill re JUMP: The idea of flagging based on baseline-dependent jumps in phase does seem counter to the AIPS philosophy. However, we think of it as follows: UV data is available in baseline-dependent terms. At time T0 and T1, two contributing stations could have atmospheric solutions that jump by reasonable amount, say +/-35 deg. However, this could correspond to 70 deg of jump on the corresponding baseline. The UV data between T0 and T1 must be calibrated in the context of this baseline-dependent phase jump (since the SN entries are differenced). If that jump is more than say 1 rad, then at times between T0 and T1, a linear interpolation could be sufficiently discrepant from the true atmospheric phase contribution as to contribute a fractional decorrelation. Conversely, two stations could jump by +70 deg. This would be grounds for flagging (if 1 rad is the threshold). But the difference is 0 and there is no jump on the corresponding baseline. This data should be OK. I could introduce a case in which the atmosphere is well correlated over two antennas but otherwise highly variable. The baseline-dependent phase jump from T0 to T1 would be 0 and the data good. The antenna dependent phase jump from T0 to T1 would be unacceptably large.