; REFLG ;--------------------------------------------------------------- ;! Attempts to compress a flag table ;# Task UV EDITING ;----------------------------------------------------------------------- ;; Copyright (C) 2011-2012 ;; Associated Universities, Inc. Washington DC, USA. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License as ;; published by the Free Software Foundation; either version 2 of ;; the License, or (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public ;; License along with this program; if not, write to the Free ;; Software Foundation, Inc., 675 Massachusetts Ave, Cambridge, ;; MA 02139, USA. ;; ;; Correspondence concerning AIPS should be addressed as follows: ;; Internet email: aipsmail@nrao.edu. ;; Postal address: AIPS Project Office ;; National Radio Astronomy Observatory ;; 520 Edgemont Road ;; Charlottesville, VA 22903-2475 USA ;----------------------------------------------------------------------- REFLG LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC REFLG Compresses flag tables INNAME Input UV file name (name) INCLASS Input UV file name (class) INSEQ 0.0 9999.0 Input UV file name (seq. #) INDISK 0.0 9.0 Input UV file disk unit # SOURCES Restrict to listed sources CALCODE Calibrator code ' ' all FLAGVER Flag table version CPARM (1) Time interval in sec IMPORTANT - see HELP (2) flag C(2) channels between flag groups (3) Flag full spectrum if fraction flagged > C(3) (4) Flag all times if fraction flagged > C(4) (5) Flag all baselines if fraction flagged > C(5) (6) Flag antenna if fraction flagged > C(6) (7) =1 -> flag cross hands if parallel flagged =2 -> flag all polariz. if any flagged (8) =1,3 -> omit combination of baselines >= 2 -> omit combination of IFs ---------------------------------------------------------------- REFLG Task: Many flagging tasks examine large amounts of data but generate flag table entries over small ranges of time and/or spectral channel. It may be possible to compress such tables by combining these ranges. REFLG attempts to do this. It also has options to delete full things if enough channels, baselines, etc. are already flagged. After the time-frequency operation, the task optionally takes another pass through the flags looking for ways to combine IFs and baselines. That may produce some gains, but is not likely to be so profitable and can actually fail badly in some cases. You should always use FGDIF to check that the input and output FG tables flag the same data - then rerun REFLG if desired using CPARM(2) - CPARM(7). A warning about sources - if the input table has flags that apply to a source other than those requested via SOURCES and CALCODE, the flags may be lost if they do not specify a specific source number > 0. Flags specifying a specific source will be copied intact at the beginiing if they are not in the list specified by SOURCES and CALCODE. Again FGDIF is a good idea with a very general source list. Adverbs: INNAME.....Input UV file name (name). Standard defaults. INCLASS....Input UV file name (class). Standard defaults. INSEQ......Input UV file name (seq. #). 0 => highest. INDISK.....Disk drive # of input UV file. 0 => any. SOURCES....List of sources to for which flags are reviewed. The task has to list all times for a source and check them against the flags. Preparing the list of times can be expensive so this adverb lets you avoid sources which are not relevant to the flag table. '*' or ' ' = all; a "-" before a source name means all except ANY source named. CALCODE....Also used to limit sources selected: '*' => any non-blank, '-CAL' => only blank, ' ' => all, otherwise must match. FLAGVER....Specifies the version of the flagging table to be compressed. <= 0 => highest numbered table. A new table is always written. CPARM......(1) Normal interval between samples (IMPORTANT) in sec. Use a little bit less than the normal interval. 0 -> 10. This parameter can have interesting and confusing consequences when data have been time averaged by say UVAVG. Let us assume you have averaged 1 second data with no flags to 5 seconds and that the data started at time 0. Thenthe average times will be 2, 7, 12, 17, etc seconds. But at the end of the scan (e.g. time 20) there could be onl;y one 1-sec record so its averaged time would be 20. Thus to separate the 17 and the 20 one must set CPARM(1) = 2.9 say. However, if the data input to UVAVG already have extensive flagging, the times may be disturbed from this nice pattern (depnding on OPCODE in UVAVG). In this case, the possibilities are endless and there is no simple answer to a correct value for CPARM(1). (One might consider TIME or even GRID in UVAVG to alleviate this issue.) The simplest thing to do in the heavy editing case might be to set CPARM(1) = 0.95. The following parameters are used on the flag information compiled by this task. That information includes only those flags that apply to specific ranges in time, channel, and IF and to only a single baseline. More general flags are not included. If CPARM(i) <= 0, the test is not performed. (2) Flag up to CPARM(6) "good" channels located between groups of flagged channels. Thus if we are to flag, for example, channels 12-14, 16-20, 23-24, and 28-30 and FPARM(7)=2, then we will flag 12-24 and 28-30. (3) If the fraction of spectral channels flagged at a particular time and baseline exceeds CPARM(3), flag all channels. (4) If the fraction of times flagged at a particular channel and baseline exceeds CPARM(4), flag all times. (5) If the fraction of baselines flagged at a particular channel and time exceeds CPARM(5), flag all baselines. Note that autocorrelations are allowed, so the maximum number of "baselines" is (Mant * (Mant+1)) / 2 where Mant is the maximum antenna number. Thus the EVLA with Mant = 28 has 406 "baselines" rather than 351 when antenna 28 is in the array. (6) If the fraction of baselines to a particular antenna flagged at a particular channel and time exceeds CPARM(6), flag all baselines to that antenna. Note that autocorrelations are allowed, so the maximum number of "baselines" to an antenna is the maximum antenna number. (7) = 1 -> if parallel hands flagged, flag the cross hands = 2 -> if any polarization flagged, flag all polarizations. (8) The final pass through the data may be unreliable with odd data sets (e.g. a single baseline). To turn off the baseline combination, set CPARM(8) to 1 or 3. To turn off the IF combination, set CPARM(8) to 2 or more. ----------------------------------------------------------------