;CLIP ;----------------------------------------------------------------------- ;! edits data based on amplitudes, phases, and weights out of range ;# TASK UV CALIBRATION EDITING ;----------------------------------------------------------------------- ;; Copyright (C) 1999-2001, 2003-2004, 2006-2007, 2009-2011, 2013-2014 ;; Associated Universities, Inc. Washington DC, USA. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License as ;; published by the Free Software Foundation; either version 2 of ;; the License, or (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public ;; License along with this program; if not, write to the Free ;; Software Foundation, Inc., 675 Massachusetts Ave, Cambridge, ;; MA 02139, USA. ;; ;; Correspondence concerning AIPS should be addressed as follows: ;; Internet email: aipsmail@nrao.edu. ;; Postal address: AIPS Project Office ;; National Radio Astronomy Observatory ;; 520 Edgemont Road ;; Charlottesville, VA 22903-2475 USA ;----------------------------------------------------------------------- CLIP LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC CLIP Flags data based on amplitudes/phases out of range INNAME Input UV file name (name) INCLASS Input UV file name (class) INSEQ 0.0 9999.0 Input UV file name (seq. #) INDISK 0.0 9.0 Input UV file disk unit # SOURCES Source list QUAL -10.0 Source qualifier -1=>all CALCODE Calibrator code ' '=>all STOKES Limit tests to this STOKES TIMERANG Time range to purge SELBAND Bandwidth to select (kHz) SELFREQ Frequency to select (MHz) FREQID Freq. ID to select. BCHAN 0.0 4096.0 Begin channel number. ECHAN 0.0 4096.0 End channel number. BIF 0.0 100.0 Lowest IF number 0=>all EIF 0.0 100.0 Highest IF number 0=>all SUBARRAY 0.0 1000.0 Subarray, 0=>all DOCALIB -1.0 101.0 > 0 calibrate data & weights > 99 do NOT calibrate weights GAINUSE CL (or SN) table to apply DOPOL -1.0 10.0 If >0 correct polarization. PDVER PD table to apply (DOPOL>0) BLVER BL table to apply. FLAGVER Flag table version: in OUTFGVER 0.0 Output FG table version DOBAND -1.0 10.0 If >0 apply bandpass cal. Method used depends on value of DOBAND (see HELP file). BPVER Bandpass table version SMOOTH Smoothing function. See HELP SMOOTH for details. ANTENNAS Antennas to consider 0=>all BASELINE Baselines with ANTENNAS UVRANGE Range spacings in klambda PHSLIMIT Max allowed parallel phase DOSCALE -1.0 3.0 > 0 => scale by SU table fluxes; = 2 do spectral index APARM Flux range by polarization: (1) = max. allowed parallel (2) = max. allowed cross-pol. (3) = min. allowed parallel (4) = min. allowed cross-pol. (5) = min allowed weight (6) = max allowed weight (7) Flag channels around bad (8) > 0 => do not flag cross because parallel flagged (9) > 0 => flag all channels if >= APARM(9) are clipped (10) > 0 => flag all IFs when one is bad BPARM (1) > 0 => flag parallel hand if cross hand is flagged (2) > 0 flag all Stokes if any one is flagged (!!) BADDISK 0.0 9999.0 Disks to avoid for scratch ---------------------------------------------------------------- CLIP Task: This task applies the calibration (optionally) to UV data and then examines the amplitudes and weights for data out of range. It writes entries in a flag table. It can handle either single- or multi-source data sets but it handles only cross-correlations. Use ACLIP to clip autocorrelations. Be aware that a very large number of flag table lines may be written. Setting APARM(9)>0 will reduce this a bit but may flag good data. Task UVCOP may be used to apply the flags and is the only task that can handle more than 30000 flags applying to a single time. Adverbs: INNAME.....Input UV file name (name). Standard defaults. INCLASS....Input UV file name (class). Standard defaults. INSEQ......Input UV file name (seq. #). 0 => highest. INDISK.....Disk drive # of input UV file. 0 => any. SOURCES....Source list. The task loops over all sources specified. '*' = all; a "-" before a source name means all except ANY source named. QUAL.......Only sources with a source qualifier number in the SU table matching QUAL will be used if QUAL is not -1. CALCODE....Sources may be selected on the basis of the calibrator code given in the SU table. ' ' => any calibrator code selected '* ' => any non blank code (cal. only) '-CAL' => blank codes only (no calibrators) anything else = calibrator code to select. NB: The CALCODE test is applied in addition to the other tests, i.e. SOURCES and QUAL, in the selection of sources to process. STOKES.....Specifies which STOKES parameters are examined for flux out of range: ' ' => 'FULL' 'I','Q','U','V', 'IV', 'IQU', 'IQUV' 'RR','LL', 'RL', 'LR', 'RRLL', 'RLLR', 'RLRL' 'XX','YY', 'XY', 'YX', 'XXYY', 'XYYX', 'XYXY' 'HALF', 'CROS', and 'FULL' have sensible interpretations depending on the Stokes present in the data. The last in each of the 3 rows above == 'FULL'. TIMERANG...Time range of the data to be copied. In order: Start day, hour, min. sec, day, hour, min. sec. Days relative to reference date. SELBAND....Bandwidth of data to be selected. If more than one IF is present SELBAND is the width of the first IF required. Units = kHz. For data which contain multiple bandwidths/frequencies the task will insist that some form of selection be made by frequency or bandwidth. SELFREQ....Frequency of data to be selected. If more than one IF is present SELFREQ is the frequency of the first IF required. Units = MHz. FREQID.....Frequency identifier to select (you may determine which is applicable from the OPTYPE='SCAN' listing produced by LISTR). If either SELBAND or SELFREQ are set, their values override that of FREQID. However, setting SELBAND and SELFREQ may result in an ambiguity. In that case, the task will request that you use FREQID. BCHAN......Begin (line) channel number. 0 => 1. ECHAN......End (line) channel number. 0 => highest. All channels are processed by only BCHAN-ECHAN are examined for setting flags. BIF........First IF to copy. 0=>all. EIF........Highest IF to copy. 0=>all higher than BIF SUBARRAY...Subarray number to copy. 0=>all. DOCALIB....If true (>0), calibrate the data using information in the specified Cal (CL) table for multi-source or SN table for single-source data. Also calibrate the weights unless DOCALIB > 99 (use this for old non-physical weights). GAINUSE....version number of the CL table to apply to multi-source files or the SN table for single-source files. 0 => highest. DOPOL......If > 0 then correct data for instrumental polarization as represented in the AN or PD table. This correction is only useful if PCAL has been run or feed polarization parameters have been otherwise obtained. See HELP DOPOL for available correction modes: 1 is normal, 2 and 3 are for VLBI. 1-3 use a PD table if available; 6, 7, 8 are the same but use the AN (continuum solution) even if a PD table is present. PDVER......PD table to apply if PCAL was run with SPECTRAL true and 0 < DOPOL < 6. <= 0 => highest. BLVER......Version number of the baseline based calibration (BL) table to appply. <0 => apply no BL table, 0 => highest. FLAGVER....specifies the version of the flagging table to be applied to the data on input. 0 -> highest, -1 -> none. OUTFGVER...Flag table version to be used on output for both single- and multi-source data sets. If OUTFGVER is <= 0 or greater than FGmax (the previously highest FG version number), then a new FG table will be created for the new flags with version FGmax+1. This new table will also contain the flags applied on input (if any) from FG version FLAGVER. If OUTFGVER specifies a pre-existing FG version, then the input flags are not copied even if OUTFGVER and FLAGVER are not equal. DOBAND.....If true (>0) then correct the data for the shape of the antenna bandpasses using the BP table specified by BPVER. The correction has five modes: (a) if DOBAND=1 all entries for an antenna in the table are averaged together before correcting the data. (b) if DOBAND=2 the entry nearest in time (including solution weights) is used to correct the data. (c) if DOBAND=3 the table entries are interpolated in time (using solution weights) and the data are then corrected. (d) if DOBAND=4 the entry nearest in time (ignoring solution weights) is used to correct the data. (e) if DOBAND=5 the table entries are interpolated in time (ignoring solution weights) and the data are then corrected. BPVER......Specifies the version of the BP table to be applied 0 => highest numbered table. <0 => no bandpass correction to be applied. SMOOTH.....Specifies the type of spectral smoothing to be applied to a uv database . The default is not to apply any smoothing. The elements of SMOOTH are as follows: SMOOTH(1) = type of smoothing to apply: 0 => no smoothing To smooth before applying bandpass calibration 1 => Hanning, 2 => Gaussian, 3 => Boxcar, 4 => Sinc To smooth after applying bandpass calibration 5 => Hanning, 6 => Gaussian, 7 => Boxcar, 8 => Sinc SMOOTH(2) = the "diameter" of the function, i.e. width between first nulls of Hanning triangle and sinc function, FWHM of Gaussian, width of Boxcar. Defaults (if < 0.1) are 4, 2, 2 and 3 channels for SMOOTH(1) = 1 - 4 and 5 - 8, resp. SMOOTH(3) = the diameter over which the convolving function has value - in channels. Defaults: 1,3,1,4 times SMOOTH(2) used when input SMOOTH(3) < net SMOOTH(2). ANTENNAS...A list of the antennas to be examined. All 0 => all. If any number is negative then all antennas listed are NOT to be copied and all others are. BASELINE...If any elements of BASELINE are non zero then all baselines between antennas named in ANTENNAS and those named in BASELINE are specified. If ANTENNAS contains a negative value then all baselines NOT specified are selected. UVRANGE....Range of baselines in kilolambda. PHSLIMIT...Maximum allowed absolute value of phase in degrees for parallel-hand polarizations. 0 => 180. DOSCALE....> 0 => scale the visibilities by the fluxes found for the source in the SU table. If there are no fluxes, the source is ignored. = 2 => scale the visibilities as above, but also determine a spectral index from the SU table fluxes and apply that as well. = 3 => scale the visibilities as above, but also determine a flux at 1 GHz, spectral index, and curvature from the SU table fluxes and apply them as well. <= 0 => Examine visibilities as they come out of the calibration routines - i.e. in Jy. APARM......Sets an allowable range in amplitudes (Jy or normalized) APARM(1) = max. parallel-hand amplitude. (IPOL, RR, or LL) 0 => 1.e6 APARM(2) = max. cross-amplitude. 0 -> 1e6. (QPOL, UPOL, VPOL, RL, or LR) APARM(3) = min. parallel-hand amplitude. 0 -> -1.e6 APARM(4) = min. cross-hand amplitude. Note that sign is significant for autocorrelation data such as single-dish data. APARM(5) = min allowed weight (after calibration and Stokes conversion are applied). With proper calibration, weight = 1/sigma**2 in 1/Jy**2. APARM(6) = max allowed weight. 0 -> very large. APARM(7) > 0 -> extend the flag +- APARM(7) spectral channels surrounding a high channel Thus if APARM(7) = 5 and channel 43 is bad, channels 38 through 48 will be flagged. This option is allowed to extend beyond the BCHAN throughh ECHAN range. APARM(8) > 0 => do not flag cross-polarization correlators when the corresponding parallel-hand ones are flagged. APARM(9) > 0 => Flag all spectral channels of a data sample (1 baseline, 1 time, 1 IF, 1 Stokes) if >= APARM(9) of the spectral channels in that 1 baseline, 1 time, 1 IF, and 1 Stokes are clipped. Note - this means all channels 1 through max, not just BCHAN through ECHAN. APARM(10) > 0 => Flag all IFs rather than a single IF whenever a new flag is written. BPARM.....(1) > 0 => flag parallel-hand polarizations if one of the cross-hand polarizations is flagged (2) > 0 => flag all polarizations if any one is flagged. **** BE CAREFUL **** BADDISK....A list of disks on which scratch files are not to be placed. ---------------------------------------------------------------- CLIP: Task to flag data whose amplitude is out of range. Related Programs: PRTUV, UVFLG, UVSUB, UVPLT, WIPER PURPOSE CLIP is a task which will flag all visibilities whose amplitudes are larger than a specified limit or are smaller than a specified limit. Some interaction between the flagging of correlators is possible. The task is the most convenient method for flagging large, spurious visibility values. Before serious imaging it is worthwhile to run UVPLT in order to display the visibility amplitude as a function of UV projected spacing. The plot should show a relatively smooth decline of amplitude versus spacing; the details of which depend of the angular extent of the source and the signal to noise. Large amplitude spikes are generally very obvious and these visibilities can be flagged by using CLIP over all or some the visibility spacing range. Task WIPER may also be used to flag the bad data seen in a UVPLT by interactive means. NOTE: UVPLT is normally run to display Stokes I - the vector average of RR and LL. This vector average can appear ok with bad, high amplitude RR and LL of different phase or can appear low (and bad) with ok amplitude RR and LL of different phase. CLIP will flag the former, but not the latter in its normal mode. However, the STOKES adverb allows the conversion to Stokes in CLIP before testing the amplitudes. For normal VLA data, if IPOL or VPOL is bad, RR and LL will be flagged (and RL and LR depending on APARM(6)) and, if QPOL or UPOL is bad, RL and LR will be flagged. For a bright radio source where the correlated flux density in the source obscures possible amplitude spikes, the following scheme is advised. After getting a reasonably CLEAN image, subtract the clean components from the visibility data using UVSUB. The data will now be more dominated by noise since the majority of flux density from the source has been removed, and the subsequent use of UVPLT and CLIP can flag bad data. Do not forget to add back in the clean components using UVSUB in the opposite sign in order to get back to the original visibility data. After execution CLIP will report on the number of visibilities which have been flagged. The correlator order (1,2,3,4) corresponds to (RR,LL,RL,LR). COMMENTS UVRANGE Set the uv-spacing range (in klambda) appropriately for sources which are dominated by large and small-scale structure. APARM (1) Clip any parallel hand vis. amplitude greater than this value. 0 => no upper limit. (10**6) (2) Clip any crossed-hand vis. amplitude greater than this value. 0 => no upper limit. (10**6) (3) Clip any parallel hand vis. amplitude less than this value. 0=>no lower limit. (-10**6) (4) Clip any crossed-hand vis. amplitude less than this value. 0=>no lower limit. (-10**6) Sign is significant - autocorrelation data may have either sign while cross-correlation data always has positive amplitude. (5) Clip any data with weights below this value. 0 => no lower limit (-10**12) (6) Clip any data with weights above this value. 0 => no upper limit (+10**12) (8) <= 0 => Flag crossed-hand correlators if parallel hand correlator(s) are flagged. > 0 => Do not flag crossed-hand correlators if parallel hand correlator(s) are flagged. (9) > 0 => Flag all spectral channels of a given IF and Stokes if APARM(8) or more of them exceed the clip limit (within a single sample - i.e. 1 baseline, 1 time) <= 0 => Flag spectral channels individually. (10) > 0 => Flag all IFs when one is bad - likely to be excessive in multi-channel synthesis <= 0 => Flag only the specific IF that apperas to be bad Use UVFND before CLIP if you wish to see which data will be flagged. ;---------------------------------------------------------------