; UVRFI ;------------------------------------------------------------------ ;! Mitigate RFI by Fourier transform or fitting the circle ;# TASK UV UTILITY ;------------------------------------------------------------------ ;; Copyright (C) 2009-2012 ;; Associated Universities, Inc. Washington DC, USA. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License as ;; published by the Free Software Foundation; either version 2 of ;; the License, or (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public ;; License along with this program; if not, write to the Free ;; Software Foundation, Inc., 675 Massachusetts Ave, Cambridge, ;; MA 02139, USA. ;; ;; Correspondence concerning AIPS should be addressed as follows: ;; Internet email: aipsmail@nrao.edu. ;; Postal address: AIPS Project Office ;; National Radio Astronomy Observatory ;; 520 Edgemont Road ;; Charlottesville, VA 22903-2475 USA ;----------------------------------------------------------------------- UVRFI LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC UVRFI Task to mitigate RFI at 'BT' (and TB ?) sorted UV data INNAME Input UV file name (name) INCLASS Input UV file name (class) INSEQ 0.0 9999.0 Input UV file name (seq. #) INDISK 0.0 9.0 Input UV file disk unit # SOURCES Source name QUAL -10.0 Calibrator qualifier -1=>all CALCODE Calibrator code ' '=>all TIMERANG Time range to use SELBAND Bandwidth to select (kHz) SELFREQ Frequency to select (MHz) FREQID Freq. ID to select. SUBARRAY 0.0 1000.0 Sub-array, 0=>all BIF Low IF number to do EIF Highest IF number to do BCHAN 0.0 First channel included ECHAN 0.0 last channel included DOCALIB -1.0 101.0 > 0 calibrate data & weights > 99 do NOT calibrate weights GAINUSE CL (or SN) table to apply DOPOL -1.0 10.0 If >0.5 correct polarization. PDVER PD table to apply (DOPOL>0) BLVER BL table to apply. FLAGVER Flag table version DOBAND -1.0 10.0 If >0.5 apply bandpass cal. Method used depends on value of DOBAND (see HELP file). BPVER Bandpass table version SMOOTH Smooth or not smoth amplitudes at the frequency (channel) axis smooth(1) = 0 => no smoothing = 3 => boxcar smooth(2) width of the boxcar in channel pixels DOACOR Include autocorrelations? OUTNAME Output UV file name (name) OUTCLASS Output UV file name (class) OUTSEQ -1.0 9999.0 Output UV file name (seq. #) OUTDISK 0.0 9.0 Output UV file disk unit #. XINC 0.0 Use every XINC'th record. YINC 0.0 Time interval to find solution (sec), FULLTIME ZINC 0.0 Time (secs) to which data were pre-averaged. It must be close to the true one. OPCODE algorithm to remove RFI: 'CEXP' - Hogbom clean of fringe rate spectrum 'CIRC' - fitting the circle with variable radius See help ' ' => 'CEXP' APARM Parameters used in the two algorithms to remove RFI. For CEXP: 1 : time interval at the input data, in sec 2 : Number of pixels at the fringe rate spectrum; 0=>1/APARM(1)/2/APARM(3) 3 : Pixel size at fringe rate spectrum, Hz <=1/2/FULLTIME 4 : Number of subtracted complex exponents 0 => 1 5 : number of pixels at the DM (near 0 fringe rate) where subtraction is not allowed to exclude subtraction of signal itself. 0 => no limit for subtraction. see help >1/(FULLTIME)/APARM(3) 6 : 0 => 10000 If the flux at given channel>APARM(6), then subtraction is allowed elsewhere,as APARM(5)=0 see help 7 : Gain in the CLEAN 0 => 0.1 8 : Minimum Clean component FLUX (Jy) 9 : number of pixels at the DM near the maximum position found at the first iteration, where subtraction is allowed 10: Number of attempts for different cleanboxs 0 => 1, see help For CIRC: 1 : number of iterations at the NLLSQ algorithm fitting the circle 0 => 3 2 : 0=> fit the radius increment 1=> no fit the radius increment OPTYPE is not used ---------------------------------------------------------------- UVRFI Task: This task mitigates RFI using the two algorithm: 'CEXP': Fourier transform of the data at each time interval, each baseline, each channel, and following complex Hogbom clean subtracting one or more components from the "dirty map", using precalculated "dirty beam". The cleaned value at zero fringe rate is used for estimation of the actual visibility. 'CIRC': Fitting the circle at the complex visibility plain. This algorithm is a modernized algorithm of Raman Athrea ApJ 696, 885, 2009, May 1. Actually the spiral may be fitted. The solution for the circle (spiral) center is used for estimation of the actual visibility. Adverbs: INNAME.....Input UV file name (name). Sort 'BT' was tested but code allows for sort of 'TB' and should now work. INCLASS....Input UV file name (class). Standard defaults. INSEQ......Input UV file name (seq. #). 0 => highest. INDISK.....Disk drive # of input UV file. 0 => any. SOURCES....Source to be considered. ' '=> all; if any starts with a '-' then all except ANY source named. QUAL.......Qualifier of source to be considered. -1 => all. CALCODE....Calibrator code of sources to consider. ' '=> all. TIMERANG...Time range of the data to be copied. In order: Start day, hour, min. sec, end day, hour, min. sec. Days relative to ref. date. SELBAND....Bandwidth of data to be selected. If more than one IF is present SELBAND is the width of the first IF required. Units = kHz. For data which contain multiple bandwidths/frequencies the task will insist that some form of selection be made by frequency or bandwidth. SELFREQ....Frequency of data to be selected. If more than one IF is present SELFREQ is the frequency of the first IF required. Units = MHz. FREQID.....Frequency identifier to select (you may determine which is applicable from the OPTYPE='SCAN' listing produced by LISTR). If either SELBAND or SELFREQ are set, their values override that of FREQID. However, setting SELBAND and SELFREQ may result in an ambiguity. In that case, the task will request that you use FREQID. SUBARRAY...Sub-array number to copy. 0=>all. BIF........First IF to include. 0 -> 1. EIF........Last IF to include. 0 -> max. BCHAN......First channel to copy. 0=>all. ECHAN......Highest channel to copy. 0=>all higher than BCHAN DOCALIB....If true (>0), calibrate the data using information in the specified Cal (CL) table for multi-source or SN table for single-source data. Also calibrate the weights unless DOCALIB > 99 (use this for old non-physical weights). If the data are not in time order, then DOCALIB must be false. GAINUSE....version number of the CL table to apply to multi-source files or the SN table for single source files. 0 => highest. DOPOL......If > 0 then correct data for instrumental polarization as represented in the AN or PD table. This correction is only useful if PCAL has been run or feed polarization parameters have been otherwise obtained. See HELP DOPOL for available correction modes: 1 is normal, 2 and 3 are for VLBI. 1-3 use a PD table if available; 6, 7, 8 are the same but use the AN (continuum solution) even if a PD table is present. PDVER......PD table to apply if PCAL was run with SPECTRAL true and 0 < DOPOL < 6. <= 0 => highest. BLVER......Version number of the baseline based calibration (BL) table to apply. <0 => apply no BL table, 0 => highest. If the data are not in time order, then BL tables may not be applied and BLVER should be set to -1 if any are present. FLAGVER....specifies the version of the flagging table to be applied. 0 => highest numbered table. <0 => no flagging to be applied. DOBAND.....If true (>0) then correct the data for the shape of the antenna bandpasses using the BP table specified by BPVER. The correction has five modes: (a) if DOBAND=1 all entries for an antenna in the table are averaged together before correcting the data. (b) if DOBAND=2 the entry nearest in time (including solution weights) is used to correct the data. (c) if DOBAND=3 the table entries are interpolated in time (using solution weights) and the data are then corrected. (d) if DOBAND=4 the entry nearest in time (ignoring solution weights) is used to correct the data. (e) if DOBAND=5 the table entries are interpolated in time (ignoring solution weights) and the data are then corrected. If the data are not in time order, then only DOBAND=1 may be used. BPVER......Specifies the version of the BP table to be applied 0 => highest numbered table. <0 => no bandpass correction to be applied. SMOOTH.....UVRFI reproduces correct phases but amplitudes still rise under the expected ones at the frequencies of very high RFI. The SMOOTH allows to improve the amplitude behavior. Smooth or not smooth amplitudes at the frequency (channel) axis smooth(1) = 0 => no smoothing = 3 => boxcar smooth(2) width of the boxcar in channel pixels DOACOR.....> 0 => include autocorrelations as well as cross correlation data. OUTNAME....Output UV file name (name). Standard defaults. OUTCLASS...Output UV file name (class). Standard defaults. OUTSEQ.....Output UV file name (seq. #). Standard defaults. OUTDISK....Disk drive # of output UV file. 0 => highest with space for the file. XINC.......Write only every XINC'th output record. YINC.......Time interval used at the RFI mitigation. ZINC.......Time (sec) to which data have already been pre-averaged. UVRFI uses this to estimate the size of the output file. OPCODE.....algorithm to remove RFI: 'CEXP' - Hogbom clean of the fringe rate spectrum Fourier transform of the stored time data is carried out. We call it the Dirty Map (DM) Fourier transform of the sampling function (Dirty Beam (DB)) is calculated. The position of the DM maximum of amplitude is found. The DB located by its maximum at the position of the maximum of amplitude is multiplied by the value of the complex DM at this position and subtracted of the DM. Then such a subtraction is repeated using the residual. Number of subtraction is given by APARM(4). Parameters of the set of visibilities and its Fourier transform are determined by APARMs. The relations between parameters given by APARM are: FULLTIME = YINC is the solution interval APARM(1) = DTIME the time interval in the input data in sec APARM(2) is number of points at the half of the Fourier transform The range of frequencies at the Fourier transform is (-0.5/DTIME, 0.5/DTIME) So maximum value of APARM(2) is 0.5/DTIME / APARM(3) It may be chosen less if the range of expected fringe rates is known. APARM(3) is the step in the Fourier transform in Hz; must be <0.5/FULLTIME APARM(4) is the number of subtracted complex exponents APARM(5) is the number of pixels (NPIX) at the DM (near zero fringe rate) where subtraction is not allowed to exclude the subtraction of the signal itself. It is recommended as =>1/(FULLTIME)/APARM(3) Example 1: FULLTIME = 10sec, DTIME = 0.1sec APARM(1,2,3,4,5)= 0.1 500 0.01 100 10 APARM(6) IF (the channel AMP .GT. APARM(6)) THEN this channel is considered as RFI, and the task allows to subtract at the FOURIER transform elsewhere including zero fringe rate. If the found fringe rate period evaluated at the first iteration appears to be smaller than half of the solution interval, then such a channel is flagged. ELSE NPIX = APARM(5) 0=> 10000; therefore the flagging is excluded APARM(7) Gain in the CLEAN 0 => 0.1 APARM(8) Minimum Clean component FLUX (Jy) APARM(9) number of pixels at the DM (near the maximum position found at the first iteration) where subtraction is allowed APARM(10) If you think that having finished the cleaning at the iteration less the given number of iterations, the task may continue finding another (actually only one more) cleanbox. So the Number of the attempts is determined by the APARM(10) Only 2 attempts are allowed. 'CIRC' - fitting the circle with variable radius In many cases the "circle" looks actually as a spiral. That is why the fitting of the "circle" with increment of radius is offered. The pure circle is the algorithm given by Ramana Athrea at ApJ, 696, 885, 2009 May 1. APARM(1) : number of iterations at the NLLSQ algorithm of the circle fitting 0 => 3 APARM(2) : 0=> fit the radius increment 1=> no fit the radius increment 'CIRC' works only at special cases, although it is faster than 'CEXP'. SO THE 'CEXP' IS MUCH MORE PREFERABLE!!! ' ' => 'CEXP' OPTYPE.....is not used --------------------------------------------------------------- UVRFI: Task to mitigate RFI at 'BT' sorted UV data DOCUMENTOR: L. R. Kogan NRAO/SO RELATED PROGRAMS: UVAVG PURPOSE The UVRFI (opcode='CEXP') mitigates the RFI by Fourier transform of the time sequence of the visibilities (for each baseline, channel, polarization and time interval) and following CLEAN of the spectral (fringe rate) components. The true visibility (from the sky source) appears at zero frequency, as the result of the fringe stopping procedure at the correlator. The RFI visibility from the ground located sources should appear at non zero frequency as the result of the fringe stopping procedure at the correlator, which shifts the initial zero fringe rate (the RFI source is immovable relatively the array). The RFI visibility from a satellite located source should appear at non zero fringe rate: difference of the fringe rate from the faster moving satellite and the result of the fringe stopping procedure at the correlator. The modify algorithm of Ramana Athrea (ApJ, 696, 885, 2009 May 1) (opcode='CIRC') fitting the spiral in the Re/Im track (for each baseline, channel, polarization and each solution time interval) is available also. 'CEXP' is much more preferable.