; UVRFI
;------------------------------------------------------------------
;! Mitigate RFI by Fourier transform or fitting the circle
;# TASK UV UTILITY
;------------------------------------------------------------------
;;  Copyright (C) 2009-2011
;;  Associated Universities, Inc. Washington DC, USA.
;;
;;  This program is free software; you can redistribute it and/or
;;  modify it under the terms of the GNU General Public License as
;;  published by the Free Software Foundation; either version 2 of
;;  the License, or (at your option) any later version.
;;
;;  This program is distributed in the hope that it will be useful,
;;  but WITHOUT ANY WARRANTY; without even the implied warranty of
;;  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;  GNU General Public License for more details.
;;
;;  You should have received a copy of the GNU General Public
;;  License along with this program; if not, write to the Free
;;  Software Foundation, Inc., 675 Massachusetts Ave, Cambridge,
;;  MA 02139, USA.
;;
;;  Correspondence concerning AIPS should be addressed as follows:
;;         Internet email: aipsmail@nrao.edu.
;;         Postal address: AIPS Project Office
;;                         National Radio Astronomy Observatory
;;                         520 Edgemont Road
;;                         Charlottesville, VA 22903-2475 USA
;-----------------------------------------------------------------------
UVRFI     LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC
UVRFI     Task to mitigate RFI at 'BT' sorted UV data.
INNAME                             Input UV file name (name)
INCLASS                            Input UV file name (class)
INSEQ             0.0     9999.0   Input UV file name (seq. #)
INDISK            0.0        9.0   Input UV file disk unit #
SOURCES                            Source name
QUAL            -10.0              Calibrator qualifier -1=>all
CALCODE                            Calibrator code '    '=>all
TIMERANG                           Time range to use
SELBAND                            Bandwidth to select (kHz)
SELFREQ                            Frequency to select (MHz)
FREQID                             Freq. ID to select.
SUBARRAY          0.0     1000.0   Sub-array, 0=>all
BIF                                Low IF number to do
EIF                                Highest IF number to do
BCHAN             0.0              First channel included
ECHAN             0.0              last channel included
DOCALIB          -1.0      101.0   > 0 calibrate data & weights
                                   > 99 do NOT calibrate weights
GAINUSE                            CL (or SN) table to apply
DOPOL            -1.0       10.0   If >0.5 correct polarization.
PDVER                              PD table to apply (DOPOL>0)
BLVER                              BL table to apply.
FLAGVER                            Flag table version
DOBAND           -1.0       10.0   If >0.5 apply bandpass cal.
                                   Method used depends on value
                                   of DOBAND (see HELP file).
BPVER                              Bandpass table version
SMOOTH                             Smooth or not smoth
                                   amplitudes at the frequency
                                   (channel) axis
                                   smooth(1) = 0 => no smoothing
                                             = 3 => boxcar
                                   smooth(2) width of the boxcar
                                   in channel pixels
DOACOR                             Include autocorrelations?
OUTNAME                            Output UV file name (name)
OUTCLASS                           Output UV file name (class)
OUTSEQ           -1.0     9999.0   Output UV file name (seq. #)
OUTDISK           0.0        9.0   Output UV file disk unit #.
XINC              0.0              Use every XINC'th record.
YINC              0.0              Time interval to find
                                   solution (sec), FULLTIME
ZINC              0.0              Time (secs) to which data
                                   were pre-averaged. It must be
                                   close to the true one.
OPCODE                             algorithm to remove RFI:
                                   'CEXP' - Hogbom clean of fringe
                                            rate spectrum
                                   'CIRC' - fitting the circle
                                            with variable radius
                                            See help
                                   '    ' => 'CEXP'
APARM                              Parameters used in the two
                                   algorithms to remove RFI.
                                   For CEXP:
                                     1 : time interval at the
                                         input data, in sec
                                     2 : Number of pixels
                                         at the fringe rate
                                         spectrum;
                                         0=>1/APARM(1)/2/APARM(3)
                                     3 : Pixel size at fringe
                                         rate spectrum, Hz
                                         <=1/2/FULLTIME
                                     4 : Number of subtracted
                                         complex exponents
                                         0 => 1	
                                     5 : number of pixels at the
                                         DM (near 0 fringe rate)
                                         where subtraction is
                                         not allowed to exclude
                                         subtraction of signal
                                         itself.
                                         0 => no limit for
                                         subtraction.
                                         see help
                                         >1/(FULLTIME)/APARM(3)
                                     6 : 0 => 10000
                                         If the flux at given
                                         channel>APARM(6), then
                                         subtraction is allowed
                                         elsewhere,as APARM(5)=0
                                         see help
                                     7 : Gain in the CLEAN
                                         0 => 0.1
                                     8 : Minimum Clean component
                                         FLUX (Jy)
                                     9 : number of pixels at the
                                         DM near the maximum
                                         position found at the
                                         first iteration,
                                         where subtraction is
                                         allowed
                                     10: Number of attempts for
                                         different cleanboxs
                                         0 => 1, see help
                                   For CIRC:
                                     1 : number of iterations
                                         at the NLLSQ algorithm
                                         fitting the circle
                                         0 => 3
                                     2 : 0=> fit the radius
                                             increment
                                         1=> no fit the radius
                                             increment
OPTYPE                             is not used
----------------------------------------------------------------
UVRFI
Task: This task mitigates RFI using the two algorithm:
      'CEXP': Fourier transform of the data at each time interval,
              each baseline, each channel,
              and following complex Hogbom clean subtracting one or
              more components from the "dirty map", using precalculated
              "dirty beam". The cleaned value at zero fringe rate is
              used for estimation of the actual visibility.
      'CIRC': Fitting the circle at the complex visibility plain.
              This algorithm is a modernized algorithm of Raman Athrea
              ApJ 696, 885, 2009, May 1.
              Actually the spiral may be fitted.
              The solution for the circle (spiral) center  is
              used for estimation of the actual visibility.
Adverbs:
  INNAME.....Input UV file name (name). Sort must be 'BT'!!!
  INCLASS....Input UV file name (class).    Standard defaults.
  INSEQ......Input UV file name (seq. #).   0 => highest.
  INDISK.....Disk drive # of input UV file. 0 => any.
  SOURCES....Source to be considered.   '  '=> all; if any starts with
             a '-' then all except ANY source named.
  QUAL.......Qualifier of source to be considered. -1 => all.
  CALCODE....Calibrator code of sources to consider. ' '=> all.
  TIMERANG...Time range of the data to be copied. In order: Start day,
             hour, min. sec, end day, hour, min. sec. Days relative to
             ref. date.
  SELBAND....Bandwidth of data to be selected. If more than one IF is
             present SELBAND is the width of the first IF required.
             Units = kHz. For data which contain multiple
             bandwidths/frequencies the task will insist that some form
             of selection be made by frequency or bandwidth.
  SELFREQ....Frequency of data to be selected. If more than one IF is
             present SELFREQ is the frequency of the first IF required.
             Units = MHz.
  FREQID.....Frequency identifier to select (you may determine which is
             applicable from the OPTYPE='SCAN' listing produced by
             LISTR). If either SELBAND or SELFREQ are set, their values
             override that of FREQID.  However, setting SELBAND and
             SELFREQ may result in an ambiguity.  In that case, the task
             will request that you use FREQID.
  SUBARRAY...Sub-array number to copy. 0=>all.
  BIF........First IF to include.  0 -> 1.
  EIF........Last IF to include.   0 -> max.
  BCHAN......First channel to copy. 0=>all.
  ECHAN......Highest channel to copy. 0=>all higher than BCHAN
  DOCALIB....If true (>0), calibrate the data using information in the
             specified Cal (CL) table for multi-source or SN table for
             single-source data.  Also calibrate the weights unless
             DOCALIB > 99 (use this for old non-physical weights).  If
             the data are not in time order, then DOCALIB must be
             false.
  GAINUSE....version number of the CL table to apply to multi-source
             files or the SN table for single source files.
             0 => highest.
  DOPOL......If > 0 then correct data for instrumental polarization as
             represented in the AN or PD table.  This correction is
             only useful if PCAL has been run or feed polarization
             parameters have been otherwise obtained.  See HELP DOPOL
             for available correction modes: 1 is normal, 2 and 3 are
             for VLBI.  1-3 use a PD table if available; 6, 7, 8 are
             the same but use the AN (continuum solution) even if a PD
             table is present.
  PDVER......PD table to apply if PCAL was run with SPECTRAL true and
             0 < DOPOL < 6.  <= 0 => highest.
  BLVER......Version number of the baseline based calibration (BL) table
             to apply. <0 => apply no BL table, 0 => highest.
             If the data are not in time order, then BL tables may not
             be applied and BLVER should be set to -1 if any are
             present.
  FLAGVER....specifies the version of the flagging table to be applied.
              0 => highest numbered table.
             <0 => no flagging to be applied.
  DOBAND.....If true (>0) then correct the data for the shape of the
             antenna bandpasses using the BP table specified by BPVER.
             The correction has five modes:
             (a) if DOBAND=1 all entries for an antenna in the table
             are averaged together before correcting the data.
             (b) if DOBAND=2 the entry nearest in time (including
             solution weights) is used to correct the data.
             (c) if DOBAND=3 the table entries are interpolated in
             time (using solution weights) and the data are then
             corrected.
             (d) if DOBAND=4 the entry nearest in time (ignoring
             solution weights) is used to correct the data.
             (e) if DOBAND=5 the table entries are interpolated in
             time (ignoring solution weights) and the data are then
             corrected.
             If the data are not in time order, then only DOBAND=1 may
             be used.
  BPVER......Specifies the version of the BP table to be applied
                0 => highest numbered table.
               <0 => no bandpass correction to be applied.


  SMOOTH.....UVRFI reproduces correct phases but amplitudes still rise
             under the expected ones at the frequencies of very high RFI.
             The SMOOTH allows to improve the amplitude behavior.

             Smooth or not smooth amplitudes at the frequency (channel)
             axis
             smooth(1) = 0 => no smoothing
                       = 3 => boxcar
             smooth(2) width of the boxcar in channel pixels


  DOACOR.....> 0 => include autocorrelations as well as cross
             correlation data.
  OUTNAME....Output UV file name (name).    Standard defaults.
  OUTCLASS...Output UV file name (class).   Standard defaults.
  OUTSEQ.....Output UV file name (seq. #).  Standard defaults.
  OUTDISK....Disk drive # of output UV file. 0 => highest with
             space for the file.
  XINC.......Write only every XINC'th output record.
  YINC.......Time interval used at the RFI mitigation.
  ZINC.......Time (sec) to which data have already been pre-averaged.
             UVRFI uses this to estimate the size of the output file.
  OPCODE.....algorithm to remove RFI:
             'CEXP' - Hogbom clean of the fringe rate spectrum
                 Fourier transform of the stored time data is carried out.
                 We call it the Dirty Map (DM)
                 Fourier transform of the sampling function (Dirty Beam (DB))
                 is calculated.
                 The position of the DM maximum of amplitude is found.
                 The DB located by its maximum at the position of the
                 maximum of amplitude is multiplied by the value
                 of the complex DM at this position and subtracted of
                 the DM. Then such a subtraction is repeated using
                 the residual. Number of subtraction is given by APARM(4).
                 Parameters of the set of visibilities and its Fourier
                 transform are determined by APARMs.
                 The relations between parameters given by APARM are:

                     FULLTIME = YINC is the solution interval
                     APARM(1) = DTIME the time interval in the input data
                                in sec
                     APARM(2) is number of points at the half of
                              the Fourier transform
                              The range of frequencies at the Fourier
                              transform is (-0.5/DTIME, 0.5/DTIME)
                              So maximum value of APARM(2) is
                              0.5/DTIME / APARM(3)
                              It may be chosen less if the range of
                              expected fringe rates is known.
                     APARM(3) is the step in the Fourier
                              transform in Hz;
                              must be <0.5/FULLTIME
                     APARM(4) is the number of subtracted
                              complex exponents	
                     APARM(5) is the number of pixels (NPIX) at the
                              DM (near zero fringe rate) where subtraction
                              is not allowed to exclude the subtraction of
                              the signal itself.
                              It is recommended as =>1/(FULLTIME)/APARM(3)
                     Example 1:
                        FULLTIME = 10sec, DTIME = 0.1sec
                        APARM(1,2,3,4,5)= 0.1 500 0.01 100 10
                     APARM(6) IF (the channel AMP .GT. APARM(6)) THEN
                                 this channel is considered as RFI,
                                 and the task allows to subtract
                                 at the FOURIER transform elsewhere
                                 including zero fringe rate.
                                 If the found fringe rate period
                                 evaluated at the first iteration appears
                                 to be smaller than half of the solution
                                 interval, then such a channel is flagged.
                              ELSE
                                 NPIX = APARM(5)
                              0=> 10000; therefore the flagging is excluded
                     APARM(7) Gain in the CLEAN
                              0 => 0.1
                     APARM(8) Minimum Clean component FLUX (Jy)
                     APARM(9) number of pixels at the DM (near
                              the maximum position found at the
                              first iteration) where subtraction
                              is allowed
                     APARM(10) If you think that having finished
                               the cleaning at the iteration less
                               the given number of iterations,
                               the task may continue finding another
                               (actually only one more) cleanbox.
                               So the Number of the attempts is
                               determined by the APARM(10)
                               Only 2 attempts are allowed.

             'CIRC' - fitting the circle with variable radius
                      In many cases the "circle" looks actually as
                      a spiral. That is why the fitting of the "circle"
                      with increment of radius is offered.
                      The pure circle is the algorithm given by
                      Ramana Athrea at ApJ, 696, 885, 2009 May 1.

                      APARM(1) : number of iterations
                                 at the NLLSQ algorithm
                                 of the circle fitting
                                 0 => 3
                      APARM(2) : 0=> fit the radius  increment
                                 1=> no fit the radius increment
                      'CIRC' works only at special cases, although
                             it is faster than 'CEXP'.
                      SO THE 'CEXP' IS MUCH MORE PREFERABLE!!!
             '    ' => 'CEXP'
  OPTYPE.....is not used
---------------------------------------------------------------
UVRFI:  Task to mitigate RFI at 'BT' sorted UV data
DOCUMENTOR: L. R. Kogan  NRAO/SO
RELATED PROGRAMS:  UVAVG

                   PURPOSE

   The UVRFI (opcode='CEXP') mitigates the RFI by Fourier transform of
   the time sequence
   of the visibilities  (for each baseline, channel, polarization and
   time interval) and following CLEAN of the  spectral (fringe rate)
   components.
   The true visibility (from the sky source) appears at zero frequency,
   as the result of the fringe stopping procedure at the correlator.
   The RFI visibility from the ground located sources should appear at
   non zero frequency as the result of the fringe stopping procedure
   at the correlator, which shifts the initial zero fringe rate
   (the RFI source is immovable relatively the array).
   The RFI visibility from a satellite  located source  should appear at
   non zero fringe rate: difference of the fringe rate from the faster
   moving satellite and the result of the fringe stopping procedure
   at the correlator.

   The modify algorithm of Ramana Athrea (ApJ, 696, 885, 2009 May 1)
   (opcode='CIRC') fitting the spiral  in the Re/Im track
   (for each baseline, channel, polarization and each solution time interval)
   is available also. 'CEXP' is much more preferable.




