; UVLIN
;---------------------------------------------------------------
;! Fits and removes continuum visibility spectrum, also can flag
;# TASK UV EDITING SPECTRAL
;-----------------------------------------------------------------------
;;  Copyright (C) 1995, 1999-2001, 2008, 2010, 2014
;;  Associated Universities, Inc. Washington DC, USA.
;;
;;  This program is free software; you can redistribute it and/or
;;  modify it under the terms of the GNU General Public License as
;;  published by the Free Software Foundation; either version 2 of
;;  the License, or (at your option) any later version.
;;
;;  This program is distributed in the hope that it will be useful,
;;  but WITHOUT ANY WARRANTY; without even the implied warranty of
;;  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;  GNU General Public License for more details.
;;
;;  You should have received a copy of the GNU General Public
;;  License along with this program; if not, write to the Free
;;  Software Foundation, Inc., 675 Massachusetts Ave, Cambridge,
;;  MA 02139, USA.
;;
;;  Correspondence concerning AIPS should be addressed as follows:
;;         Internet email: aipsmail@nrao.edu.
;;         Postal address: AIPS Project Office
;;                         National Radio Astronomy Observatory
;;                         520 Edgemont Road
;;                         Charlottesville, VA 22903-2475 USA
;-----------------------------------------------------------------------
UVLIN     LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC
UVLIN     Fits and removes continuum visibility, also can flag
INNAME                             Input UV file name (name)
INCLASS                            Input UV file name (class)
INSEQ             0.0     9999.0   Input UV file name (seq. #)
INDISK            0.0        9.0   Input UV file disk unit #
SOURCES                            Source name
QUAL            -10.0              Calibrator qualifier -1=>all
CALCODE                            Calibrator code '    '=>all
TIMERANG                           Time range to use
SELBAND                            Bandwidth to select (kHz)
SELFREQ                            Frequency to select (MHz)
FREQID                             Freq. ID to select.
SUBARRAY          0.0     1000.0   Sub-array, 0=>all
BIF                                Low IF number to do
EIF                                Highest IF number to do
DOCALIB          -1.0      101.0   > 0 calibrate data & weights
                                   > 99 do NOT calibrate weights
GAINUSE                            CL (or SN) table to apply
DOPOL            -1.0       10.0   If >0.5 correct polarization.
PDVER                              PD table to apply (DOPOL>0)
BLVER                              BL table to apply.
FLAGVER                            Flag table version
DOBAND           -1.0       10.0   If >0.5 apply bandpass cal.
                                   Method used depends on value
                                   of DOBAND (see HELP file).
BPVER                              Bandpass table version
SMOOTH                             Smoothing function. See
                                   HELP SMOOTH for details.
DOACOR                             Include autocorrelations?
INTEXT
                                   File with channel weights
OUTNAME                            Output UV file name (name)
OUTCLASS                           Output UV file name (class)
OUTSEQ           -1.0     9999.0   Output UV file name (seq. #)
OUTDISK           0.0        9.0   Output UV file disk unit #.
SHIFT                              Shift in asec at ref position
FLUX              0.0              Threshold for unity weight
DOCONT                             Retain continuum if > 0
ICHANSEL                           Select channels to fit: NOTE
                                   this is start,end,increment
                                   and IF for each region
ORDER             0.0        1.0   Order of fit line (0 -> DC)
PRTLEV           -1.0        2.0   Print level: 0 very little
                                   1 flag summary, 2 channels
                                   used and flagged
FQCENTER                           >= 0 -> center frequency axis
----------------------------------------------------------------
UVLIN
Task:  This task does a continuum emission subtraction by making
       linear fits to the real and imaginary components versus channel
       number for each visibility and subtracting the appropriate
       values from all channels.  It uses the residual values on the
       designated channels to decide on flagging.  The user specifies
       the threshold appropriate for a 10 sec integration and the test
       is done taking into account the weight of the data point.  Be
       conservative (6 - 8 times the theoretical sigma is a good
       default choice).

       UVMLN does the same flagging operations on a multi-source data
       set but does not write out the continuum-subtracted
       visibilities.    UVLSF is similar to UVLIN, but offers a wider
       range of input options, somewhat different flagging options,
       and is able to write an output "continuum" data set based on
       the fit.  These data are very useful for continuum imaging,
       producing self-calibration SN and FG tables which may then be
       applied to the line data.  UVBAS is similar to UVLIN but uses
       amplitude and phase as so is normally inferior.  IMLIN is the
       image plane analog of UVLIN but it is almost always better to
       remove the continuum prior to imaging the line channels.
Adverbs:
  INNAME.....Input UV file name (name).      Standard defaults.
  INCLASS....Input UV file name (class).     Standard defaults.
  INSEQ......Input UV file name (seq. #).    0 => highest.
  INDISK.....Disk drive # of input UV file.  0 => any.
  SOURCES....Source to be baselined.   '  '=> all; if any starts with
             a '-' then all except ANY source named.   Only one source
             may be done at a time.
  QUAL.......Qualifier of source to be baselined. -1 => all.
  CALCODE....Calibrator code of sources to baseline. ' '=> all.
  TIMERANG...Time range of the data to be copied. In order: Start day,
             hour, min. sec, end day, hour, min. sec. Days relative to
             ref. date.
  SELBAND....Bandwidth of data to be selected. If more than one IF is
             present SELBAND is the width of the first IF required.
             Units = kHz. For data which contain multiple
             bandwidths/frequencies the task will insist that some form
             of selection be made by frequency or bandwidth.
  SELFREQ....Frequency of data to be selected. If more than one IF is
             present SELFREQ is the frequency of the first IF required.
             Units = MHz.
  FREQID.....Frequency identifier to select (you may determine which is
             applicable from the OPTYPE='SCAN' listing produced by
             LISTR). If either SELBAND or SELFREQ are set, their values
             override that of FREQID.  However, setting SELBAND and
             SELFREQ may result in an ambiguity.  In that case, the task
             will request that you use FREQID.
  SUBARRAY...Sub-array number to copy. 0=>all.
  BIF........First IF to include.  0 -> 1.
  EIF........Last IF to include.   0 -> max.
  DOCALIB....If true (>0), calibrate the data using information in the
             specified Cal (CL) table for multi-source or SN table for
             single-source data.  Also calibrate the weights unless
             DOCALIB > 99 (use this for old non-physical weights).
  GAINUSE....version number of the CL table to apply to multi-source
             files or the SN table for single source files.
             0 => highest.
  DOPOL......If > 0 then correct data for instrumental polarization as
             represented in the AN or PD table.  This correction is
             only useful if PCAL has been run or feed polarization
             parameters have been otherwise obtained.  See HELP DOPOL
             for available correction modes: 1 is normal, 2 and 3 are
             for VLBI.  1-3 use a PD table if available; 6, 7, 8 are
             the same but use the AN (continuum solution) even if a PD
             table is present.
  PDVER......PD table to apply if PCAL was run with SPECTRAL true and
             0 < DOPOL < 6.  <= 0 => highest.
  BLVER......Version number of the baseline based calibration (BL) table
             to apply. <0 => apply no BL table, 0 => highest.
  FLAGVER....specifies the version of the flagging table to be applied.
              0 => highest numbered table.
             <0 => no flagging to be applied.
  DOBAND.....If true (>0) then correct the data for the shape of the
             antenna bandpasses using the BP table specified by BPVER.
             The correction has five modes:
             (a) if DOBAND=1 all entries for an antenna in the table
             are averaged together before correcting the data.
             (b) if DOBAND=2 the entry nearest in time (including
             solution weights) is used to correct the data.
             (c) if DOBAND=3 the table entries are interpolated in
             time (using solution weights) and the data are then
             corrected.
             (d) if DOBAND=4 the entry nearest in time (ignoring
             solution weights) is used to correct the data.
             (e) if DOBAND=5 the table entries are interpolated in
             time (ignoring solution weights) and the data are then
             corrected.
             IMAGR uses DOBAND as the nearest integer; 0.1 is therefore
             "false".
  BPVER......Specifies the version of the BP table to be applied
                0 => highest numbered table.
               <0 => no bandpass correction to be applied.
  SMOOTH.....Specifies the type of spectral smoothing to be applied to
             a uv database . The default is not to apply any smoothing.
             The elements of SMOOTH are as follows:
             SMOOTH(1) = type of smoothing to apply: 0 => no smoothing
               To smooth before applying bandpass calibration
                 1 => Hanning, 2 => Gaussian, 3 => Boxcar, 4 => Sinc
               To smooth after applying bandpass calibration
                 5 => Hanning, 6 => Gaussian, 7 => Boxcar, 8 => Sinc
             SMOOTH(2) = the "diameter" of the function, i.e. width
               between first nulls of Hanning triangle and sinc
               function, FWHM of Gaussian, width of Boxcar. Defaults
               (if < 0.1) are 4, 2, 2 and 3 channels for SMOOTH(1) =
               1 - 4 and 5 - 8, resp.
             SMOOTH(3) = the diameter over which the convolving
               function has value - in channels.  Defaults: 1,3,1,4
               times SMOOTH(2) used when input SMOOTH(3) < net
               SMOOTH(2).
  DOACOR.....> 0 => include autocorrelations as well as cross
             correlation data.
  INTEXT.....File with channel weights. Specify path in usual way e.g.
             LOGICAL:FILE.  If this is blank then ICHANSEL is used.
             The format of the file is one row per channel containing
             channel number, IF number, and a weight which has to be 1
             to use the channel for fitting and interference testing
             or 0 if the channel should be ignored (these unwanted
             channels can be omitted).  Use at least one blank
             character at the beginning of the line and one in between
             the channel number, the IF number, and the weight (for an
             example, see EXPLAIN).  An IF number of 0 is taken to
             apply to all IFs.
  OUTNAME....Output UV file name (name).     Standard defaults.
  OUTCLASS...Output UV file name (class).    Standard defaults.
  OUTSEQ.....Output UV file name (seq. #).   0 => highest unique
  OUTDISK....Disk drive # of output UV file. 0 => highest with
             space for the file.
  SHIFT......Shift in arcseconds during fitting process. The data
             are shifted, fitted, flagged and then shifted back.
             These are shifts in arc seconds at the reference point -
             RA = RA0 + SHIFT(1)/cos(DEC0), DEC = DEC0 + SHIFT(2)
  FLUX.......Max. residual flux allowed for unity weight.  If the
             data weights represent 1/rms**2 as expected, then FLUX
             should be 6-8 since a weight 1 channel has an rms of 1
             Jy.  The test is actually, if
                    ABS(residual) > FLUX * SQRT (weight)
             then all channels are flagged for that correlator.  The
             threshold is applied to the residuals after subtracting
             the best linear fit.
  DOCONT.....Retain continuum if > 0. If so, the routine will simply
             flag bad data.  If <= 0, it will write the residual
             values after subtracting the best line fit to the real
             and imaginary components of each vis., flagging as well
             as flagging noisy data as above.
  ICHANSEL...Select up to 20 groups of channels/IF(s) to fit as sets
             of (Start,end,inc,IF), i.e., ICHANSEL = 6,37,1,0,
             92,123,1,0 for two regions applyingto all IFs.  The first
             group for which ICHANSEL(2,i) <= 0 ends the list.
             Defaults: Any IF having no group assigned to it, gets a
             group including all channels.
             ICHANSEL(1,j) defaults to 1,
             0 < ICHANSEL(2,j) < ICHANSEL(1,j) defaults to Nchan.
             ICHANSEL(3,j) < 1 or > ICHANSEL(2,j)-ICHANSEL(1,j)+1
             defaults to 1.
             ICHANSEL(4) <= 0 => this group applies to all IFs.
  ORDER......Normally the fit should be done with a first order
             polynomial, namely a DC term and a slope.  If only one box
             is used, the slope may not be adequately defined and a
             simple DC term for the real and for the imaginary parts
             would be more reliable.
  PRTLEV.....Print level. > 0 => print summary of data flagged.
             > 1.5 => print mask of channels used for baseline and
             print summary by channel of data flagged.
  FQCENTER...>  0 => Change frequency axis reference pixel to
                     Nchan / 2 + 1
             else => do not change reference pixel
----------------------------------------------------------------
UVLIN: Task which subtracts continuum from channels in UV-plane

			PURPOSE

UVLIN fits and removes the continuum emission in the UV-plane.
The fit is performed using the specified weights (allowing one
to ignore frequencies for which lines are present). UVLIN fits
in real and imaginary parts of the visibility and is therefore
much superior to UVBAS which fits in amplitude and phase.

			PARAMETERS

SHIFT: since UVLIN works perfectly for a single point source at
the phase center, one should shift any dominant point source to
the phase center prior to the fitting process. Note that UVFIX
does this incorrectly since it ignores the relative phase shift
between channels. THIS SHIFT IS UNDONE AFTER THE FIT. The
sense of the shift is the same as in UVFIX. The code for
computing the shift is:

         DTORAD = 3.14159265358979326D0 / 180.0D0
         RA = RA0 + SHIFT(1) / 3600./ COS (DTORAD * DEC0)
         DEC = DEC0 + SHIFT(2) / 3600.
         DXC = SIN (DTORAD * (RA - RA0)) * COS (DEC * DTORAD)
         DYC = COS (DEC0 * DTORAD) * SIN (DEC * DTORAD) -
     *      SIN (DEC0 * DTORAD) * COS (DEC * DTORAD) *
     *      COS ((RA - RA0) * DTORAD)
         DZC = SIN (DTORAD * DEC0) * SIN (DTORAD * DEC) +
     *      COS (DTORAD * DEC0) * COS (DTORAD * DEC) +
     *      COS (DTORAD * (RA - RA0)) - 1.0D0

The necessary values for SHIFT can be measured from an image in arc
seconds or have position routines return true coordinates which must
be corrected by multiplying by cos(declination) in RA.

FLAGGING of the data on the basis of discrepancy in the fit can be
performed using FLUX: the maximum error allowed per channel for
for a weight of 1 (Jy). This is very useful for removing narrow-band
interference. The number specified is the limit per weight.  This is
adjusted by 1/sqrt[visibility weight] to correct for integration time,
bandwidth, and receiver differences.  Note that although flags are
only triggered by the channels for which the fitting is done (i.e.,
those designated by the user with INTEXT or the array BOX), the
flagging is performed on all channels for a given correlator. This is
done to avoid changes in the synthesized beam as a function of
frequency that could result in spurious spectral features.  Flagging
will be minimized by setting FLUX to a suitably large number (the
default, 0, corresponds to 1.0E20). We recommend using TVFLG on any
channel of the output of UVLIN to find out what flagging has been
done. For example, if 80% of a baseline is flagged, the user might
want to delete such baseline entirely unless there is a clear reason
why UVLIN has done such flagging (if all flags are contiguous maybe
the baseline had some problem that was fixed like interference from a
transmitter that only operated during some specific time interval).

CONTINUUM subtraction is enabled by setting DOCONT less than or equal
to zero.  Otherwise the program passes the continuum and only flags
the bad points.

WEIGHTs can be specified by either using NBOXES and BOX or by an input
file. The fit is performed using only those channels with non-zero
weights but the subtraction is done for all channels.

Example of weights file:

   1   0     0
   2   1     1
   3   0  1
   4   0   1
 5  0     1
 6     0     1
 7   1      0
 8    1      0
   9   0    1
  10  0    1
  11   0      1
  12   0  1
  14   1  1
  15   0    0
  7   2    1
  8   2    1

where channel 13 is missing, so its weight will be zero.  The format
is free but do not use tabs nor other control characters except for
new-line (return). The last line must end with a new-line (return).
To accomplish the same thing with ICHANSEL:

ICHANSEL = 3,6,1,0, 9,12,1,0, 2,2,1,1, 14,14,1,1, 7,8,1,2
If INTEXT is blank and ICHANSEL is 0 for an IF, then all channels are
used for that IF.

PRTLEV is used to write flagging statistics to message file if >0.
The total number of visibilities present per correlator is printed
as well as the percentage flagged.  In addition, the program reports
how often each channel has triggered the flagging (PRTLEV>1.5).  If
one or a few channels are responsible for most of the flags, the user
might decide to give up on those channels and exclude them from the
test.  Images of the remaining channels should then have lower noise
as more visibilities will be used in making them.  This option used to
be restricted to the VLA; it no longer has such a short-sighted limit.


			REFERENCES

See Cornwell, Uson and Haddad (Astron. Astrophys. 258, 583; 1992) for a
detailed discussion which includes estimates of errors. Here is a brief
excerpt which summarizes the key attributes of UVLIN and a complementary
task, IMLIN.

	- If the continuum emission is spread over a sufficiently small
field of view, either the IMLIN or the UVLIN image will represent the
line emission well. Both may be deconvolved to produce high dynamic
range in the line. The noise level of IMLIN images varies within
position, whereas that of the UVLIN image is approximately constant.
In the presence of a point source of continuum strength S, the errors
for a small field of view are

IMLIN:

                         (theta)^2
         S (sigma_B)^2  -----------
                        (theta_F)^2

UVLIN:

                        (theta_0)^2
         S (sigma_B)^2  -----------
                        (theta_F)^2


where theta is the distance from the point source, theta_0 is the
distance of the point source to the phase center, the angular distance
theta_F is the ratio of the observing-frequency to the bandwidth times
the full-width at half maximum of the synthesized beam and sigma_B is
the sidelobe level of the synthesized beam.

	- For these methods to work well, the continuum emission must
therefore lie within a field of view theta_F, centered on the strongest
source for IMLIN, and centered on the phase tracking center for UVLIN.
This field of view may be extended by using UVSUB to pre-subtract the
brightest sources.
	- For larger fields of view, both IMLIN and UVLIN might fail
completely.
	- When imaging small fields in the presence of instrumental
errors or time-variable sources, both IMLIN and UVLIN are quite robust.
	- Both IMLIN and UVLIN have a considerable speed advantage over
UVSUB, especially for low frequency observations because of the extent
of sidelobe confusion.

There seems little in this list of attributes of IMLIN and UVLIN to
favor conclusively one method over the other. The clearest advantage
occurs in the case of weak line emission in the same field as a very
much brighter point continuum source, where UVLIN with a phase shift of
the point source to the phase center will be almost error free. In the
case of continuum power spread fairly uniformly over the field, neither
will win out decisively.  The difference in failure modes makes the
methods complementary and so both should probably be used, each as a
check on the other. In any event, the error levels can be estimated
using our formulae.

A key advantage of UVLIN occurs in the presence of interference (RFI).
UVLIN's check for consistency of the channel residuals to the linear
fit has proven to be a very effective filter of corrupted data.  We
recommend using UVLIN in both its modes of retaining the continuum as
well as discarding it (i.e. DOCONT = 1,-1).  The first output can then
be used to make an image cube and subsequently use IMLIN to obtain
both continuum and spectral images.  This, of course, images and
Cleans the continuum (differently) for each spectral channel.  The
continuum image should be made from a continuum data set, such as the
one written by UVLSF.  Imaging of the second output results directly
in the spectral image cube - the better route.
