; UVLIN ;--------------------------------------------------------------- ;! Fits and removes continuum visibility spectrum, also can flag ;# TASK UV EDITING SPECTRAL ;----------------------------------------------------------------------- ;; Copyright (C) 1995, 1999-2001, 2008, 2010, 2014 ;; Associated Universities, Inc. Washington DC, USA. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License as ;; published by the Free Software Foundation; either version 2 of ;; the License, or (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public ;; License along with this program; if not, write to the Free ;; Software Foundation, Inc., 675 Massachusetts Ave, Cambridge, ;; MA 02139, USA. ;; ;; Correspondence concerning AIPS should be addressed as follows: ;; Internet email: aipsmail@nrao.edu. ;; Postal address: AIPS Project Office ;; National Radio Astronomy Observatory ;; 520 Edgemont Road ;; Charlottesville, VA 22903-2475 USA ;----------------------------------------------------------------------- UVLIN LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC UVLIN Fits and removes continuum visibility, also can flag INNAME Input UV file name (name) INCLASS Input UV file name (class) INSEQ 0.0 9999.0 Input UV file name (seq. #) INDISK 0.0 9.0 Input UV file disk unit # SOURCES Source name QUAL -10.0 Calibrator qualifier -1=>all CALCODE Calibrator code ' '=>all TIMERANG Time range to use SELBAND Bandwidth to select (kHz) SELFREQ Frequency to select (MHz) FREQID Freq. ID to select. SUBARRAY 0.0 1000.0 Sub-array, 0=>all BIF Low IF number to do EIF Highest IF number to do DOCALIB -1.0 101.0 > 0 calibrate data & weights > 99 do NOT calibrate weights GAINUSE CL (or SN) table to apply DOPOL -1.0 10.0 If >0.5 correct polarization. PDVER PD table to apply (DOPOL>0) BLVER BL table to apply. FLAGVER Flag table version DOBAND -1.0 10.0 If >0.5 apply bandpass cal. Method used depends on value of DOBAND (see HELP file). BPVER Bandpass table version SMOOTH Smoothing function. See HELP SMOOTH for details. DOACOR Include autocorrelations? INTEXT File with channel weights OUTNAME Output UV file name (name) OUTCLASS Output UV file name (class) OUTSEQ -1.0 9999.0 Output UV file name (seq. #) OUTDISK 0.0 9.0 Output UV file disk unit #. SHIFT Shift in asec at ref position FLUX 0.0 Threshold for unity weight DOCONT Retain continuum if > 0 ICHANSEL Select channels to fit: NOTE this is start,end,increment and IF for each region ORDER 0.0 1.0 Order of fit line (0 -> DC) PRTLEV -1.0 2.0 Print level: 0 very little 1 flag summary, 2 channels used and flagged FQCENTER >= 0 -> center frequency axis ---------------------------------------------------------------- UVLIN Task: This task does a continuum emission subtraction by making linear fits to the real and imaginary components versus channel number for each visibility and subtracting the appropriate values from all channels. It uses the residual values on the designated channels to decide on flagging. The user specifies the threshold appropriate for a 10 sec integration and the test is done taking into account the weight of the data point. Be conservative (6 - 8 times the theoretical sigma is a good default choice). UVMLN does the same flagging operations on a multi-source data set but does not write out the continuum-subtracted visibilities. UVLSF is similar to UVLIN, but offers a wider range of input options, somewhat different flagging options, and is able to write an output "continuum" data set based on the fit. These data are very useful for continuum imaging, producing self-calibration SN and FG tables which may then be applied to the line data. UVBAS is similar to UVLIN but uses amplitude and phase as so is normally inferior. IMLIN is the image plane analog of UVLIN but it is almost always better to remove the continuum prior to imaging the line channels. Adverbs: INNAME.....Input UV file name (name). Standard defaults. INCLASS....Input UV file name (class). Standard defaults. INSEQ......Input UV file name (seq. #). 0 => highest. INDISK.....Disk drive # of input UV file. 0 => any. SOURCES....Source to be baselined. ' '=> all; if any starts with a '-' then all except ANY source named. Only one source may be done at a time. QUAL.......Qualifier of source to be baselined. -1 => all. CALCODE....Calibrator code of sources to baseline. ' '=> all. TIMERANG...Time range of the data to be copied. In order: Start day, hour, min. sec, end day, hour, min. sec. Days relative to ref. date. SELBAND....Bandwidth of data to be selected. If more than one IF is present SELBAND is the width of the first IF required. Units = kHz. For data which contain multiple bandwidths/frequencies the task will insist that some form of selection be made by frequency or bandwidth. SELFREQ....Frequency of data to be selected. If more than one IF is present SELFREQ is the frequency of the first IF required. Units = MHz. FREQID.....Frequency identifier to select (you may determine which is applicable from the OPTYPE='SCAN' listing produced by LISTR). If either SELBAND or SELFREQ are set, their values override that of FREQID. However, setting SELBAND and SELFREQ may result in an ambiguity. In that case, the task will request that you use FREQID. SUBARRAY...Sub-array number to copy. 0=>all. BIF........First IF to include. 0 -> 1. EIF........Last IF to include. 0 -> max. DOCALIB....If true (>0), calibrate the data using information in the specified Cal (CL) table for multi-source or SN table for single-source data. Also calibrate the weights unless DOCALIB > 99 (use this for old non-physical weights). GAINUSE....version number of the CL table to apply to multi-source files or the SN table for single source files. 0 => highest. DOPOL......If > 0 then correct data for instrumental polarization as represented in the AN or PD table. This correction is only useful if PCAL has been run or feed polarization parameters have been otherwise obtained. See HELP DOPOL for available correction modes: 1 is normal, 2 and 3 are for VLBI. 1-3 use a PD table if available; 6, 7, 8 are the same but use the AN (continuum solution) even if a PD table is present. PDVER......PD table to apply if PCAL was run with SPECTRAL true and 0 < DOPOL < 6. <= 0 => highest. BLVER......Version number of the baseline based calibration (BL) table to apply. <0 => apply no BL table, 0 => highest. FLAGVER....specifies the version of the flagging table to be applied. 0 => highest numbered table. <0 => no flagging to be applied. DOBAND.....If true (>0) then correct the data for the shape of the antenna bandpasses using the BP table specified by BPVER. The correction has five modes: (a) if DOBAND=1 all entries for an antenna in the table are averaged together before correcting the data. (b) if DOBAND=2 the entry nearest in time (including solution weights) is used to correct the data. (c) if DOBAND=3 the table entries are interpolated in time (using solution weights) and the data are then corrected. (d) if DOBAND=4 the entry nearest in time (ignoring solution weights) is used to correct the data. (e) if DOBAND=5 the table entries are interpolated in time (ignoring solution weights) and the data are then corrected. IMAGR uses DOBAND as the nearest integer; 0.1 is therefore "false". BPVER......Specifies the version of the BP table to be applied 0 => highest numbered table. <0 => no bandpass correction to be applied. SMOOTH.....Specifies the type of spectral smoothing to be applied to a uv database . The default is not to apply any smoothing. The elements of SMOOTH are as follows: SMOOTH(1) = type of smoothing to apply: 0 => no smoothing To smooth before applying bandpass calibration 1 => Hanning, 2 => Gaussian, 3 => Boxcar, 4 => Sinc To smooth after applying bandpass calibration 5 => Hanning, 6 => Gaussian, 7 => Boxcar, 8 => Sinc SMOOTH(2) = the "diameter" of the function, i.e. width between first nulls of Hanning triangle and sinc function, FWHM of Gaussian, width of Boxcar. Defaults (if < 0.1) are 4, 2, 2 and 3 channels for SMOOTH(1) = 1 - 4 and 5 - 8, resp. SMOOTH(3) = the diameter over which the convolving function has value - in channels. Defaults: 1,3,1,4 times SMOOTH(2) used when input SMOOTH(3) < net SMOOTH(2). DOACOR.....> 0 => include autocorrelations as well as cross correlation data. INTEXT.....File with channel weights. Specify path in usual way e.g. LOGICAL:FILE. If this is blank then ICHANSEL is used. The format of the file is one row per channel containing channel number, IF number, and a weight which has to be 1 to use the channel for fitting and interference testing or 0 if the channel should be ignored (these unwanted channels can be omitted). Use at least one blank character at the beginning of the line and one in between the channel number, the IF number, and the weight (for an example, see EXPLAIN). An IF number of 0 is taken to apply to all IFs. OUTNAME....Output UV file name (name). Standard defaults. OUTCLASS...Output UV file name (class). Standard defaults. OUTSEQ.....Output UV file name (seq. #). 0 => highest unique OUTDISK....Disk drive # of output UV file. 0 => highest with space for the file. SHIFT......Shift in arcseconds during fitting process. The data are shifted, fitted, flagged and then shifted back. These are shifts in arc seconds at the reference point - RA = RA0 + SHIFT(1)/cos(DEC0), DEC = DEC0 + SHIFT(2) FLUX.......Max. residual flux allowed for unity weight. If the data weights represent 1/rms**2 as expected, then FLUX should be 6-8 since a weight 1 channel has an rms of 1 Jy. The test is actually, if ABS(residual) > FLUX * SQRT (weight) then all channels are flagged for that correlator. The threshold is applied to the residuals after subtracting the best linear fit. DOCONT.....Retain continuum if > 0. If so, the routine will simply flag bad data. If <= 0, it will write the residual values after subtracting the best line fit to the real and imaginary components of each vis., flagging as well as flagging noisy data as above. ICHANSEL...Select up to 20 groups of channels/IF(s) to fit as sets of (Start,end,inc,IF), i.e., ICHANSEL = 6,37,1,0, 92,123,1,0 for two regions applyingto all IFs. The first group for which ICHANSEL(2,i) <= 0 ends the list. Defaults: Any IF having no group assigned to it, gets a group including all channels. ICHANSEL(1,j) defaults to 1, 0 < ICHANSEL(2,j) < ICHANSEL(1,j) defaults to Nchan. ICHANSEL(3,j) < 1 or > ICHANSEL(2,j)-ICHANSEL(1,j)+1 defaults to 1. ICHANSEL(4) <= 0 => this group applies to all IFs. ORDER......Normally the fit should be done with a first order polynomial, namely a DC term and a slope. If only one box is used, the slope may not be adequately defined and a simple DC term for the real and for the imaginary parts would be more reliable. PRTLEV.....Print level. > 0 => print summary of data flagged. > 1.5 => print mask of channels used for baseline and print summary by channel of data flagged. FQCENTER...> 0 => Change frequency axis reference pixel to Nchan / 2 + 1 else => do not change reference pixel ---------------------------------------------------------------- UVLIN: Task which subtracts continuum from channels in UV-plane PURPOSE UVLIN fits and removes the continuum emission in the UV-plane. The fit is performed using the specified weights (allowing one to ignore frequencies for which lines are present). UVLIN fits in real and imaginary parts of the visibility and is therefore much superior to UVBAS which fits in amplitude and phase. PARAMETERS SHIFT: since UVLIN works perfectly for a single point source at the phase center, one should shift any dominant point source to the phase center prior to the fitting process. Note that UVFIX does this incorrectly since it ignores the relative phase shift between channels. THIS SHIFT IS UNDONE AFTER THE FIT. The sense of the shift is the same as in UVFIX. The code for computing the shift is: DTORAD = 3.14159265358979326D0 / 180.0D0 RA = RA0 + SHIFT(1) / 3600./ COS (DTORAD * DEC0) DEC = DEC0 + SHIFT(2) / 3600. DXC = SIN (DTORAD * (RA - RA0)) * COS (DEC * DTORAD) DYC = COS (DEC0 * DTORAD) * SIN (DEC * DTORAD) - * SIN (DEC0 * DTORAD) * COS (DEC * DTORAD) * * COS ((RA - RA0) * DTORAD) DZC = SIN (DTORAD * DEC0) * SIN (DTORAD * DEC) + * COS (DTORAD * DEC0) * COS (DTORAD * DEC) + * COS (DTORAD * (RA - RA0)) - 1.0D0 The necessary values for SHIFT can be measured from an image in arc seconds or have position routines return true coordinates which must be corrected by multiplying by cos(declination) in RA. FLAGGING of the data on the basis of discrepancy in the fit can be performed using FLUX: the maximum error allowed per channel for for a weight of 1 (Jy). This is very useful for removing narrow-band interference. The number specified is the limit per weight. This is adjusted by 1/sqrt[visibility weight] to correct for integration time, bandwidth, and receiver differences. Note that although flags are only triggered by the channels for which the fitting is done (i.e., those designated by the user with INTEXT or the array BOX), the flagging is performed on all channels for a given correlator. This is done to avoid changes in the synthesized beam as a function of frequency that could result in spurious spectral features. Flagging will be minimized by setting FLUX to a suitably large number (the default, 0, corresponds to 1.0E20). We recommend using TVFLG on any channel of the output of UVLIN to find out what flagging has been done. For example, if 80% of a baseline is flagged, the user might want to delete such baseline entirely unless there is a clear reason why UVLIN has done such flagging (if all flags are contiguous maybe the baseline had some problem that was fixed like interference from a transmitter that only operated during some specific time interval). CONTINUUM subtraction is enabled by setting DOCONT less than or equal to zero. Otherwise the program passes the continuum and only flags the bad points. WEIGHTs can be specified by either using NBOXES and BOX or by an input file. The fit is performed using only those channels with non-zero weights but the subtraction is done for all channels. Example of weights file: 1 0 0 2 1 1 3 0 1 4 0 1 5 0 1 6 0 1 7 1 0 8 1 0 9 0 1 10 0 1 11 0 1 12 0 1 14 1 1 15 0 0 7 2 1 8 2 1 where channel 13 is missing, so its weight will be zero. The format is free but do not use tabs nor other control characters except for new-line (return). The last line must end with a new-line (return). To accomplish the same thing with ICHANSEL: ICHANSEL = 3,6,1,0, 9,12,1,0, 2,2,1,1, 14,14,1,1, 7,8,1,2 If INTEXT is blank and ICHANSEL is 0 for an IF, then all channels are used for that IF. PRTLEV is used to write flagging statistics to message file if >0. The total number of visibilities present per correlator is printed as well as the percentage flagged. In addition, the program reports how often each channel has triggered the flagging (PRTLEV>1.5). If one or a few channels are responsible for most of the flags, the user might decide to give up on those channels and exclude them from the test. Images of the remaining channels should then have lower noise as more visibilities will be used in making them. This option used to be restricted to the VLA; it no longer has such a short-sighted limit. REFERENCES See Cornwell, Uson and Haddad (Astron. Astrophys. 258, 583; 1992) for a detailed discussion which includes estimates of errors. Here is a brief excerpt which summarizes the key attributes of UVLIN and a complementary task, IMLIN. - If the continuum emission is spread over a sufficiently small field of view, either the IMLIN or the UVLIN image will represent the line emission well. Both may be deconvolved to produce high dynamic range in the line. The noise level of IMLIN images varies within position, whereas that of the UVLIN image is approximately constant. In the presence of a point source of continuum strength S, the errors for a small field of view are IMLIN: (theta)^2 S (sigma_B)^2 ----------- (theta_F)^2 UVLIN: (theta_0)^2 S (sigma_B)^2 ----------- (theta_F)^2 where theta is the distance from the point source, theta_0 is the distance of the point source to the phase center, the angular distance theta_F is the ratio of the observing-frequency to the bandwidth times the full-width at half maximum of the synthesized beam and sigma_B is the sidelobe level of the synthesized beam. - For these methods to work well, the continuum emission must therefore lie within a field of view theta_F, centered on the strongest source for IMLIN, and centered on the phase tracking center for UVLIN. This field of view may be extended by using UVSUB to pre-subtract the brightest sources. - For larger fields of view, both IMLIN and UVLIN might fail completely. - When imaging small fields in the presence of instrumental errors or time-variable sources, both IMLIN and UVLIN are quite robust. - Both IMLIN and UVLIN have a considerable speed advantage over UVSUB, especially for low frequency observations because of the extent of sidelobe confusion. There seems little in this list of attributes of IMLIN and UVLIN to favor conclusively one method over the other. The clearest advantage occurs in the case of weak line emission in the same field as a very much brighter point continuum source, where UVLIN with a phase shift of the point source to the phase center will be almost error free. In the case of continuum power spread fairly uniformly over the field, neither will win out decisively. The difference in failure modes makes the methods complementary and so both should probably be used, each as a check on the other. In any event, the error levels can be estimated using our formulae. A key advantage of UVLIN occurs in the presence of interference (RFI). UVLIN's check for consistency of the channel residuals to the linear fit has proven to be a very effective filter of corrupted data. We recommend using UVLIN in both its modes of retaining the continuum as well as discarding it (i.e. DOCONT = 1,-1). The first output can then be used to make an image cube and subsequently use IMLIN to obtain both continuum and spectral images. This, of course, images and Cleans the continuum (differently) for each spectral channel. The continuum image should be made from a continuum data set, such as the one written by UVLSF. Imaging of the second output results directly in the spectral image cube - the better route.