AIPS HELP file for UVMLN in 31DEC24
As of Fri Apr 19 12:04:54 2024
UVMLN: Flags data based on the rms of uv-data spectra
INPUTS
INNAME Input UV file name (name)
INCLASS Input UV file name (class)
INSEQ 0.0 9999.0 Input UV file name (seq. #)
INDISK 0.0 9.0 Input UV file disk unit #
SOURCES Source list
QUAL -10.0 Source qualifier -1=>all
CALCODE Calibrator code ' '=>all
TIMERANG Time range to purge
SELBAND Bandwidth to select (kHz)
SELFREQ Frequency to select (MHz)
FREQID Freq. ID to select.
BIF 0.0 100.0 Lowest IF number 0=>all
EIF 0.0 100.0 Highest IF number 0=>all
SUBARRAY 0.0 1000.0 Subarray, 0=>all
DOCALIB -1.0 101.0 > 0 calibrate data & weights
> 99 do NOT calibrate weights
GAINUSE CL (or SN) table to apply
DOPOL -1.0 10.0 If >0 correct polarization.
PDVER PD table to apply (DOPOL>0)
BLVER BL table to apply.
FLAGVER Flag table version
OUTFGVER 0.0 Output FG table version
DOBAND -1.0 10.0 If >0 apply bandpass cal.
Method used depends on value
of DOBAND (see HELP file).
BPVER Bandpass table version
SMOOTH Smoothing function. See
HELP SMOOTH for details.
FLUX 0.0 Threshold for unity weight
ICHANSEL Select channels to fit: NOTE
this is start,end,increment
and IF for each region
ORDER 0.0 1.0 Order of fit line (0 -> DC)
BADDISK 0.0 9999.0 Disks to avoid for scratch
HELP SECTION
UVMLN
Task: This task applies the calibration to spectral-line data and then
makes a linear fit to the real and imaginary spectra using a
specified set of the spectral channels. If the residual rms in
the designated channels from this fit exceeds a noise-based
cutoff an entry is made in a flag table. The user specifies the
threshold appropriate for a 10 sec integration and the test is
done taking into account the weight of the data point. Be
conservative (i.e. 6 - 8 times the theoretical sigma is a good
default choice). Multiple sources may be processed in a single
run and calibration and editing, if present, should be applied.
The task is used to clean-up RFI prior to determining BP tables.
Adverbs:
INNAME.....Input UV file name (name). Standard defaults.
INCLASS....Input UV file name (class). Standard defaults.
INSEQ......Input UV file name (seq. #). 0 => highest.
INDISK.....Disk drive # of input UV file. 0 => any.
SOURCES....Source list. The task loops over all sources specified.
'*' = all; a "-" before a source name means all except ANY
source named.
QUAL.......Only sources with a source qualifier number in the SU table
matching QUAL will be used if QUAL is not -1.
CALCODE....Sources may be selected on the basis of the calibrator code
given in the SU table.
' ' => any calibrator code selected
'* ' => any non blank code (cal. only)
'-CAL' => blank codes only (no calibrators)
anything else = calibrator code to select.
NB: The CALCODE test is applied in addition to the other
tests, i.e. SOURCES and QUAL, in the selection of sources
to process.
TIMERANG...Time range of the data to be copied. In order: Start day,
hour, min. sec, day, hour, min. sec. Days relative to
reference date.
SELBAND....Bandwidth of data to be selected. If more than one IF is
present SELBAND is the width of the first IF required.
Units = kHz. For data which contain multiple
bandwidths/frequencies the task will insist that some form
of selection be made by frequency or bandwidth.
SELFREQ....Frequency of data to be selected. If more than one IF is
present SELFREQ is the frequency of the first IF required.
Units = MHz.
FREQID.....Frequency identifier to select (you may determine which is
applicable from the OPTYPE='SCAN' listing produced by
LISTR). If either SELBAND or SELFREQ are set, their values
override that of FREQID. However, setting SELBAND and
SELFREQ may result in an ambiguity. In that case, the task
will request that you use FREQID.
BIF........First IF to copy. 0=>all.
EIF........Highest IF to copy. 0=>all higher than BIF
SUBARRAY...Subarray number to copy. 0=>all.
DOCALIB....If true (>0), calibrate the data using information in the
specified Cal (CL) table for multi-source or SN table for
single-source data. Also calibrate the weights unless
DOCALIB > 99 (use this for old non-physical weights).
GAINUSE....version number of the CL table to apply to multi-source
files or the SN table for single-source files.
0 => highest.
DOPOL......If > 0 then correct data for instrumental polarization as
represented in the AN or PD table. This correction is
only useful if PCAL has been run or feed polarization
parameters have been otherwise obtained. See HELP DOPOL
for available correction modes: 1 is normal, 2 and 3 are
for VLBI. 1-3 use a PD table if available; 6, 7, 8 are
the same but use the AN (continuum solution) even if a PD
table is present.
PDVER......PD table to apply if PCAL was run with SPECTRAL true and
0 < DOPOL < 6. <= 0 => highest.
BLVER......Version number of the baseline based calibration
(BL) table to appply. <0 => apply no BL table,
0 => highest.
FLAGVER....specifies the version of the flagging table to be applied
to the data on input. 0 -> highest, -1 -> none.
OUTFGVER...Flag table version to be used on output for both single-
and multi-source data sets. If OUTFGVER is <= 0 or
greater than FGmax (the previously highest FG version
number), then a new FG table will be created for the new
flags with version FGmax+1. This new table will also
contain the flags applied on input (if any) from FG
version FLAGVER. If OUTFGVER specifies a pre-existing FG
version, then the input flags are not copied even if
OUTFGVER and FLAGVER are not equal.
DOBAND.....If true (>0) then correct the data for the shape of the
antenna bandpasses using the BP table specified by BPVER.
The correction has five modes:
(a) if DOBAND=1 all entries for an antenna in the table
are averaged together before correcting the data.
(b) if DOBAND=2 the entry nearest in time (including
solution weights) is used to correct the data.
(c) if DOBAND=3 the table entries are interpolated in
time (using solution weights) and the data are then
corrected.
(d) if DOBAND=4 the entry nearest in time (ignoring
solution weights) is used to correct the data.
(e) if DOBAND=5 the table entries are interpolated in
time (ignoring solution weights) and the data are then
corrected.
BPVER......Specifies the version of the BP table to be applied
0 => highest numbered table.
<0 => no bandpass correction to be applied.
SMOOTH.....Specifies the type of spectral smoothing to be applied to
a uv database . The default is not to apply any smoothing.
The elements of SMOOTH are as follows:
SMOOTH(1) = type of smoothing to apply: 0 => no smoothing
To smooth before applying bandpass calibration
1 => Hanning, 2 => Gaussian, 3 => Boxcar, 4 => Sinc
To smooth after applying bandpass calibration
5 => Hanning, 6 => Gaussian, 7 => Boxcar, 8 => Sinc
SMOOTH(2) = the "diameter" of the function, i.e. width
between first nulls of Hanning triangle and sinc
function, FWHM of Gaussian, width of Boxcar. Defaults
(if < 0.1) are 4, 2, 2 and 3 channels for SMOOTH(1) =
1 - 4 and 5 - 8, resp.
SMOOTH(3) = the diameter over which the convolving
function has value - in channels. Defaults: 1,3,1,4
times SMOOTH(2) used when input SMOOTH(3) < net
SMOOTH(2).
FLUX.......Max. residual flux allowed for unity weight. This should
be 6-8 times the rms noise for a single channel in a 10
second integration.
ICHANSEL...Select up to 20 groups of channels/IF(s) to fit as sets
of (Start,end,inc,IF), i.e., ICHANSEL = 6,37,1,0,
92,123,1,0 for two regions applyingto all IFs. The first
group for which ICHANSEL(2,i) <= 0 ends the list.
Defaults: Any IF having no group assigned to it, gets a
group including all channels.
ICHANSEL(1,j) defaults to 1,
0 < ICHANSEL(2,j) < ICHANSEL(1,j) defaults to Nchan.
ICHANSEL(3,j) < 1 or > ICHANSEL(2,j)-ICHANSEL(1,j)+1
defaults to 1.
ICHANSEL(4) <= 0 => this group applies to all IFs.
ORDER......Normally the fit should be done with a first order
polynomial, namely a DC term and a slope. If only one box
is used, the slope may not be adequately defined and a
simple DC term for the real and for the imaginary parts
would be more reliable.
BADDISK....A list of disks on which scratch files are not to be
placed.
EXPLAIN SECTION
UVMLN: Task which subtracts continuum from channels in UV-plane to
determine entries in a flag table from scatter of residuals.
Documenter: Juan M. Uson (juson@nrao.edu)
PURPOSE
UVMLN fits and removes the continuum emission in the UV-plane. UVMLN
fits to the real and imaginary parts of the visibility. The residuals
are checked in order to generate flags if the scatter is excessive.
This is done in order to clean-up the data prior to the generation of
BP tables to be used for calibration. The fit is performed using the
specified channels.
FLAGGING of the data on the basis on discrepancy in the fit is performed
using FLUX: the maximum error allowed (Jy). This is very useful for
removing narrow band interference. The number specified is the limit
per unit weight. This is adjusted by 1/sqrt[visibility weight] to
correct for integration time differences. If FILLM has been run so that
the weight of the data does NOT take into account the system
temperature, this weight is normally (tint/10 sec). UVMLN will assume
this to be the case. If not, you might have to adjust FLUX
accordingly although in the case of the VLA, the Tsys values are only
OK as relative values for any given antenna over the course of the run
and are not absolutely calibrated (they can be off by factors of up to
three depending on the actual frequency used).
Channel selection for fitting and flagging is now done with ICHANSEL
sets of channel numbers (range limits and increment) plus IF, e.g.
ICHANSEL =2, 6, 1, 0, 9, 12, 1, 0, 14, 14, 1, 0
specifies that channels 2, 3, 4, 5, 6, 9, 10, 11, 12, and 14 are used
for the fit and then examined for the rms of the residual in all IFs.
Note that the flagging is applied to all channels of the visibility
point being examined (e.g., baseline 7 with 8, Stokes LL, all channels
for a given time stamp). This is done in order to obey the
assumptions underlying the calibration of the data.
Notes on STRATEGY: UVMLN should be used to clean-up RFI from the multi-
source data prior to generating the BP tables which will be used for the
calibration. However, running the task efficiently requires an initial
BP table as otherwise the bandpass shape will determine the residuals to
the fit to a large extent. This initial BP table can be generated with
SOLINT=-1 (in BPASS). If RFI is pervasive, even this initial table can
be so contaminated that it will induce large residuals in most
visibility points. In that case, start with a high value for FLUX in
order to eliminate the worst part of the data, generate a new BP table
and iterate as needed.
UVMLN will report to the message file on visibilities present as well as
on the percentage that it flags. It also reports on how many times any
given channel triggered a flag. Notice that any one visibility might be
flagged on triggers by several channels. Use the output to decide on
threshold level and trigger channels as discussed in the next paragraph.
If a channel suffers heavily from RFI, it might cause a great number of
flags. If you need that channel to do the science, you might have to
accept the flagging of a large number of visibilities in order to
generate an interference-free bandpass table. However, if that
particular frequency is not needed for the research, it might be best to
exclude it from the fit and flagging. As long as the data have been
Hanning-smoothed (on-line or off-line), the RFI should not spread its
effect to neighboring channels and all that will be lost will be that
particular one, which will have to be ignored in the spectral cube.
Other tasks can be used to help with a contaminated, but needed,
channel. (See for example UVNOU).
UVMLN will work on single-source data but it is best to use UVLIN on
such data.
REFERENCES
See Cornwell, Uson and Haddad (Astron. Astrophys. 258, 583; 1992) for a
detailed discussion of UVLIN, (on which this task is based) which
includes estimates of errors.