; UFLAG ;--------------------------------------------------------------- ;! Plots and edits data using a uv-plane grid and the TV ;# Task UV Plot Editing interactive TV-APPL ;----------------------------------------------------------------------- ;; Copyright (C) 2016 ;; Associated Universities, Inc. Washington DC, USA. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License as ;; published by the Free Software Foundation; either version 2 of ;; the License, or (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public ;; License along with this program; if not, write to the Free ;; Software Foundation, Inc., 675 Massachusetts Ave, Cambridge, ;; MA 02139, USA. ;; ;; Correspondence concerning AIPS should be addressed as follows: ;; Internet email: aipsmail@nrao.edu. ;; Postal address: AIPS Project Office ;; National Radio Astronomy Observatory ;; 520 Edgemont Road ;; Charlottesville, VA 22903-2475 USA ;----------------------------------------------------------------------- UFLAG LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC UFLAG Plots and edits data from a uv data base using TV INNAME Input UV file name (name) INCLASS Input UV file name (class) INSEQ 0.0 9999.0 Input UV file name (seq. #) INDISK Input UV file disk unit # SOURCES Sources to plot, ' '=>all. ISCALIB > 0 => treat as point source QUAL -10.0 Qualifier -1=>all CALCODE Calibrator code ' '=>all STOKES Stokes type to select. SELBAND Bandwidth to select (kHz) SELFREQ Frequency to select (MHz) FREQID Freq. ID to select. TIMERANG Time range to select ANTENNAS Antennas to plot BASELINE Baselines with ANTENNAS UVRANGE UV range in kilolambda. SUBARRAY 0.0 1000.0 Subarray, 0 => all BCHAN 0.0 4096.0 1st spectral channel # ECHAN 0.0 4096.0 Last spectral channel # NCHAV 0.0 4096.0 # of channels to average CHINC 0.0 4096.0 Increment in channel # BIF Low IF number to plot EIF Highest IF number to plot DOCALIB -1.0 101.0 > 0 calibrate data & weights > 99 do NOT calibrate weights GAINUSE CAL (CL or SN) table to apply DOPOL -1.0 10.0 If >0 correct polarization. PDVER PD table to apply (DOPOL>0) BLVER BL table to apply. FLAGVER Flag table version OUTFGVER 0.0 Output FG table version DOBAND -1.0 10.0 If >0 apply bandpass cal. Method used depends on value of DOBAND (see HELP file). BPVER Bandpass table version SMOOTH Smoothing function. See HELP SMOOTH for details. CELLSIZE 1.E-12 (X,Y) size of image cell in asec IMSIZE IMAGR image size in pixels DOWEIGHT >0 -> weighted sum into each cell APARM (1) Factor times rms to examine cells in AUTO (2) Factor times rms to flag amplitudes in AUTO (3) Factor times rms to flag phases in AUTO DOCENTER -1.0 1.0 Place curvalue display at left, center, right: -1, 1, 0 DOALL -1.0 7.0 bit 1 flag all channels bit 2 flag all IFs if 1 bit 3 flag all parallel pol bit 4 do NOT flag cross-hand pol if flag parallel PRTLEV > 1 -> generate lengthy flag report > 0 -> report on all pixels auto corrected BADDISK Disk to avoid for scratch. ---------------------------------------------------------------- UFLAG Type: Task Use: Plots data from a u,v data base, with calibration if desired, as a gray-scale plot on the TV gridded in u and v. The image goes from -Umax to +Umax in U and from 0 to +Vmax in V (taking advantage of the Hermitian property of visibilities) and is kept in memory. The cell size in the array is set by the cell size used in IMAGR when given adverbs IMSIZE and CELLSIZE. This array is then displayed in whole, or in part, on the TV and various interactive editing options are available. The pixels in the plot are, at the end, converted into flags in a new flag table. Lines one pixel outside the plot, surrounding it, and a line at U and V value 0.0 are drawn when possible. The plots are labeled when either the full image or a sub-image is plotted. During interactive editing, the image pixel coordinates, the U and V physical coordinates, the value of the pixel, and the first two baselines contributing to the pixel are plotted for the pixels selected by the TV cursor. A plus sign is also shown if there are more than two baselines contributing. Details of the interactive operation of this task are given in the EXPLAIN file. See also AIPS Memo 121, "Editing on a uv grid in AIPS", September 2016 for a more detailed description of the use of this task. Adverbs: INNAME.....Input UV file name (name) Standard defaults. INCLASS....Input UV file name (class) Standard defaults. INSEQ......Input UV file name (seq. #) 0 => highest. INDISK.....Disk drive # of input UV file. 0 => any. SOURCES....List of sources to be plotted. ' '=> all; if any starts with a '-' then all except ANY source named. It is best if only one source is selected. ISCALIB....> 0 => the source is a calibrator and so should have phases of zero and a limited range of amplitudes. The AUTO FLAG VIS function flags calibrators on phase and both low and high amplitudes. Non calibrators are not flagged on phase or on low amplitudes. QUAL.......Qualifier of source to be plotted. -1 => all. CALCODE....Calibrator code of sources to plot. ' '=> all. STOKES.....The desired single Stokes type of the plotted data: 'I','Q','U','V','RR','LL', 'RL', 'LR', 'XX','YY', 'XY', and 'YX', are allowed depending on the Stokes present in the data. SELBAND....Bandwidth of data to be selected. If more than one IF is present SELBAND is the width of the first IF required. Units = kHz. For data which contain multiple bandwidths or frequencies the task will insist that some form of selection be made by frequency or bandwidth. SELFREQ....Frequency of data to be selected. If more than one IF is present SELFREQ is the frequency of the first IF required. Units = MHz. FREQID.....Frequency identifier to select (you may determine which is applicable from the OPTYPE='SCAN' listing produced by LISTR). If either SELBAND or SELFREQ are set, their values override that of FREQID. However, setting SELBAND and SELFREQ may result in an ambiguity. In that case, the task will request that you use FREQID. If all SELBAND, SELFREQ and FREQID are not specified (<= 0) then the task will loop over the frequency ID plotting all of them! TIMERANG...Time range of the data to be plotted. In order: Start day, hour, min. sec, end day, hour, min. sec. Days relative to reference date. ANTENNAS...A list of the antennas to plot. If any number is negative then all antennas listed are NOT desired and all others are. All 0 => list all. BASELINE...Baselines are specified using BASELINE. Eg. for baselines 1-6,1-8, 2-6 and 2-8 use ANTENNAS=1,2; BASELINE=6,8. UVRANGE....Range of projected spacings to be plotted in 1000's of wavelengths. 0 => 1, 1.E10 SUBARRAY...Subarray number to plot. 0 => all - the task will loop over subarray number. BCHAN......Beginning spectral line channel number. 0 => 1. ECHAN......Ending spectral line channel number. 0 => max. NCHAV......Number of spectral channels to average before plotting. 0 => 1. ECHAN will be adjusted downwards so that all averages will include the same number of spectral channels. CHINC......Increment in spectral line channel number. 0 => NCHAV Channels i through i+CHINC-1 are flagged if channel i is flagged. All channels may be flagged by setting DOALL. CHINC is ignored if NCHAV >= ECHAN-BCHAN+1. BIF........First IF number to plot. 0 => 1 EIF........Last IF number to plot. 0 => highest DOCALIB....If true (>0), calibrate the data using information in the specified Cal (CL) table for multi-source or SN table for single-source data. Also calibrate the weights unless DOCALIB > 99 (use this for old non-physical weights). GAINUSE....Version number of the Cal. table to apply to the data if DOCALIB=1. Refers to a CL table for multi-source data or an SN table for single-source. 0 => highest. DOPOL......If > 0 then correct data for instrumental polarization as represented in the AN or PD table. This correction is only useful if PCAL has been run or feed polarization parameters have been otherwise obtained. See HELP DOPOL for available correction modes: 1 is normal, 2 and 3 are for VLBI. 1-3 use a PD table if available; 6, 7, 8 are the same but use the AN (continuum solution) even if a PD table is present. PDVER......PD table to apply if PCAL was run with SPECTRAL true and 0 < DOPOL < 6. <= 0 => highest. BLVER......Version number of the baseline based calibration (BL) table to apply. <0 => apply no BL table, 0 => highest, if any. FLAGVER....Specifies the version of the flagging table to be applied. 0 => highest numbered table. <0 => no flagging to be applied. If FLAGVER < 0, no input FG table is used or copied. OUTFGVER...Flag table version to be used on output for both single- and multi-source data sets. If OUTFGVER is <= 0 or greater than FGmax (the previously highest FG version number), then a new FG table will be created for the new flags with version FGmax+1. This new table will also contain the flags applied on input (if any) from FG version FLAGVER. If OUTFGVER specifies a pre-existing FG version, then the input flags are not copied even if OUTFGVER and FLAGVER are not equal. DOBAND.....(multi-source) If true (>0), then correct the data for the shape of the antenna bandpasses using the BP table specified by BPVER. The correction has five modes: (a) if DOBAND=1 all entries for an antenna in the table are averaged together before correcting the data. (b) if DOBAND=2 the entry nearest in time (including solution weights) is used to correct the data. (c) if DOBAND=3 the table entries are interpolated in time (using solution weights) and the data are then corrected. (d) if DOBAND=4 the entry nearest in time (ignoring solution weights) is used to correct the data. (e) if DOBAND=5 the table entries are interpolated in time (ignoring solution weights) and the data are then corrected. BPVER......(multi-source) specifies the version of the BP table to be applied. 0 => highest numbered table. SMOOTH.....Specifies the type of spectral smoothing to be applied to a uv database . The default is not to apply any smoothing. The elements of SMOOTH are as follows: SMOOTH(1) = type of smoothing to apply: 0 => no smoothing To smooth before applying bandpass calibration 1 => Hanning, 2 => Gaussian, 3 => Boxcar, 4 => Sinc To smooth after applying bandpass calibration 5 => Hanning, 6 => Gaussian, 7 => Boxcar, 8 => Sinc SMOOTH(2) = the "diameter" of the function, i.e. width between first nulls of Hanning triangle and sinc function, FWHM of Gaussian, width of Boxcar. Defaults (if < 0.1) are 4, 2, 2 and 3 channels for SMOOTH(1) = 1 - 4 and 5 - 8, resp. SMOOTH(3) = the diameter over which the convolving function has value - in channels. Defaults: 1,3,1,4 times SMOOTH(2) used when input SMOOTH(3) < net SMOOTH(2). CELLSIZE...IMAGR (X,Y) pixel separation in asec. IMSIZE.....IMAGR (X,Y) The desired size of the image in pixels. UFLAG sets the U cell size as (3600 * 180) / (pi * CELLSIZE(1) * IMSIZE(1)) in wavelengths with a similar formula for the V cell size. The gridded image is made a little bigger than the minimum to hold all values of U and V. DOWEIGHT...> 0 => use data weights when summing into cells. APARM......The AUTO FLAG VIS function uses a factor times the rms in setting value ranges to examine and to flag. The user may employ APARM to set the factors. (1) is the factor for the examination range 0 => 3 (2) is the factor for the amplitude flag range 0 => 3 (3) is the factor for the phase flag range 0 => 4 For target sources (ISCALIB <= 0), all unflagged cells above the mean plus APARM(1) * rms will be examined. Visibility amplitudes above the median plus APARM(2) * rms are flagged. For calibrator sources, all unflagged cells more than APARM(1) * rms from the mean are examined. Visibility amplitudes more than APARM(2) * rms aways from the median are flagged. Visibility phases more than APARM(3) * rms / median (in radians) away from zero are flagged. DOCENTER...Rounding to nearest integer, if DOCENTER = -1 the curvalue-like display will appear at the upper left, = 1 at the upper center, and = 0 at the upper right. DOALL......Bit pattern: = 1,3,5,7,9,11,13,15 -> flag all spectral channels in a sample if any one is flagged (e.g. if you plotted only one) = 2,3,7,11,12,15 -> flag all IFs if any 1 is flagged. = 4,5,6,7,12,13,14,15 -> flag both parallel polarizations when flagging one (set true on Stokes I plots) = 8,9,10,11,12,13,14,15 -> do NOT flag all cross-hand polarizations if a parallel-hand is flaggeD AND do NOT flag all parallel-hand polarizations when a cross-hand polarization is flagged PRTLEV.....> 1 => print matices showing the amount of flagging by baseline and IF > 0 => print info about each pixel addressed by the auto flag options BADDISK....Disk numbers to avoid for scratch files. Scratch files may be created by the sorting routines if calibration or flagging is applied. ---------------------------------------------------------------- UFLAG: Plots and edits data from a u,v data base. RELATED PROGRAMS: UVPLT, EDITR, TVFLG, SPFLG PURPOSE UFLAG is used to display, in gray-scale, visibility amplitudes gridded in projected baseline coordinates u and v. Since visibilities have the Hermitian property, the full range in u, but only positive v, are plotted. Bad data may be visible with pixels well above (or below in the case of calibrator sources) the neighboring pixels. The task constructs an image from covering the full range of u and abs (v) using standard imaging adverbs to set the u and v cell sizes. The images available for viewing and editing are the averages of the samples falling in each uv cell (pill-box convolution) of the scalar average amplitude, the vector average amplitude, the vector average phase, and the difference between scalar and vector average amplitudes. Up to 25 baseline values and visibility numbers are also kept for each pixel as well as a complete list of the visibility sample numbers contributing to that pixel. The actual data being flagged (i.e. spectral channel(s), IFs) cannot be identified during interactive, TV-based editing functions. However, a special data examination function is available which shows the individual visibilities in detail for all samples contributing to the pixel. This function lets you specify which of those visibilities should be flagged. Data may be marked bad in this way and/or by marking pixels as bad, either on a pixel-by-pixel basis or in a rectangular area. (The "fat wiper" of the WIPER task is not offered here.) When you select "FLAG + EXIT", "FLAG + REPEAT", or "FLAG + SWITCH", the task re-reads the uv file (applying the same adverbs as at the beginning) and writes a flag table flagging all samples falling in pixels marked as bad, all baselines listed as bad, and any specified visibilities specified in the detailed examination routines. The new "FLAG + REPEAT" function does the operations of "FLAG + EXIT" but, instead of exiting the task, it changes FLAGVER to the OUTFGVER, sets OUTFGVER to write a new flag table, makes an entirely new image, and then resumes the editing functions. "FLAG + SWITCH" is like repeat except that it also switches polarization. If the uv grid is too large top fit on the full TV display, the plot will be shown on the TV with plot pixels averaged. In this case, you are only offered one "option" - to select a sub-image to be displayed at full resolution. The smoothed image and the selected images will have plot labels displayed. When the image or selected sub-image is smaller than the TV (when expanded to its maximum size), you are offered the following menu panels. Hit button A, B, or C to get the desired option, hit button D to get helpful information about the option. The left panel includes ----------------- | ABORT | exits the task with no flagging saved | FLAG + EXIT | stops plotting, converts flagging into FG table and exits TASK | FLAG + REPEAT | stops plotting, converts flagging into FG table and restarts task using the new flag table | FLAG + SWITCH | stops plotting, converts flagging into FG table and restarts task using the new flag table and switching the polarization | OFF ZOOM | turn off any zoom magnification | TVZOOM | interactive zoom magnification and center | OFF TRANS | turn off any black & white enhancement | OFF COLOR | turn off any pseudo-coloring | TVTRANSF | black & white enhancement as in AIPS | TVPSEUDO | many pseudo-colorings as in AIPS | TVPHLAME | flame-like pseudo-colorings as in AIPS | SET PIXRANGE | enter the range of gray-scale values to be displayed for the image type being viewed | LOAD xxxx | change the scaling function of grey-scale display to xxxx | VIEW yyyy | change the image being viewed/edited to yyyy | VIEW ONLY | sets a value range to limit the pixels viewed to only those in the selected range | VIEW ALL | resets the intensity range allowed to all values | FLAG PTS zzz | turns off/on the viewing of which pixels are flagged (in the cyan graphics channel) ----------------- The image is displayed with a step wedge above. The wedge is labeled, from left to right, with the minimum image value, the image median value, the image type and polarization (in parentheses), the image mean and rms (in parentheses), and the image maximum. The four images which may be viewed are in order VECTOR, PHASE, SCAL-VEC, and SCALAR. The "VIEW yyyy" command will display the next type yyyy. The "LOAD xxxx" command will reload the image to the TV with the specified "xxxx" function type where the types are in order LIN, LOG, SQRT, and LOG2. The right panel includes ----------------- | SET WINDOW | interactive window setting to limit edit/display area; if possible pixels will be replicated | RESET WINDOW | turn off any windowing - may force you to select a window before you can edit again | FLAG POINT | flags plot cells one at a time on buttons A, B, C | FLAG AREA | flags rectangles of cells set in the usual manner | FLAG FAST | flags any cell at which the cursor is pointed | FLAG BASELINE | Records flagging for a pair of antennas and marks cells appropriately (0 for one => all) | UNFLAG POINT | unflags flagged cells one at a time with buttons | UNFLAG AREA | unflags flagged cells in rectangles | UNFLAG FAST | unflags any flagged cell at which cursor is pointed | UNFLAG BASEL | removes record of flagging for a pair of antennas and unmarks cells appropriately (0 for one => all) | EXAMINE VIS | displays all visibilities contributing to the interactively selected pixel, allows some to be flagged and the remaining cell value restored | USER FLAG VIS | auto-flags visibilities based on value ranges entered on the terminal | AUTO FLAG VIS | auto-flags visibilities based on value ranges set by the image statistics | FLAG ALL | flags all pixels in the VIEW ONLY range | FLAG ALL WIN | flags all pixels in the VIEW ONLY range in the current sub-image | UNDO FLAG ALL | unflags a previous FLAG ALL range | UNDO FLAG WIN | unflags a previous FLAG ALL range but only in the current sub-image ----------------- The interactive flagging (unflagging) options are of three types: (1) The most basic flags unflagged points (unflags flagged points) toward which the cursor is first pointed and then button A, B, or C is "pushed". C causes an exit to the menu after flagging (unflagging) the point. (2) The area type has the user set a rectangular area with the cursor, first setting a bottom left corner and then hitting button A to set the opposite corner. Button B or C will cause all unflagged (flagged) points in the rectangle to be flagged (unflagged). C causes an exit to the menu after flagging (unflagging) the points. (3) The fast mode requires a button push to get started after which no further button pushes are required to flag (unflag) points. Any unflagged (flagged) point selected by the cursor (with left mouse button down) will be flagged (unflagged). After the first button hit, hit any button to exit to the menu. In all modes, button D exits to the menu with no further flagging. The FLAG BASELINE and UNFLAG BASEL ask you to enter 2 integers for the two antennas in the baseline. A 0 for one will flag all antennas to the other one - both 0 exits. They baseline(s) affected are recorded in a list. The plotted points are examined and any affected by the baseline(s) have their attached baseline list modified to record the flagging or unflagging. If the plotted point has any "good" baselines attached after this operation, it will be displayed as good (AND STILL INCLUDE THE DATA FROM THE FLAGGED BASELINE). Otherwise it will be shown as flagged. The plot is fully re-drawn after these operations. The "VIEW ONLY" function allows you to select with the TV cursor along the plotted step wedge the value range to be displayed at present with all values below and above the range shown as black. The cursor starts at the right of the wedge (highest value) and buttons A and B let you switch the setting of the value range between the high value and the low value. Button C exits the function with the specified values, button D exits with the values not set. When a "VIEW ONLY" range has been specified, options "FLAG ALL" and, if a sub-image has been displayed, "FLAG ALL WIN" will appear. They allow you to flag all pixels in the "VIEW ONLY" range over the full image or over only the current sub-image. The "VIEW ONLY" range is reset if one of these flag options is selected or if the image being viewed is changed. After a "VIEW ONLY" range has been flagged, options "UNDO FLAG ALL" and, if a sub-image is displayed, "UNDO FLAG WIN" are displayed. (Image types are handled separately in this matter.) They will show you all the ranges that have been flagged and, if there is more than one, allow you to select the range you wish to unflag. They unflag all flagged pixels in the specified range over the full image or over only the sub-image being viewed. The "EXAMINE VIS" function starts an interaction where you select a pixel by positioning the TV cursor and hitting buttons A, B, or C. The task then rereads the visibility records involved, performs the calibration and gridding functions, and lists all samples contributing to the pixel. It then asks for a sample number to flag and, if you enter a valid number, it marks that sample and asks again for another to flag (if there are any more). Enter 0 to stop flagging. If there are remaining unflagged pixels, they are averaged and the image(s) updated. If there are no more, the pixel is set to empty in the images. All flagged visibilities are written to a flag command file (FC table) which will be applied to the output flag (FG) table on "FLAG + EXIT", "FLAG + REPEAT", AND "FLAG + SWITCH". The "USER FLAG VIS" function allows some of the pixels in the current window to be examined with visibility records that are out of range being flagged. This function asks for input from the terminal to provide the range of pixel values to be examined, the minimum and maximum acceptable visibility amplitude and phase, and whether only flagged, only unflagged, or both pixels will be examined. The "AUTO FLAG VIS" sets these parameters for you and only works when viewing SCALAR or VECTOR. PHASE and SCALAR-VECTOR are bad ideas with automatic parameters, but you can use USER FLAG VIS in that mode. If ISCALIB <= 0, they are set so that only unflagged pixels above a value equal to the mean + 3 times the rms are examined and visiibility amplitudes greater than the median + 3 times the rms are flagged. If ISCALIB > 0, they are set so that pixels below the median - 3 times the rms and pixels above the mean + 3 times the rms are examined, visibilities below the median - 3 times the rms and above the median + 3 times the rms in amplitude are flagged, and, when not looking at PHASE, visibilities with phase in absolute value greater than 4 times the rms divided by the median are flagged. These flags go to an FC table and are applied on the "FLAG +" commands. The values in the image pixels are corrected, leaving out the flagged values. Note that you cannot unflag a visibility once it has appeared in the FC table except by choosing the ABORT option (or discarding the new output flag table). The median, mean, and rms are among the values displayed above the step wedge image and are updated whenevr an image is reloaded (after flagging or changing to another image or transfer type). These operations can be very slow if there are a lot visibilities, so progress messages are given about every two minutes. COMMENTS BCHAN, ECHAN, NCHAV, CHINC, BIF, EIF: More than one frequency may be plotted from multi-channel, multi-IF data sets. The data will be plotted at u,v,w values corrected to the frequency of the individual channel and IF. This is useful in seeing the effect of bandwidth synthesis. IF-dependent and channel-dependent errors may be spotted and edited in this way. However, they may be difficult to untangle. Displaying a single range of channels "averaged" may be better - NCHAV averages the data before they are gridded (unlike IMAGR). Note that multiple subarrays and multiple FREQIDs may be plotted and flagged. Again this may be rather hard to disentangle. SCALING : The plot scale is chosen by the task to encompass all of the data included by the adverbs. You may select the size of a pixel in the image by specifying CELLSIZE and IMSIZE. It is very helpful if the image fits on the TV screen. It is likely to be at least twice as wide as high.