; EDITR ;--------------------------------------------------------------- ;! Interactive baseline-oriented visibility editor using the TV ;# TASK UV VLBI CALIBRATION EDITING TV-APPL INTERACTIVE OOP AP ;----------------------------------------------------------------------- ;; Copyright (C) 1996-2000, 2004, 2006-2008, 2010 ;; Associated Universities, Inc. Washington DC, USA. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License as ;; published by the Free Software Foundation; either version 2 of ;; the License, or (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public ;; License along with this program; if not, write to the Free ;; Software Foundation, Inc., 675 Massachusetts Ave, Cambridge, ;; MA 02139, USA. ;; ;; Correspondence concerning AIPS should be addressed as follows: ;; Internet email: aipsmail@nrao.edu. ;; Postal address: AIPS Project Office ;; National Radio Astronomy Observatory ;; 520 Edgemont Road ;; Charlottesville, VA 22903-2475 USA ;--------------------------------------------------------------- EDITR LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC EDITR: Interactive baseline-oriented UV data editor using the TV INNAME Input UV data (name) INCLASS Input UV data (class) INSEQ Input UV data (seq. #) INDISK Input UV data disk drive # SOURCES Source list QUAL -10.0 Calibrator qualifier -1=>all CALCODE Calibrator code ' '=>all TIMERANG Time range to use SELBAND Bandwidth to select (kHz) SELFREQ Frequency to select (MHz) FREQID Freq. ID to select. SUBARRAY 0.0 1000.0 Subarray, 0=>1 DOCALIB -1.0 101.0 > 0 calibrate data & weights > 99 do NOT calibrate weights GAINUSE CL (or SN) table to apply DOPOL -1.0 10.0 If >0 correct polarization. PDVER PD table to apply (DOPOL>0) BLVER BL table to apply. FLAGVER Flag table version OUTFGVER 0.0 Output FG table version DOBAND -1.0 10.0 If >0 apply bandpass cal. Method used depends on value of DOBAND (see HELP file). BPVER Bandpass table version SMOOTH Smoothing function. See HELP SMOOTH for details. STOKES Use 'I' to edit RR and LL combined, else task will set BCHAN 0.0 8192.0 Low freq. channel in average ECHAN 0.0 8192.0 High freq channel in average BIF First IF displayed EIF Last IF displayed UVRANGE 0. Min & max baseline (klambda) ANTENNAS List of antennas to be used IN2NAME 2nd UV file name IN2CLASS 2nd UV file class IN2SEQ 2nd UV file sequence IN2DISK 2nd UV file disk DOHIST -3.0 1.0 Record flags in history file SOLINT 0.0 Data averaging time in mins DETIME 0.0 Break interval (min) DOWEIGHT 0.0 Rescale weights by DOWEIGHT DOTWO -1.0 1.0 True => do second observable plot of main baseline EXPERT -1.0 1.0 > 0 start in expert mode CROWDED -1.0 1.0 > 0 => allow plots with all polarizations and IFs REASON Initial reason string ANTUSE 0.0 90.0 Initial displayed antennas BADDISK -1.0 1000.0 Disks to avoid for scratch. ---------------------------------------------------------------- EDITR Type: Task Use: To edit visibility data interactively using the TV graphics planes. The visibility amplitude or phase or the amplitude of the visibility with a running average subtracted may be displayed in an edit window. A second observable from the selected baseline is shown in a display adjacent to the edit window. Optionally, data in the primary observable for 1 to 10 other baselines to the selected antenna may be displayed over the same time range in areas above the second window. The data are shown in a low-resolution, full time-range display or they may be shown with higher resolution by interactively selecting a "frame" or window of shorter time duration. A second uv data set may also be displayed along with the first. These data are not used for editing but may help you to select the data to be deleted. A "normal" choice for the second data set would be the residuals after Cleaning or UVSUB. A menu-like control interface is available to select the data, antenna, and time range to be edited and to select various forms of editing. An "expert mode" allows you to enter single-letter menu selections from the keyboard (xterm) while this expert mode is enabled. EDITR creates a Flag Command file attached to the input UV data file. If one already exists - due to a prior power failure or program failure - it will be used. This is a bit dangerous and should be done only if the data selection adverbs have the values they had when the FC file was created. When EDITR finishes, the FC table is translated into a standard Flag (FG) table. EDITR is for editing continuum data from one or more IFs. Multiple spectral channels may be averaged on input with the vector average used for display and editing; multiple IFs are kept separate. The data may also be averaged over time as they are read into memory. This is useful for improved signal-to-noise and to help squeeze the data into memory, but will cause the data flags to be less selective in time. The program is more efficient if all data fit in memory, so EDITR will try to read all data into memory if it can. Failing that it will try to read all data for one IF or all data for one antenna for all or one IF. It will fail if this last case does not fit and you will need to use TIMERANGE or SOLINT to reduce the amount of data. Adverbs: INNAME.....Input UV data file (name). Standard defaults. INCLASS....Input UV data file (class). Standard defaults. INSEQ......Input UV data file (seq. #). 0 => highest. INDISK.....Input UV data file disk drive #. 0 => any. SOURCES....Source list. '*' = all; a "-" before a source name means all except ANY source named. QUAL.......Only sources with a source qualifier number in the SU table matching QUAL will be used if QUAL is not -1. CALCODE....Sources may be selected on the basis of the calibrator code given in the SU table. ' ' => any calibrator code selected '* ' => any non blank code (cal. only) '-CAL' => blank codes only (no calibrators) anything else = calibrator code to select. NB: The CALCODE test is applied in addition to the other tests, i.e. SOURCS and QUAL, in the selection of sources to process. TIMERANG...Time range of the data to be copied. In order: Start day, hour, min. sec, end day, hour, min. sec. Days relative to reference date. SELBAND....Bandwidth of data to be selected. If more than one IF is present SELBAND is the width of the first IF required. Units = kHz. For data which contain multiple bandwidths/frequencies the task will insist that some form of selection be made by frequency or bandwidth. SELFREQ....Frequency of data to be selected. If more than one IF is present SELFREQ is the frequency of the first IF required. Units = MHz. FREQID.....Frequency identifier to select (you may determine which is applicable from the OPTYPE='SCAN' listing produced by LISTR). If either SELBAND or SELFREQ are set, their values override that of FREQID. However, setting SELBAND and SELFREQ may result in an ambiguity. In that case, the task will request that you use FREQID. SUBARRAY...Subarray number to copy. 0=>all. DOCALIB....If true (>0), calibrate the data using information in the specified Cal (CL) table for multi-source or SN table for single-source data. Also calibrate the weights unless DOCALIB > 99 (use this for old non-physical weights). GAINUSE....Version number of the CL table to apply to multi-source files or the SN table for single-source files. 0 => highest. DOPOL......If > 0.5 then correct data for instrumental polarization as represented in the AN or PD table. This correction is only useful if PCAL has been run or feed polarization parameters have been otherwise obtained. See HELP DOPOL for available correction modes: 1 is normal, 2 and 3 are for VLBI. 1-3 use a PD table if available; 6, 7, 8 are the same but use the AN (continuum solution) even if a PD table is present. PDVER......PD table to apply if PCAL was run with SPECTRAL true and 0 < DOPOL < 6. <= 0 => highest. BLVER......Version number of the baseline based calibration (BL) table to apply. <0 => apply no BL table, 0 => highest. FLAGVER....specifies the version of the flagging table to be applied to the data on input. 0 -> highest, -1 -> none. OUTFGVER...Flag table version to be used on output for both single- and multi-source data sets. If OUTFGVER is <= 0 or greater than FGmax (the previously highest FG version number), then a new FG table will be created for the new flags with version FGmax+1. This new table will also contain the flags applied on input (if any) from FG version FLAGVER. If OUTFGVER specifies a pre-existing FG version, then the input flags are not copied even if OUTFGVER and FLAGVER are not equal. DOBAND.....If true (>0) then correct the data for the shape of the antenna bandpasses using the BP table specified by BPVER. The correction has five modes: (a) if DOBAND=1 all entries for an antenna in the table are averaged together before correcting the data. (b) if DOBAND=2 the entry nearest in time (including solution weights) is used to correct the data. (c) if DOBAND=3 the table entries are interpolated in time (using solution weights) and the data are then corrected. (d) if DOBAND=4 the entry nearest in time (ignoring solution weights) is used to correct the data. (e) if DOBAND=5 the table entries are interpolated in time (ignoring solution weights) and the data are then corrected. BPVER......Specifies the version of the BP table to be applied (if DOBAND > 0). 0 => highest numbered table. SMOOTH.....Specifies the type of spectral smoothing to be applied to a uv database . The default is not to apply any smoothing. The elements of SMOOTH are as follows: SMOOTH(1) = type of smoothing to apply: 0 => no smoothing To smooth before applying bandpass calibration 1 => Hanning, 2 => Gaussian, 3 => Boxcar, 4 => Sinc To smooth after applying bandpass calibration 5 => Hanning, 6 => Gaussian, 7 => Boxcar, 8 => Sinc SMOOTH(2) = the "diameter" of the function, i.e. width between first nulls of Hanning triangle and sinc function, FWHM of Gaussian, width of Boxcar. Defaults (if < 0.1) are 4, 2, 2 and 3 channels for SMOOTH(1) = 1 - 4 and 5 - 8, resp. SMOOTH(3) = the diameter over which the convolving function has value - in channels. Defaults: 1,3,1,4 times SMOOTH(2) used when input SMOOTH(3) < net SMOOTH(2). STOKES.....Set STOKES='I' to force editing of RR and LL as one - this saves memory and will make the task faster. Otherwise, the task sets STOKES to 'HALF' unless the first (or if present the 2nd) data set is already converted to I. In that case, 'I' is used for both. BCHAN......First channel number to include. 0 => 1. Channels are averaged before being displayed. ECHAN......Highest channel number to to include in displayed average, 0 => max BIF........The lowest numbered IF to include. IFs may be edited separately in one run. 0 => 1. EIF........The highest numbered IF to include. 0 =>highest. Note: not all data sets will have IFs. UVRANGE....Include data from UVRANGE(1) through UVRANGE(2) kilowavelengths. 0,0 => all. ANTENNAS...A list of antennas to be included (if all are > 0), data involving any other antenna is excluded. If any one of the numbers is < 0, then ANTENNAS becomes a list of antennas to be excluded; data involving two antennas both not in ANTENNAS are included. IN2NAME....2nd UV data file name. This file will be displayed along with the first file, although only the first file may be used for editing. All calibration is turned off for this second file. BCHAN, ECHAN, BIF, and EIF are applied to the second file iff the second file has the same number of input channels and IFs as the first. Otherwise, channels 1 through ECHAN-BCHAN+1, IFs 1 through EIF-BIF+1 are included. ' ' => no 2nd UV data set. This option is normally used for a "residual" uv data set set as that produced by IMAGR or UVSUB. IN2CLASS...2nd UV file class. ' ' => no 2nd UV data set. IN2SEQ.....2nd UV data file sequence number. < 0 => do not use this option; = 0 => highest existing. IN2DISK....2nd UV data file disk number. = 0 => any. DOHIST.....> 0 => record task execution and flagging info in the history file (this can be a lot!). <= 0.0 means to omit the flagging info and < -1.5 means to omit the execution information as well. SOLINT.....Data are averaged in time over SOLINT minutes to produce better signal to noise and to reduce memory requirements. 0 => 1/6000(0.01 s). If you want averaging, set this number appropriately. Editing times are offset by SOLINT/2 which may cause confusion if no averaging occurs. When you do this, you must set a DETIME value which correctly represents your data; the default can put a gap between every sample which is costly in memory and data access. DETIME.....Samples more than DETIME (minutes) from the previous will be considered to imply a break in the time sequence and will be assigned a time interval after a gap of 1 interval or 2 intervals if the sample is more than 2 * DETIME later. < SOLINT => MAX (2, 5*SOLINT). DETIME is used as the initial estimate of the scan length as well - this may be changed interactively. DOWEIGHT...The display of error bars (restricted to amplitudes) depends on the data weights being 1/(sigma**2) in 1 (Jy**2). Set DOWEIGHT to get the data weights into this scale, multiplying the input weights by DOWEIGHT. 0 -> 1. DOTWO......> 0 (true) means to do the secondary plot of a second observable (phase, amplitude, diff amplitude) from the primary baseline. <= 0 means to plot only the primary observable used for the editing. True is highly recommended. EXPERT.....> 0 => the task begins in "expert mode" in which commands are entered with single characters at the keyboard rather than by TV menu selection. You can switch in and out of expert mode interactively after the task begins without regard for the value of EXPERT. If the task begins in EXPRT mode, it starts with the amplitudes and difference amplitudesplotted from 0 to maximum and phases from -180 to 180 and with the ALL IF and ALL POL flags =NOT.CROWDED rather than false. These choices may be changed interactively. CROWDED....If true (> 0), the task will be allowed to plot and edit all IFs and/or all polarizations at once. Otherwise, only one polarization and one IF are plotted at a time and much of the editing is based only on the values of that polarization/IF. REASON.....The initial reason for flagging recorded in the FG table. ' ' => task name, date and time. This can be changed interactively. ANTUSE.....Lists the antennas to appear in the initial display: baseline ANTUSE(1)-ANTUSE(2) is the initial edit baseline and ANTUSE(1)-ANTUSE(3) through ANTUSE(1)-ANTUSE(12) are the comparison baselines. The default is equivalent to ANTUSE=1,2,3,4 (or whatever are the 4 lowest numbered antennas). BADDISK....This array contains the numbers of disks on which it is desired that scratch files not be located. ---------------------------------------------------------------- EDITR is a TV graphic editor for uv data using the uv data themselves and optionally a second data set (e.g. residuals). It works with the XAS-TV display graphics planes (7 of them) to allow you to flag uv data based on their values. It does not change the input data set, but will write a flag table. EDITR averages spectral channels and over a specified time interval as it reads the data set(s). It attempts to read the entire data set into the pseudo-AP memory when if starts up. If it cannot fit all the data, it attempts to read just one IF at a time and/or all baselines to one antenna at a time. In that case, when you switch IFs or main antennas, EDITR has to read in the next IF or set of baselines. EDITR will tell you as it starts whether "All data will reside in memory" or various other combinations. Judicious choices of SOLINT and TIMERANGE can help the task to read all selected data and thus to be more efficient There are 6 parts to EDITR's display: (1) A menu of operations displayed in two columns, one each at the left and right sides of the screen. When the menu is displayed, move the cursor to the desired item and press buttons A, B, or C. To get on-line help in the message window about a given option, move the cursor to the desired menu item and press button D. (2) A plot at the bottom of the screen in a bright color (usually) of the data which may be actively edited at present. It is of the chosen type for the selected baseline, polarization(s), and IF(s) and is limited in time range by the current frame (which can be all or a portion of the total time range). Flagged data are shown in a different color. The time range of the frame may be made small to expand the time axis, but may also be made so large as to cause serious crowding of the plotted data points. (Increase your XAS window width to increase the plot scale.) (3) A plot above the edit area plot in the same bright color of the data from the selected baseline, polarization(s), and IF(s). This plot is in a second observable (e.g. if amplitude is in the edit plot, phase is often the second observable). Data in this second area may not be used for editing but should be helpful in choosing which data to delete in the edit area. Flagged data are shown in a different color. This plot may be suppressed by adverb DOTWO. (4) 0-10 plots of data of the current data type, polarization(s), and IF(s) for a list of 0-10 other baselines to the main antenna are shown above the second plot. They are shown for comparison using another color, while flagged data are shown in a different color. (5) If a second uv data set was specified, its data will be shown in another color in each of the windows (2, 3, 4) above. (6) Several text areas also appear including (a) the start and end times for the current frame's time axis, (b) the selected data type, IF, and polarization at the bottom of the screen, (c) the antenna shown in each plot at the top right of each plot, (d) the current all-IF, all-polarization, and all-antenna flags, (e) the y-axis tick values on each of the frame plots, and (f) while interactively setting time and/or value ranges, the time and/or value to which you are currently pointing and the associated source name. You may change the size of the XAS window at any time. EDITR will not allow it to become too small, but will adjust its display for all reasonable sizes. The left-hand menu can contain ------------------- | FLAG TIME | To delete one time at a time. | FLAG TIME RANGE | To delete one or more time ranges. | FLAG BELOW | To delete all displayed times with data below a cutoff value. | FLAG ABOVE | To delete all displayed times with data above a cutoff value. | FLAG AREA | To delete one or more areas in the data-value - time plane. | FLAG POINT | To delete one sample at a time using both horizontal and vertical cursor position. | FLAG QUICKLY | To delete samples using only mouse clicks | ENTER AMPL RNG | To select the display range for amplitude plots: 0 -1 for 0 to maximum, 0 0 for min to max | ENTER PHASE RNG | To select the display range for phase plots. | ENTER DAMP RNG | To select the display range for plots of the amplitude of the visibility minus a running vector average visibility. | PLOT ERROR BARS | To plot error bars based on data weights | SET SCAN LENGTH | To set the averaging time used to determine the running average in seconds. | LIST FLAGS | To list all flags now in the Flag Command table. | UNDO FLAGS | To undo one of the flag operations in the FC table | REDO FLAGS | To reapply all remaining flags after one or more have been undone (see note below) | SET REASON | To set the 24-character "reason" string to be put in the uv-data flag table | USE EXPERT MODE | To control the task from the keyboard instead of the menu. | HOLD TV LOAD | To stop updating the TV display with every change of parameter; change several, then select | DO TV LOAD | To update the TV display now and with each change of display parameter. | REPLOT | To do the current plot over again, recomputing the differences from the running mean if appropriate. | EXIT | To exit EDITR, moving the FC table to a uv-data FG table | ABORT | To exit EDITR, deleting the FC table ------------------- The right-hand menu can contain ------------------- | NEXT POL/IF | To switch to viewing the next correlator, switching to the other polarization and, if needed, incrementing the IF. | SWITCH POLARIZ | To switch to viewing and editing the other polarization (can cycle through all). | SWITCH ALL POL | To switch functions from applying to one polarization to applying to both polarizations or vice versa. | ENTER IF | To select which IF is viewed and edited. This can force a read of data if all IFs did not fit in memory. 0 -> all. | SWITCH ALL IF | To switch functions from applying to one IF to applying to all IFs or vice versa. | SWITCH ALL TIME | To switch FLAG ABOVE and FLAG BELOW between all times and the time range of the frame | ROTATE ALL ANT | To rotate functions from applying to (a) one baseline, (b) all baselines to the main antenna, and (c) all baselines. | SWITCH ALL SOURC| To switch between flagging the current source and all sources. | ENTER ANTENNA | To select the main antenna, baselines to which are displayed on the screen. | ENTER OTHER ANT | To select up to 11 other antennas to define the baselines to be displayed; enter 10 numbers, 0's are then ignored (to plot 5 enter the 5 plus 6 0's). The first one is used for the edit area. | NEXT BASELINE | To rotate the list of other antennas, selecting the next one for the edit area. | NEXT ANTENNA | To select a new main antenna, one higher than the current main antenna. The "others" MAY also be adjusted. | PLOT ALL TIMES | To display all data for the selected baselines. | SELECT FRAME | To select a window into the current data interactively. | NEXT FRAME | To select the next time-range window of the same size as the current frame. | PREVIOUS FRAME | To select the previous time-range window of the same size as the current frame. | SHOW AMPLITUDE | To display and edit amplitudes. | SHOW PHASE | To display and edit phases. | SHOW DIFF AMPL | To display and edit the amplitudes of the vector difference between the sample and its running mean | SHOW ALSO AMPL | To display amplitudes of the edit baseline for reference with the phase or difference amplitude edit window. | SHOW ALSO PHASE | To display phases of the edit baseline for reference with the amplitude or difference amplitude edit window. | SHOW ALSO DAMP | To display difference amplitudes of the edit baseline for reference with the phase or amplitude edit window. | TV ZOOM | To alter the display zoom used while in the flag functions. | OFF ZOOM | To turn off any zooming. | 2ND UV OFF | To disable the display of the 2nd uv data set. | 2ND UV ON | To enable the display of the 2nd uv data set. ------------------- The menus will not show all of these options every time. The HOLD TV LOAD option is shown until invoked, and then is replaced with the DO TV LOAD. When that is invoked, the HOLD TV LOAD option reappears. The SWITCH POLARIZ and SWITCH ALL POL options appear only if there are two polarizations in the data. The ENTER IF option appears only if there is more than one IF in the data and the SWITCH ALL IF option appears only if there is more than one IF and all IFs fit in the program memory. Only one of the three SHOW ALSO choices appears at any one time. Plots of the difference between the visibility and its running mean can be particularly sensitive to short term disturbances while ignoring slow changes due to gradually changing source structure and the like. The running mean is not carried between sources and, as a result, is not normally carried across scan boundaries. Note that value-dependent flagging (FLAG BELOW, ABOVE, AREA) use the values currently plotted to make a list of value-independent flag commands (i.e., a single time for the specified antennas, IFs, polarizations, etc.). When a value-dependent FLAG operation is undone (UNDO FLAGS) or redone (REDO FLAGS), it is these value-independent flags which are undone or redone. You may have to undo more commands and then repeat FLAG commands to get the results you could have gotten by doing the now desired value-dependent command in the first place. You need also to be careful with the ROTATE ALL ANT setting with these value-dependent commands. If one baseline is set, then the commands only apply to the current baseline. If one antenna is set, the commands apply to all baselines to the current main antenna, while if all antennas is set, the commands apply to all baselines. The first two set a clip level, below or above which data are deleted, based on the value of the observable in each baseline independently. The FLAG AREA command, however, only looks at the values of the observable in the main edit baseline and flags those samples from all applicable baselines. Be careful when choosing EXIT versus ABORT. The former applies the flag commands to a flag table attached to the input uv data, the latter causes the flag commands to disappear without a trace. After EXIT, of course, one may use, edit, or ignore the output flag (FG) table. For single-source files, it may be necessary to run SPLIT to apply the FG table to the data since only some tasks know how to apply FG tables (those with FLAGVER as an adverb). The colors used by EDITR are those of the various graphics planes when it begins to run. You may change them with the AIPS verb GWRITE to more desirable colors. The planes are: Plane Default RGB Use 1 1.00 1.00 0.00 Main editing and secondary windows 2 0.06 1.00 0.00 Comparison baseline data windows 3 1.00 0.67 1.00 Menu highlight 4 0.00 1.00 1.00 Edit and frame window boundaries 5 1.00 0.18 0.18 Flagged data in all windows 6 0.60 0.60 1.00 Menu foreground 7 1.00 0.80 0.40 Second uv data set if present You may wish to change the colors to ones that you can see better. In "expert" mode, you are prompted with a set of 4-character codes. You type in the letter within that code that is shown in upper case (left-justified with a carriage return) to get the desired function. The expert functions allowed are Code Prompt Full menu OP name T Time FLAG TIME R Rang FLAG TIME RANGE B Belo FLAG BELOW A Abov FLAG ABOVE E arEa FLAG AREA O pOin FLAG POINT Q Quik FLAG QUICKLY L List LIST FLAGS U Undo UNDO FLAGS S baSl NEXT BASELINE C nCor NEXT POL/IF N Nant NEXT ANTENNA F Fram SELECT FRAME M aMpl SHOW AMPLITUDE P Phas SHOW PHASE D aDif SHOW DIFF AMPL X eXit EXIT EXPERT MODE H Help THIS HELP LIST although some may be suppressed depending on what is currently displayed. ------------------------------------------------------------------------