; UVAVG ;--------------------------------------------------------------- ;! Average or merge a sorted (BT, TB) uv database ;# TASK UV UTILITY ;----------------------------------------------------------------------- ;; Copyright (C) 1995, 2006-2007 ;; Associated Universities, Inc. Washington DC, USA. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License as ;; published by the Free Software Foundation; either version 2 of ;; the License, or (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public ;; License along with this program; if not, write to the Free ;; Software Foundation, Inc., 675 Massachusetts Ave, Cambridge, ;; MA 02139, USA. ;; ;; Correspondence concerning AIPS should be addressed as follows: ;; Internet email: aipsmail@nrao.edu. ;; Postal address: AIPS Project Office ;; National Radio Astronomy Observatory ;; 520 Edgemont Road ;; Charlottesville, VA 22903-2475 USA ;----------------------------------------------------------------------- UVAVG LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC UVAVG Task to average or merge 'BT' or 'TB' sorted UV data. INNAME Input UV file name (name) INCLASS Input UV file name (class) INSEQ 0.0 9999.0 Input UV file name (seq. #) INDISK 0.0 9.0 Input UV file disk unit # SOURCES Source name QUAL -10.0 Calibrator qualifier -1=>all CALCODE Calibrator code ' '=>all TIMERANG Time range to use SELBAND Bandwidth to select (kHz) SELFREQ Frequency to select (MHz) FREQID Freq. ID to select. SUBARRAY 0.0 1000.0 Sub-array, 0=>all BIF Low IF number to do EIF Highest IF number to do BCHAN 0.0 First channel included ECHAN 0.0 last channel included DOCALIB -1.0 101.0 > 0 calibrate data & weights > 99 do NOT calibrate weights GAINUSE CL (or SN) table to apply DOPOL -1.0 10.0 If >0.5 correct polarization. BLVER BL table to apply. FLAGVER Flag table version DOBAND -1.0 10.0 If >0.5 apply bandpass cal. Method used depends on value of DOBAND (see HELP file). BPVER Bandpass table version SMOOTH Smoothing function. See HELP SMOOTH for details. DOACOR Include autocorrelations? OUTNAME Output UV file name (name) OUTCLASS Output UV file name (class) OUTSEQ -1.0 9999.0 Output UV file name (seq. #) OUTDISK 0.0 9.0 Output UV file disk unit #. XINC 0.0 Write every XINC'th record. YINC 0.0 Averaging time (sec). ZINC 0.0 Time (secs) to which data were pre-averaged. It must be close to the true one. 0 => ignore OPCODE 'MERG' for Merge function. Anything else for average. OPTYPE 'CROS' - merge xc data 'AUTO' - merge ac data Anything else => merge all. ac data requires DOACOR > 0 ---------------------------------------------------------------- UVAVG Task: This task will average or merge a 'BT' or 'TB' sorted uv data set in time. The input UV file can be either single- or multi-source. In the latter case, UVAVG averages each source separately. Adverbs: INNAME.....Input UV file name (name). Sort must be 'BT' or 'TB'. Standard defaults. INCLASS....Input UV file name (class). Standard defaults. INSEQ......Input UV file name (seq. #). 0 => highest. INDISK.....Disk drive # of input UV file. 0 => any. SOURCES....Source to be baselined. ' '=> all; if any starts with a '-' then all except ANY source named. QUAL.......Qualifier of source to be baselined. -1 => all. CALCODE....Calibrator code of sources to baseline. ' '=> all. TIMERANG...Time range of the data to be copied. In order: Start day, hour, min. sec, end day, hour, min. sec. Days relative to ref. date. SELBAND....Bandwidth of data to be selected. If more than one IF is present SELBAND is the width of the first IF required. Units = kHz. For data which contain multiple bandwidths/frequencies the task will insist that some form of selection be made by frequency or bandwidth. SELFREQ....Frequency of data to be selected. If more than one IF is present SELFREQ is the frequency of the first IF required. Units = MHz. FREQID.....Frequency identifier to select (you may determine which is applicable from the OPTYPE='SCAN' listing produced by LISTR). If either SELBAND or SELFREQ are set, their values override that of FREQID. However, setting SELBAND and SELFREQ may result in an ambiguity. In that case, the task will request that you use FREQID. SUBARRAY...Sub-array number to copy. 0=>all. BIF........First IF to include. 0 -> 1. EIF........Last IF to include. 0 -> max. BCHAN......First channel to copy. 0=>all. ECHAN......Highest channel to copy. 0=>all higher than BCHAN DOCALIB....If true (>0), calibrate the data using information in the specified Cal (CL) table for multi-source or SN table for single-source data. Also calibrate the weights unless DOCALIB > 99 (use this for old non-physical weights). If the data are not in time order, then DOCALIB must be false. GAINUSE....version number of the CL table to apply to multi-source files or the SN table for single source files. 0 => highest. DOPOL......If > 0.5 then correct data for instrumental polarization as represented in the AN table. This correction is only useful if PCAL has been run or feed polarization parameters have been otherwise obtained. If DOPOL > 0.5, then you can do only one sub-array at a time. If you have more than one sub-array, it would be best to calibrate and SPLIT each separately and then DBCON them back together before IMAGRing. See HELP DOPOL for available correction modes. BLVER......Version number of the baseline based calibration (BL) table to apply. <0 => apply no BL table, 0 => highest. If the data are not in time order, then BL tables may not be applied and BLVER should be set to -1 if any are present. FLAGVER....specifies the version of the flagging table to be applied. 0 => highest numbered table. <0 => no flagging to be applied. DOBAND.....If true (>0) then correct the data for the shape of the antenna bandpasses using the BP table specified by BPVER. The correction has five modes: (a) if DOBAND=1 all entries for an antenna in the table are averaged together before correcting the data. (b) if DOBAND=2 the entry nearest in time (including solution weights) is used to correct the data. (c) if DOBAND=3 the table entries are interpolated in time (using solution weights) and the data are then corrected. (d) if DOBAND=4 the entry nearest in time (ignoring solution weights) is used to correct the data. (e) if DOBAND=5 the table entries are interpolated in time (ignoring solution weights) and the data are then corrected. If the data are not in time order, then only DOBAND=1 may be used. BPVER......Specifies the version of the BP table to be applied 0 => highest numbered table. <0 => no bandpass correction to be applied. SMOOTH.....Specifies the type of spectral smoothing to be applied to a uv database . The default is not to apply any smoothing. The elements of SMOOTH are as follows: SMOOTH(1) = type of smoothing to apply: 0 => no smoothing To smooth before applying bandpass calibration 1 => Hanning, 2 => Gaussian, 3 => Boxcar, 4 => Sinc To smooth after applying bandpass calibration 5 => Hanning, 6 => Gaussian, 7 => Boxcar, 8 => Sinc SMOOTH(2) = the "diameter" of the function, i.e. width between first nulls of Hanning triangle and sinc function, FWHM of Gaussian, width of Boxcar. Defaults (if < 0.1) are 4, 2, 2 and 3 channels for SMOOTH(1) = 1 - 4 and 5 - 8, resp. SMOOTH(3) = the diameter over which the convolving function has value - in channels. Defaults: 1,3,1,4 times SMOOTH(2) used when input SMOOTH(3) < net SMOOTH(2). DOACOR.....> 0 => include autocorrelations as well as cross correlation data. OUTNAME....Output UV file name (name). Standard defaults. OUTCLASS...Output UV file name (class). Standard defaults. OUTSEQ.....Output UV file name (seq. #). Standard defaults. OUTDISK....Disk drive # of output UV file. 0 => highest with space for the file. XINC.......Write only every XINC'th output record. YINC.......Averaging time (sec). < 0.2 => 0.2 The intervals are handled differently for TB and BT data - see the EXPLAIN file for details. The latter attempts to make intervals at integer*YINC spacing through the day; the former averages data within YINC of the first sample in the interval. If you have data at 1.0 sec intervals and want 10.0 sec intervals out, do NOT set YINC to 10, expecially for TB-sorted data. Set it to e.g. 9.5 so that the first sample 10.0 sec from the first sample in the current interval will cause the start of a new interval. ZINC.......Time (sec) to which data have already been pre-averaged. UVAVG uses this to estimate the size of the output file. 0 => ignore. This must be the correct input average time for the merging function. OPCODE.....'MERG' selects the data with the highest weight within each YINC seconds. This is used primarily on VLB data where some baselines are repeated in processing and for putting together polarization or spectral line data sets. Any other OPCODE, simply averages the input data over YINC seconds. Note that the MERG operation is particularly sensitive to the choice of YINC and you MUST set YINC less than the interval between samples that you wish to retain. Thus if your data are at 10.0 sec intervals and you set YINC to 10.0, you will discard half your data and end up with a data set on 20 second intervals. With BT sorted data the affect is less predictable and you may get more or less what you want or you may not. If data to be merged are e.g. within 1 sec of each other whilst data to be retained are, say 2 sec apart, then set YINC to 1.1 or even 1.5 but NOT 2.0. OPTYPE.....'AUTO' - just merge autocorrelation spectra; 'CROS' - just merge cross-power spectra; ' ' - merge both. Adverb only used in MERG mode. NOTE: DOACOR must be true or auto-correlation data are not passed through the task at all. DOACOR is forced to be true when MERG and AUTO are specified. --------------------------------------------------------------- UVAVG: Task to average 'BT' or 'TB' sorted UV data DOCUMENTOR: R. C. Walker NRAO/CV RELATED PROGRAMS: UVSRT PURPOSE UVAVG averages or merges 'BT' or 'BT' sorted uv data bases. The average function is especially useful if a large data base can be reduced in size to make it more manageable. Care should be taken that the average time is not so long that 'time average smearing' occurs - ie. that the longer baselines sweep through enough uv space in an integration interval to smear significant structure. Averaging is often used after VBFIT on VLBI data. Longer averages are allowed at that point because relatively high residual fringe rates due to atmosphere etc have been removed. The merge function in UVAVG replaces VBMRG and works on 'TB' or 'BT' data with the same restrictions as for averaging. This function is used to eliminate duplicate data after multi-pass VLBI processing and to form full polarization or many channel spectral line data bases from separate channel inputs. The input data sets must have records long enough to hold the full number of channels - see DBCON if changes need to be made. UVAVG can be used to thin out a data set while leaving it properly configured for self cal. If XINC greater than 1 is specified, UVAVG will only write every XINC'th output. If the input data is in 'TB' sort order, all baselines for every XINC'th time interval are written so that complete data sets for each time remain and can be self-cal'ed. This function should be useful, for example, if a small data set sharing about the uv coverage of the main data set and not subject to time average smearing is desired for the first few iterations of hybrid mapping or self cal. The function works in either the merge or average modes. The time windows used for merging or averaging are of a length set by YINC and are timed such that one record begins at 0 hours UT (not IAT) when the data are in BT order. There should no longer be a limitation on the product of the number of baselines and the number of words per visibility record that can be averaged if the input sort order is 'TB'. Dynamic memory is used to an appropriate buffer is allocated. There must be adequate virtual memory in your machine to handle the problem. UVAVG marks the data with a time that is the weighted average time of the data when averaging and is the time of the record containing the last accepted correlator data when merging. For most purposes, this is proper, but it can cause trouble when VLBI data is transfered to the Caltech package. The times for data points corresponding to the same average interval are not the same so they are treated as separate points by VBCIT. This does not work well in the Caltech format. To avoid the problem, the data can be averaged in the Caltech package or UVAVG output can be run through the Caltech averaging program with the same average time that was used in UVAVG. The records should be properly combined in this way. COMMENTS XINC: This controls the output. It should be an integer and causes only every XINC'th record to be written. For 'TB' sorted data, all baselines for every XINC'th time interval are written so the data set may still be self-caled. YINC: This sets the averaging time in seconds. For averaging, specify the output average time. For merging, specify less than the input average time. If YINC is not correctly specified, the merging function may not behave correctly. If it is long enough to just encompass two samples you wish to retain, then one will be discarded and you will not get what you expect. The averaging intervals are managed in different ways for data in BT and TB order. For TB order, an interval starts with the first sample and includes all samples within YINC seconds. The next interval begins with the first sample beyond the end of the first interval and goes for another YINC seconds, and so forth. In BT order, however, we attempt to align the averaging intervals for all baselines by averaging in intervals of YINC seconds beginning at time 0 for the given day. This may cause there to be two output records for a particular baseline when you ask for an averaging interval somewhat greater than a scan length if the times span these integer times YINC intervals. When merging, YINC should be rather less than the averaging time for TB data and should be just a hair less than the averaging time for BT data. ZINC: ZINC defines the pre-average time interval in seconds, i.e. the time to which the data in the input file have already been averaged. If ZINC is set to 0 then the output file created will initially be as large as the input file. After UVAVG has finished the unused disc space allocated to the output file will then be released. If ZINC is nonzero then UVAVG will estimate the size of the output file needed (plus a little extra, just in case..) and will still shrink it after it has finished if necessary. The use of ZINC is recommended if particularly large files are being averaged. OPCODE: If OPCODE='MERG' is specified, UVAVG selects data from one input record for each YINC time interval. The data selected for different correlators (polarizations, spectral channels etc.) can come from different input records. Normally either the record with the highest weight, or, if the weights are equal, the last record in, are selected. If the weights are equal and the amplitudes differ by more than a factor of two, the record with the highest amplitude is selected. The u, v, w, time, etc in the output record are those of the last bit of data selected, regardless of polarization etc, for that output record. If OPCODE not equal to 'MERG', the data is averaged, weighted by the data weights. OPTYPE: When MERG mode is specified a problem may arise if the autocorrelation data have a different averaging time from the cross-correlation data (common in VLBI experiments). In this case the YINC will be different for the two sorts of data. When this occurs you must run UVAVG in MERG mode on the data twice, setting YINC to the appropriate value for the XC or AC data and then setting OPTYPE to either 'CROS' or 'AUTO'. In this case only the relevant data are merged, the other data type is passed through unscathed. SORT ORDER: The input data base must be in 'BT' or 'TB' sort order. See above discussion of data set size limitations for 'TB' sorted data. EXECUTION TIMES: The run time on an otherwise empty VAX 11-785 is 3.0 * (D / 100,000) cpu minutes where D is the number of input UV points. (This run time is for AVER and may be out of date - RCW)