; DBCON ;--------------------------------------------------------------- ;! concatenates two UV data sets ;# Task UV ;----------------------------------------------------------------------- ;; Copyright (C) 1995, 1998, 2000-2002, 2004, 2007 ;; Associated Universities, Inc. Washington DC, USA. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License as ;; published by the Free Software Foundation; either version 2 of ;; the License, or (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public ;; License along with this program; if not, write to the Free ;; Software Foundation, Inc., 675 Massachusetts Ave, Cambridge, ;; MA 02139, USA. ;; ;; Correspondence concerning AIPS should be addressed as follows: ;; Internet email: aipsmail@nrao.edu. ;; Postal address: AIPS Project Office ;; National Radio Astronomy Observatory ;; 520 Edgemont Road ;; Charlottesville, VA 22903-2475 USA ;----------------------------------------------------------------------- DBCON LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC DBCON Task which concatenates two uv data bases . INNAME Input UV file name (name) INCLASS Input UV file name (class) INSEQ 0.0 9999.0 Input UV file name (seq. #) INDISK 0.0 9.0 Input UV file disk unit # IN2NAME 2nd input file name. IN2CLASS 2nd input file class. IN2SEQ 0.0 9999.0 2nd input file seq. # IN2DISK 0.0 9.0 2nd input file disk number REWEIGHT 0.0 Weight factors. OUTNAME Output UV file name (name) OUTCLASS Output UV file name (class) OUTSEQ -1.0 9999.0 Output UV file name (seq. #) OUTDISK 0.0 9.0 Output UV file disk unit #. DOPOS If (1,1) true (+1) check pos. will shift second if nec. If (2,1) true check freq. DOARRAY If true (+1) output data will be the same subarray as input Forced to +1 for multi-source FQTOL -1.0 > 0 tolerance to not renumber FQs of dataset 2 (in kHz) -1 => no renumbering. ---------------------------------------------------------------- DBCON Type: Task Use: To concatenate two uv databases. After concatenation the two single-source input files are considered (optionally) to consist of data from separate arrays. The times are offset by the array number minus 1 times 5 days. Information peculiar to an array is stored in the AN file whose version number corresponds to the array number. Use PRTAN to access this information. For multi-source data files the FQ and source numbers of the second input file are converted to a system consistent with the first and the source tables are appended. Also, any CL, FG, TY, WX, IM, MC, PC, AT, CT, OB, or GC tables with version=1 will have their source numbers translated and appended to the end of the corresponding table (if any) from the first file. Any CL tables of version > 1 are copied from the first file; those belonging to the second file are ignored. The same happens to all versions (> 1) of BP and SN tables. Multi-source files are always considered to have the same array numbers in the two data sets. If the subarrays of one do not match the subarrays of the other, you should not be using DBCON on them. Concatenate single-source data after the separate calibrations and SPLITs have been done. The phases of the data are not changed for different source positions either. If the coordinates do not match, the second source name is altered. If the two input data bases are in the same sort order the output file will be in the same sort order. The input data sets may be compressed or normal. If one or both are compressed, the the output will be compressed. The reweighting, phase correction, etc may be applied to compressed as well as uncompressed uv data. If you are combining a number of data sets into one, it is recommended that you avoid repetitive addition of numbers to the time and subarray number. To do this, have the first file be the one with the larger number of subarrays each time. Adverbs: INNAME.....First input UV file name (name). Standard defaults INCLASS....First input UV file name (class). Standard defaults INSEQ......First input UV file name (seq. #). 0 => highest. INDISK.....Disk drive # of first input UV file. 0 => any. IN2NAME....Second input UV file name (name). Standard defaults IN2CLASS...Second input UV file name (class).Standard defaults IN2SEQ.....Second input UV file name (seq. #). 0 => highest. IN2DISK....Disk drive # of second input UV file. 0 => any. REWEIGHT...Weight scaling factors for the first and second uv data files. 0 => 1.0. See Explain DBCON for an explanation of how to determine these values. OUTNAME....Output UV file name (name). Standard defaults OUTCLASS...Output UV file name (class). Standard defaults OUTSEQ.....Output UV file name (seq. #). 0 = > highest unique OUTDISK....Disk drive # of output UV. 0 => highest with space DOPOS......If the first value (1,1) is true (>0) then the position of the second is shifted to agree with that of the first data set. This is always carried out using the correct frequency for each visibility. If the second value (2,1) is true then the frequency of the second data set must agree with that of the first. Phase shifting is not done for multi-source data sets. DOARRAY....If true (>0) the output vis. records will have the same subarray number as the input records. Thus the antenna numbers must be the same (DBCON does only minimal checks). The reference days will be adjusted. Also the antenna file(s) for the second input data base are not copied. If false (<0) then the time will have (subarray no. - 1) * 5 days added. DOARRAY is forced to TRUE for multi-source data sets. FQTOL......Tolerance for IF frequencies and total bandwidths in kHz to accept the FQ values as the same in the two data sets. SPECIAL VALUE: -1 => do not renumber any FQ entries When combining single-source, single FQ continuum data sets, this gets it to ignore minor differences, e.g. bandwidth. 0 -> 1 kHz. ---------------------------------------------------------------- DBCON: Task to concatenate two data bases. DOCUMENTOR: Jim Ulvestad NRAO/CV RELATED PROGRAMS: PRTAN, BTCOP PURPOSE DBCON is a relatively straightforward program whose sole purpose in life is to combine two data sets taken at different times, possibly using different arrays. The problem is that what seems straightforward for any 2 data sets is not simple in general. The idea is that these data sets should contain observations of roughly the same frequency. The data may be obtained in two different sessions, perhaps with the first data set consisting of old "resurrected" data to be added to newer data. Combination of data sets obtained using different arrays may provide increased uv coverage and a wider range of spacings (hence sensitivity to a wider range of size scales) than is found in a single set of data. If the two data bases have the same sort order then the two files will be merged so that the output file has the same sort order. COMMENTS IN2NAME: Like most adverbs in this program, IN2NAME is reasonbaly self-explanatory. However, this seems as good a place as any to mention that IN2NAME does not necessarily have to be data at exactly the same frequency as the data in INNAME. For example, in studying polarization and Faraday rotation, it may be desirable to spend some observing time at 18 cm and some at 20 cm. But the highest sensitivity total intensity map can be made by combining the two sets of observations into a single LBAND data base, if the user is willing to accept the uncertainties caused by spatial variations in spectral index within the radio source. If IN2NAME consists of data taken at a time much later or earlier than the data in INNAME, temporal variations in source properties can also screw things up. REWEIGHT: All uv data have associated weights which express the relative value of the data. Internally, these weights are treated as statistical weights (i.e. 1 / variance (amplitude)). These weights are used in making images but are modified by uniform weighting and tapering. In practice, the weights coming into AIPS from non-AIPS calibration systems are in arbitrary units so a renormalization of the weights may be necessary when data sets are combined. The adverb REWEIGHT can be used to modify the relative weights of the two input data sets. The simplest way to determine the appropriate relative weighting is to image each set of data independently in the same fashion as the desired final image (i.e. using the same IMSIZE, CELLSIZE, UVWTFN, UVBOX, and UVTAPER but not necessarily doing a deconvolution). All AIPS imaging tasks report the "sum of the gridding weights" in the message file. This is the sum of the weights after all corrections are applied (this value has been written into the history file since 15JUL89). REWEIGHT is a pair of values to be multiplied by the input weights for the first (1) and second (2) input uv data file. These values should be set such that the two data sets will have the desired relative sum of weights. For example, if the two data sets before DBCON give a sum of gridding weights of 1.0E5 and 2.0E2 when imaged by MX AND you wish to make them contribute equally to the final image then use: REWEIGHT = 1.0,1.0e5/2.0e2 (or 1, 500). OUTDISK: The specified OUTDISK must contain enough empty space to hold both of the input data sets (see below). DOPOS: This input provides for the spatial and frequency registering of the two data sets. If the user wants the second set shifted so that the phase centers coincide exactly then the first value DOPOS(1,1) should be set to +1. The default (0) or a negative value means no shifting is desired. Note: the phase center but not the tangent point is changed; if large shifts are desired then UVFIX should be used prior to DBCON to shift the tangent point. The value of DOPOS(2,1) is used to specify if the reference frequencies in the two headers should be checked, if there is more than one frequency channel present. A value of +1 means that the frequencies at pixel 1 should agree to better than 0.1% of the first frequency. This option should be used for combining spectral line data as this tolerance allows for changes in the doppler tracking. For combining continuum data, i.e., multiple-frequency data sets to be used for bandwidth synthesis a zero or negative value of DOPOS(2,1) will disable this check, which is unnecessary in this case. DOARRAY: This input allows the user to force DBCON not to mark the output data as subarrays unless the input data was already divided into subarrays. The user should be sure that the two input data bases do in fact have the same antenna numbers refer to the same antennas. The reference days will be adjusted in the output uv data file and in the first CL and FG table associated with each input file. If DOARRAY is false (<0) then the data to be combined are marked as separate subarrays. The most visible indicators of this are that the times of the data will have (subarray no. - 1) * 5 days added and there will be one antenna file per subarray. The purpose of this time offset is primarily to keep ASCAL and friends from getting confused. If DOARRAY=TRUE then this time offset is not added. If the antenna numbers in the data sets to be combined refer to different physical antennas or the data were obtained at different times then DBCON should be run with DOARRAY=FALSE. If the data were obtained with the same antennas at the same time then it MAY be desirable to use DOARRAY=TRUE. With DOARRAY=TRUE the task will try to consolidate the AN tables of the two input files for each AN version number. This will not be attempted if the two AN table headers disagree in significant ways (eg array centers do not agree, different polzn calibration). The output table will be the union of the two input tables, thus avoiding incomplete AN tables which sometimes result for VLBI data after using MK3IN or VLBIN. If the AN tables disagree the table from input file #1 will be written to the output file. If only file#2 has an AN table it will be written as a last resort. Multisource data files are always combined with DOARRAY true. MULTISOURCE FILES: If both input files are multisource files the output file will be also. Single and multi source files cannot be concatenated at the present. The source numbers of the second file will be translated to be consistent with the source numbers of the first table and the source table associated with the output table should be correct. The source numbers in any version=1 CL or FG table associated with the second input file will have a similar translation done and the contents of these tables appended to the end of a copy the corresponding table from the first input file. Higher version number tables from the either input file will not be copied. If necessary TABED can copy and modify higher numbered tables. If there is overlap between the CL tables from the first and second CL tables (usually the case for VLBI but not otherwise) then the resultant CL table need be compressed using TAMRG. After DBCON is run, a new index (NX) table needs to be constructed using INDXR. It is intended that multisource data bases be concatenated before a substantial amount of calibration or editing is done. INTERACTION WITH OTHER PROGRAMS: The AN file stores the properties (e.g., antenna position) peculiar to each array in the concatenated data base; this information can be accessed by using PRTAN. If ASCAL is given data combined by DBCON with DOARRAY=TRUE then all data within a given solution interval with the same antenna numbers specifying the baseline are averaged before the solution. If DBCON was run with DOARRAY=FALSE then ASCAL treats data from each subarray independently. When combining VLA data from the AC and BD IFs, DOARRAY will determine whether combined or separate gain solutions are determined for the two "IF pairs". EXECUTION TIMES: This program is unlike the DBCON program at the VLA in that it offsets the times by 5 day intervals rather than interleaving them. That process greatly reduces the CPU time involved. A typical execution time in an otherwise empty VAX 11-780 is about 5 min to concatenate two 50,000 visbility data sets. DISK SPACE: The amount of disk space used up by the final data set is approximately the sum of the disk spaces used by the initial data sets. A data set containing about 50,000 visibility records will occupy about 3,500 blocks of disk space.