; CLCAL ;--------------------------------------------------------------- ;! merges and smooths SN tables, applies them to CL tables ;# Task Calibration ;----------------------------------------------------------------------- ;; Copyright (C) 1995, 1997-1998, 2002-2006 ;; Associated Universities, Inc. Washington DC, USA. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License as ;; published by the Free Software Foundation; either version 2 of ;; the License, or (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public ;; License along with this program; if not, write to the Free ;; Software Foundation, Inc., 675 Massachusetts Ave, Cambridge, ;; MA 02139, USA. ;; ;; Correspondence concerning AIPS should be addressed as follows: ;; Internet email: aipsmail@nrao.edu. ;; Postal address: AIPS Project Office ;; National Radio Astronomy Observatory ;; 520 Edgemont Road ;; Charlottesville, VA 22903-2475 USA ;----------------------------------------------------------------------- CLCAL LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC CLCAL Task to manage SN and CL calibration tables INNAME Input UV file name (name) INCLASS Input UV file name (class) INSEQ 0.0 9999.0 Input UV file name (seq. #) INDISK 0.0 9.0 Input UV file disk unit # SOURCES Source list to calibrate SOUCODE Source "Cal codes" CALSOUR Cal sources for calibration QUAL -10.0 Source qualifier -1=>all CALCODE Calibrator code ' '=>all TIMERANG Time range to calibrate SUBARRAY -32000.0 1000.0 Subarray, 0=>all, ANTENNAS Antennas selected, 0=> all SELBAND Bandwidth to select (kHz) SELFREQ Frequency to select (MHz) FREQID Freq. ID to select. OPCODE Operation 'MERG','CALI', 'CALP'; ' ' => 'CALI' INTERPOL Interpolation function, choices are: '2PT','SIMP', 'AMBG','CUBE','SELF','POLY', 'SELN'; see HELP for details CUTOFF Interpolation limit in time (min); 0=> no limit. SAMPTYPE Smoothing function BPARM Smoothing parameters ICUT 0.0 Cutoff for functional forms DOBLANK Blanked value interpolation DOBTWEEN -1.0 1.0 > 0 -> smooth all sources together; else separate them SMOTYPE Data to smooth SNVER Input SN table, 0=>all. INVERS Upper SN table vers in a range. 0=>SNVER GAINVER Input Cal table 0=>high GAINUSE Output CAL table 0=>high+1 REFANT Reference antenna 0=>pick. BADDISK 0.0 9999.0 Disks to avoid for scratch ---------------------------------------------------------------- CLCAL Task: Applies solutions from a set of SN tables to selected entries in one CL table and writes them into another CL table. The input CL table should be the one applied to the data when determining the solutions and the output CL table is the one to apply when calibrating the data. Other operations on SN tables are also performed. For OPCODE='MERG', the SN tables will be merged and re-referenced to a common reference antenna. The result may then be smoothed. This OPCODE may be used with single-source as well as multi-source files. For OPCODE='CALI' selected SN tables are merged into a temporary SN table, smoothed if requested, applied to a specified CL table and written to the specified output CL table. This is for multi-source files only. OPCODE='CALP' is identical to 'CALI' except that uncalibrated data are passed through instead of being dropped. Considerable care should be exercised in using this task. If GAINUSE is not pre-existing, then the output file will contain data only for the specified range of adverbs (e.g. time-range, source list, etc). The data from the SN table(s) will be interpolated and extrapolated as needed to cover this range. They are then applied to the input CL values to produce the output CL values. If GAINUSE pre-exists, then the CL values for the range of adverbs is replaced by the data from the input CL tables times the data from the SN tables and the rest of the pre-existing CL table is unchanged. Note that setting GAINUSE=0, creates and fills a new CL table that then is "pre-existing" in the above sense. See the EXPLAIN CLCAL for additional information. Adverbs: INNAME.....Input UV file name (name). Standard defaults. INCLASS....Input UV file name (class). Standard defaults. INSEQ......Input UV file name (seq. #). 0 => highest. INDISK.....Disk drive # of input UV file. 0 => any. SOURCES....List of sources whose CL table entries are to be updated. '*' or blank = all; a "-" before a source name means all except ANY source named. SOUCODE....Sources may be selected on the basis of the calibrator code given in the SU table. ' ' => any calibrator code selected '* ' => any non blank code (cal. only) '-CAL' => blank codes only (no calibrators) anything else = calibrator code to select. NB: The SOUCODE test is applied in addition to the other tests, i.e. SOURCES and QUAL, in the selection of sources to process. CALSOUR....List of calibrators to use for updating the CL table. '*' or blank = all; a "-" before a calibrator name means all except ANY calibrator named. QUAL.......Only sources with a source qualifier number in the SU table matching QUAL will be used if QUAL is not -1. QUAL affects the selection of both the list of sources and the list of calibrators. CALCODE....Calibrators may be selected on the basis of the calibrator code given in the SU table. ' ' => any calibrator code selected '* ' => any non blank code (cal. only) '-CAL' => blank codes only (no calibrators) anything else = calibrator code to select. NB: The CALCODE test is applied in addition to the other tests, i.e. CALSOUR and QUAL, in the selection of sources to process. CALCODE affects only the selection of calibrators. TIMERANG...Time range of the data to be calibrated. In order: Start day, hour, min. sec, end day, hour, min. sec. Days relative to ref. date. SUBARRAY...Subarray number to calibrate. 0=>all. VLBI users may have a situation in which all AIPS subarrays actually contain the same antennas. CLCAL may be used to combines the separate SN gain tables, writing a single output table for "subarray 0". Later calibration routines will understand that this applies to all subarrays. Set SUBARRAY = -32000 to request this operation; it will be reset to 0 if the antennas files are not identical. ANTENNAS...A list of the antennas to be processed. If any number is negative then all antennas listed are NOT to be included. All 0 => use all. Note: when processing subsets of antennas separately it is very important always to use the same reference antenna (REFANT). SELBAND....Bandwidth of data to be selected. If more than one IF is present SELBAND is the width of the first IF required. Units = kHz. For data which contain multiple bandwidths/frequencies the task will insist that some form of selection be made by frequency or bandwidth. SELFREQ....Frequency of data to be selected. If more than one IF is present SELFREQ is the frequency of the first IF required. Units = MHz. FREQID.....Frequency identifier to select (you may determine which is applicable from the OPTYPE='SCAN' listing produced by LISTR). If either SELBAND or SELFREQ are set, their values override that of FREQID. However, setting SELBAND and SELFREQ may result in an ambiguity. In that case, the task will request that you use FREQID. NOTE: CLCAL should be run once per FREQID. OPCODE.....The desired operation: 'MERG' requests that all selected SN tables be merged and the phase-like parameters (phase, delay and rate) re-referenced to a common reference antenna. The merged SN table will be smoothed if SAMPTYPE is not blank. The output will be a new SN table one higher than the previous maximum. Note that this concatenates the SN tables, it DOES NOT multiply, divide, or replace entries. It is intended to be used for SN tables with solutions for different time ranges or sources rather than incremental calibration of the same source(s). If the tables have the significantly different solutions for the same (similar) time then you probably don't want to MERG them. 'CALI' requests that the merge operation be done if necessary and the solutions be (optionally) smoothed and the selected CL table entries be updated from the smoothed SN table. 'CALP' is identical to 'CALI' except that uncalibrated data is passed through instead of being discarded. INTERPOL...The type of interpolation to be applied to the SN table when computing the CL table entries (CALI, CALP only) All methods will extrapolate solutions if necessary. All of them limit the degree of extrapolation to no more than 1/2 of the time difference between the two calibrations being extrapolated after which the nearest calibration is used. WARNING: The CUBE method cannot be limited in this way and should not be used when extrapolation is required. '2PT ' = linear vector interpolation - the default 'SELF' = do 2PT but only using SN entries from the same source. (New mode starting 20 April 2004.) 'SELN' = Use only SN solution from same source which is closest in time. (This mode was called SELF before 20 April.) 'SIMP' = Simple linear phase connection between SN phase entries, assumes phase difference less than 180 degrees. 'AMBG' = Linear phase connection using rates to resolve phase ambiguities. 'CUBE' = As AMBG but fit third order polynomial to phases and rates. 'POLY' = Fit a polynomial to the SN rates and delays. Use the integral of the rate polynomial for the phases. (CUTOFF sets the degree.) CUTOFF.....Interpolation limit in time (minutes). This specifies the maximum time interval over which SN records will be interpolated to calibrate a given CL record. 0 -> infinite INTERPOL='POLY' does not use this limit. Instead CUTOFF sets the degree of the polynomial <= 10. SAMPTYPE...The type of smoothing: ' ' => none 'BOX ' = boxcar smoothing 'MWF ' = Median window filter 'GAUS' = Gaussian 'EXP ' = Exponential 'LINE' = Linear (1 - abs(t-t0)/sigma) '2PT ' = Two-point '2PTH' = Two-point + "Hanning" BPARM......Parameters for smoothing function. Function support full width in hours. (1) => support time for amplitudes, (2) => support time for phase, (3) => support time for rates (4) => support time for singleband delay (5) => support time for multiband delay Added parameter (FWHM) for GAUS, EXP, LINE in hours (6) => smoothing FWHM time for amplitudes, (7) => smoothing FWHM time for phase, (8) => smoothing FWHM time for rates (9) => smoothing FWHM time for singleband delay (10) => smoothing FWHM time for multiband delay In all cases, the substitution for blanked and good solutions is governed by DOBLANK (see below) ICUT.......Cutoff for GAUS, EXP, LINE. The sum of the weighting function in the support region must exceed ICUT for the smoothed value to be regarded as valid. Be careful, a value of 1.5 means that the sample itself must be good and the sum over other good samples in the support range must exceed 0.5. < 1.e-6 => 1.e-6. DOBLANK....Blanked value interpolation: > 0: replace previously blanked values with smoothed values, leave previously good values unchanged. = 0: replace previously blanked and previously good values with smoothed values. < 0: replace previously good values with smoothed values, leave previously blanked values blanked. Note that DOBLANK >= 0 requires a SMOTYPE that smooths both phase and amplitude, namely BOTH or FULL NOTE: >=0 Forces SMOTYPE = 'FULL' unless BOTH is specified. DOBTWEEN...> 0 => smooth all SN values regardless of source. <= 0 => smooth only SN values from the same source. Well-separated calibrators may have different phases just because of different atmosphere, so one would not want to smooth them together. This can be achieved via doing one source at a time, but DOBTWEEN allows one to do all sources at once - at least if the choice is a simple one. SMOTYPE....Specified the data to be smoothed. ' '=>'FULL' 'AMPL' = amplitude smoothing only, 'PHAS' = phase smoothing only, 'BOTH' = amplitude and phase 'VLBI' = delay and rate, 'FULL' = amplitude, phase, delay and rate. SMOTYPE = 'FULL' forced if DOBLANK >= 0 SNVER......First SN table to use. 0 -> 1. INVERS.....Last SN table to use. 0 -> SNVER EXCEPT if both are <= 0 then use all SN tables. This will concatenate the SN tables and then apply them. It DOES NOT multiply, divide, or replace the entries in the SN tables. It is intended to be used for SN tables with solutions for different time ranges or sources rather than incremental calibration of the same source(s). If the SN tables have different solutions for the same/similar source/time then you probably don't want to do this. GAINVER....Specifies the input version of the (CL) table to which the SN tables will be applied. 0 => highest CL table GAINUSE....Version number of the output CL table. Old entries in the CL table will be replaced. Note: modifying version 1 is disallowed. 0 => highest + 1 with the full GAINVER table copied to GAINUSE before the process begins. Note that GAINUSE may be set to highest + 1 instead and no advance copying is done. REFANT.....Reference antenna to use. All phase-like values in the combined SN table will be referenced to this antenna. 0=> use the one used in the most solutions. BADDISK....A list of disks on which scratch files are not to be placed. This will not affect the output file. ---------------------------------------------------------------- The process of CLCAL consists of 3 steps: (1) The selected SN tables are merged. Adverbs which control this step are SNVER - first SN to use INVERS - last SN to use REFANT - re-reference phases to this reference antenna Beginning on 19 May 2005 this merger process became more complete. Previously, "merged" tables were simply concatenated tables sorted into proper order. Thus, if one of the contributing SN tables had solutions for a subset of the IFs and polarizations and another SN table had the solutions for the other IFs and polarizations, the two sets of records would both appear in the merged SN table. Half of the records would have flagged solutions for some of the data and the other half would have flagged solutions for the rest. This mixture of failed solutions could be corrected by smoothing (with DOBLANK >= 0) but would otherwise complicate (disrupt) step 3 below. Now, if two records have the same time, antenna, etc., the solutions are truely merged with blanked solutions replaced by good values. (2) The merged SN table is smoothed in time. Adverbs that control this process are SAMPTYPE - type of smoothing function ' ' => none BPARM - parameters of smoothing function - support size (time over which function has value) and function width ICUT - function set to 0 when value < ICUT DOBLANK - which SN table entries are replaced by smoothed values - blanked ones, good ones, or both DOBTWEEN - does smoothing go over all sources or is each source smoothed individually SMOTYPE - which of amp, phase, etc are smoothed. (3) The merged and smoothed SN is applied to the output CL table, except for OPCODE 'MERG'. This is a confusing step: a) If GAINUSE=0, GAINVER is copied in its entirety to a new CL table. Otherwise GAINUSE version is used as is (including starting with an empty file). b) Data selection adverbs are used to delete the SELECTED data from GAINUSE leaving any unselected data. Adverbs are SOURCES, SOUCODE, SUBARRAY, ANTENNAS, SELBAND, SELFREQ, FREQID, TIMERANGE, QUAL C) Data from GAINVER are calibrated by interpolating selected (merged and smoothed) SN entries to the times in the CL GAINVER and then written to the GAINUSE. Relevant adverbs for interpolation are INTERPOL - function used to interpolate SN to CL CUTOFF - max time allowed between an SN entry and the CL entry to which it might apply And all of the above selection adverbs plus CALSOUR and CALCODE are also used. OPCODE 'CALP' allows data in GAINUSE which are selected but for which no SN table entry exists to be passed through unchanged. Otherwise they are omitted.