; ACSCL ;--------------------------------------------------------------- ;! Corrects cross amplitudes using auto correlation measurements ;# Task UV VLBI CALIBRATION ;----------------------------------------------------------------------- ;; Copyright (C) 2014, 2016 ;; Associated Universities, Inc. Washington DC, USA. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License as ;; published by the Free Software Foundation; either version 2 of ;; the License, or (at your option) any later version. ;; ;; This program is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;; GNU General Public License for more details. ;; ;; You should have received a copy of the GNU General Public ;; License along with this program; if not, write to the Free ;; Software Foundation, Inc., 675 Massachusetts Ave, Cambridge, ;; MA 02139, USA. ;; ;; Correspondence concerning AIPS should be addressed as follows: ;; Internet email: aipsmail@nrao.edu. ;; Postal address: AIPS Project Office ;; National Radio Astronomy Observatory ;; 520 Edgemont Road ;; Charlottesville, VA 22903-2475 USA ;----------------------------------------------------------------------- ACSCL LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC ACSCL Correct cross correlations using auto correlations INNAME Input UV file name (name) INCLASS Input UV file name (class) INSEQ 0.0 9999.0 Input UV file name (seq. #) INDISK 0.0 9.0 Input UV file disk unit # FLAGVER -1.0 Flag table version to apply 0 -> highest TIMERANG Timerange: 1-4 = start day,hr,min,sec 5-8 = end day,hr,min,sec SOLINT Solution interval (min) > 0 => ignore scans < 0 => SOLINT=ABS(SOLINT) and average is terminated at the end of scans 0 => -2 min; see help SOURCES Source list QUAL -10.0 Source qualifier -1=>all CALCODE Calibrator code ' '=>all SELBAND Bandwidth to select (kHz) SELFREQ Frequency to select (MHz) FREQID Freq. ID to select. SUBARRAY 0.0 Subarray, 0 -> all DOCALIB 1.0 101.0 > 0 calibrate data & weights > 99 do NOT calibrate weights GAINUSE CL (or SN) table to apply DOPOL -1.0 10.0 If >0 correct polarization. PDVER PD table to apply (DOPOL>0) BLVER BL table to apply. DOBAND 1.0 10.0 If >0 apply bandpass cal. Method used depends on value of DOBAND (see HELP file). BPVER Bandpass table version SMOOTH Smoothing function. See HELP SMOOTH for details. ICHANSEL Array of start and stop chn numbers, plus a channel increment and IF to be used for channel selection in the averaging. See HELP ICHANSEL. Default = center 75% of band. ---------------------------------------------------------------- ACSCL Type: Task Use: ACCOR corrects amplitudes in cross-correlation spectra due to errors in sampler thresholds using measurements of auto-correlation spectra. Some correlators, e.g. JIVE, apply this correction before writing out the data. For them, ACCOR should not be needed. On the other hand, ACCOR should also be benign since the auto-correlations should end up scaled to 1.0 and so cause essentially no correction to be made by ACCOR. For the VLBA correlators - both the older hardware correlator and new software DiFX correlator - this correction is not done prior to reading the data into AIPS. For such data, ACCOR is required. The scaling of autocorrelation data may drift away from 1.0 later in the processing, for example, because of the choice of normalization in BPASS. ACSCL is designed to correct this drift by applying all calibrations and making sure that the autocorrelation data average 1.0. Numerous "calibration" adverbs are offered, although DOCALIB and DOBAND are forced to true. Otherwise, this task is ACCOR in disguise. Note that this task should be used before APCAL since that will make a CL table designed to scale the autocorrelations to the SEFD. Actually GAINUSE may point at a CL table prior to that affected by APCAL even when APCAL has already been used. The added scaling may then be applied to the highest CL table version with CLCAL. Adverbs: INNAME.....Input UV file name (name). Standard defaults. INCLASS....Input UV file name (class). Standard defaults. INSEQ......Input UV file name (seq. #). 0 => highest. INDISK.....Disk drive # of input UV file. 0 => any. FLAGVER....Specifies the version of the flagging table to be applied. 0 => highest numbered table. <0 => no flagging to be applied. TIMERANG...Time range of the data to be used. In order: Start day, hour, min. sec, End day, hour, min. sec. Days relative to reference date. TIMERANGE = 0 => All available data are used SOLINT.....The solution interval (minutes). The mean values of spectra are averaged through the SOLUTION intervals following the start time in TIMERANGE. If SOLINT is more than the range of times in TIMERANG, the time of averaging is equal to the range of times in TIMERANG. If SOLINT > 0, the averaging does not pay attention to scan boundaries and several scans inside SOLINT may be averaged together. The averaging in this case may include more than one source ID or/and frequency ID. The source ID and frequency ID written to the output SN table is the same for the all rows of the SN table and equal 0. Use CALSOUR = '' when running CLCAL to convert this SN table to the relevant CL table. If SOLINT < 0, ABS(SOLINT) is used as the average interval and averaging is terminating at the end of the scan. The correct source IDs and frequency IDs appear in the output SN table for the each averaged interval. CALSOUR can be used in CLCAL to select calibrators in this case. Because of the gaps between scans, small intervals can appear at scan ends. Intervals less than 0.1*MIN(SOLINT, scan length) are not recorded to the output SN table. 0 => -2 min SOURCES....Source list. The task loops over all sources specified. '*' = all; a "-" before a source name means all except ANY source named. QUAL.......Only sources with a source qualifier number in the SU table matching QUAL will be used if QUAL is not -1. CALCODE....Sources may be selected on the basis of the calibrator code given in the SU table. ' ' => any calibrator code selected '* ' => any non blank code (cal. only) '-CAL' => blank codes only (no calibrators) anything else = calibrator code to select. NB: The CALCODE test is applied in addition to the other tests, i.e. SOURCES and QUAL, in the selection of sources to process. SELBAND....Bandwidth of data to be selected. If more than one IF is present SELBAND is the width of the first IF required. Units = kHz. For data which contain multiple bandwidths/frequencies the task will insist that some form of selection be made by frequency or bandwidth. SELFREQ....Frequency of data to be selected. If more than one IF is present SELFREQ is the frequency of the first IF required. Units = MHz. FREQID.....Frequency identifier to select (you may determine which is applicable from the OPTYPE='SCAN' listing produced by LISTR). If either SELBAND or SELFREQ are set, their values override that of FREQID. However, setting SELBAND and SELFREQ may result in an ambiguity. In that case, the task will request that you use FREQID. SUBARRAY...Desired subarray - 0 => loop over all subarrays. DOCALIB....If true (>0), calibrate the data using information in the specified Cal (CL) table for multi-source or SN table for single-source data. Also calibrate the weights unless DOCALIB > 99 (use this for old non-physical weights). GAINUSE....version number of the CL table to apply to multi-source files or the SN table for single-source files. 0 => highest. DOPOL......If > 0 then correct data for instrumental polarization as represented in the AN or PD table. This correction is only useful if PCAL has been run or feed polarization parameters have been otherwise obtained. See HELP DOPOL for available correction modes: 1 is normal, 2 and 3 are for VLBI. 1-3 use a PD table if available; 6, 7, 8 are the same but use the AN (continuum solution) even if a PD table is present. PDVER......PD table to apply if PCAL was run with SPECTRAL true and 0 < DOPOL < 6. <= 0 => highest. BLVER......Version number of the baseline based calibration (BL) table to appply. <0 => apply no BL table, 0 => highest. DOBAND.....> 0 is not an error if there is no BP table, but this task is really designed to be used after there is a bandpas correction. If true (>0) then correct the data for the shape of the antenna bandpasses using the BP table specified by BPVER. The correction has five modes: (a) if DOBAND=1 all entries for an antenna in the table are averaged together before correcting the data. (b) if DOBAND=2 the entry nearest in time (including solution weights) is used to correct the data. (c) if DOBAND=3 the table entries are interpolated in time (using solution weights) and the data are then corrected. (d) if DOBAND=4 the entry nearest in time (ignoring solution weights) is used to correct the data. (e) if DOBAND=5 the table entries are interpolated in time (ignoring solution weights) and the data are then corrected. BPVER......Specifies the version of the BP table to be applied 0 => highest numbered table. <0 => no bandpass correction to be applied. SMOOTH.....Specifies the type of spectral smoothing to be applied to a uv database . The default is not to apply any smoothing. The elements of SMOOTH are as follows: SMOOTH(1) = type of smoothing to apply: 0 => no smoothing To smooth before applying bandpass calibration 1 => Hanning, 2 => Gaussian, 3 => Boxcar, 4 => Sinc To smooth after applying bandpass calibration 5 => Hanning, 6 => Gaussian, 7 => Boxcar, 8 => Sinc SMOOTH(2) = the "diameter" of the function, i.e. width between first nulls of Hanning triangle and sinc function, FWHM of Gaussian, width of Boxcar. Defaults (if < 0.1) are 4, 2, 2 and 3 channels for SMOOTH(1) = 1 - 4 and 5 - 8, resp. SMOOTH(3) = the diameter over which the convolving function has value - in channels. Defaults: 1,3,1,4 times SMOOTH(2) used when input SMOOTH(3) < net SMOOTH(2). ICHANSEL...Array of start and stop channels plus a channel increment and IF, used to select the channels to be averaged. For instance, if you wished to exclude channels 1 - 10 and 121 - 128 because of bandpass effects, and channels 56 - 80 of IF 1 but not IF 2 because of interference, then you would set ICHANSEL = 11,55,1,1, 81,121,1,1, 11,121,1,2. If you only wished to use every other channel from the second IF then you would set ICHANSEL = 11,55,1,1, 81,121,1,1, 11,121,2,2. Up to 20 groups of start, stop and increment channel numbers plus IF numbers can be specified. The default (ICHANSEL = 0) is to average the center 75% of the band, i.e. ICHANSEL(1,1) = (# channels)/8 + 1 For example: # channels=16 => ICHANSEL(1,1)=3 ICHANSEL(2,1) = (# channels + 1)*7/8 For example: # channels=16 => ICHANSEL(2,1)=14 ICHANSEL(3,1) = 1 ICHANSEL(4,1) = 0 (meaning all IFs). If ICHANSEL describes averaging explicitly for some IFs, but skips other IFs, then the center 75% of the band is averaged for the skipped IFs. For example: ICHANSEL=2,6,1,2 => The channels 2-6 will be averaged for IF=2 and the center 75% of the band will be averaged for the rest of the IFs. ---------------------------------------------------------------- ACSCL: Task to correct amplitudes distorted by errors of samplers. Documentor: L.R. Kogan RELATED PROGRAMS: FITLD, ACCOR PURPOSE Calibration of a cross correlation spectrum supposes knowing parameters of samplers (digitizers in particular). Unfortunately these parameters are not identical to optimal ones and their real values are not known with enough accuracy. As a result the cross correlation spectrum amplitudes are measured with errors. These errors can be estimated having measured the mean value of auto correlation spectrum of a given interferometer's antennas. The theory of this correction is given by L. Kogan in VLBA SCIENTIFIC MEMO #9, 1995. The task estimates the mean values of the auto correlation spectra of all antennas of a given VLBI network and records them in a new SN table. The following application of the correction is provided by standard procedure of applying the SN table to an CL table (CLCAL) and following application this table with DOCALIB = 1 in relevant tasks. Some correlators apply this correction to the data before writing them out - notably the EVN JIVE correlator. The VLBA hardware and software (DiFX) correlators do NOT apply this correction to the data. Therefore, ACSCL is required for the VLBA correlators and any others that do not apply the correction. ACSCL should be benign (do nothing) for those correlators that do apply the correction prior to reading the data into AIPS. Note that you can always run ACSCL and look at the SN table produced with PRTAB or SNPLT to see if it was benign or not. COMMENTS This method of correction is especially effective in the case of TWO bits (FOUR levels) digitizers. Uncorrected amplitudes of cross correlation spectra can differ from real ones in this case up to 25%. The correction can minimize the errors to a bit of percent. The behavior of the samplers' errors is unknown. In many cases it is stable for hours for a given sampler. But cases of minute changing can not be excluded. The default value of SOLINT is 60 minutes. But smaller value 5-10 minutes can be recommended as well. The correction quality can be checked by POSSM (for example) with DOCALIB=1 and GAINUSE equaled to the CL table number gotten from ACSCL's SN table. The mean values of all auto correlation spectra would be close to unit. The task corrects other small errors of spectrum amplitudes caused by other errors besides samplers errors. The task requires TB order of data. If it is not the task UVSRT has to be applied to the data early. COMMENTS FOR ACSCL BY CRAIG WALKER The data from the correlator are correlation coefficients - the ratio of the correlated power to the noise power. The apriori calibration process can be considered to have two major steps. The first is to adjust the data for all of the effects introduced by the various bits of hardware in the system, mainly the analog and digital filters. The second step is to multiply by the SEFD - the ratio of Ts/gain which converts power to flux density. That second step might also include some corrections for propagation effects like opacity. The autocorrelations are a measure of the total power in the system. For the second step to work properly, the net effect of the various corrections for instrumental effects should produce autocorrelations that are unity across the portion of the bandpass of interest. Then, after multiplying by the SEFD, the autocorrelation amplitudes will be the SEFD, which is the desired situation. The same scaling of cross correlations will produce the calibrated flux density. One of the first calibration steps for VLBI data is to run ACCOR. That has the effect of making the average of the autocorrelations unity, but it does not correct for spectral shape. With BPASS in its current form, one can get unity calibration results for the autocorrelations by normalizing by power over the full band using autocorrelation based bandpasses. Note that one might actually not want to normalize at all to force the results to unity, but it doesn't really matter because ACCOR has already forced the average to unity. The problem with this method is that the autocorrelations do not contain information about the phase bandpass, and one really wants to calibrate the phases along with the amplitude. If one uses strong-source cross correlation data to derive the bandpasses, complete with phases, and normalize on power across the full band, you get close. But there are aliasing effects in the edge channels that cause the bandpass shape in the autocorrelations to be somewhat different, in those channels, from the cross correlation bandpass. The autocorrelations are higher in the edge channels because the aliased signals don't correlate. This effect is stronger for the PFB, with soft edge filters, than the DDC or the legacy system with harder edged filters. The result is that, when the cross-correlation derived bandpass is applied to the autocorrelations, the values over the central channels that you want to keep are slightly suppressed. In my test data, the average value is 0.977 for the PFB and 0.993 for the DDC. One can use CLCOR with the power gain option (new) to correct that and get good calibration. But it would be nice to not have to do the CLCOR correction. In principle, one could run BPASS with autocorrelation based bandpasses, apply that, and make an incremental cross-correlation bandpass and it should work. But AIPS is not set up to do that conveniently. One would have to make a new file with the calibration applied after the first BPASS. What would be most convenient, I believe, is for BPASS to make a cross correlation based bandpass, apply it to a the autocorrelations, and determine the average result over the selected channels, probably not including the edge channels. Then the normalization would be to cause those autocorrelation channels to come out 1.00 on average (and individually to within the noise). This would introduce the complexity of having to read the crosscorrelations, determine the bandpass shapes, read the autocorrelations, apply the bandpass, determine the average in the selected channels, then apply to the bandpass the normalization required to make that average unity. The program would thus have to deal with both cross and autocorrelations, which it probably does not do now. But the end results, I think, is a proper calibration to do in preparation for the scaling by the SEFD to convert correlation coefficients to Jy.