AIPS HELP file for ASTROMET in 31DEC24
As of Fri Apr 19 14:56:03 2024
HELP SECTION
GUIDE FOR AIPS REDUCTION OF ASTROMETRIC/GEODETIC VLBI DATA
by David Gordon (dgg@leo.gsfc.nasa.gov, 301-614-6107)
and Ed Fomalont (efomalon@nrao.edu)
Updated by Amy Mioduszewski (amiodusz@nrao.edu) on
March 9, 2004
The following guide gives a brief cookbook recipe for processing VLBA
correlated geodetic and/or astrometric VLBI data into Haystack-like
fringe files that can be made into Mark III databases. This guide is
suitable for users with only minimal experience with AIPS. Complete
AIPS beginners should at least read parts of the NRAO AIPS Cookbook.
The introductory chapters and Appendix C (Step-by-step Recipe for
VLBA Data Calibration in AIPS) and possibly the chapter 9 (Calibration
and Reducing VLBI Data).
Discussions of task and verb usages given here are not meant to be
complete. We suggest that users first study the explain files (EXPLAIN
FITLD, etc) for any tasks/verbs they are not familiar with. Also, this
guide assumes you are working with X/S multifrequency data, with
4 IF's in each band. For single frequency data, adjust IF numbers and
other instructions as needed. Table version numbers given are intended
only as a guide. You may need to adjust the SN, CL, and other table
input/output version numbers, as necessary, for steps you might skip,
or other calibrations you may add.
About Astrometry (tasks and verbs in AIPS associated with
astrometry/geodesy)
FITLD Task to load data from VLBA correlator into AIPS.
USUBA Task to assign subarray numbers.
INDXR Task to create NX and CL tables after USUBA.
LISTR Task to print contents of datasets and tables.
PRTAN Task to print contents of the ANtenna table.
PRTAB Task to list contents of the extension tables.
AVSPC Task to average UV data in the frequency domain.
ANTAB Read amplitude information in AIPS GC, TY tables.
ANCAL Determine SN table from GC and TY tables.
PCLOD Read phase tone information into AIPS PC table.
PCCOR Determine SN table from PC table.
CLCAL Interpolating SN table into revised CL table.
POSSM Task to plot total and cross-power spectra.
BPASS Task to create a BP (bandpass) table containing the bandpass
response functions of the antennas.
GETHEAD Verb to read image header values.
PUTHEAD Verb to modify image header values.
GETTHEAD Verb to read table header values.
PUTTHEAD Verb to modify table header values.
REFREQ Run file to move reference pixels to other parts of the IF bands
and to modify frequency offsets correctly.
SNCOR Task to zero out phase rates (or other quantities) in an SN table.
UVCOP Copies all/subsets of AIPS UV data files .
TBOUT Task to write an AIPS table into an external text file.
TBIN Task to create an AIPS table from an external text file.
UVFLG Flagging bad data from correlator or afterwards.
VLOG Formats VLBA Flagging, Pulse Cal, Tsys, and weather information.
mk4flags (Non-AIPS, available from GSFC) Fortran program to create a
flagging file for Mark III/IV antennas.
FRING Method for determining phase, delay rate and delay residuals.
KRING Method for determining phase, delay rate and delay residuals.
MBDLY Method for determining multi-band delay from FRING results.
CL2HF Converts AIPS (Earth-centered) astrometric qualities
into Haystack Format (HF table) telescope-reference
quantities. Results are put in AIPS HF table.
HF2SV Coverts data in HF table to a UNIX file which can be
exported into Calc/Solve.
HFPRT Prints out selected quantities in HF table.
Calc/SOLVE (Non-AIPS) Software supported by GSFC and USNO for analyzing
astrometric/geodetic data. It is supported on HP
workstations at GSFC, USNO, NRAO/Socorro, and elsewhere.
Procedures that might be useful (from the VLBAUTIL set of procedures).
VLBALOAD Loads, and merges the TY, GC and PC tables.
VLBAMCAL Merges the TY GC and PC tables, only needed if VLBALOAD was
not used.
VLBAFIX Splits data into different frequency files, sorts and indexes
the data and deals with subarrays.
VLBASUMM Gives a summary of the dataset.
for more info on these scripts see HELP VLBAUTIL
1. Reading in the VLBA Export Tape
The VLBA tape is read into AIPS with task 'FITLD', or possibly VLBALOAD,
which runs FITLD with simpler inputs and also merges the TY, GC and PC
tables for VLBA data that includes calibration transfer. It will typically
contain from a few hundred Mbytes to one or more Gbytes of data, so you
may need a lot of empty disk space. Typical data sets will contain 8 IF's
(each corresponding to a BBC), with IF's 1 to 4 at S-band and IF's 5 to 8
at X-band. Usually each IF will contain 16 spectral channels (pixels),
averaged down from 256 at the correlator. The integration time is usually
2 or 4 seconds. We have found 4 seconds to be adequate and results in half
as much data. [If you want to change the integration time, you need to
contact the VLBA before your data is correlated.]
The tape is mounted with the 'MOUNT' verb, and dismounted with the
'DISMOUNT' verb. Now load the data using VLBALOAD or FITLD. Most
adverb default values are okay. For VLBA LOAD you also need to
specify OUTNAME, OUTDISK, NFILES, NCOUNT, DOUVCOMP, and CLINT. Set
CLINT (CL table interval) to a value which is about 25 percent of the typical
scan length, or no longer than about 0.5 minute. If you are running
FITLD you should additionally set DOCONCAT and DIGICOR to 1. The
VLBA recommends flagging (not using) data with weights below ~70-75 percent,
so set WTTHRESH = .75 or so.
*** VLBALOAD Example ***
MOUNT
TASK 'VLBALOAD'
OUTNAME 'RDV11'; OUTDISK 2; INTAPE 1; CLINT 0.5
NCOUNT 12 (# of files on the tape - should be on a
listing sent with the tape.
INP
VLBALOAD (to run)
DISMOUNT (when finished)
*** FITLD Example ****
MOUNT
TASK 'FITLD'
INPUT (to review the inputs)
OUTNAME 'RDV11'; OUTCLASS 'FITLD'; OUTDISK 2
NCOUNT 12 (# of files on the tape - should be on a
listing sent with the tape.)
DOCONCAT 1; NFILES 0; DIGICOR 1; DATAIN ' '; OPTYPE ' '; CLINT 0.5
INTAPE 1; WTTHRESH .75; DELCORR 1
BCHAN 0; ECHAN 0; BIF 0; EIF 0
INP (to check inputs, make corrections)
GO
DISMOUNT (when finished)
As FITLD runs, you may see messages that a subarray condition was
encountered. If so, the CL and NX tables will be deleted and you will
later have to run tasks USUBA and INDXR. More on that later.
If there is a second tape, just change NCOUNT and repeat. The data should
all go into the same AIPS file.
**MERGING TY, GC and PC TABLES** If your data was correlated after 1 April
1999, then it contains calibration tables from the VLBA antennas. If you
loaded more than one file then the TY, GC and PC tables will have to be
merged. If you used VLBALOAD this has already been done, if not then
you have to run VLBAMCAL. See HELP VLBAMCAL.
After running FITLD or VLBALOAD, type 'IMHEAD' to see the header values in
the data base. It should contain several hundred thousand or more
visibilities, with 8 IF's and 16 frequencies. Sort order should be 'TB'
(time/baseline). If not, you will need to run task MSORT or UVSRT,
setting SORT 'TB'. If you run VLBAFIX (step 3.5) sorting will be done
automatically, if it is needed.
2. Incorrect AN/AT Table Frequencies
The AIPS files contain reference frequencies in the UV header and in the
AN and AT tables. For S/X data these frequencies may not agree due to the
reassignment of IF numbers in FITLD. The UV header frequency is
usually correctly set to the lowest S-band frequency. But the AN and AT
table frequencies will usually be set to the lowest X-band frequency.
The AN table frequency is used in some of the programs here so it will
need to be corrected. Although the AT table frequency does not get used
in the geodesy processing, it is probably a good idea to correct it also.
Do the following:
a) Get the AN table frequency:
INEXT 'AN'; INVERS 1; KEYWORD 'FREQ'; GETTHEAD
TYPE KEYVALUE ----> may give something like
8105990144 -144
(The two parts get added together.)
b) Get the UV header frequency:
KEYWORD 'CRVAL3'; GETHEAD
TYPE KEYVALUE ----> may give something like
2220989952 48
If the two agree, then you can skip to the next section. But if they
don't agree, then leave KEYVALUE unchanged from step b), and
c) Insert keyvalue from UV header into AN table:
INEXT 'AN'; INVERS 1; KEYWORD 'FREQ'; PUTTHEAD
Repeat step a) to verify. This could all be done later, but there may be
more than one AN table at that time. If so, then use the INVERS adverb to
correct all AN table versions.
For the AT table, repeat the above using INEXT 'AT' and KEYWORD 'REF_FREQ'.
Tables can also be checked using task 'PRTAB'.
3. Controlling the Effective Fringing Frequency
Before you begin fringing your data, you need to decide where in
the bandpass to do the fringing. The frequency within each IF where
the fringing is done (in task FRING) is controlled by the value of
a reference pixel number in the image header. (This controls the true
reference frequency of the residual phases, and has some effect on
delays and rates.) For recently correlated data, the reference pixel
number will initially be set at 0.53125, which is the lower frequency
edge of each IF. It is okay to leave this as is. However, many geodesy
and astrometry users prefer to fringe at the center of each IF and put
the reference frequency at the center of the first IF. You should no
longer try to reset pixel numbers or dataset frequencies manually
(except in the case of incorrect AN table frequencies mentioned
earlier). Instead, use the runfile 'REFREQ'. This will prevent
problems latter on if you didn't make all the necessary changes. To
move to the middle of the IF, with 16 channel data, use: INVER 1;
SUBARRAY 0; PIXVAL 8.0; RUN REFREQ.
REFREQ still does not change the AT table. For completeness you may want
to fix it up, although I don't think it gets used anywhere here. Do the
the procedure before running it.
following:
KEYW 'CRPIX3'; GETHEAD; TYPE KEYV (will say 8.5313 0)
INEXT 'AT'; KEYW 'REF_PIXL'; PUTTHEAD
3.5 VLBAFIX
VLBAFIX is designed to take a VLBI dataset and "fix" it. If necessary
it sorts, fixes the subarrays, splits into different frequency files
and indexes the data. So if you run VLBAFIX you can skip sections 4
and 9. All you have to do is set CLINT and SUBARRAY. For this case
you probably want to set SUBARRAY 2. DO NOT USE VLBAFIX EARLIER THAN
28 FEBRUARY 2004 UNLESS YOU HAVE NO SUBARRAYS.
4. Subarray Condition, USUBA and INDXR
Very likely FITLD will encounter a 'subarray condition', and will be
unable to create the NX and CL tables. It will tell you to run tasks
USUBA and INDXR. If not, you can skip to the next section. Run USUBA
with OPCODE 'AUTO'; SUBARRAY 0; INFILE ' '. There should then be
several AN tables, equal to the number of subarrays created.
Then run task INDXR with CPARM(3) = .5 or 1. (CL table interval, minutes),
CPARM(4) = 1 (calculate CL table geometric delay coefficients), and
CPARM(5) = 1 (calculate CL table clock and atmosphere delays and rates).
Also set CPARM(1) (maximum data gap) if you make repeat observations of
the same source less than 10 minutes apart; otherwise you may join two
or more scans that should remain separate. You might also need to set
CPARM(2). INDXR will create a CL and an NX table. [Many of the following
tasks cannot be run until you have an NX table. Some of them will
delete the NX table though, requiring you to run INDXR again.]
I've had minor problems with both of these tasks. USUBA occasionally
splits data for a single scan into two different subarrays. And INDXR
also occasionally splits a single scan into two parts. I've heard you can
hand-edit the NX table, but it looks pretty messy.
5. Creating the Flagging Table
Flagging tables are now being written at the correlator for VLBA
stations and a few others. If you use only these stations you can
skip to the next section.
Flagging files for the VLBA before 1 April 1999: There should be
flagging information available that will tell some of the AIPS programs
when to ignore data (antennas slewing, etc.). For the VLBA antennas,
the flagging times are written into the VLBA log file for your experiment.
These can be obtained from the aspen workstation at the AOC. (Contact
the AOC for the account and password.) It will have a name like
xxxxxcal.vlba, where xxxxx is the experiment designation. It will also
contain weather information, cable cal measurements, and pulse (phase)
cal measurements. Use the task 'VLOG', this will split all the
calibration files out in the correct format to be loaded by AIPS tasks.
Flagging files for Mark III/IV/V stations: If you have Mark III/IV/V
stations in your experiment, you should create flagging files for them
also. These can be constructed using the 'onsource/TRACKING' and
'data stop' lines in the Mark III/IV/V log files. You are welcome to use
the Fortran program 'mk4flags' (not part of AIPS), available via
anonymous FTP from gemini.gsfc.nasa.gov, directory
/pub/misc/dgg/AIPS/mk4flags. Make sure the two-letter codes match those
in your AIPS dataset. NRAO also has a script to do this, I think. Make
sure the lines end in a slash (/) if you use it. The VLBA flagging
files and Mark III/IV/V flagging files can be combined into a single
file, or you can run UVFLG twice.
To put this flagging information into the AIPS data base, run task
'UVFLG' and specify the INTEXT 'FITS:FFF', where FFF is the file name
of the flagging information in the $FITS directory. Set OPCODE 'FLAG';
FLAGVER 1; SUBARRAY 0. UVFLG will create an 'FG' table version 1.
UVFLG can also be used to flag individual IF's, if there is a bad one at
a particular antenna. This has to be done separately from the above step,
using INTEXT = ' ', and setting the ANTENNAS, BIF, and EIF adverbs. The
same FG version number can be used, i.e. FLAGVER 1. [If you flag an IF,
be aware that you will probably need to set APARM(4) = 3 (min. no. IFs)
when you run MBDLY.] You can also flag individual channels in an IF, if
necessary.
6. Scan and Antenna Summaries
You can run the task LISTR to get a time listing of the scans. Set
OPTYPE 'SCAN'. With DOCRT 132 you will get the information on the screen.
With DOCRT -1 and OUTPRINT ' ', it will come out on the printer, with
maybe a lot of paper wasted. With DOCRT -1 and OUTPRINT = 'FITS:NAME',
the output will be put in the $FITS area in the file NAME. If you send
it to a file, you can edit it before printing and maybe save a few
trees, as it seems to contain a lot of unneeded lines.
Task PRTAN (INVERS 1 and NCOUNT 0) will print out the antenna locations
and numbers, either to the screen, the printer, or to a file, using
DOCRT and OUTPRINT as above.
Or run VLBASUMM which runs LISTR and PRTAN as above with the minimum
of fuss.
7. CQ Table Fixup
Your dataset should have a CQ (correlator parameter frequency table)
which is used later in processing to correct for delay decorrelation
losses in the VLBA Correlator. After running USUBA and INDXR, it will
probably have only a single row for one of the subarrays in your data.
This used to cause errors later on, but I've been told it shouldn't any
longer. The following fix is left in, just in case it's needed. You
need a row for EACH subarray. One way is to fix it manually, easy but
tedious. First, run task 'TBOUT' with inext 'CQ'; INVER 1; OUTFILE
'FITS:CQ' (for example); DOCRT 132. Then edit the ASCII file, adding
records for each subarray, and modifying the 'Number of entries in
table'. This is fairly easy to figure out. Then delete CQ version 1
(inext 'CQ'; INVER 1; EXTDEST). Then run task 'TBIN' setting infile
'FITS:CQ' or whatever file name; also carefully set Outname, Outclass,
and OUTDISK.
8. Data Averaging - No Longer Recommended
You may want to average your data to reduce the data volume and
reduce the processing time. However, this is no longer recommended.
If you desperately need to do this use AVSPC to average spectral
channels and/or UVAVG to average in time. See the help files for
more details.
9. Splitting X and S Bands
If you ran VLBAFIX this has already been done.
Now you should split your dataset into separate S and X band datasets.
You will not be able to use measured phase cals otherwise. Run task
'UVCOP' twice, setting OUTNAME and OUTCLASS for each output file. For
the S-band, use BIF 1; EIF 4. For X-band, use BIF 5; EIF 8. For both,
use Flagver 1; UVCOPPRM 0. This will remove flagged data and the FG
table will disappear. The NX table will probably disappear also, so you
will need to rerun INDXR on both output files. Set CPARM(1) and CPARM(2)
as before. CPARM(3), CPARM(4), and CPARM(5) should be zero, since you
will still have CL tables.
AT THIS POINT ONWARD, ALL STEPS WILL HAVE TO BE RUN TWICE,
ONCE FOR X-BAND AND ONCE FOR S-BAND!!!
If you have only single channel data, skip or modify this step. You might
still want to copy all IF's to another dataset if there is a lot of
flagged data to be removed. The resulting smaller dataset may be easier
to work with. If not, some of the following tasks will require specifying
the flagging table version number, FLAGVER.
10. Phase Calibration
The VLBA correlator does not extract phase calibration phases, as does
the Haystack and other Mark III/IIIA correlators. Some type of phase
calibration must be done, in order to flatten out the bandpasses.
VLBA stations routinely extract phase cal phases in real time and
record them in the monitor data. For data after 1 April 1999 the pulse
cals for the VLBA are already attached to the data. For data before
this, you probably used VLOG to extract the flagging info (step 5) from
the calibration file. VLOG will have also extracted the pulse cal file
with the extension PCAL ready for loading with PCLOD.
It is recommended that pulse cals be used if they are available, as
opposed to using manual phase cals. [It looks like manual phase cals
add about 5-10 psec of noise, in an RSS manner.]
None of the non-VLBA stations extract phase cals yet, but this is
supposed to be implemented in a few years. (Some new hardware will be
required.) Until then, you can only use manual phase cals at these
stations. In the following discussion, we will assume that a mixture of
measured and manual phase cals is being used.
10a. Phase Calibration Using Measured Phase Cals
The data MUST be split into separate X-band and S-band datasets. In addition
to phase cal phases, phase cal single band delays and a phase cal group
(multi-band) delay will also be computed and applied in fringing, and the
CL tables have room for only one MBD value.
If your data is from after 1 April 1999, then the pulse cals are already
in the PC table and you can skip PCLOD and you can skip directly to PCCOR,
PROVIDING YOU HAVE RUN VLBAMCAL (see step 1). If you have data from before
this and have run VLOG then you should have a file with the extension .PCAL
that can be loaded with PCLOD. Use this file for the CALIN adverb of task
PCLOD. Run task PCLOD with OUTVER 1. You will have to run it once for each
subarray in your dataset, setting SUBARRAY each time, but keeping OUTVER 1.
This will create PC table #1, with all subarrays.
Next you want to use task PCCOR to create an SN table containing IF
phase offsets, IF single band delays (SBD's) and a multiband delay (MBD)
for each observation for each antenna. But there's a lot of preliminary
work before you can do this. First you need to select a calibrator to
resolve the MBD ambiguities, preferably a reasonably strong, unresolved
source. It also must be a scan in which all antennas participated (or at
least all VLBA antennas). [You should read ahead the section on running
FRING to select a calibrator for manual phase calibration. The same source
could be used for manual phase calibration of the non-VLBA antennas
later.] Once you have the calibrator, and a time range for its use
(~1 minute is good), then set up PCCOR. Set CALSOUR and TIMERANG for the
calibrator. Set REFANT to a VLBA antenna to use as a reference. (It must
be used later as the reference antenna for the manual phase cal step.)
Set SUBARRAY 0 to do all subarrays at one time. I'm not sure what the
CUTOFF adverb really does, but I have used CUTOFF 1 (turns off cable cal).
Now run PCCOR. It will create an SN table, usually version 1, with only
the VLBA antennas present.
Next you want to merge this SN table into the CL table, using task CLCAL.
This CL table must have all antennas present, not just the VLBA antennas.
Use OPCODE 'CALP' to do this. (If you have only VLBA antennas, use OPCODE
'CALI'.) Also INTERP '2PT'; ANTENN 0; TIMERANG 0. Also set REFANT, GAINVER,
GAINUSE, and SNVER to appropriate values. Run CLCAL once for each
subarray, resetting SUBARRAY each time. Then check the output CL table to
see that it has all antennas, and real phase cal values for the VLBA
antennas. You can use task 'PRTAB' with Inext 'CL'. Setting BOX to the
relevant columns will print a full row at a time (BOX 1 3 4 5 12 17 18
19 20, for example). MBDELAY1, REAL1, IMAG1, RATE1, and DELAY1 (columns
12, 17, 18, 19, and 20) should have real, non-zero values for the VLBA
antennas. You should not see 'INDE' anywhere!
If you have only VLBA antennas in your experiment, then you are finished
with phase calibration and you can skip the rest of this section. If not,
then read on.
10b. Manual Phase Calibration
With manual phase calibration, you run FRING on a short segment of good
data (a scan of a minute or so with all antennas and all IF's present)
to get phase offsets for each IF of each antenna, relative to a
reference antenna. This will define the multiband delay residual for
this particular observation to be zero. These phase offsets are then
applied to (subtracted from) all observations throughout the experiment.
First, set up the task 'FRING'. Set the TIMERANG appropriate for the
one scan selected, or even just one minute of that scan. Some adverb
values of consideration are: REFANT N where N is a good antenna near the
array center; DPARM 0 0 0 4 0 (4 seconds integration time); SNVER 0
(output SN table, probably #2 here); APARM 0; GAINUSE 0; SUBARRAY I
(where I is the appropriate subarray number). If you are going to
combine manual phase cals with measured phase cals from the previous
step, then REFANT should be the same VLBA antenna as was used in PCCOR.
ANTENN should be a list of the non-VLBA antennas plus the REFANT
antenna. Run FRING. It will create the next SN table. There should be
(#ant) x (#IF) good solutions. [The TIMERANG adverb may be confusing at
first. The days part is usually 0 or 1. To fringe from 21:16:10 to
21:18:00 on the first day of the experiment, you would use TIMER
0 21 16 10 0 21 18 0.]
You can examine these phases using TASK 'LISTR' with OPTYPE 'GAIN';
INEXT 'SN'; INVER 2; DPARM(1) 1; DPARM(8) 1; and specifying SUBARRAY.
You can only list one subarray at a time. Usually screen output is all
you will need (DOCRT 132, etc.). To see the SNR's, set DPARM 8 0. Make
sure you have a scan in which all antennas participated, and which gives
phases and SNR's for each IF. If not, delete that SN table (inext
'SN'; INVER 2; EXTDEST) and try another scan until you get a good one.
[If you have no scan in which all antennas participated, you will
have to 'manufacture' one by combining two or more scans, using
'TBOUT' and 'TBIN' and some hand editing, as in section 7.]
The results of this fringe fit (phase, delay, single-band delays,
and phases for each IF are stored in the SN table. We recommend that
you do NOT zero out the single band delays. These represent real
instrumental effects that should be calibrated. Next you will run
CLCAL to insert the manual phase cals into a CL table, but there's a
hitch here because CLCAL will not work across subarray boundaries. If
your dataset has subarrays, the same manual phase cals MUST be applied
to all subarrays, so you must duplicate your phase cal scan for all
subarrays. This can be done using TBOUT, hand editing, and TBIN. You can
either produce multiple SN tables, each for a different subarray; or one
larger SN table, with rows for all subarrays. (Running FRING multiple
times will work, then you edit out the minus signs in the row numbers
and edit the subarray columns.) There are also some copying and editing
tasks, but I'm not familiar with them. Either way is tedious but not
difficult. Contact me (DG) if you need some advice. [An alternative is
to set SUBARRAY -32000 and run CLCAL only once. I have had problems later
on in the processing with this option though, so I don't recommend it.]
When ready, run CLCAL once for each subarray. Assuming you want to
merge the manual phase cals in with the measured phase cals from the
previous step, you set OPCODE 'CALP'. If not, Opcode 'CALI' will work.
Also set ANTENN 0; INTERP 'BOX'; TIMER 0; SNVER N; GAINVER 2; GAINUSE 3;
where N is the input SN table version(s). Run CLCAL for each subarray,
changing SUBARRAY, and SNVER if necessary, each time.
You can do a quick check of the phases in the CL table using task 'LISTR'
with OPTYPE 'GAIN'; INEXT 'CL'; INVER 3; TIMER 0; DPARM 1 0; SUBARRAY 1
(or whatever is correct); GO. For the manual phase cal antennas, the
phases should be constant with time for each IF. For the measured phase
cal antennas, the phases should vary slowly with time. For a more
detailed look, use task 'PRTAB' with Inext 'CL'; BOX 1 3 4 5 12 17 18
19 20; etc. MBDELAY1 should be zero for manual phase cal antennas,
non-zero for measured. REAL1 and IMAG1 should have real numbers, RATE1
should be zeros for manual phase cals, non-zero constants for measured.
DELAY1 should have real values also. You should not see 'INDE' anywhere
(unless an IF is flagged as bad)!
11. Bandpass Calibration
Bandpass calibration may improve the quality of the FRING fits. To
examine the IF bandpasses, use task 'POSSM'. Set TIMERANG for a single
scan, it can be the same scan that was used for measured and/or
manual phase calibration; and set the appropriate SUBARRAY #. Also set
CALCODE ' '; DOCALIB 2; GAINUSE 3; DOBAND -1; NCOUNT 4; DOTV 1;
LTYPE 3; BCHAN 0; ECHAN 0; BIF 0; EIF 0; APARM 0; OUTTEXT ' '. You
will get 4 plots at a time. Amplitudes will probably drop off
dramatically at the edge channels, but that's usually normal. Phases
will typically show about 10-20 degrees of scatter. If the bandpass
phases are all linear, you do not need a bandpass calibration, but
this is unlikely. See 'EXPLAIN POSSM' for more details.
You create a bandpass (BP) calibration table with task 'BPASS'. Set
adverbs for the chosen calibrator, time range, subarray, and a
reference antenna. For example: CALSOUR '0552+398' ' '; SUBARRAY 1;
TIMERANG 1 12 57 10 1 12 58 20; DOCALIB 2; GAINUSE 3; REFANT 7;
BPVER 1; CHANSEL 1 16; ANTENN 0; BPASSPRM 0 0 1 0 1 0 0 1 0 1 1.
See 'EXPLAIN BPASS' for more details. BPASS will create BP table #1,
with rows for each antenna for one subarray only. Unfortunately you
will need a table with rows for all subarrays. You can run BPASS N
times, for N subarrays, and it will write identical blocks N times.
Then use TBOUT to write to an ASCII file. Edit it and change the subarray
column for the second, third, etc, sets of rows. Then delete the BP
table, and run TBIN to replace it with the edited rows. There might be
better ways of doing this using the table editing functions.
Some of the columns in the bandpass table (such as channel frequency
offsets) for the last station may look wrong, but they are really
okay. They are not calculated and not used for the 'remote' station,
and the last station is always the remote (as opposed to the reference)
station on all baselines.
Because of the frequency shifting when a bandpass calibration is used,
it is necessary to drop the first and last spectral channels (pixels)
when fringing. This is one reason we no longer recommend spectral
averaging of pixels.
12. Amplitude Calibrations
Amplitude calibration is usually not necessary for geodesy work. Some
astrometric users need to or prefer to calibrate amplitudes though. The
current version of CL2HF will not compute SNR's correctly in this case.
13. Polarizations
The S/X geodesy and astrometry runs will normally contain only one
polarization (RCP usually). CL2HF is currently unable to handle two
polarizations.
14. Further Processing
At this point you are ready to process the full data set using one of
several possible schemes, only two of which are described here. The
first uses 'Global', or 'Station' fringing. The second uses single
baseline fringing. Both fringe with task 'FRING'. [Supposedly task
'KRING' can now be used in place of 'FRING'. I have not had time yet
to check this out, but plan to soon.] In global fringing, all baselines
are fringed together, and phase closure is enforced, even if there is
source structure. This is somewhat unappealing for geodetic analysis.
The single baseline mode is closest to what the geodesy community has
gotten from the Mark III/IIIA correlators for many years, and what we
are most comfortable with. Therefore, this is what we are currently
using at GSFC. We also find that the data yield (number of good
observations) is greater with single baseline processing. The drawback
is that it takes MUCH longer and is more tedious. It is necessary to
run FRING once for each baseline (153 times for an 18 station exp),
and then again for the other frequency band. For Global fringing, do
steps 15 - 19. For single baseline fringing, study these steps, then
follow the directions in step 20. From here on, the processing can be
very time consuming, so regardless of how you process your data, you
should learn how to set up run files, procedure files, and batch jobs
so that you can process your data overnight, or over the weekend.
15. Global Fringing
Set up FRING for the whole data set. Suggested parameters are:
TIMERANG 0; DOCALIB 1; GAINUSE 3 (final phase cal table);
FLAGVER -1 (if no FG table); REFANT N (your pick); SUBARRAY 0;
APARM 0 0 0 0 0 0 3 0; DPARM 0; SNVER 3; ANTENNAS 0. If you are
doing bandpass calibration, set DOBAND 1; BPVER 1; BCHAN 2;
ECHAN 15. You may want to set some of the APARM or DPARM options
differently. Study the 'Explain FRING' file. SUBARRAY 0 will go
through all subarrays. Set the BADDISK array if you have disks with
little space or that are on other machines, otherwise it may take
forever. FRING may run for several hours for a 24-hour experiment.
It will do a "global" fringe fit in this example. FRING will determine
the phase rate, single-band delay, and phase offset at a central time
for each scan, for each IF, and for each antenna.
The output SN table will contain the solution values. These can be
looked at to see if there are lots of bad data. This is done by
running LISTR with OPTYPE 'GAIN'; INEXT 'SN'; INVER 2; TIMERANG 0;
DOCRT 132. Set DPARM N 0 to determine what kind of information to
display. [N=1 (phase), 6 (SBD), 7(phase rate), 8(SNR), 11(elevation).]
The SNR for the reference antenna is not meaningful; it is equal to
the specified minimum SNR + 1 for some unknown reason.
You might want to do some further editing using UVFLG. For example, if
a long stretch of data associated with an antenna is obviously bad, you
might as well delete it. Some of the tasks yet to be run look for
consistency amongst the IF solutions and will do some reasonable self
editing.
16. Multi-Band Delays
The multi-band delays (MBD's or group delays) are normally computed
using task MBDLY. MBDLY fits a linear phase versus frequency to the
IF phases for each antenna in the SN table output by FRINGE. The
output of MBDLY is placed in the next SN table and it will be nearly
identical to the previous SN table but with the multi-band delays
inserted. Set BIF 0; EIF 0; INVERS 3; OUTVERS 4 (adjust as appropriate.
APARM = 0 is reasonable. Some non-default values may be preferred
though. APARM(6) = 0 leaves phases in the calibrated UV data while
APARM(6) = 1 puts them in the SN table. CL2HF supports either mode,
but the APARM(6) = 1 mode has not been thoroughly checked out.
MBDLY runs fast, a few minutes or less even for large data sets. The
solutions will fly by too quick to read. They can be looked at later
using the verb PRTMSG with PRTASK 'MBDLY'. Each line corresponds to an
antenna solution with average rate and SBD over the IF's. The RMS phase
should not be more than a few degrees. The range is the MBD ambiguity.
The Res: line shows the departure of each IF from the linear fit.
17. Applying FRING Fits To the CL Table
The results of the FRING and MBDLY fits must be applied to the CL table
in order to calibrate the data. This is done using task CLCAL. INTERPOL
'SELF' must be used. This means that a solution for any scan in the SN
table found by FRING/MBDLY will only be used to correct that one scan,
i.e., there will be no interpolation between scans. Also use OPCODE ' ';
SNVER 4; GAINVER 3; GAINUSE 4. You must run CLCAL once for each subarray,
resetting the SUBARRAY adverb each time. The program runs fairly fast
though, taking only a few minutes or so even on huge databases.
18. Obtaining Haystack Reference Frame Results
Task CL2HF converts the Earth-centered rates, delays and phases to
the Haystack reference antenna system. The output is placed in an
AIPS extension 'HF' table. CL2HF has been modified extensively
at the Goddard Space Flight Center. [An alternate version, SN2HF, has
been developed at the Center For Astrophysics by Dan Lebach. The steps
for using SN2HF are somewhat different than those outlined here. Plans
are to consolidate these two version eventually.]
For CL2HF, APARM(1) controls how the UTC reference time tags will be
determined. To use the CL table times (rounded to the nearest integer
second) set APARM(1) = 0. APARM(1) = 1 will use the SN table time.
APARM(1) = 2 will use an input file of time tags, which must be
specified using the INFILE adverb. (This file should contain one line
for each scan, with the year (last 2 digits), month, day, hour, minute,
second, and source name of the desired time tags using a (6(I2,1X),A8)
format. The source name is optional.) APARM(1) = 3 will use the Mark
III schedule file, which must be specified using the INFILE adverb.
(The midpoint of the shortest scheduled duration will be used, almost
identical to the Haystack convention.) APARM(1) = 4 will use the file
output by LISTR from step #6 earlier - but it must be edited first.
Remove ALL lines which do not have a time range in them, but do not
edit the remaining lines. The program will take the midpoint of this
time range as the UTC reference time tag. Beware though with this
option, sometimes these time-ranges include dead-time when data is not
really being taken. If you want an option to read some other type of
file contact me (DG) and I'll add it as time permits.
APARM(2) sets a minimum elevation cutoff, in degrees. This was put
in because of problems with very low elevation observations, that we
didn't expect would be processed. Permitted values of APARM(2) are
between 0.0 and 10.0 (degrees). We recommend setting APARM(2) to 3.0
or 4.0 degrees. A value of 0.0 means no cutoff (negative elevations
allowed), as requested by some VSOP users.)
APARM(5) tells CL2HF where to get phases. If APARM(5) = 0, phases will
be from the calibrated UV data. If APARM(5) = 1, phases will be from
the SN table. This option must be coordinated with APARM(6) in MBDLY.
Zero is recommended for both at this time.
DPARM(1) and DPARM(2) is currently used when bandpass calibration was
applied in fringing, in order to calculate SNR's correctly. Set
DPARM(1) = 14 and DPARM(2) = 16 if the end channels (out of 16 channels)
were dropped. Set them to zero if bandpass calibration was not used and
no channels were dropped.
Currently the output reference frequency will be the frequency in
the first IF corresponding to the reference pixel number, initially
set by FITLD, and perhaps modified by RUN REFREQ. An option will be
added to move this around, say to other IF's.
CL2HF has been restructured in how it matches SN and CL records, and
consequently runs about 2 to 3 times faster than the previous version.
It should take about 1/3 as long to run as FRING. More recent mods
have been made for subarrays and to handle different CL table times
for the various stations. Previously all stations had the same CL
table times.
For CL2HF use: INVER 4 (input CL table from CLCAL); IN2VER 3 (phase cal
table); SNVER 4 (output by MBDLY); SUBARRAY N; OUTVERS N, where
N is the subarray number. You must now specify the subarray number and
run it once for each subarray. You should also specify APARM, DPARM and
INFILE. To compute reference time tags using a Mark III schedule and a
3 degree elevation cutoff, set APARM 3 3 0; INFILE 'FITS:X.SKD', where
X.SKD is the schedule file name. To use the CL table times, set APARM
0 3 0; INFILE ' '.
----------------------------------------------------------------
Ignore the following for now:
Options that will be available when CL2HF/SN2HF are consolidated:
APARM(3) - Determines whether residuals should be taken from
the SN table (version # specified by SNVER)
or from the CL table (version # specified by
INVERS). If residuals are taken from the CL
table, then residual MBDs and phases are
extrapolated to the SN table time (which is what
task CLCAL does).
<= 0 Get residuals from SN table.
> 0 Get residuals from CL table. This mode is
intended for use when the CL table used as input
when the SN table is created is something other
than the phase cal. CL table, e.g., when the SN
table is created with CALIB when using an input
CL table that is based on residuals from a
reference source.
APARM(5) - The value of APARM(5) (<= 0 or > 0) used in the
FRING solution.
This parameter identifies which mode of FRING was
used to create the SN table that is used as input
here or that was used by MBDLY to create the SN
table used here. It is needed so that IFs with
no or unused data are identified, and rate
estimates corrected, when FRING was run in a
mode to combine IFs (i.e., with APARM(5) > 0).
<= 0 FRING was run to make separate solutions
for each IF (and MBDLY probably then run
prior to input here).
> 0 FRING was run to combine IFs in solution.
DPARM(2) - The delay window parameter used in FRING.
DPARM(3) - The rate window parameter used in FRING.
DPARM(4) - The integration time (accumulation period, in
Haystack terminology) of the correlated data, in
seconds. Same as DPARM(4) in FRING.
<= 0 implies 1.0 seconds.
The correct value should be supplied.
DPARM(5) - The frequency adjustment, in MHz, to be added to
to the AN table frequency to obtain the HF table
reference frequency.
DPARM(7) - Determines whether to write results to separate
HF tables for each polarization.
<= 0 Write all results to a single table.
> 0 Write L polarization results to a separate
HF table (OUTVERS+1).
DPARM(8) - The SNR threshold below which a quality factor
code of "0" is assigned.
----------------------------------------------------------------
19. Putting HF Data Into Mark III Fringe Files For Dbedit
The HF files can be converted into the proper tree file structure
system that Dbedit (part of the GSFC/USNO Calc/Solve system) reads
using the AIPS task HF2SV. First, get into the $FITS (/data28/AIPS/FITS)
area (or any directory that has been defined by an environment variable)
and make two directories - SRESULTS and XRESULTS, for example, for the
S-band and X-band outputs. [It is okay to use the same directory for
both S and X files, but probably better if you keep them separate.]
Then, back in AIPS run HF2SV with INEXT 'HF'; INVER N; OUTFILE
'FITS:SRESULTS' or OUTFILE 'FITS:XRESULTS', GO. Run once for each HF
version number. This step should only take a few minutes of real time.
HF2SV may (if doing single baseline fringing) give you many error
messages like:
mkdir: cannot create /data28/RDV971.X/031-185835: File exists
These aren't really errors and you can just ignore them. HF2SV will
happily continue.
You are now ready for Dbedit/Calc/Dbcal/Solve.
20. Single Baseline Fringe Fitting
To do single baseline processing, you simply run through steps 15 - 19
separately for each baseline, using the ANTENNAS adverb in FRING to
select a single baseline each time. It will take much longer to
process an experiment, but the result will be a database in which all
the baselines are independent, and can be analyzed correctly using the
current Mark III software package (Calc/SOLVE, etc.).
There are so many steps repeated over and over that you simply have no
choice but to create a RUN file and submit it as a batch job. Below is
an example run file. It will process the 1-2 baseline, then 1-3, 1-4,
etc, through 18-19. You will have to tailor it somewhat for each
experiment and for your own needs. In this example, the SN, CL, and HF
files are deleted away each time to save space. Other ways are possible
though. Study the sections on RUN files, procedure files, and Batch
jobs in the AIPS Cookbook before calling Ed Fomalont or Craig Walker
at home for help. With typical astrometry/geodesy experiments, it takes
~20-30 minutes per baseline per frequency band on an HP780 (AIPS mark ~7).
************************** Example Runfile ******************************
*
* Run file for RDV16 X-band Baseline-by-baseline fringing, 19 stations.
* Measured/manual P-cals and Bandpass calibrations.
*
STORE 1
RESTORE 0
VERSION ' '
*
INDISK 5; OUTDISK 5; INSEQ 1; TIMERANG 0; INCLASS 'CH16'; OUTCLASS 'CH16'
USERID 5; BADDISK 1 2 3 4; FREQID 1
*
*---------------------------------------------------------------------------
PROC BB(I,J)
*
INNAME 'RDV16X'; OUTNAME 'RDV16X'
*
VERSION 'TST'; TASK 'FRING'; SUBARRAY 0; DOBAND 1; BPVER 2; BCHAN 2; ECHAN 15
DOCALIB 1; GAINUSE 4; FLAGVER -1; APARM 2 0 0 0 0 0 3 0
DPARM 1 0 0 0 0; ANTENNAS I J; REFANT I; SNVER 3; GO; WAIT
*
VERSION 'TST'; TASK 'MBDLY'; APARM 0 0 0 0 0 0; SUBARRAY 0
INVERS 3; OUTVERS 4; BIF 0; EIF 0; GO; WAIT
*
VERSION 'TST'; TASK 'CLCAL'; INTERPOL 'SELF'; OPCODE 'CALI'; GAINVER 4;
ANTENN 0; SNVER 4; GAINUSE 5; REFANT 0
SUBARRAY 1; GO; WAIT
SUBARRAY 2; GO; WAIT
SUBARRAY 3; GO; WAIT
SUBARRAY 4; GO; WAIT
SUBARRAY 5; GO; WAIT
SUBARRAY 6; GO; WAIT
SUBARRAY 7; GO; WAIT
*
VERSION 'TST'; TASK 'CL2HF'; IN2VER 4; APARM 3 3 0; FLAGVER 0
INFILE 'FITS:RDV16.SKD'; INVERS 5; SNVER 4; DPARM 14 16 0
SUBARR 1; OUTVERS 1; GO; WAIT
SUBARR 2; OUTVERS 2; GO; WAIT
SUBARR 3; OUTVERS 3; GO; WAIT
SUBARR 4; OUTVERS 4; GO; WAIT
SUBARR 5; OUTVERS 5; GO; WAIT
SUBARR 6; OUTVERS 6; GO; WAIT
SUBARR 7; OUTVERS 7; GO; WAIT
*
VERSION 'TST'; TASK 'HF2SV'; INEXT 'HF'; BCOUNT 0; ECOUNT 0
OUTFILE 'BOX2TMP:RDV16X'
INVERS 1; GO; WAIT
INVERS 2; GO; WAIT
INVERS 3; GO; WAIT
INVERS 4; GO; WAIT
INVERS 5; GO; WAIT
INVERS 6; GO; WAIT
INVERS 7; GO; WAIT
*
INEXT 'SN'; INVERS 4; EXTDEST; INVERS 3; EXTDEST
INEXT 'CL'; INVERS 5; EXTDEST
INEXT 'HF'; INVERS 1; EXTDEST; INVERS 2; EXTDEST; INVERS 3; EXTDEST
INVERS 4; EXTDEST; INVERS 5; EXTDEST; INVERS 6; EXTDEST
INVERS 7; EXTDEST
*
FINISH
*---------------------------------------------------------------------------
*
*---------------------------------------------------------------------------
PROC CC(I,J)
*
INNAME 'RDV16S'; OUTNAME 'RDV16S'
*
VERSION 'TST'; TASK 'FRING'; SUBARRAY 0; DOBAND 1; BPVER 2; BCHAN 2; ECHAN 15
DOCALIB 1; GAINUSE 4; FLAGVER -1; APARM 2 0 0 0 0 0 3 0
DPARM 1 0 0 0 0; ANTENNAS I J; REFANT I; SNVER 3; GO; WAIT
*
VERSION 'TST'; TASK 'MBDLY'; APARM 0 0 0 0 0 0; SUBARRAY 0
INVERS 3; OUTVERS 4; BIF 0; EIF 0; GO; WAIT
*
VERSION 'TST'; TASK 'CLCAL'; INTERPOL 'SELF'; OPCODE 'CALI'; GAINVER 4;
ANTENN 0; SNVER 4; GAINUSE 5; REFANT 0
SUBARRAY 1; GO; WAIT
SUBARRAY 2; GO; WAIT
SUBARRAY 3; GO; WAIT
SUBARRAY 4; GO; WAIT
SUBARRAY 5; GO; WAIT
SUBARRAY 6; GO; WAIT
SUBARRAY 7; GO; WAIT
*
VERSION 'TST'; TASK 'CL2HF'; IN2VER 4; FLAGVER 0; APARM 3 3 0
INFILE 'FITS:RDV16.SKD'; INVERS 5; SNVER 4; DPARM 14 16 0
SUBARR 1; OUTVERS 1; GO; WAIT
SUBARR 2; OUTVERS 2; GO; WAIT
SUBARR 3; OUTVERS 3; GO; WAIT
SUBARR 4; OUTVERS 4; GO; WAIT
SUBARR 5; OUTVERS 5; GO; WAIT
SUBARR 6; OUTVERS 6; GO; WAIT
SUBARR 7; OUTVERS 7; GO; WAIT
*
VERSION 'TST'; TASK 'HF2SV'; INEXT 'HF'; BCOUNT 0; ECOUNT 0
OUTFILE 'BOX2TMP:RDV16S'
INVERS 1; GO; WAIT
INVERS 2; GO; WAIT
INVERS 3; GO; WAIT
INVERS 4; GO; WAIT
INVERS 5; GO; WAIT
INVERS 6; GO; WAIT
INVERS 7; GO; WAIT
*
INEXT 'SN'; INVERS 4; EXTDEST; INVERS 3; EXTDEST
INEXT 'CL'; INVERS 5; EXTDEST
INEXT 'HF'; INVERS 1; EXTDEST; INVERS 2; EXTDEST; INVERS 3; EXTDEST
INVERS 4; EXTDEST; INVERS 5; EXTDEST; INVERS 6; EXTDEST
INVERS 7; EXTDEST
*
FINISH
*---------------------------------------------------------------------------
*
* Skip baselines 4-6 and 4-10 (no data)
*
FOR I = 1 TO 3; FOR J = I+1 TO 19; BB(I,J); CC(I,J); END; END
FOR I = 4 TO 4; FOR J = 5 TO 5; BB(I,J); CC(I,J); END; END
FOR I = 4 TO 4; FOR J = 7 TO 9; BB(I,J); CC(I,J); END; END
FOR I = 4 TO 4; FOR J = 11 TO 19; BB(I,J); CC(I,J); END; END
FOR I = 5 TO 18; FOR J = I+1 TO 19; BB(I,J); CC(I,J); END; END
*
*
RESTORE 1
****************************************************************************
Appendix A: Submitting a Batch Job
1) Put your RUN file in the $FITS directory with a name like
'BJOB.nnn', where nnn is your AIPS user number.
(BJOB.007 for user #7, etc.)
2) Edit the Batch queue: (AIPS Cookbook section 12.10)
Batque = 2; batclear; batch
VERSION = 'FITS'
RUN BJOB
endbatch
batlist (to list batch queue)
3) In interactive AIPS, type
SUBMIT
If there are no errors in your RUN file, the job will begin running in
the background. You can check its progress by periodically examining
the message file, using PRTMSG. After the batch job starts, you can
terminate your AIPS session or leave it running. You should not leave
AIPS suspended though (control Z) as this may also suspend the
background job at some installations.