AIPS HELP file for VLBAPIPE in 31DEC13
As of Tue May 21 11:02:49 2013
VLBAPIPE: Applies amplitude and phase calibration to VLBA data !
To run this procedure you MUST type RUN VLBAUTIL and
RUN VLBAPIPE first to define the procedures in AIPS.
This is 'the' input dialog to the VLBA pipeline.
There are NO DEFAULTS! Note the special use of
almost all variables !
-> READ THE HELP/EXPLAIN FILE FOR 'CLEAN' STARTING
CONDITIONS AND SPECIFIC USE OF ALL THE VARIABLES.
When you run VLBAPIPE with DOALL > 0 the procedure
will ask about loading tapes. You will have to type
0 to confirm that the tape has been loaded
into the drive. When it is done with loading data
it will run to completion on its own.
- WHEN IT GOES, IT GOES !!! -
DOALL Tape unit #, < 1 to skip load
-> READ THE HELP/EXPLAIN FILE
NCOUNT Number of distribution tapes.
APARM Number of files on each tape.
DOUVCOMP Compress data? Use -1, 0 or 1
OUTNAME VLBA program name (eg BZ099A)
Restarts use this as INNAME!
First 6=outclass cal'd data.
OUTDISK Working disk with ample space
OPTYPE For CONT, PSEU or SPEC (LINE)
CLINT CL table interval in minutes.
REFANT Reference antenna #, or use 0
SORT and eg. SORT 'PT' if unknown
Specify either REFANT or SORT
INFILE 1st IONEX file name for TECOR
NFILES Number of TECOR files to use;
NFILES < 1: don't run TECOR.
TIMERANG A good band-pass scan, or use
scan# in TIMERANG(1)+rest=0.
CALSOUR List ALL Fring fit & CAL sour
FIRST is bandpass calibrator
in TIMERANG specified above.
If CALSOUR(2)='*', then will
take every source in SUtable
Put all strong sources here.
SOURCES Source * PAIRS * to calibrate
START each pair with Ph-Ref.
2nd='*': all non-calibrators
are phase-referenced to 1st.
SOURCES ONLY USED FOR PH-REFS
SOLINT Time interval for fringe-fit,
BPARM or : SOLINT per frequency-ID.
Use either SOLINT /or/ BPARM
INTERPOL Interpolation: '2PT', 'SIMP',
'AMBG', or 'CUBE' (in CLCAL)
IMSIZE Image size for 'even' sources
FACTOR others are FACTORx128 square
IMSIZE ONLY USED FOR SOURCES
IMAGING CALSOUR USES FACTOR
DOTV Plot type: >0 then plot on TV
=0 make PL file, <0 make PL
files and send to printer.
Recommend =0, discourage =-1
BADDISK Disks to avoid for scratch
VLBAPIPE is defined in the standard AIPS runfile area.
Use: VLBAPIPE is the procedure that applies the VLBA calibration procedures
(VLBAUTIL) for a-priori amplitude and phase corrections, and also
performs fringe fitting on single strong sources and phase-references
weak sources. Simple visibility flagging, bandpass calibration, and
ionospheric (total electron content) corrections are applied before
fringe fitting and averaging. It may attempt to self-cal the
calibrators and will image the targets.
READ THE HELP/EXPLAIN FILE THOROUGHLY BEFORE USING SINCE NONE OF THE
AIPS INPUTS ARE USED IN THE USUAL WAY.
NOTE: currently this procedure is intended for simple VLBA-only experiments!
More info on http://www.aoc.nrao.edu/vlba/html/vlbahome/observer.html
under 'Calibration and Imaging' of NRAO's VLBA web page.
Type RUN VLBAUTIL and RUN VLBAPIPE to define the VLBAPIPE procedures;
it is wise to type RESTORE 0 (before the 'run' commands above) or
DEFAULT (after the 'run' commands, and setting TASK 'VLBAPIPE') to
set the default task values. Default values should be MEANINGLESS here.
Note that a 'RESTORE 0' after RUN VLBAPIPE deletes all the procedures!
The procedure is run by typing : (GO) VLBAPIPE
The procedure will check the inputs (as far as it can) to avoid later
interruptions, and then proceed directly, if the data is already on
disk, or after the last tape has been loaded if the data is not on disk.
Some manual operation (loading tapes and typing a zero) is needed.
CLEAN STARTING CONDITIONS
I prefer to use one project per user number, as otherwise you may get
into troubles if different projects use the same source names. And I
use a different user number than for my other non-VLBA projects. This
means I can type RESTORE 0 without it affecting my tget's and catalogs
that I use for my other non-VLBA projects. But this is up to you.
It is not recommended to run two pipelines with the same user number
simultaneously, nor overlapping at any time during the processes.
Clean up all files resulting from previous attempts of running the
procedure, except for the original UVDATA and perhaps, when restarting
a specific frequency-ID, also not the F(X)POL files. When restarting
from disk, make sure the UV data has no extra SN, CL, FG etc tables.
Use the procedure P_RESTART (which is defined in VLBAPIPE) to do this.
When you start the procedure on a new data set which includes sources
that have the same name in a previous run, then RENAME the images from
the prevoius run. IMAGR otherwise attempts to overwrite the earlier
images with the same source name and dies. If you wish to flag some
additional data (because the procedure did not flag all the bad data
automatically), make sure you put the flags in FG table number 1.
The procedure uses the CALIBRATION TRANSFER FILEs that come with the
data on the VLBA data distribution tape. Occasionally a VLA antenna,
and/or the Effelsberg or Green Bank telescopes also participated. For
now that means that if you want these antennas also to contribute to
the calibrated data set, you would want to load all the data with
VLBALOAD, and include the external calibration of the non-VLBA antennas
in the old-fashioned way with ANTAB. Write the external calibration
system temperatures and gain curves in TYVER=1 and GCVER=1 (and BLVER=1).
It is a VERY good idea to run TASAV before you do this to keep the
original TY, GC, BL and also FG tables number 1 in case of disasters.
After you loaded the data with VLBALOAD and put the external calibration
and flagging information for the non-VLBA antennas in the appropriate
tables number one, start the VLBAPIPEline with DOALL=0 and OUTNAME and
OUTDISK pointing to the file VLBALOAD created (thus not INNAME, INDISK).
Make sure there is enough disk space available on OUTDISK, about 4 to
5 times the expected compressed UVDATA set. For spectral line probably
you need much more because it will make spectral line image cubes.
DOALL is supposed to set the starting point for the procedure. A positive
number is used to start with loading the data from tape, and using
INTAPE=DOALL for the tape drive. A DOALL=0 means, the data is loaded
to disk (and has name 'OUTNAME'.UVDATA.1 on 'OUTDISK' -see below), but
nothing else has been done to it. DOALL=-1, -2, -3, etc is used to
restart at frequency-ID 1, 2, 3, etc. and continue to higher numbers,
if they exist. The files 'OUTNAME'.FQ-(i).1 or 'OUTNAME'.FPOL(i).1
(for a single freq. UVDATA or FXPOL) should exist! Restarting at a
given point may be fragile and may break the procedure.
* testing: If < 1000, then ONLY do Freq-ID = (-1*DOALL)-1000; ie. -1004->4
NCOUNT is the number of distribution tapes to be loaded. The number of
files per tape is specified in APARM.
APARM is the number of files to be loaded from each distribution tape
- files at the beginning of a tape cannot be skipped. For more
control use VLBALOAD. Make sure that APARM lists the number of
files per tape in the right order ;-), that the unused APARMs are
set to zero, and the tapes are loaded into the tape drive in the
DOUVCOMP determines whether all UV data will be compressed (DOUVCOMP=1),
not compressed at all (DOUVCOMP=-1), or whether you want the UV data
loaded in uncompressed multi-source form with FITLD, and compressed
in the calibrated single source files after SPLIT (using DOUVCOMP=0).
OUTNAME is used to create files (e.g. from tape), AND to search for as an
INNAME for files that are already on disk (which will be OUTDISK).
OUTNAME='VLBA-experiment name' (eg BZ99Z) is a good unique name to
use. All final calibrated single source UV files will use the
first 6 characters of OUTNAME as their OUTCLASS, so for multiple
passes make OUTNAME unique in the first six characters.
OUTDISK is the disk that is used for ALL files produced by the procedure.
It may use other disks for scratch, which can be regulated with the
BADDISK parameter below. OUTDISK is the working disk; it needs to
be spacey - clean up if needed (before you start vlbapipe).
OPTYPE should reflect the type of observation and requested output files:
'CONT' for plain continuum observations using the PC (pulse-cal)
table calibration; all data is averaged over IFs and channels
and continuum images are made from the averaged data.
'PSEU' for pseudo-continuum observations using the PC table calibration
the data is NOT averaged over IFs nor over channels, but the
resulting images are continuum images - used for large FOVs
'SPEC' for spectral line observations - there should not be a PC-table!
Manual pulse-cal is performed on the bandpass calibrator in the
timerange given (see below). Data is unaveraged and images are
spectral line cubes of the phase referenced target sources ONLY.
(see below in CALSOUR for strong spectral line sources that can
be fringed - essentially you have to make your own cube later)
Calibrator sources will be made continuum images even in 'SPEC'!
'LINE' is the same as 'SPEC'.
NOTE that the imaging is only done with the intention to check whether
the calibration has worked and whether maybe some more data needs
to be flagged (in FG#1). Always re-image your sources for your
scientific use (imaging, analysis) - these won't be good enough!
CLINT is the CL table interval (minutes) used throughout the whole experiment
For VLBA data you would generally choose a CLINT between 0.1-2 minutes.
REFANT or SORT is used to select the reference antenna. Because the antenna
number for REFANT is usually unknown before loading, SORT can be used
to supply the two letter code; e.g. SORT='PT' for Pie Town. Use
either REFANT or SORT, not both! Set REFANT to zero if you are using
SORT; blank SORT if using REFANT.
SORT Use either REFANT or SORT, not both; see REFANT above.
INFILE The name of the file containing a model for the total electron content
(TEC) in the ionosphere which is used by TECOR to correct delays caused
by the ionosphere. It is recommended to run TECOR for ALL frequencies
up to 15 GHz. The filename should be given in the usual AIPS
style as a logical directory name followed by a colon and the name of
the file. The entire name will be converted to upper-case. The file
should contain TEC maps spanning the time range covered by the input CL
table and should be in IONEX format. If NFILES > 1 then the name MUST
be in standard format CCCCdddC.yyC where C can be any character, ddd
is the day number, and yy is the year. Also the name given in INFILE
must be the first day.
The IONEX/TECOR files can be downloaded through an anonymous ftp to
cddisa.gsfc.nasa.gov and following the directory structure:
/gps/products/ionex// (take doy from your *.sum).
Also note that if your observation starts before 01:00 UT, then you
need the IONEX/TECOR for the preceding day too, and point INFILE to
this first file. For observations ending after 23:00 UT, you need the
next day's file as well, and in both cases add one to NFILES. TECOR is
not run and INFILE is ignored for NFILES < 1, e.g. for 43 GHz data.
For more details, see EXPLAIN TECOR.
NFILES is the number of IONEX files to use, starting at INFILE. Note that
if NFILES > 1 then the INFILE must be in a standard format and the
file in INFILE must be the first day of those to be loaded.
TIMERANG is a time range on which ALL antennas have good data on the bandpass
calibrator (see below, CALSOUR(1)). This interval is used for pulse-cal,
either with the PC table, or manually if there is none (as in spectral
line data). For multiple frequency data it is VERY nice to have
TIMERANGE to span the whole time range in which you flip through all
the observed frequencies - otherwise the procedure will choke.
Alternatively, if you know the scan number, you can use the scan number
in TIMERANGE=66,0 to get the time range associated with scan 66.
CALSOUR is the parameter to name all the sources you want to be fringe fitted
e.g. your calibrator sources, strong (target), or survey sources. The
first source MUST BE named, and will be used for pulse-cal and bandpass
calibration. Use a simple source with good signal to noise and with all
antennas present (note: phase-reference calibrators may be good too)
in the TIMERANGE specified above. If you want all sources fringe fitted
then you may use CALSOUR(2)='*' but remember to also specify CALSOUR(1)
to get the pulse-cal and bandpass calibration. If you want to attempt a
self-cal for stronger sources, CALSOUR(2)='*' is not allowed, so you
have to specify your calibrator sources (up to 30). The source names
that start with an additional minus will be self-cal'd, but be warned
that this is a frequent point of failure which may break the procedure.
The sources that are fringe fitted will be imaged in (pseudo) continuum
mode, so if your spectral line target is strong (e.g. when looking for
absorption) you will only get a continuum image (not UV data set) which
is used to asses the calibration - Re-image for the spectral line cube.
SOURCES holds the phase-referencing scheme in PAIRS with the phase-reference
calibrator in the odd index. Each phase-reference target is in an even
index, and calibrated with the preceding phase-reference calibrator.
There is a maximum of 15 pairs, a total of 30 source fields.
If your are not phase-referencing any source, then leave SOURCES blank.
Do not repeat sources that are in CALSOUR that are not actually used in
the phase-referencing scheme. If all targets are to be referenced to
only one source, use SOURCES = 'PRCAL', '*' to get all non-calibrators
phase referenced to 'PRCAL' - sources in CALSOUR are left alone, as is.
E.g. SOURCES='PRCAL1','TARG_1','PRCAL2','TARG_2','PRCAL1','TARG_3' will
reference target source 'TARG_1' and 'TARG_3' to phase-reference source
'PRCAL1' and target source 'TARG_2' will be phase-referenced to source
'PRCAL2'. All phase reference sources 'PRCAL1', etc MUST also be
present in the CALSOUR list, so they will be fringe fitted.
SOLINT or BPARM is used to specify the length of the fringe fit intervals.
They should be given in minutes. If SOLINT > 0, the whole experiment
uses the same fringe fit interval, regardless of frequency-ID. If you
set SOLINT=-1 then you may choose different fringe fit intervals for
each frequency-ID (e.g. 3 minutes for 5 GHz and 0.5 minutes for 43 GHz)
with BPARM. But you need to know the order (usually order of observing)
and you need to specify a number (maximum 10 values) for each separate
frequency-ID in BPARM. SOLINT is also used in the self-cal of CALSOUR.
BPARM Use either SOLINT or BPARM, not both; see SOLINT above.
INTERPOL is the interpolation all sources. The options are '2PT', 'SIMP',
'AMBG', or 'CUBE' as defined by CLCAL.
The procedure will also image all the sources, but note that the images are
produced in order to check the calibration. By no means, are the choices
made in the imaging stage represent the best possible imaging that could
be performed with this data. Those selections should be made by the
individual observer, in pursuit or their own scientific goals. The images
should allow you to get an idea about the success of the calibration, and
to find data that needs flagging before restarting it.
IMSIZE is used to define the output image sizes. You will want a big field
for your phase-referenced target sources with a minimum IMSIZE of 512
pixels and up to 8192. A larger image is slower, but accounts for
possible position inaccuracies if it is known to less accuracy than
500/Frequency(GHz) mas. Sources listed in CALSOUR are automatically
imaged with a size of 128 by 128 unless FACTOR is set.
Pixels automatically scale with frequency: 2.25/Frequency(GHz) mas.
FACTOR can increase the image size of CALSOURs. It is assumed that your
calibrators are well-known, so they do not need to be imaged in more
than 128 pixels. However you may want to increase this for complicated
sources. If FACTOR is used then the resulting image size for the
sources listed in CALSOUR is FACTOR*128. The maximum FACTOR is 64.
DOTV is used to direct all the 'instructive' plots that the procedure
makes to show you how the calibration is progressing. A DOTV < 0 value,
which is highly DISCOURAGED, will divert plots directly to the printer.
A positive value for DOTV send the plots to the TV and thus is
interactive. This can be slow and the user should be present the
entire time the procedure is running. DOTV=0 is recommended, this makes
PLot tables. Afterwards all the PLot tables can be cycled through with
the procedure P_ALLPLOT, maybe P_ALLPRINT, which are defined in the
BADDISK is the well-known parameter specifying the disks to avoid for scratch.
VLBAPIPE: Procedure that attempts to calibrate VLBA data blindly.
Documentor: Lorant Sjouwerman, email@example.com.
Related Programs: All VLBAUTIL procs, TECOR, BPASS, UVMLN, SPLIT, IMAGR,..
CLIPM, POSSM, VPLOT and many more
VLBAPIPE applies VLBAUTIL procedures to simple VLBA-only experiments, with
simple flagging and Ionospheric Total Electron Content correction before
bandpass calibration. Sources are split after calibration, and strong and
phase-reference sources are self-called and target sources are imaged as a
first order quality check.
READ THE HELP/EXPLAIN FILE THOROUGHLY BEFORE USING SINCE NONE OF THE
AIPS INPUTS ARE USED IN THE USUAL WAY.