AIPS HELP file for PIPEAIPS in 31DEC25
As of Wed Apr 23 14:25:48 2025
PIPEAIPS: LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC
INPUTS
PIPEAIPS: Amp+phase calibration for conn. elem. interferometers
** Type RUN PIPEAIPS to load this procedure **
WORKDISK Working disk, thus in/outdisk
CATNUM Catalog number of the UV-file
INNAME Input UV file name (CATNUM<1)
INCLASS Input UV file class(CATNUM<1)
INSEQ Input UV file seq. (CATNUM<1)
TINT Re-average visibilities (sec)
or corr.avg. time if known
FACTOR Frequency averaging? 0,2 or 4
FASTSW > 0 Correct fast-sw source
names
VLANTCOR Run VLANT? Only do once!
AUTOFLAG Level of automatic flags used
< 0: no automatic flagging.
= 0: default FLAGR
= 1: flag beginning of scans
PHAINT phase solution interval (min)
AMPINT ampl. solution interval (min)
BASEBAND Data has N equal basebands:
see help and FRING aparm(5)
BBMAXIF For unequal IFs/baseband set
BASEBAND=-1 and fill up to 4
indices of max IFs. See help
BPNORM -2.5 1.5 Bandpass 'divide-by' control
see help, BPASS bpassprm(5)
REFANT Reference antenna number
DOMODEL > 0 Use standard flux
calibrator models
Note that most standard flux
calibrators have models but
the absence of a model will
make this CRASH. (see HELP)
VLAOBS Alternative MODEL frequency,
see explain file for CALRD
UVRANGE UV range for flux calibrator
(may be used if no model)
** use next 2 lines if flux calib. is NOT standard **
AMPCAL Alternative flux calib. name
FLUX Flux calib. total flux dens.
PHACAL Phase calibrators
'*' = any CALCODE (continuum)
All others are your targets
BNDCAL Bandpass calibrators (max 5)
NOPAUSE > 0 no pause after GETJY
AUTOPLOT > 0 make diagnostic plots
** the following lines are for auto-imaging **
DOIMAGES > 0 apply calibration and
image
IMGTYPE 'SIMP','CONT','PSEU' & 'LINE'
for continuum (single image)
from "Ch.0" or "line" cube,
pseudo cont (image per IF)
or line (full image cube)
ARRYSIZE Max baseline in kilometers
= 0 let procedure find array
IMSIZE Square size of image (pixels)
NITER Max number of iterations/chan
= 0 recommended for LINE
CUTOFF Clean threshold (Jy)
ALLIMG > 0 also image continuum
calibrators
> 1 line calibrators too
DOEVAUV Only for IMGTYPE='PSEU' and
if > 0 then run 'EVAUV' too
** interactive self calibration mode
-- not recommended for beginners **
SLFCAL > 0 do interactive selfcal
ABS(SLFCAL) the number of
iterations
See HELP PIPEAIPS and
EXPLAIN SCIMG
BADDISK Disks to avoid for scratch
QCREATE See help/explain file!
HELP SECTION
PIPEAIPS
Type: Procedure
Use: PIPEAIPS is a procedure that does quick and dirty VLA
calibration and imaging of line and continuum data,
including high frequency data. Full polarization
calibration is not included.
NOTE: This PIPEAIPS code originates from earlier VLARUN/DOOSRO versions
To load the procedure into AIPS type:
(this is only required once per AIPS session)
> RESTORE 0 (recommended, not required)
> RUN PIPEAIPS
> COMPRESS
To review inputs type:
> TASK'PIPEAIPS';INPUTS
To execute it type:
> PIPEAIPS
Adverbs:
CATNUM, or all of INNAME, INCLASS, INSEQ (and WORKDISK for both):
The catalog number of the file to calibrate; for VLA data loaded
with BDF2AIPS this would be UVEVLA for a yet unprocessed file.
Add flagging info on missing receivers, etc., already here in FG1.
PIPEAIPS converts this into a SPLAT0/SPLATL pair to keep the data
in spectral window form but also using a channel 0 for speed. If
a first instance of PIPEAIPS has run, the 'SPLAT0' should be
chosen. If INNAME etc are used, then set CATNUM to a non-positive
value (<=0) i.e. CATNUM=0;GETNAME Do not use both CATNUM
and IN<*>
TINT:
Some (VLA) data sets have unnecessary short (i.e., 1 sec)
visibility integrations. Set this to a larger value (>=1,
typically an integer multiple of the original correlator
integration interval) to decrease the data size by this factor.
Leave zero to do no averaging, though if the correlator averaging
time is known, setting it to that interval (in sec) will take out
the guess work in finding a value from the data.
FACTOR:
Reduce data size by averaging the frequency axis by a factor 2 or
4. Note that this is done after fringe fitting (i.e., correcting
delay) when creating the SPLATL file. The TYAPL file remains
unaveraged.
FASTSW:
Sometimes, with fast-switching, single calibrator may have more
than one name. Set FASTSW positive to check the positions (closer
than 3 mas in R.A. and DEC.), qualifier and calcode. If sources
have the same position, PIPEAIPS will rename them all with the
shortest name. This will prevent multiple images of the same
source. Up to 100 source names can be modified.
VLANTCOR:
Skip running VLANT if < 0 or for an input file with class 'UVLANT'
or for an input file with class 'SPLAT*' (i.e. 'SPLAT0' and
'SPLATL'). VLANT should be run only once so set negative if you
know it was run. If set to 0, the pipeline runs VLANT and will
change the class of the input file (e.g. 'UVEVLA' or 'UVDATA') to
'UVLANT' and thus skip it on successive runs of PIPEAIPS.
AUTOFLAG:
< 0 -> no automated flagging.
NOTE: if an FG table is detected, it WILL NOT DO any form of
automated flagging on the data. You may not want to do
autoflagging when you have very short scans (i.e. in
fast-switching mode) and flag by hand. The highest FG-table is
always kept.
0 -> Perform FLAGR (OPTYPE'TIME') on the raw multi-source data set
PIPEAIPS estimates the integration time and SOLINT, in FLAGR,
is set to 3 times the integration time.
1 -> Also flag beginning of scans (usually necessary). Uses QUACK
(OPCODE'BEG'; APARM(2)=MIN(20 SEC, 3 int. times).
Again, you may not want to do this if you have short scans
(30s).
PHAINT, AMPINT:
PHAINT is the solution interval in minutes for phase calibration
- typically the scan length on your phase calibrator, but much
shorter (0.2-0.5) for high-frequency data.
AMPINT is the solution interval for amplitude calibration
- probably best between 1-10 times the value of PHAINT.
PIPEAIPS first calibrates the phases, then the amplitudes
(including phase), i.e., the second SN table should have phases
very close to 0.
BASEBAND, BBMAXIF:
In FRING set aparm(5) to make solutions for combinations of 1, 2
or 4 equal divisions of the IFs. Thus APARM(5) will be set to 1,
3 or 4. If left to 0 it will assume default wideband continuum
and choose two basebands for bands up to K (18 GHz), four for K,
Ka and Q bands. If an unequal number of subbands per basebands
WAS used (eg 11,15,14 and 14) use BASEBAND=-1 (i.e. to set
aparm(5)=-1) and fill BBMAXIF with the *ending* IF numbers (i.e.,
to set bparm, see help FRING), in the example:
BASEBAND=-1; BBMAXIF=11,26,40,54
(that is: 11,11+15,11+15+14,11+15+14+14).
BBMAXIF uses up to 4 values, derive them from LISTR, using OPTYPE
'scan'.
BPNORM:
The value of BPASSPRM(5) in BPASS for line data; Defaults to zero,
but some data sets are better off using, e.g, -2. See EXPLAIN
BPASS
REFANT:
# of antenna to be used as reference antenna, should be well
behaving antenna towards to middle of array. AIPS will make a
choice if left zero.
DOMODEL:
> 0 -> CALIB will use a standard model for any of the five
standard flux density calibrators (3C48, 3C138, 3C147, 3C286,
3C295, or their standardized IAU names - see EXPLAIN PIPEAIPS),
assuming they exist. Currently they exist for most calibrators
and bands, and more are added all the time. For the latest
available models type 'CALDIR' and/or 'EXPLAIN CALRD'. For lower
frequencies, and for smaller arrays, it is usually sufficient to
use a non-positive DOMODEL, i.e., to use a point source model for
the standard flux density calibrators.
!! NOTE THAT IF YOU SELECT THIS MODE, PIPEAIPS WILL CRASH IF !!
!! NO MODEL IS AVAILABLE FOR YOUR PARTICULAR CALIBRATOR. !!
!! Check this with 'CALDIR' and/or 'EXPLAIN CALRD'. !!
VLAOBS:
If an alternative model available (with CALRD) is preferred over
the standard flux calibrator model for the band (the typical L, C,
K, etc models), provide the string that combines the model with
the source in VLAOBS. That is, provide the string '2.7' to use
3C138_2.7.MODEL.
AMPCAL:
If none of the standard flux calibrators is observed (see EXPLAIN
PIPEAIPS), then enter the name of an alternative source here.
NOTE: if specified it will be used by force even if a standard
calibrator present! So you can force your own flux scale if
desired.
FLUX:
Total flux density of amplitude calibrator specified in AMPCAL.
UVRANGE:
UV range for which flux calibrator (standard or not) is a point
source. Leave zero if you are using a flux calibrator model
(DOMODEL>0), or want a point source model approximation for the
one in AMPCAL.
PHACAL:
Specify your continuum phase calibrators. Limited to a total of
20. If there are more, use '*' in the FIRST argument to select
sources with any calcode. Include your flux and/or bandpass
calibrators if you want them to be calibrated. Any source not
appearing in PHACAL are considered targets.
BNDCAL:
Specify up to five bandpass calibrator sources.
Bandpass calibrators defined here can be any (but maximum 5)
sources in your data. If multiple sources are specified then an
average of the sources is used.
NOPAUSE:
< 0 then PIPEAIPS will pause after GETJY calculated the fluxes
for the secondary calibrators. This allows you to check the flux
densities (specifically their errors) of your secondary
calibrators, which is a good diagnostic of the calibration up to
this point. Simply press to continue.
AUTOPLOT:
Make diagnostic plots:
< 0 -> No plots
1 -> Only make plot files of SN tables
2 -> Also include plots of CL tables
3 -> (LINE data) add plots of the BP table
4 -> (LINE data) also plot BP calibrators with CL and BP table
applied
DOIMAGES:
> 0 -> continue with imaging.
< 0 -> do not image.
You may check the calibration (in a different window) and
continue with imaging by typing:
"IMAPIPE(1,,)" in the PIPEAIPS window
directly after the calibration part has finished. The '1'
in the first argument of IMAPIPE means to start with SPLIT,
but if you have SPLIT the data already in the other window,
use a 'zero' instead.
IMGTYPE:
Choice of 'SIMP', 'CONT', 'PSEU', or 'LINE' (no default). For a
single image averaged over all channels and IFs (spectral windows)
use 'SIMP' or 'CONT'. Here 'SIMP' uses the IF-channel averaged
data set (i.e. "CH 0" or "SPLAT0"), whereas 'CONT' uses all
(frequency averaged) channels (a.k.a. "LINE"or "SPLATL") data thus
reducing the potential effects of bandwidth smearing (within an IF
bandwidth) at the cost of slower operation. Note that 'CONT' is a
proper on-the-fly averaging of the visibilities. For spectral
line applications, 'LINE' will make separate cubes retaining all
channels for each IF/spectral window (producing one cube per IF).
The 'PSEU' will make so-called pseudo continuum or reference
images; a channel averaged continuum image for each IF/spectral
window, combined in an image cube. For example, an 8 IF visibility
data set with 256 channels per IF will result in 8 pseudo-continuum
images, each averaged over the inner 240 (256*30/32) channels.
1/32 of each edge is skipped for cubes of more than 128 channels,
1/16 of each edge for fewer.
ARRYSIZE (if DOIMAGES>0):
Maximum baseline length in kilometers; this sets the resolution.
The pixel size will be 17.5/freq(GHz)/ARRYSIZE. For the VLA arrays
use 35.4 (A-array), 10.8 (B), 3.3 (C), 1 (D), or 73 (PT-link)
kilometer. If =0, PIPEAIPS will guess the maximum unprojected
baseline from the antenna file and set ARRYSIZE to about the above
values but this code is fragile. If <0, SETFC will be used to
figure out the best cell size.
IMSIZE (if DOIMAGES>0):
Target image size in pixels, with a minimum of 128. For
calibrators the default is 256.
< 0, AIPS will try to image the full primary beam (using SETFC).
< -9, then this also applies to calibrator sources.
NITER (if DOIMAGES>0):
The number of clean components to be used in the imaging. Because
spectral line data can take a very long time to image you may want
to use zero or a low number of iterations and check that there is
no bad data or a faulty calibration. Then you can re-imaging with
more iterations. If ALLIMG >0 (i.e., image calibrators), 500
iterations is used as there should be no need for deep cleans for
calibration check images. However if IMSIZE<-9, then the number of
iterations=NITER, for all sources. If negative, NITER=6e6,
effectively using CUTOFF as the only criterion of when to stop.
Note that NITER=0 means zero iterations, i.e., make the dirty image
(and ignore CUTOFF).
CUTOFF (if DOIMAGES>0):
Stop cleaning when this level (in Jy) is reached in the residual
image.
< 0 and NITER >>> 0, then it will stop cleaning after the first
negative clean component. Same for calibrators and targets,
but also competes with NITER, i.e., whichever is reached first
will determine when clean stops.
< 0 and NITER < 0 then PIPEAIPS will estimate a conservative value
forthe noise level from the visibility data header, and stop
at three times this noise level. NOTE that for strong sources
the dynamic range limitation may prevent it from reaching this
limit - it is VERY wise to set either NITER or CUTOFF (or both)
on a first trial of using the pipeline on a particular data
set.
ALLIMG (if DOIMAGES>0):
> 0 -> image calibrators also.
> 0 ALLIMG >= 1 -> only image continuum calibrators in spectral
line dataset.
> 1 -> image both continuum and line calibrators.
Set IMSIZE to <-9 if you want larger images than 256 pixels squared
and deeper cleans than 500 iterations for the calibrators.
DOEVAUV (if DOIMAGES>0):
Only used when IMGTYPE='PSEU'. If > 0 then after the imaging,
compare the pseudo continuum image with the SPLAT0 continuum UV
data set. That is, per IF run task 'EVAUV' and dump the
statistics/results to the terminal. If > 1 or if AUTOPLOT > 0 then
make histograms.
SLFCAL (if DOIMAGES>0):
<> 0 -> do self-calibration for targets only in continuum data
(SLFCAL is ignored for spectral line data).
> 0 -> you wish to interact with the image using the AIPS TV, this
includes UV data editing options.
< 0 -> only display at the end of each clean.
ABS(SLFCAL) specifies how many cycles of self-cal to do.
This option should be used with EXTREME care. Care should be taken
that there is enough flux in the field for self-cal to work. Note
that the input PHAINT is taken as the solution interval, which in
most cases will be an invalid assumption
!! USE OF SLFCAL BY NOVICES IS HIGHLY DISCOURAGED. !!
BADDISK
Disk not to use for scratch files. Usually only used if there is a
read-only or 100 percent full disk.
QCREATE
Only use if you understand its implications! Set > 0 to speed up
output file creation. If set > 0, it will crash the pipeline if not
enough disk space on the working disk is available. Suppose the
original data set is X GB in size; another X is needed for TYAPL
(if not skipped, see hint 6 below), as well as X/Tavg for SPLATL,
plus X/Tavg for SPLITs (where Tavg is the amount of averaging of
visibilities). That is, X+2X/Tavg more than the original. This
still excludes the disk space needed for imaging, so only use
QCREATE if you know the original data sise and have made sure
more than the expected disk space is available on the working disk.
But of course this disk space requirement also applies for
QCREATE <= 0 ...
EXPLAIN SECTION
PIPEAIPS : Procedure to calibrate (VLA) interferometer data blindly.
Documenter: Lorant Sjouwerman, Amy Mioduszewski
Related Programs: (VLA) calibration, imaging and plotting routines
Type RUN PIPEAIPS to define the PIPEAIPS procedure (once is enough)
BEFORE RUNNING PIPEAIPS:
0) Load data with BDF2AIPS (using Obit) or FITLD/UVLOD
1) Recommended to use TABPUT to place calibrator codes on your
calibrators
2) Flag the known bad data and add the flags into FG table 1. If a
run of PIPEAIPS has produced a SPLAT0/SPLATL pair these inital
flags will be applied. For additional flagging, add to FG 1
attached to the SPLAT0 file; if there is a flag table then
AUTOFLAGing is disallowed.
3) If preprocessed data, compile all flagging in the continuum
(SPLAT0) flag table #1. Also add line flags here to the first
flag table attached to the SPLAT0 dataset.
4) Check for long (>12 char) source names - typically in mozaics and
for eg. full IRAS or CXO names. PIPEAIPS expects names to be <=
12 characters, so they need to be changed if longer than 12
char. If you don't care about the exact names you can get
approximate/reversed short names by running the predefined
procedure 'shortnames' on the data file (note, if EVLA data then
run it both on SPLAT0 and SPLATL).
5) Check and rename the dummy sources in the SU table with
TABPUT/TABED (ie, source enties that appear twice: rename the
very first one) or flag scans that are typically <1 min at the
start before a longer one.
6) If you want to speed things up at the start, opt to skip applying
the SysPower corrections and thus skip to recompute all
visibilities by deleting (after TASAV) the SY table.
7) If you want to speed things up at the imaging stage, opt to
average the data more than you'd normally do, using TINT.
Note that flag tables with numbers higher than 1 get deleted at
restarts.
CLEAN STARTING CONDITIONS
Make sure only one frequency-ID is present (otherwise use UVCOP to
split them apart and INDXR to re-index the files). It can process
both continuum and line data, but the line files must have inclass
SPLAT0' and 'SPLATL'. If any, flags must be made in FG table
number 1 on the continuum or 'SPLAT0' files, so if you flag the
'SPLATL' data you have to copy the line flags/flag-table to the
'SPLAT0' file. Restarting is straightforward - fluxes are reset
and tables are deleted although naming back fast-switching
sources is irreversible (FASTSW>0).
OTHER NOTES:
- There are NO DEFAULTS!
- Assumes you have observed 3C286 or other standard VLA amplitude
calibrator (see below) for absolute amplitude calibration.
- Will use up to 5 sources (maybe 3C286 etc) for bandpass in SPLATL,
- Will use up to 20 possible phase calibrators (or more if '*').
PRIMARY FLUX CALIBRATORS:
List of all names recognized as a VLA amplitude calibrator in
SETJY:
- 3C48 is also known as: 0134+329, 0137+331, or J0137+3309
- 3C138 is also known as: 0518+165, 0521+166, or J0521+1638
- 3C147 is also known as: 0538+498, 0542+498, or J0542+4951
- 3C286 is also known as: 1328+307, 1331+305, or J1331+3030
- 3C295 is also known as: 1409+524, 1411+522, or J1411+5212
At least one of these names (3C or other) must be in the data set
for the flux calibration. If a different source is used, then you
have to specify this other source using AMPCAL and FLUX (and
maybe UVRANGE).
ADVICE!
The best results are obtained if flagging of bad data is recorded
in FG table 1 before starting PIPEAIPS. Use UVFLG for known bad
antennas or basebands (from the operator log), a semi-automated
procedure like RFLAG, FGSPW, or manual usage of e.g. WIPER, CLIP,
TVFLG, SPFLG, etc. Make sure that these flags end up in FG#1
(outfgver=1) or they will be lost on reruns. A TASAV before and
after flagging is good practice!
When using PIPEAIPS for the first time on a data set (or on line
data) do not do any cleaning (NITER=0), or self-cal (very
fragile!). It will go much faster and tell you right away if you
should expect problems. If you switch on autoflagging, you'll
see where extra flags are needed and whether it did what you
expected and intended (i.e., if there are any typo's in the
input; if you can you do self-cal; if you should find a better
reference antenna, different PHAINT, etc).