AIPS HELP file for FITLD in 31DEC09
As of Sat Nov 21 0:40:38 2009
FITLD: Task to store an image or UV data from a FITS tape
INPUTS
INTAPE 0.0 9.0 Input tape drive # (0 => 1)
NFILES -9999.0 9999.0 # of files to advance on tape
DATAIN Disk file name
OUTNAME File name (name)
OUTCLASS File name (class)
OUTSEQ -1.0 9999.0 File name (seq. #)
0 => highest unique number
=> matching (on VLBA)
-1 => FITS tape value
OUTDISK 0.0 9.0 Disk drive # (0 => any)
OPTYPE Type of data to load,
' ' => all types
'UV' => UV data
'IM' => images
NCOUNT 0.0 9999.0 Number of files to load.
DOTABLE -1.0 1.0 True (1.0) means load tables
for images.
DOUVCOMP -1.0 1.0 >0 => compressed data (FITS)
DOCONCAT -1.0 1.0 >0 -> if VLBA correlator data
append data to existing
files, or if no appropriate
files exist create a new file
and append all data to that
file.
*****************************
Following adverbs are useful
only when reading VLBA
distribution tapes.
*****************************
CLINT CL entry interval (min)
0 => 1 minute.
SOURCES Source list to accept.
QUAL -10.0 Source qualifier -1=>all
TIMERANG Timerange selected
BCHAN 0.0 8192.0 Lowest spectral channel
number to select in each IF.
0=>1
ECHAN 0.0 8192.0 Highest spectral channel
number to select in each IF.
0=>highest
BIF 0.0 100.0 Lowest IF number 0=>1
EIF 0.0 100.0 Highest IF number 0=>all
DIGICOR -1.0 2.0 Controls the application of
the VLBA correlator's digital
correction.
0 or 1 => perform corrections
2 => perform cross-power
corrections, do total-power
only if zero padding used.
-1 => do NOT perform
corrections.
SEE HELP for more details
SELBAND Bandwidth to select (kHz)
SELFREQ Frequency to select (MHz).
See HELP.
FQTOL Frequency tolerance assigned
to SELFREQ, FITLD will select
data with freq. of SELFREQ
+/- FQTOL.
The unit of FQTOL is kHz.
<=0 => 10 kHz
WTTHRESH Flagging threshold based on
weights. See HELP.
0 => no flagging
1 => will flag ALL data
OPCODE Specify if wish to keep VT
(VLBA Tape Statistics) table
and other tables. See HELP.
' ' => do not save tables
NPIECE 0.0 90.0 Maximum uv table piece to
load (ignored for tape unless
NCOUNT = 1)
ERROR -1.0 3.0 >= 2 -> do not use AIPS
history in the FITS file
HELP SECTION
FITLD
Type: Task
Use: FITLD loads both maps and UV data from tape (or disc) to disc. It
will only load FITS files, if any other type of file exist on the
tape the task will fail, so users must skip over non-FITS files.
The aim of this task is to read, in one pass, a tape written by
FITTP. However a user may specify that all 'UV' files are to be
loaded, or all map files, or (default) all types of FITS file.
If IMAGE extensions are present, they are written to separate
catalog entries, but all TABLES are attached to the primary
catalog entry. They may only occur with a real or empty image
in the main FITS HDU.
Until the end of July 1999, UV data sets written out with FITAB,
read back in with UVLOD or FITLD, processed, and then written out
with FITTP could contain AIPS history records which will define
the data set falsely to the standard reading tasks in AIPS. If
you attempt to read a data set or image with FITLD (NOT a VLBA
correlator data set) and encounter error 4 (premature end of file)
or other failure, try reading the FITS data set with PRTTP with
ERROR set to 2. If this works, use UVLOD for UV data and IMLOD
for images with ERROR=2.
FITLD (and IMLOD, UVLOD, PRTTP, TPHEAD) now understand and
interpret the standard world coordinate system keywords CDi_j
and PCi_j. They are used if possible and problems reported.
Adverbs:
INTAPE......The input tape drive #. 0 => 1.
NFILES......# of files to skip on tape before reading data. For
example, 1 => start of next file, -1 => start of previous
file. 0 => no tape movement at all. (AIPS verbs AVFILE
plus TPHEAD allow you to position and check the tape in
advance; be sure to reset NFILES to 0 for FITLD after
them.)
DATAIN......48-character name of the disk file from which to read a
FITS file. It must be in the form
:
or
:::
where is the remote computer name, is the
environment variable (logical name) for the disk area in
which the file named is stored. is
usually omitted when the file is local to the current
computer. If DATAIN is not found, the task will try
DATAIN with the character 1 appended.
Beginning 2003-Oct-16 FITLD can read more than one disk
file at a time. In that case, they must all have the
same name except that the last letter(s) are the
sequence number 1 through NCOUNT with no leading zeros.
DATAIN must give the base name only, omitting the
sequence number. Then either
DATAIN = 'MYDATA:file_
or
DATAIN = 'MYDATA:file_1
can access files in $MYDATA named file_1, file_2,
file_3, etc. Note that this is similar to the pieces of
a FITS pieced UV-table data set, but only one UV-table
data set may be read at a time.
OUTNAME.....The output filename (name). Standard behavior with
default = the original file name (FITS), the source name,
or 'NONE'.
OUTCLASS....The output filename (class). Standard behavior with
default = the original map class (FITS) or a class of the
form 'STYP ', where S = (I, Q, U, V) Stokes parameter of
the image and TYP = (BEM, CLN, MAP) for a beam, clean map,
or anything else. Special combination images (OPTD et
al.) are also recognized.
OUTSEQ......The output filename (seq #). 0 => Find the lowest number
which produces a unique mapname. -1 => use the sequence
number on the tape (if any - works only on tapes written
by FITTP). SEE ALSO the remarks below under DOCONCAT.
OUTDISK.....The disk drive #. 0 => highest with space
OPTYPE......The type of data files to load, ' ' (default) => load uv
and images; 'UV' => load uv files only; 'IM' => load
images only.
NCOUNT......The number of consecutive files to load from tape or
disk. 0 -> 10000 for tape data, 0 -> 1 for disk FITS
files, but it is better to give a correct number to
allow the program to manage messages, file status, etc.
If you want to load only up to NPIECE pieces of a uv
table set, then NCOUNT must be 1 (or 0 if disk since
that defaults to 1). Note that the multiple pieces of a
uv-table set are counted as 1 for the purposes of NCOUNT
(but not NFILES).
DOTABLE.....True (1.0) means load all tables. (FITS images only).
DOUVCOMP...If true (DOUVCOMP > 0) the output data is written in
compressed format which can result in a substantial
reduction in disk space needed. Some loss of spectral
dynamic range and differences in weights between IFs and
spectral channels within a record are lost.
Only used for FITS files of the random-groups type (e.g.
written by FITTP) and those written in the IDI convention
for binary tables. Binary tables such as those written
by FITAB have the compression selected by the compression
state of the table.
Only used for FITS UV files
DOCONCAT...If true (>0) and the input data are VLBA binary tables
(directly from the correlator) then data are to be
appended to an existing file if possible. Possible
output files can be selected by OUTNAME, OUTDISK and
OUTSEQ. Use this option with CAUTION; it is REQUIRED (to
be true) if you wish to concatenate the data from
multiple files being read in the same execution of FITLD
for VLBA correlator data. (UV table pieces from FITAB
are concatenated to each other even if DOCONCAT is false.)
If DOCONCAT is false and OUTSEQ is specified, FITLD will
fail at the start of file 2. If DOCONCAT is false and
OUTSEQ is not specified, then FITLD will make separate disk
files for each tape file. These disk files may be
concatenated later with DBCON, but perhaps only after
they have been MATCHed. FITLD will write one or, if it
finds data that do not match in e.g. number of channels,
more output files. If DOCONCAT is TRUE then FITLD will
look over all files matching the OUTNAME, OUTCLAS,
OUTSEQ, OUTDISK parameters to try to find one that it can
append data to. If it does not it will create one based
on these name parameters. That will generally fail if
OUTSEQ is specified. Thus, for DOCONCAT true, set OUTSEQ=0.
CLINT......To define the interval between CL entries for antennas for
data from the VLBA correlator only. This defines the
fundamental time-scale for calibration, 1 minute is a good
value for most frequencies. Shorter intervals may be
relevant for high frequency programs.
SOURCES....Source list. List of sources to accept from the VLBA
distribution tape (or file).
QUAL.......Only sources with a source qualifier number in the SU table
matching QUAL will be used if QUAL is not -1.
TIMERANG...Range of UTC times to be read wrt the first day number in
the first file read. 0=>all.
BCHAN......First spectral channel number to select from each IF. 0=>1.
ECHAN......Highest spectral channel number to select from each IF.
0=>all higher than BCHAN.
BIF........First IF to copy. 0=>1
EIF........Highest IF to copy. 0=>all higher than BIF. WARNING: IF
selection will work differently in FITLD where it is
based on the input IF numbers and in later tasks where it
is based on IF numbers reordered by FITLD to frequencu
order. These two selections will be the same only if the
data from the correlator are in increasing frequency
order.
DIGICOR....Data processed in the VLBA correlator are affected by a
variety of phenomena that we collectively refer to as
digital effects. FITLD is able to apply the corrections
needed to scale the data to the values that would be
returned by a purely analogue correlator. DIGICOR = -1
will turn off the application of these correction,
DIGICOR = 0 or 1 will apply them to both cross and total
power, DIGICOR = 2 will apply the corrections to
cross-power and only to total power if zero-padding was
used in the correlator. The latter case is designed to
deal with a situation in which the total-power spectrum
is dominated by multiple, strong spectral lines. In that
case, if zero-padding was not used, the reconstructed
total-power spectrum, after an FFT, is inaccurate at some
level. In all continuum cases DIGICOR = 1 should be the
default. In nearly all line cases DIGICOR = 1 is fine.
only if the Tant in the line is about equal to the Tsys
should DIGICOR = 2 be used. One word of caution, for
data correlated prior to Aug 1 1994 the correlator was
incorrectly scaling the data by a factor of 1.57 (i.e.
PI/2) . So for that data, DIGICOR can be used but a
BFACTOR of 0.6369 (i.e. 1/1.57) should be applied in
APCAL or to the final images. Unfortunately FITLD had no
knowledge of the date of correlation so cannot perform
this correction. The code detects data from the DifX
correlator and applies a different (1.0) saturation
correction in that case.
SELBAND....Bandwidth of data to be selected. If more than one IF is
present SELBAND is the width of the lowest frequency IF
required. Units = kHz, 0=> all
SELFREQ....Frequency of data to be selected. If more than one IF is
present SELFREQ is the frequency of the lowest frequency
IF required. FITLD will sort the IF channels into
increasing frequency order as the data are read from the
tape so all selection will be performed on the frequency
and bandwidth values of the output IF 1.
Units = MHz, 0=> all
FQTOL......The frequency tolerance used in selecting data using
SELFREQ. Data whose output IF 1 frequency lies in the
range SELFREQ +/- FQTOL will be read. Units = kHz, =< 0
= 10kHz
WTTHRESH...Weight based flagging threshold. If WTTHRESH
> 0 the weights for each visibility are examined,
if weights for all IFs are =< WTTHRESH the
visibility record is flagged. The flagging is
permanent, i.e. the record is not copied to the
output file.
If a weight of some IFs (not all) =< WTTHRESH such
a visibility is recorded but the weight of these
IFs put to zero.
Weights lie between 0 and 1. If you wish to apply
weight based flagging a suggested value for
WTTHRESH is 0.8. A value of 0 switches off the
flagging.
OPCODE.....By default, the input tables that are translated into
AIPS tables are all removed and the large VT (VLBA Tape
Statistics) table is neither translated nor saved. If
you wish to keep the former and/or latter enter:
'VT' => Keep VT tables, not the rest
'ALL' => Drop VT tables, keep rest
'VALL' => Keep VT and other tables
NPIECE.....Maximum piece number to read in a multi-piece uv-table FITS
data set. 0 => maximum in the data. Note, if NCOUNT > 1,
NPIECE is set to 999 by FITLD.
ERROR......>= 2 => do not parse AIPS history cards found in the FITS
header. Do not use this option until you have tried IMLOD
without it. If you get mysterious errors (usually error 4
which implies premature end of file), then try IMLOD with
ERROR set to 2. For a period of time, UV data sets written
out with FITAB, read back in, processed, and then written
out with FITTP could have so many AIPS history records in
their header as to cause IMLOD to read the data wrongly.
This can also occur when AIPS data are written to another
package and then brought back to AIPS.
EXPLAIN SECTION
FITLD: Task to load FITS format data
Documentor: Chris Flatters
Related Programs: FITTP, FITAB, IMLOD, UVLOD, TAMRG
-----------------------------------------
Disk file reading: DATAIN
To read multiple disk files in one pass of FITLD, the file names must
all be predictable from the user supplied value of DATAIN. This
requirement has led to the requirement that they be named with a
consecutive postfix number beginning with 1. It may seem onerous to
have to do this, so we suggest using link files. Thus 4 disk files of
consecutive VLBA correlator data may be linked to a simple name in the
local directory ($MYWORK) by
primate<142>$ ln -s /home/euro/aipsdvlp/WB/AO-0/32400001.FITS Lorant_1
primate<143>$ ln -s /home/euro/aipsdvlp/WB/AO-0/32410001.FITS Lorant_2
primate<144>$ ln -s /home/euro/aipsdvlp/WB/AO-0/32420001.FITS Lorant_3
primate<145>$ ln -s /home/euro/aipsdvlp/WB/AO-0/32430001.FITS Lorant_4
Then, inside AIPS, one sets
NCOUNT = 4; DATAIN = 'MYWORK:Lorant_
where you should note (a) DATAIN= is the last command on the line and
is typed with no close quote so that the case is preserved, (b) the
number 1 may be left off the name (FITLD will append it), and (c) the
environment vaiable $MYWORK must be known before AIPS is started and
is typed with a colon following but no $.
-----------------------------------------
VLBA CALIBRATION TRANSFER
FITLD will load flags, gain curves, system temperature data, and
pulse-cal data from VLBA format FITS files. These data will be
loaded into version 1 of the FG, GC, TY, and PC tables
respectively.
If several input files are concatenated into one AIPS file then
the FG, GC, TY, and PC tables will contain the calibration data
from all of the inputs files that were concatenated. This may
lead these tables containing duplicated records.
I recommend that you remove all of the duplicate records in the
GC, TY, and PC tables after you have loaded all of the data for
a given experiment using the procedure MERGECAL (see HELP
MERGECAL).
KNOWN DEFICIENCIES WITH RESPECT TO INTERFEROMETRY DATA
INTERCHANGE CONVENTIONS
Reference dates.
FITLD assumes that the reference date for the data loaded from
FITS-IDI files is the first reference date encountered in any
of the tables in the file. If the file conforms the the table
ordering conventions in AIPS Memo 102 then this should be the
reference date for the ARRAY_GEOMETRY table for the first
array. This is normally the correct reference date.
If any table has a reference date (RDATE) earlier than that
chosen by FITLD or if data occur before 0h on the chosen
reference date then AIPS may not be able to handle the data
output by FITLD.
It is also possible for problems to occur if the first table
in the file that has an RDATE keyword is not an ARRAY_GEOMETRY
table and the value of that keyword differs from the RDATE of
of the first ARRAY_GEOMETRY table since the reference date
will not then be the same as the reference date for subarray
1 as is assumed in some parts of AIPS.
This adverb has been removed and made always true. It is turned off
for correlators other than the VLBA.
DELCORR....This parameter determines whether the second order
amplitude corrections caused by delay decorrelation
in the VLBA correlator can be applied after fringe
fitting or not. These corrections are necessary for
obtaining extremely high dynamic range images. If
they are not applied excellent images can still be
obtained. The corrections are applied when a CQ
table is attached to the data, the CQ table
contains information required to perform the
correction - see AIPS Memo 90 by Athol Kemball for
more details. Under some circumstances (e.g.
spectral averaging in the correlator on data that
contains narrow spectral lines) the assumptions
under which the 2nd-order corrections can be
applied are false. In that case the CQ table should
not be generated.
DELCORR = -1 => do not generate CQ table
DELCORR >= 0 => generate CQ table.