AIPS NRAO AIPS HELP file for Y2K in 31DEC18



As of Sun Apr 22 17:48:58 2018


Y2K: Verification/timing test; see Y2KLOAD.001, Y2KEXEC.001

INPUTS

TCODE                              INIT, TEST, READ, or WRIT
TMODE                              T or M; Test or Master input
TMASK           1.0       127.0    test selection bit mask
Y2KSIZE                            SMALL, MEDIUM, LARGE test
DDISK           1.0        35.0    Disk drive #: master UVDATA
MDISK           1.0        35.0    Disk drive #: other masters
TDISK           1.0        35.0    Disk drive #: test images
IOTAPE          1.0         9.0    Input/Output tape drive #.
Y2KDISK                            Disk area for READ and WRIT
EDGSKP          0.0        64.0    Pixels to skip at edges
TERSE          -1.0        99.0    > 0 => reduced output
OUTPRINT
                                   Printer disk file to save
BADDISK                            Disks to avoid
VERSION

HELP SECTION

Y2K
Type: RUN file
Use:  Y2K performs the AIPS verification test on an AIPS system.
      See RUN files Y2KLOAD.001 to compile a set of procedures and
      Y2KEXEC.001 to run them.  This input file is used to prepare the
      parameters needed to run Y2KEXEC.  Set the adverb values, do a
      TPUT Y2K, and then RUN Y2KEXEC.

      The idea is to run a sequence of the most commonly used tasks in
      AIPS on a standard UV data set.  The tasks are UVSRT, IMAGR,
      APCLN, CALIB, and VTESS as the main tasks with tasks SUBIM,
      COMB, UVDIF, CCMRG, FITLD, and FITAB also being exercised.  The
      test runs each task and compares the results with standard
      answers prepared when the test was devised or last revised.  The
      total time it takes to execute may be used to compare computer
      performance and the degree to which the current answers agree
      with the official answers compares compilers/computers and code
      reliability/constancy.
Adverbs:
 TCODE    String*4      'INIT' - create new master files
                        'TEST' - test tasks against master files
                        'READ' - read master files from tape/disk
                        'WRIT' - write master files to tape/disk
                        User sites would first READ the master files
                        in and then TEST them.
 TMODE    String*1      'T' - test output from previous task used as
                              input to next task.
                        'M' requests master input to all tasks.
 TMASK    Scalar        Task selection bit mask (1-63) - use 63
                        normally.
                        +1=>SORT,  +2=>IMAGR, +4=>APCLN, +8=>CALIB,
                        +16=>IMAGR-Clean, +32=>VTESS
                        For example, TMASK=54 would execute all but
                        sort and CALIB.
 Y2KSIZE  String*8      'SMALL', 'MEDIUM', 'LARGE' test.  The LARGE
                        test is used to measure performance.  There
                        is also a 'HUGE' test.
 DDISK    Scalar        Disk drive # (1-9) master UVDATA
 MDISK    Scalar        Disk drive # (1-9) other master files
 TDISK    Scalar        Disk drive # (1-9) test files
 IOTAPE   Scalar        Input or output tape drive #. (1-9)
 Y2KDISK  String*8      Logical name of disk area to be used for
                        FITS-disk files in READ or WRIT instead
                        of actual magnetic tape.
                        ' ' => use magnetic tape.
 EDGSKP   Scalar        Number of pixels to skip at edges in
                        comparing test and master images
 TERSE    Scalar        > 0 => reduced output:
                          > 0 = drops INPUTS for all tasks and
                                TPHEAD when reading tape
                          > 2 = also drops CATALOG listings
                          > 3 = also drops IMHEADER listings
                        Use -1 on all new or uncertain systems,
                        or when determining the AIPSMark(93).
 OUTPRINT String*48     Disk file name in which to save the line
                        printer output.   ' ' => use scratch and
                        print immediately for interactive jobs -
                        batch jobs use OUTPRINT =
                        'PRTFIL:BATCHjjj.nnn' (jjj = job #, nnn
                        = user #).  When OUTPRINT is not blank,
                        multiple outputs are concatenated and
                        the file is not actually printed.
 BADDISK  Array         Any bad disks to avoid for scratch files
 VERSION  String*48     Version of tasks to use - normally blank to
                        use TST.  Other values include OLD, NEW, and
                        private dirs; use | to separate entries in a
                        search path.

Y2K runs in four different modes: (TCODE)
   INIT = starts from the original master UV dataset, and
           (re)computes master files (does ALLDEST).
           The "mask" TMASK controls which steps execute.
   TEST = re-does the INIT computation but stores the new
           files under different names and compares against
           masters. The "mask" TMASK controls which steps
           execute. TEST steps use either the TEST run output
           of the previous step (TMODE=T) or the INIT run
           output (TMODE=M).
   WRIT = write master or test files to tape (TMODE matters)
   READ = read master files from tape.

Input data files for test steps are selected by the TMODE
   adverb which has two possible values: 'T' for those made
   in previous steps of the present test or 'M' for the
   master "correct" data sets input from tape.

The adverb Y2KSIZE controls the size of the test performed:
   'S'    512x512 image, 5000 clean components, 15000 vis
   'M'    1024x1024 image, 15000 clean components, 75000 vis
   'L'    2048x2048 image, 25000/75000 Clean components,
          425000 vis
   'H'    4096x4096 images, APCLN not used, 4-resolution IMAGR
          used with 35000/200000 Clean components
           2674490 vis.

EXPLAIN SECTION

Explanation of Y2K:

Y2K exercises important AIPS tasks using data for one of four
sources.  The four data sets provide four sizes of "real" AIPS
reduction: 512x512 with 13000 vis. and 5000 components; 1024x1024 with
77000 vis. and 15000 components; 2048x2048 images with 425000 vis
and 25000/75000 Clean components and 4096x4096 images with a
4-resolution Clean on 2674490 vis with 35000 and 200000 Clean
components.  The sources for the "small" problem was originally a
background sources for an HI absorption experiment by Liszt and
Greisen, the "medium" problem uses data supplied by Alan Bridle, and
the "large" and "huge" problems use data supplied by Rick Perley and
Chris Carilli.

The Y2K tests are intended to certify the correct execution of all
AIPS systems.  They are an enhancement of the DDT tests devised a
decade earlier for the same purpose on computers of more modest
capabilities.  About DDT it was said: "The smallest is small enough to
run in less than one day on all machines which will ever run
AIPS."   Its execution time is significant and easily measurable
on modest systems such as the Sparc IPX, and approximate timings
can also be obtained for faster systems.  The AIPSmark(93) is
based on the total elapsed time for the Large DDT suite; see
AIPS memo 85 for full details.  The AIPSmark(2000) has been defined in
AIPS Memo 104 using the large Y2K test in a similar manner.  This memo
is available from our web site and is shipped with AIPS.
================================================================
Steps to execute the Y2K Test on a new computer:

1. Log in to AIPS under the user number which will own the data files
   and the SAVE/GET files associated with the test.  It is probably
   convenient to clear your AIPS message file before proceeding:
   PRNUM=-1; (PRTMSG possibly) ; CLRMSG

2. Now do the POPS command "RUN Y2KLOAD".  This initiates the
   compilation process, which on most modern systems will not take
   more than a few minutes.

3. The next step is to read the data files from the tape, or from
   disk.  On tape, these are files 1 - 10 for the small problem, files
   11 - 20 for the medium problem, and files 21 - 39 for the large
   problem (which breaks the UV files into pieces).  The HUGE test has
   never been put on tape, a large-capacity tape would be required.
   Alternately, if you have the files on disk, merely set the Y2KDISK
   adverb to point at where the files reside (you need to define or
   use a "logical" or environment variable with an uppercase name
   before starting AIPS to use this feature; e.g. FITS).  The Y2K
   files are available via anonymous ftp to ftp.aoc.nrao.edu in
   directory
              cd pub/software/aips/FITS/31DEC04
              prompt
              binary
              MGET Y2KL*                  (or Y2KS*, Y2KM*, Y2KH*)

   On tapes, the first one in each set is the original visibility data
   set of the Y2K problem.  Subsequent files are the "master" versions
   of files computed from the original dataset.  The read process
   consists of the following steps:

      CLRMSG
      GET Y2KINIT
      Specify DDISK, MDISK, TDISK, and IOTAPE or Y2KDISK
      Also specify BADDISK as needed
      Y2KSIZE='SMALL' or 'MEDIUM' or 'LARGE' or 'HUGE' (desired problem)
      TCODE='READ'; TMASK=63
      TPUT Y2K                        (TPUT saves adverb values)
      MOUNT the tape unless you have the disk files
      RUN Y2KEXEC
      DISMOUNT the tape or remove/backup the disk files
      PRTMSG                       (documents data read process)

   In the case of a Y2K tape, the READ mode will rewind the tape
   and skip over the data for the smaller problems (if 'SMALL'
   wasn't given).  It will then execute FITLD to read the UV and
   image files to disk.  At the end of the process, it rewinds
   the tape and prints a directory of the disk files.

   NOTE: Y2KEXEC will insist on having non-default values for
   IOTAPE, DDISK, MDISK, and TDISK.

4. Now it is time to actually perform the test.  The 'TEST' mode
   recomputes the various files.  All map results are compared to the
   master versions which were read from tape in the previous step.
   There are two ways to perform the calculations: with each step
   computed from the result of the previous test calculation in the
   sequence, or else from the result of the previous MASTER
   computation in the sequence.  The first mode (TMODE='T') tests how
   well the system under test is able to reproduce a complex sequence
   of calculations beginning from a designated starting point.  That
   is, it measures the accumulation of error in the system.  The
   second mode (TMODE='M') tests the steps of the sequence separately
   so that errors created in each step can be isolated.  It is also
   the step used for calculating the AIPSmark(2000).  Both tests are
   important, and both should be performed to certify fully the
   scientific adequacy of an AIPS installation, but it is generally
   best to use TMODE='M' as the initial (and primary) test of all
   systems.  The steps are:

      CLRMSG                       (prepare for test results)
      GET Y2KINIT
      TGET Y2K
      TCODE='TEST'; TMODE='M'; TMASK=63
      Y2KSIZE='SMALL' or 'MEDIUM' or 'LARGE'  (desired problem)
      EDGSKP=4, 8, or 16 for small, medium or large/huge Y2KSIZE
      TPUT Y2K                     (saves the adverb values)
      RUN Y2KEXEC
      PRTMSG                       (print out the test results)

   For the AIPSmark values, TERSE should be set to -1.  The message
   file from Y2KEXEC will be more than 2000 lines long.  It is
   intended to be the major item documenting the test procedures and
   results.  We usually test with messages coming to the xterm running
   the test - in other words, we kill the message server if one comes
   up.

   Making a second run with TMODE='T' is not required, but it may be
   advisable to evaluate the accumulation of error.  This process also
   checks the manipulations of visibilities, antenna files, and gain
   files; these processes are not really tested in the 'M' mode
   because AIPS currently has no comparison operators for files of
   these types.  The 'T' mode tests them by their effect on subsequent
   map results.  This is particularly important for the output of the
   self-calibration program CALIB.

   One can also rerun the tests with a different value for the
   VERSION adverb.  One can also use this option to test
   experimental versions of specific tasks.

   In the HUGE tests, the accuracy of the residual images after the
   first Clean is not as good as that of the other steps.  This is
   normal and should not be a great concern.

   Typical inputs look like
   Y2K  :  Verification/timing test; see Y2KLOAD.001, Y2KEXEC.001
   Adverbs     Values                 Comments
   ----------------------------------------------------------------
   TCODE      'TEST'                  INIT, TEST, READ, or WRIT
   TMODE      'M'                     T or M; Test or Master input
   TMASK        63                    test selection bit mask
   Y2KSIZE    'LARGE   '              SMALL, MEDIUM, LARGE test
   DDISK         2                    Disk drive #: master UVDATA
   MDISK         2                    Disk drive #: other masters
   TDISK         2                    Disk drive #: test images
   IOTAPE        1                    Input/Output tape drive #.
   Y2KDISK    'MYDDT   '              Disk area for READ and WRIT
   EDGSKP        4                    Pixels to skip at edges
   TERSE        -1                    > 0 => reduced output
   OUTPRINT   'FITS:Y2K.15Jan05                                '
                                      Printer disk file to save
   BADDISK    *all 0                  Disks to avoid
   VERSION    '                                                '
   where environment variable MYDDT is the data area used for the FITS
   disk files for DDT and Y2K data.

   A typical number of bits looks like:
   'IMAGE'           13.1018839    15.531579
   'IMBEAM'          13.8289413    16.6617813
   'APCLN'           12.7427092    16.7804108
   'APRES'           17.8541603    24.5266685
   'IMCLN'            9.6245136    16.6369858
   'VTESS'           15.2701616    23.2641563
   'VTESSC'          16.6828175    20.8412228

   The important timing lines are:
   15:18:41     AIPS      run y2ke
             ....
   15:33:42     AIPS      'PRINTING MESSAGES SUGGESTIVE OF ERROR: PRIO > 5'
   giving 15:01 seconds elapsed time (901 sec or aipsmark 44)
   The PRTAC information summary confirms
      Version:         ALL
      # tasks           20
      Tot real        14.9
      Tot cpu       10.231
      Av cp/re      .41414
   the total real time and shows that the cpu/real performance of this
   machine was not particularly good.

5. The 'WRIT' mode tests task FITAB, while 'READ' tests FITLD.  The
   basic idea is to produce a tape (or FITS disk files), and then to
   read that tape or those files on an AIPS system to verify that it
   yields correct results.  It follows that generating a new tape with
   the 'WRIT' mode and then executing 'READ' and 'TEST' on it will
   also effectively test the tape tasks.  Better yet, take the tape to
   another AIPS and run Y2K on it!  This verifies the
   intercommunication of the systems.  Obviously, use of FITS disk
   files will do little to test your tape I/O.

   Note that Y2K assumes that the tape consists of the 10 SMALL data
   files, followed by 10 MEDIUM data files, followed by 10 LARGE data
   files.  You should first run Y2K in WRIT mode for the SMALL
   problem.  If generating a tape, having done this, you may then
   (optionally) run Y2K in WRIT mode for the medium problem and,
   having done all this, you may then WRIT the LARGE problem.

   The steps are:

      CLRMSG
      GET Y2KINIT
      TGET Y2K
      Y2KSIZE='SMALL' or 'MEDIUM' or 'LARGE'  (desired problem)
      TCODE='WRIT';TMODE='T'
      TPUT Y2K
      RUN Y2KEXEC
      PRTMSG                       (print out the test results)

   Note that when we say "tape", we mean either magnetic tape in the
   usual sense, or pseudo-tape disk files.  The latter are selected by
   setting Y2KDISK to a non-blank value which is the logical name
   (environmental variable) for the disk area to be used.  The disk
   files are given a fixed and obvious pattern of names (Y2Ksnnnnnn,
   where s is S, M,or  L and nnnnnn is the image name related to the
   task name).  The LARGE test divides the UV data files into "pieces"
   as defined by FITAB.

6. Finally, one wants to know the accounting statistics from the test
   runs.  The accounting listing can be correlated with the message
   file listings in order to analyze the performance of the tested
   system.  Y2K now runs task PRTAC to produce the desired
   information.  Note that PRTAC has exclusive access to the
   accounting file until it finishes, and so various tasks may
   complain while they are waiting to get access to the file.

   In TEST mode, the listing produced by PRTAC will show only the last
   sequence of tasks suitable for the Y2K test.  In other modes, it
   will list all tasks run in the last day or two.  The columns of the
   listing give the taskname, POPS#, user#, start-time, real-time,
   CPU-time, and "IOCNT".  Asterisks mark any run that aborted.  After
   this basic sequential listing comes a listing by task name giving
   the totals accumulated by each task.  The listing is given in two
   sort orders.

================================================================
Regarding "cloning":

It is obvious that Y2KLOAD and Y2KEXEC are cloneable.  First, a
clone of Y2K could process another dataset; this would be done
by adding additional allowed string values for Y2KSIZE and
renaming the input UV data set DY2K?, where ? is the first
character in the new allowed string value.  The adverb values
needed for the new data set, are initialized in the Y2KPROC
procedure which is SAVEd in Y2KSAVE.  Second, the general
pattern of executing and testing the tasks in succession will
work for a wide variety of AIPS tasks and data types.

================================================================

            Interpretation of Test Results

The results of the mapping tests are recorded as the number of BITS of
accuracy.  The BITS of accuracy are defined in two ways, by the MAX
difference in the Master image and the Test image and by the RMS
differences in the images.  The BITS measured by the the base 2
logarithm of the ratio of PEAK and DIFFERENCE of the images.

The BITS are calculated as -3.3219 * LOG(DIFF/PEAK).  (LOG is the Base
10 Logarithm; 1/LOG(2) = 3.3219)

The expected BITS of accuracy for the tasks are in the range of 10 to
20 BITS.  The RMS BITS of accuracy is always greater than the PEAK
BITS of accuracy.

For further information on the interpretation of the results, refer to
AIPS memo 85, 104, and other memos in the AIPS memo series that relate
to the Y2K tests.  These are available via anonymous ftp and the world
wide web on the internet; ftp to ftp.aoc.nrao.edu, cd to
pub/software/aips/TEXT/PUBL/ .  The web page is
http://www.aips.nrao.edu/

AIPS