; PFT
;---------------------------------------------------------------
;! The Perley-Feigelson Test; see PFTLOAD.RUN, PFTEXEC.RUN
;# PROCEDURE OBSOLETE
;-----------------------------------------------------------------------
;;  Copyright (C) 1995
;;  Associated Universities, Inc. Washington DC, USA.
;;
;;  This program is free software; you can redistribute it and/or
;;  modify it under the terms of the GNU General Public License as
;;  published by the Free Software Foundation; either version 2 of
;;  the License, or (at your option) any later version.
;;
;;  This program is distributed in the hope that it will be useful,
;;  but WITHOUT ANY WARRANTY; without even the implied warranty of
;;  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;;  GNU General Public License for more details.
;;
;;  You should have received a copy of the GNU General Public
;;  License along with this program; if not, write to the Free
;;  Software Foundation, Inc., 675 Massachusetts Ave, Cambridge,
;;  MA 02139, USA.
;;
;;  Correspondence concerning AIPS should be addressed as follows:
;;         Internet email: aipsmail@nrao.edu.
;;         Postal address: AIPS Project Office
;;                         National Radio Astronomy Observatory
;;                         520 Edgemont Road
;;                         Charlottesville, VA 22903-2475 USA
;-----------------------------------------------------------------------
;---------------------------------------------------------------
PFT       LLLLLLLLLLLLUUUUUUUUUUUU CCCCCCCCCCCCCCCCCCCCCCCCCCCCC
PFT  :  The Perley-Feigelson Test; see PFTLOAD.RUN, PFTEXEC.RUN
TCODE                              INIT,TEST,READ, or WRIT
TMODE                              T or M
TMASK           1.0       127.0    test selection bit mask
INNAME
OUTNAME
INDISK          1.0         9.0    Input disk drive #.
IN2DISK         1.0         9.0    Input disk drive #.
OUTDISK         1.0         9.0    Output image disk drive #
INTAPE          1.0         9.0    Input tape drive #.
OUTTAPE         1.0         9.0    Output image tape drive #
BADDISK                            (1,3 good on CVAX)
VERSION
----------------------------------------------------------------
PFT
Type: RUN file.
Use:  PFT performs the Perley-Feigelson Test of an AIPS system.
      See RUN files PFTLOAD.RUN and PFTEXEC.RUN.
      The purpose of the inputs file is to perform bounds
      checks on the parameters and to enable the use of
      TPUT and TGET.
Adverbs:
 TCODE    String*4      INIT,TEST,READ, or WRIT
 TMODE    String*1      T or M
 TMASK    Scalar        test selection bit mask (1-127)
 INNAME   String*12
 OUTNAME  String*6
 INDISK   Scalar        Input disk drive #. (1-9)
 IN2DISK  Scalar        Input disk drive #. (1-9)
 OUTDISK  Scalar        Output image disk drive # (1-9)
 INTAPE   Scalar        Input tape drive #. (1-9)
 OUTTAPE  Scalar        Output image tape drive # (1-9)
 BADDISK  Array         (1,3 good on CVAX)
 VERSION  String*40     OLD,NEW,OLDPSAP,NEWPSAP,LOCAL
                        (also private dirs; use | to separate)
PFT runs in four different modes: (TCODE)
   INIT = starts from the original master UV dataset, and
           (re)computes master files (does ALLDEST).
           The "mask" TMASK controls which steps execute.
   TEST = re-does the INIT computation but stores the new
           files under different names and compares against
           masters. The "mask" TMASK controls which steps
           execute. TEST steps use either the TEST run output
           of the previous step (TMODE=T) or the INIT run
           output (TMODE=M).
   WRIT = write out all master files to   tape.
   READ = read  in  all master files from tape.
A binary mask controls which steps of the test execute:
   TMASK +1=>SORT,  +2=>UVMAP, +4=>APCLN,
         +8=>ASCAL, +16=>MXmap,  +32=>MXclean,
         +64=>VM
   For example, TMASK=54 would execute UVMAP,APCLN,MXmap,and
   MXclean.
Input data files for test steps are selected by the TMODE
   adverb which has two possible values: 'T' or 'M'.
----------------------------------------------------------------
Explanation of PFT:               (DCW, Feb/Mar85)

PFT exercises the "Dirty Dozen" of AIPS tasks using D-array
20cm data of Virgo-A which was acquired in June 1983 by Rick
Perley. This data set was used by Eric Feigelson in 1984 as a
test case for the PSAIPS project.

The PFT problem is intended to be used to certify the correct
execution of all AIPS systems. (It is small enough to run in
less than one day on all machines which will ever run AIPS.)
Its execution time is significant and easily measurable on
VAX+AP systems, and approximate timings can also be obtained
for machines 10-100x faster, like the Cray. In order to
properly test the performance of these machines which are much
faster than a VAX+AP, we will need to construct clones of PFT
with datasets having sizes of the order of 10x and 100x larger
than PFT.
=============================================================
Steps to execute the Perley-Feigelson Test on a new computer:

1. Read the first three files from the tape to the appropriate
   directories of the system. These files contain 80-character
   ASCII records (1 record per block); the three files are
   separated by tapemarks. The first file is this HELP file; it
   should be copied to HLPFIL:PFT.HLP. The second and third
   files are the RUN files which perform the test; they should
   be copied to RUNFIL:PFTLOAD.RUN and RUNFIL:PFTEXEC.RUN,
   respectively. Techniques for copying card image files from
   tape to disk vary from system to system. A collection of
   recipes appear in a later section of this file.

2. Log in to AIPS under the user number which will own the data
   files and the SAVE/GET files associated with the test. It is
   probably convenient to clear your AIPS message file before
   proceeding: PRNUM=-1;CLRMSG (or else PRTMSG)

3. Now do the POPS command "RUN PFTLOAD". This initiates the
   compilation process, which can take as long as 15 minutes
   on a heavily loaded VAX-11/780. You may want to execute
   the POPS command "PRTMSG" afterwards to document the
   compilation of the test procedures.

4. The next step is to read the data files from the tape. These
   are the 4th thru Nth files. The first one (the 4th physical
   file on the tape) is the original visibility data set of the
   PFT. Subsequent files are the "master" versions of files
   computed from the original dataset. The read process
   consists of the following steps:
      CLRMSG                       (prepare for tape read steps)
      GET PFTINIT
      Specify INDISK, IN2DISK, OUTDISK, INTAPE, and OUTTAPE.
      Also specify BADDISK as needed
      TCODE='READ'; TPUT PFT         (TPUT saves adverb values)
      MOUNT the tape
      RUN PFTEXEC
      DISMOUNT the tape
      PRTMSG                       (documents tape read process)
   The READ mode of PFT will rewind the tape and will skip over
   the three text files. It will then execute UVLOD and IMLOD
   in the appropriate pattern to read the UV and map files to
   disk. At the end of the process it rewinds the tape and
   prints a directory of the disk files.

   NOTE: PFTEXEC will insist on having non-default values for
   INTAPE, OUTTAPE, INDISK, INDISK2, and OUTDISK.

5. Now it is time to actually perform the test. The 'TEST' mode
   recomputes the various files. All map results are compared
   to the master versions which were read from tape in the
   previous step. There are two ways to perform the
   calculations: with each step computed from the result of the
   previous test calculation in the sequence, or else from the
   result of the previous MASTER computation in the sequence.
   The first mode (TMODE='T') tests how well the system under
   test is able to reproduce a complex sequence of calculations
   beginning from a designated starting point. That is, it
   measures the accumulation of error in the system. The second
   mode (TMODE='M') tests the steps of the sequence separately
   so that errors created in each step can be isolated. Both
   tests are important, and both should be performed to fully
   certify the scientific adequacy of an AIPS installation, but
   it is generally best to use TMODE='M' as the initial (and
   primary) test of all systems. The steps are:

      CLRMSG                       (prepare for test results)
      GET PFTINIT
      TGET PFT
      TCODE='TEST';TMODE='M';TMASK=127
      TPUT PFT                     (saves the adverb values)
      RUN PFTEXEC
      PRTMSG                       (print out the test results)

   The message file from PFTEXEC will be more than 2000 lines
   long. It is intended to be the major item documenting the
   test procedures and results.

   Making a second run with TMODE='T' is generally advisable to
   evaluate the accumulation of error. This process also checks
   the manipulations of visibilities, antenna files, and gain
   files; these processes are not really tested in the 'M' mode
   because AIPS currently has no comparison operators for files
   of these types. The 'T' mode tests them by their effect on
   subsequent map results.

   One can also rerun the tests with a different value for the
   VERSION adverb. Generally this is most interesting in the
   comparison of the NEWPSAP library with the AP library. One
   can also use this option to test experimental versions of
   specific tasks.

6. It is also possible to test task FITTP. The basic idea is to
   run PFT in the 'WRIT' mode to produce a tape, and then to
   read that tape on an AIPS system to verify that it yields
   correct results. Note that the combination of the 'READ' and
   'TEST' modes effectively verify the correctness of IMLOD and
   UVLOD on the current system. It follows that generating a
   new tape with the 'WRIT' mode and then executing 'READ' and
   'TEST' on it will effectively test task FITTP. Better yet,
   take the tape to another AIPS and run PFT on it! This
   verifies the intercommunication of the systems. The steps
   are:

      CLRMSG
      GET PFTINIT
      TGET PFT
      TCODE='WRIT';TMODE='T

7. Finally, one wants to know the accounting statistics from
   the test runs. The accounting listing can be correlated with
   the message file listings in order to analyze the
   performance of the tested system. It would be nice if all
   AIPS tasks would put the accounting information into the
   message file when they exit, but currently they don't.
   See the PRTACC discussion below to find out how to produce
   the desired listing.

=============================================================
How to list the accounting data after PFT runs (DCW, 19Mar85)

The accounting data is in the 'AC' file. It is listed by a
utility program called PRTACC, whose executable image is
in the "LOAD" directory (e.g. UMA0:[AIPS.15APR85.LOAD] on the
CVAX machine at this time). For VMS the operations are:

RUN LOAD:PRTACC         (using logical symbol LOAD)
SOPT                    (PRTACC opcode; must be upper case!)
                        (blanks ==> show all tasks)
0                       (0      ==> show all POPS #'s)
xxxxyyyy                (range of user #'s, can be xxxxxxxx)
850101                  (starting date yymmdd)
0.                      (minimum CPU time)
TFFF                    (flags, selects simple listing)
GO                      (prepare the commanded listing)
QUIT                    (dispose listing to printer & quit)

The listing should come out at this point. Note that PRTACC
has exclusive access to the accounting file until the QUIT is
issued, and so various tasks may complain while they are
waiting to get access to the file.

The listing produced by PRTACC will have one line for each
execution of any task or program AIPS. The columns of the
listing give the taskname, POPS#, user#, start-time, real-time,
CPU-time, and "IOCNT". Asterisks mark any run that aborted.

PRTACC is a machine independent program; this means it can be
run on all AIPS systems and will accept the same user responses
shown above, although the "RUN" command will be different. The
IOCNT field of the listing does not have the same meaning on
all machines.

============================================================
Regarding "cloning":

It is obvious that PFTLOAD and PFTEXEC are cloneable. First, a
clone of PFT could process another dataset; this would be done
by changing only a few adverb values here and there. Second,
the general pattern of executing and testing the tasks in
succession will work for a wide variety of AIPS tasks and data
types. The first step of the clone process is to substitute a
new test name string for "PFT" everywhere in both PFTLOAD and
PFTEXEC.

==============================================================
Interpretation of Test Results

This section will be added later and will discuss issues of
numerical precision. It will also address various issues
associated with benchmarking. We need some beta-testing before
this section can be constructed.
