-----------------------------------------------------------------------
;  Copyright (C) 1995
;  Associated Universities, Inc. Washington DC, USA.
;
;  This program is free software; you can redistribute it and/or
;  modify it under the terms of the GNU General Public License as
;  published by the Free Software Foundation; either version 2 of
;  the License, or (at your option) any later version.
;
;  This program is distributed in the hope that it will be useful,
;  but WITHOUT ANY WARRANTY; without even the implied warranty of
;  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
;  GNU General Public License for more details.
;
;  You should have received a copy of the GNU General Public
;  License along with this program; if not, write to the Free
;  Software Foundation, Inc., 675 Massachusetts Ave, Cambridge,
;  MA 02139, USA.
;
;  Correspondence concerning AIPS should be addressed as follows:
;          Internet email: aipsmail@nrao.edu.
;          Postal address: AIPS Project Office
;                          National Radio Astronomy Observatory
;                          520 Edgemont Road
;                          Charlottesville, VA 22903-2475 USA
-----------------------------------------------------------------------


                       AIPS DDT benchmark results

                                  for

                   SUN's SPARCstation 2GX (SUN 4/75)


                           November 28, 1990


                     Mark Calabretta, Henrietta May
                 Australia Telescope National Facility



1. Introduction
---------------

   AIPS memo 65 presented DDT benchmark results for the IBM RISC system 6000
model 320 and compared them against a Convex C210.  DDT results for several
SUN systems were also presented but, as emphasised by the authors, these were
not obtained under comparable benchmarking conditions.

   On 26-27/Nov/1990 we benchmarked a SUN SPARCstation 2GX (SUN 4/75) under
conditions suitable for comparison with those of the IBM machine.  The basic
configuration was as follows

   1x 669Mb SUN0669 disk drive for AIPS with a separate drive for unix
   1x Exabyte tape drive used for installing AIPS
   32Mb memory
   Graphics accelerator (standard on the 2GX)
   SunOS 4.1.1_BETA1.0
   FORTRAN f77 v1.4 (pre-release)

   In the first instance we simply copied the complete AIPS directory tree
and DDT data files to the benchmark machine from ATNF Epping using tar.  The
executables, which were completely unoptimized, were compiled under SunOS 4.1,
f77 v1.2.  After adding the benchmark machine to the list of known AIPS hosts,
and enabling the AIPS account and AIPS TV services, we had a fully functioning
system, thereby demonstrating compatibility with the SPARCstation 1.  This
unoptimized system was used to run the small and medium DDT benchmarks.

   At the request of the SUN representative, we rebuilt the AIPS libraries
and DDT executables (INSTEP3) from scratch using the "-fast -O4" optimization
level in the f77 v1.4 FORTRAN compiler.  This proceded without problems except
for $APLNOT/DATBND,DATCAL,GACSIN, and GAININ for which the compiler complained
about non-alignment of variables in equivalence statements.  These were
recompiled without optimization.  The compiler also found an invalid integer
constant -2147483648 in $APGNOT/FITTP,IMLOD, and UVLOD.  However, these tasks
are not used in the benchmark, and the problem was ignored.  The optimized
system was again used to run the small and medium DDTs.

   The results of the optimized and unoptimized DDT are presented below,
together with those of the Convex C210 (selectively optimized) and IBM 320
(optimized) for comparison.  The importance of the optimizing compiler for
SPARC systems is immediately apparent, as it is for the Convex and, we believe,
the IBM RISC systems.  AIPS managers will therefore have to confront the
problems of the reliability of the optimizer, and the debugging of optimized
code.

   As for the benchmark timings, the ratio of the total real time taken by the
SUN to that of the IBM was 1.24 for both the small and large tests.  On the
other hand, the ratio of CPU times was relatively smaller, being 1.17 for the
small DDT, and 1.12 for the medium DDT.  This indicates that the SPARC 2 is
slightly I/O bound by comparison with the IBM.  We believe that SUN will
counter this by introducing disk striping in the near future.  However,
SPARC 2s may be disadvantaged in a networked system where they must rely on an
NFS disk server for AIPS data storage.

   The accuracy measurements quoted in AIPS memo 65 for the C210, IBM 320, and
SUN 4/60 (SPARC 1) are reproduced below, together with that of the SPARC 2.
There was no significant difference between the optimized and unoptimized
versions.  Although the results are acceptible, it is interesting to note that
the accuracy for the SPARC 2 is systematically less than for the SPARC 1.

   In subjective terms the SPARC 2 handles beautifully.  Not surprisingly, it
seems significantly faster in interactive use.  Graphics handling, which
includes scrolling and AIPS TV display (SSS), benefits noticably from the GX
accelerator.  The FORTRAN compiler is also much faster, despite the extra
overhead of code optimization.


2. Benchmark timings
--------------------

SMALL TEST
            Convex C210      IBM 320       SUN 4/75       SUN 4/75*
            real    CPU    real    CPU    real    CPU    real    CPU
UVSRT        11    6.70     14    3.98     19    5.34     16    7.80
UVDIF         7    2.77     10    2.49      6    2.47      6    4.14
CCMRG         4    1.97      2    0.91      2    0.87      2    1.12
SUBIM         3    2.42      2    1.49      2    1.32      3    1.72
COMB         16   10.17     15    6.62     22    6.44     11    7.77

UVMAP         7    5.04      9    4.97     12    6.94     16   12.45
APCLN cln    17   14.25     60   57.97     59   53.36    148  144.87
APCLN res     6    3.36      4    3.05      6    3.62     10    8.36
ASCAL        10    8.05     26   24.54     43   39.67     52   49.63
MX    map    11    7.42      8    5.47     14    8.90     23   19.37
MX    cln    36   30.82     89   82.54    109   97.13    176  168.01
VTESS        25   16.98     20   15.54     27   18.76     51   45.19

total       153  109.95    259  209.57    321  244.82    514  470.43
ratio      1.00    1.00   1.69    1.91   2.10    2.23   3.36    4.28


MEDIUM TEST
            Convex C210      IBM 320       SUN 4/75       SUN 4/75*
            real    CPU    real    CPU    real    CPU    real    CPU
UVSRT        14    9.31     21    5.87     58    8.91     54   12.55
UVDIF         8    4.38     12    4.12      7    3.83     10    6.90
CCMRG         6    3.10      3    1.60      3    1.65      4    2.44
SUBIM         5    3.84      5    3.95      7    3.10      6    4.38
COMB         22   16.33     38   15.75     77   14.14     73   18.20

UVMAP        14   10.87     26   14.22     45   22.50     64   41.15
APCLN cln    84   77.14    529  517.12    492  455.13   1403 1367.76
APCLN res    14   10.56     17   11.91     40   16.72     53   38.00
ASCAL        20   17.70     77   74.83    268  261.93    322  316.14
MX    map    22   17.42     25   17.26     55   30.23     86   65.32
MX    cln   124  112.96    611  586.12    625  570.66   1108 1058.31
VTESS        47   39.18     68   48.26     95   63.06    182  145.44

total       380  322.79   1432 1301.01   1772 1451.86   3365 3076.59
ratio      1.00    1.00   3.77    4.03   4.66    4.50   8.86    9.53


Notes:
*) The results for UVSRT are the sum of 2 runs, likewise UVDIF 2 runs,
   SUBIM 2 runs, and COMB 8 runs.
*) Real times are measured to the nearest second.
*) The results for SUN 4/75* are for the unoptimized FORTRAN compiler.
*) The ratios quoted are referred to the Convex.


3. Determination of accuracy
----------------------------

SMALL TEST:
                    Peak                      Rms
          C210  IBM   4/60  4/75     C210  IBM   4/60  4/75
UVMAP     13.0  13.1  13.3  12.4     18.8  18.2  18.8  17.9
UVBEAM    10.1  10.2   9.9   8.9     16.2  15.8  16.2  15.3
APCLN     18.8  13.6  14.1  14.1     24.1  16.4  20.7  20.7
APRES     16.9  17.0  17.0  17.0     22.2  22.4  22.4  22.3
MXMAP     12.6  12.3  12.9  11.8     18.5  18.1  18.7  17.9
MXBEAM    13.9  11.1  13.7  11.4     19.3  17.7  19.3  17.6
MXCLN     14.8  14.3  14.3  14.3     17.6  17.6  17.6  17.6
VTESS      4.1   4.1   4.1   4.1     10.9  10.9  10.9  10.8

MEDIUM TEST:
                    Peak                      Rms
          C210  IBM   4/60  4/75     C210  IBM   4/60  4/75
UVMAP     13.5  12.9  13.5  12.6     17.8  18.0  18.1  17.9
UVBEAM    14.0  13.0  13.9  12.7     17.8  18.3  18.5  18.3
APCLN     17.3  12.0  11.8  11.7     23.7  14.8  14.7  14.7
APRES     15.4  15.4  15.4  15.3     21.2  21.4  21.3  21.3
MXMAP     13.0  13.3  12.6  13.0     17.8  18.0  18.1  17.9
MXBEAM    14.3  13.0  13.9  12.2     17.8  18.4  18.5  18.2
MXCLN     10.3  10.0  10.3  10.3     14.2  14.2  14.2  14.2
VTESS      3.3   3.3   3.3   3.2     10.8  10.8  10.8  10.8

