NRAO
 

Astronomical Image Processing System

[AIPS]

Solaris binary support is ending

The support for the Solaris operating system in Socorro will be terminated at some, yet to be determined, time. The binary installation of Solaris (and of Mac PPC) is not provided in the 31DEC18 release. They are available in the 31DEC17 and earlier releases. The computer that provided AIPS compilations has died and been replaced with an older, and very slow, computer. When this computer dies, or when support is formally withdrawn, we will no longer be able to offer a binary installation for DEC Solaris computers. If you have this sort of computer, you may still take AIPS in text form and compile it locally. The code was corrected a while back to survive modern Fortran compilers and the Solaris scripts for compiling AIPS have been used for a great many years.

Warning about modern Linux operating systems

Good news: I tried compiling the LNX64 binary installation with the latest Intel compiler (18.0.0, 20170811). Everything ran well on NRAO computers, but the installation failed in Leiden without even getting to the Nan issue. The binary installation has been rebuilt, therefore, with gcc and gfortran (version 6.4). That installation works normally in Leiden. The following is left, for now, to document the issue.

Computers that ran AIPS perfectly well with the binary installation have suddenly run into mysterious issues after an operating system upgrade. These issues are discussed below. Both Leiden and Jive report that the issues go away if they compile from source using gcc and gfortran. I (Eric) use gcc version 6.3 on my machine and some of the gcc versions 5.x also worked fine (latest here was 5.4). The problem is being investigated.

One of our users had his computer "upgraded" to the Fedora OS version 25. Instantly, multiple AIPS tasks started producing NaNs scattered amongst the correct values. This happened in multiple versions of AIPS using binaries downloaded from NRAO over a long span of time. The results do depend on the inputs, but if the inputs are not changed, the results even from multiple versions of AIPS remain the same. We will attempt to investigate this further, but, for the moment, we recommend strongly that you do not install this version of Fedora. Note that AIPS worked fine under previous versions, 24 and less.

And the contagion is spredding: A second site has now reported the same problems on both Fedora 25 and 26. The RedHat 7 system in Leiden has suddenly developed the same issues after having worked well for quite some time. And now Jive has reported NaN issues with Arch Linux 4.12.12-1-ARCH kkernel 4.12.12) and Debian 9. Yet another site has reported this failure with Scientific Linux 7.4, while SL6 still works fine.

Further study in Leiden has found that computers with varied but rather modern hardware fail under both Fedora and RedHat 7, while somewhat older computers run AIPS perfectly well under both. A local compilation with gfortran 6.? was done on one of the failing Fedora machines and the resulting AIPS worked properly. I am concerned that the Intel compiler we use for 64-bit Linux is too old; it dates from January 2009. We will be looking into something more modern.

AIPS has now been tested with a Fedora 25 virtual machine and the tasks that failed in Leiden worked fine on the same data when run in Socorro. Perhaps the only issue was something wrong with some installations of Fedora 25 including the one in Leiden.

Another user has reported problems with IMAGR under Ubuntu 17.04 that went away when she reverted to Ubuntu 16.04 LTS.

New helpdesk

NRAO has instituted a "helpdesk" system similar to those at other astronomical institutions. AIPS will participate in this effort which is intended to avoid having problems fall between the cracks. The helpdesk is accessed via https://help.nrao.edu. Note that the help site will forward you to my.nrao to create a login if you do not already have one and the my.nrao site will then offer a tab to reach the helpdesk. The system for the AIPS "department" will make use of the daip@nrao.edu list manager system that we have used for years. Thus, unlike the rest of the NRAO helpdesk, messages to the AIPS portion will be made public in the daip archive. This archive is accessible via the web and is well known to Google among other things. Please switch to the new helpdesk to allow more systematic handling of your AIPS queries and bug reports.

Scheduled Releases

The development version 31DEC18 is available, the previous development version 31DEC17 is "slushy" (except for patches), and the old frozen version 31DEC16 may not be available for installation much longer. (A MNJ on 31DEC16, if it has not been run since July 11, 2017, might still be useful.) The new cvs form of the Midnight Job has been very easy to use and effective in keeping numerous sites up to date. In 2018, the MNJ will work sometimes on 31DEC17 if there has been a released patch. The 31DEC18 and 31DEC17 releases are also available as binary installations (and MNJs). The 31DEC16 and later MNJs no longer require cvs for binary installations, just rsync.

TeK and message server problems

Users have encountered problems getting the TeK and message servers to work in binary installations. These servers are programs which run in an xterm and that xterm must come up with some knowledge of AIPS libraries. This requires users to modify their login scripts (e.g. .cshrc file for c shell, .bashrc file for bash). See for details.

Linux compiler troubles

There appear to be problems with many of the versions of the gcc/g77/gfortran compiler suites. GNU compiler versions 2.* and 3.* are now obsolete and the performance under g77 left much to be desired. The 31DEC15 version of AIPS has received attention so that it may be compiled under gfortran/gcc. Versions 4.4.7, 4.8.3, 4.9.2, and 5.1.0 have been tested and appear to work. Versions 4.8 and 4.9 require changes to make the pseudo-AP fixed, rather than dynamic, memory and to reduce the optimization of APCLN. See the 31DEC18 release instructions for the specifics of these changes. Versions 5.1.0 and 5.3.0 also appear to work and no longer have the issues with the pseudo-AP.


WHAT IS AIPS?

Also, see the Users' FAQ: answers to Frequently Asked Questions, the AIPS Managers' FAQ: answers to AIPS Managers (and installers) Frequent Questions; and our page on who is in the AIPS group. The AIPS Manager FAQ page is especially helpful if you are having trouble getting the AIPS TV to run properly or are encountering other general difficulties or wish to configure new machines and disks in your AIPS environment.

THE CURRENT RELEASE

  • 31DEC18 is the new primary development version of AIPS. It is ready for installation.
  • 31DEC17 is the previous development version of AIPS. It is available for installation. The 6-month progress report on 31DEC17 is available as 31DEC17 AIPSLetter dated 30 June 2017 (PS, 902569 bytes). It is also available gzipped, (419491 bytes) and PDF, (159212 bytes).
  • 31DEC16 is the previous development version of AIPS. It will be updated regulary in December 2016 and occasionally in 2017 for patches. It is available for installation. The final report on 31DEC15 is available as 31DEC16 AIPSLetter dated 31 December 2016 (PS, 957655 bytes). It is also available gzipped, (431102 bytes) and PDF, (185628 bytes). The 6-month progress report on 31DEC16 is available as 31DEC16 AIPSLetter dated 30 June 2016 (PS, 907865 bytes). It is also available gzipped, (414726 bytes) and PDF, (156135 bytes).
  • 31DEC15 is now obsolete. It will no longer be updated. It is available for installation for a limited time. It is the final version for which the Mac binaries will work on 32-bit computers. The final report on 31DEC15 is available as 31DEC15 AIPSLetter dated 31 December 2015 (PS, 1001941 bytes). It is also available gzipped, (451446 bytes) and PDF, (203207 bytes). The 6-month progress report on 31DEC15 is available as 31DEC15 AIPSLetter dated 30 June 2015 (PS, 914635 bytes). It is also available gzipped, (417682 bytes) and PDF, (169427 bytes).
  • 31DEC14 will be available for installation only in 2015, after which it will no longer be available. The last patch top 31DEC14 was applied September 14, 2015. The final report on 31DEC14 is available as 31DEC14 AIPSLetter dated 31 December 2014 (PS, 916765 bytes). It is also available gzipped, (425794 bytes) and PDF, (160987 bytes). The 6-month progress report on 31DEC14 is available as 31DEC14 AIPSLetter dated 30 June 2014 (PS, 765430 bytes). It is also available gzipped, (329413 bytes) and PDF, (167320 bytes).
  • 31DEC06 and later releases contain a port to the MacIntosh OS/X system running on Intel cpus. This includes a full binary installation produced with Intel's compiler. An AIPSMark of 150 was obtained on our iMac which will be used to support this "new" architecture.
  • Since the end of January 2009, Linux systems running on 64-bit hardware with 64-bit operating systems have the choice of using LNX64 rather than LINUX as their AIPS architecture. The binary version for LNX64 is compiled with a recent 64-bit Intel compiler and runs somewhat faster than the LINUX version run on the same machines.
  • Since September 1, 2005 the binary versions for Linux are compiled with the Intel compiler version 9.0. They are compiled and linked with options that make the load modules rather large but allow them to run as well as possible on the latest threaded Pentium IVs, older Pentium IVs, and on general machines such as AMDs. Since November 30, 2004, binary installations for MacIntosh OS/X (IBM xlf compiler), Solaris Ultra (SUN compiler), and Linux (Intel 9.0) have been available. They are on the 31DEC05 and later versions. At present, all versions provide considerable advantages over systems done with free compilers.
  • If you own a MacIntosh OS X computer, you may now run AIPS. Get either of the above versions; all contain the new MACPPC and MACINT ports. Note that the somewhat expensive IBM and Intel compilers produce code which runs about 50% faster than the code produced by the GNU compiler. Binaries compiled with the IBM and Intel compilers may be installed and updated from our ftp site. That site is maintained by our Mac computers in Socorro. Read also the special Mac installation instructions which are supplements to the A Guide to the Install Wizard instructions.
  • When you get 31DEC17 or 31DEC16, use the new install wizard; we have a good on-line guide that will help you through it. Read it!
  • The 15OCT99, 31DEC00, 31DEC01, 31DEC02, 31DEC03 31DEC04 31DEC05 31DEC06 31DEC07 31DEC08 31DEC09 31DEC10 31DEC11 31DEC12 31DEC13 31DEC14 releases of AIPS are no longer available; the number of bug fixes and differences between them and the 31DEC17 version have become too great. The continued significant demand for a frozen, even if obsolete, version was the main reason we decided to resume regular releases.

TWO VLBI TUTORIALS

Two extensive tutorials on VLBI data reduction in AIPS have been prepared, complete with data sets, introductory material about AIPS, and detailed instructions. They are simple VLBA project including self-calibration and spectral-line VLBA project plus astrometry. Users new to, or rusty in, VLBI data reduction are encouraged to try these tutorials. Appendix C and Chapter 9 of the AIPS CookBook are also recommended.

PROBLEMS AND WARNINGS

There are sometimes considerations which users of older versions of AIPS must take into account due to changes in the latest version. In addition, there are sometimes bugs introduced into the latest version which then get repaired in that version. Such matters may affect those users who keep their AIPS up to date. These bugs often do not get reported in the AIPSLetter since they were temporary but might be of concern for diligent users. They are reported in CHANGE.DOC but those reports are sometimes fragmentary and confusing. We have begun a new web page for problems of this sort. It is hoped that we will keep it up to date.

PATCHES

You really should be getting 31DEC17 or 31DEC18 instead of patching an older version!

With previous releases, we did not change the frozen release tar ball; user sites were required to download the text files and do any compilations themselves. But, beginning with 31DEC04, we provide binary releases for which user sites are not required to have compilers. Therefore, we have reversed our policy. These patches have been applied to the official code of 31DEC17 and may be downloaded using a 31DEC17 "Midnight Job". Any tarball taken after the change date listed below will already have the patches applied. Of course, you may still download and compile the files as before.

There are no patches for 31DEC17 as yet.
  1. 2018-??-??
There are fifteen 31DEC16 patches.
  1. POSSM would not label correlation functions in seconds. 2017-01-04
  2. ATLOD would not open disk files properly. 2017-01-06
  3. CCRES did not honor blanked pixels 2017-01-12
  4. DFTPL did not use the correct frequencies when BIF > 1. 2017-01-31
  5. DBCON scaled uvw wrongly when combining data sets of different frequencies. 2017-02-24
  6. DFTPL did not address data correctly when the input had more than one Stokes. 2017-02-28
  7. BPASS did not normalize linear polarization bandpasses correctly for normalization type 1. 2017-03-08
  8. Automatic spectral index finding from the SU table had problems when one or more IFs had no flux value. 2017-03-21
  9. UVMOD did not handle frequencies correctly with spectral index 2017-03-21
  10. SPIXR did not label the output images with the frequency for which they were determined (1 GHz). 2017-03-21
  11. PCAL miscomputed the flux at 1 GHz for known low-frequency sources. 2017-06-06
  12. SQASH mishandled BLC, TRC 2017-06-21
  13. UVLSF copied tables twice to the output continuum data set. 2017-06-30
  14. SCMAP and SCIMG messed up the self-cal step when the number of IFs was not the same as the number of polarizations. 2017-07-03
  15. PCAL left a file open when an IF had no data in spectral mode, causing failure on the next IF. 2017-07-11
There are three messy and nine normal 31DEC15 patch so far:
  1. GC tables used to allow 200 values of gain curve, so restored this limit used by some correlators. 2016-01-07, 2016-01-15
  2. PBEAM scaled Stokes I data incorrectly when adding right and left data files. 2016-01-12
  3. DBCON re-instated flagged table rows. 2016-01-28
  4. DTSUM did not handle new internal format correctly. 2016-02-08
  5. PCAL typo caused errors in antennas used with the new ANTENNA1, ANTENNA2 format. 2016-04-15
  6. PCAL lost Faraday Rotation calibration when doing SPECTRAL solutions. 2016-04-29
  7. IMAGR found automatic boxes in only part of the image when IMSIZE(2) < IMSIZE(1)
  8. OOP editing had trouble with source number zero sometimes found in tables. 2016-05-19
  9. UVFIX used the actual observing frequency rather than the one in the header to scale UVW. 2016-06-20
  10. OOSUB and other model subtraction/division could get the scaling between frequency channels wrong. 2016-06-24
  11. UVLOD, FITLD did not find the subarray value accurately enough. 2016-08-25
  12. SPLIT did not write the correct header frequency when averaging IFs with BIF greater than 1. 2016-08-26
  13. FITLD and UVLOD did not test data flagging correctly when writing and reading UV tables in compressed format. 2016-08-31
There were nineteen 31DEC14 patches:
  1. DOFARS help and procedure contained OUT2NAME et al. left over from when FARS used those adverbs. 2015-01-08
  2. RMFIT failed to copy the FQ table to the output residual images. 2015-01-13
  3. FTFLG messed up antenna numbers in the output FG table whena single antena or baseline was used 2015-01-16
  4. VLBATECR had trouble around the endof the year deciding what files needed to be downloaded. 2015-01-19
  5. VLBARUN requires convert to make the html output files. Added tests and a control on DOTV. 2015-01-19
  6. BPASS had an array and history writing code which could not handle more than 32 antennas. 2015-01-20
  7. MORIF messed up bandpass and other spectral tables. 2015-01-29
  8. TYSMO did not clip bad values without any median-window editing. 2015-02-11
  9. BPASS did not understand that low-level VLB routines expected left-handed antenna coordinates. 2015-02-21
  10. UV data disk I/O had an issue if the full file fit into the first of the two buffers and the disk file was exactly the right size! 2015-04-24
  11. START_AIPS, START_TVSERVERS needed a grammar change to support Mac yosemite systems 2015-04-24
  12. Basic calibration subroutines mis-computed the channel wavelengths and did not always do dispersion correction 2015-05-19
  13. XAS needed a change to support Mac yosemite systems and also to write a black character background 2015-05-20
  14. KRING help file adverb list did not match that of the Fortran. 2015-06-17
  15. IMAGR did not do the baseline-based time averaging properly in the last baselines written to the work file 2015-06-22
  16. PCAL did not detect fully flagged channels when using a source model and so died. 2015-08-14
  17. SETJY had bad formats capable of aborting the task 2015-08-22
  18. SAD and TVSAD hid the new model-fit table when making a stars table. 2015-09-07
  19. PCAL did not do mode SPECTRAL false correctly, leaving files open and then quitting. 2015-09-14

CONTRIBUTED CODE

With the advent of the binary versions of AIPS, it has become more difficult to compile special local tasks. If one wants to do this and also take advantage of the improved performance of the NRAO-purchased compilers (Intel for LINUX and MACINT; GNU for LNX64), the user must purchase the compiler(s) for himself. This can be rather expensive. There are two ways out of this bind. If the local task is of sufficient interest, you could try to pursuade the AIPS group to include it in the general AIPS distribution. This requires the code to be written up to a least some minimal standard. A recent example is ATLOD which translates older Autralia Telescope data into AIPS. The alternative is to send the code and help file to the AIPS group and ask them to compile the routines, leaving the results on the AIPS ftp site. This has now been done for 4 tasks: HAP and ORFIT in a CfA directory and SETVY and CALIV in a WSRT directory. Both of these site directories are located in a CONTRIB directory at the main AIPS public ftp site.

The load modules are stored under the name task.EXE.arch to identify the operating system (arch) to which the load module corresponds. To install them in your system, copy the help file (task.HLP) to $TST/HELP/ and copy the appropriate load module to $TST/$ARCH/LOAD/task.EXE. Note that one has to rename the load module dropping the architecture string. For LNX64 machines two load modules are provided, one compiled with the gnu compiler and one with the intel compiler. In general, for a binary installation, you will want the intel version.

THE MIDNIGHT JOB

... will be of interest to you if you want the latest 31DEC17 and 31DEC16 versions updated nightly, weekly, or occasionally. It can do either text or binary updates depending on which sort you did with install.pl.

NVSS and WENSS/WISH Survey files for AIPS

The WENSS/WISH files have been corrected (2 July 2004) to omit the sources which are the sum of other "components" also appearing in the lists. Note that the survey files shipped with AIPS releases include the shorter lists of the stronger sources. The full lists are only available from this ftp site.

DOCUMENTATION

AIPS Memos, Reports, old AIPSLetters, the Cookbook, Going AIPS, access to the newest versions of the HELP files, search CHANGE.DOC and more. The most recent AIPS Memos are:

  • number 122, discussing the usage of AGAUS and ZAMAN spectral absorption modeling tasks, dated March 7, 2017,
  • number 121, discussing the usage of the UFLAG gridded data flagging task, dated September 12, 2016,
  • number 120, discussing the usage of the TVSPC data exploration task, revision dated July 22, 2017,
  • number 119, discussing the usage of the TVSAD Gaussian modeling task, dated December 15, 2014,
  • number 118, discussing the usage of XGAUS, ZEMAN, and RMFIT spectral modeling tasks revision dated November 8, 2017,
  • number 117, discussing the detailed FITS format used by AIPS, revision dated February 16, 2016,
  • number 116, discussing new RFI mitigation options in AIPS, dated June 2010,
  • number 115, discussing new auto-boxing for Clean capabilites in AIPS, dated June 29, 2009,
  • number 114, re-stating the FITS-IDI convention with corrections and added tables, revision dated February 16, 2016,
  • number 113, discussing a "new" geometric way to handle the W problem, dated May 22, 2009,
  • number 112, discussing the AIPS pipeline for VLA data reduction, dated March 19, 2007,
  • number 111, discussing phase referencing with more than one calibrator using ATMCA, dated January 6, 2005,
  • number 110, discussing strategy for removing troposppheric and clock errors using DELZN, revised October 21, 2009, dated August 31, 2004,
  • number 109, discussing AIPS on DVDs and read-only data systems, dated January 20, 2004,
  • number 108, discussing weights for VLA data, dated January 21, 2003,
  • number 107, discussing the differences between FRING and KRING, dated April 8, 2002,
  • number 106, discussing the Brandeis scheme for making movies in AIPS dated June 6, 2001,
  • number 105, discussing the new RUN file to simplify VLBA data reduction dated February 28, 2001 and April 26, 2001,
  • number 104, discussing the new Y2K performance test dated September 8, 2000, and
  • number 103, discussing data weighting in AIPS dated 2000-03-21.

MAIL GROUPS

The AIPS group maintains three e-mail exploder lists. The first, called daip, is restricted to immediate members of the AIPS group and a few NRAO scientist advisors. The other two are, however, open to any suitable person. Membership is moderated, of course, because SPAM has become so annoying.

The group intended for discussion of AIPS issues is called bananas and is used, among other things, to announce patches to the "frozen" versions of AIPS. Traffic is light. You can subscribe yourself to this list on-line.

The group intended for discussion of problems related to the "Midnight Job" is called mnj and is used to announce AIPS Manager things related to the current MNJ. Traffic is light, but the announcements can have a serious impact on any site attempting to stay current with the development version of AIPS. You can subscribe yourself to this list on-line.

FITS-IDI

The FITS Interferometry Data Interchange format is a set of FITS binary table definitions and a set of conventions for using them to transport radio interferometry data between different software packages.

The FITS Interferometry Data Interchange Convention (formerly Format) has been accepted as an IAU-acknowledged convention. This convention was documented by Chris Flatters in AIPS Memo 102 which, unfortunately, was not accompanied by an editable text version. Therefore, Eric Greisen had to re-type the document. While doing so, he updated some matters, corrected minor wording issues, and added 5 new sections for 5 new tables, 3 of which are actually in use. Interested parties should review AIPS Memo 114 revised (ps) or AIPS Memo 114 revised (pdf); it has been submitted to the IAU FITS Committee. Greisen has attempted to highlight the changed parts in red in this draft. The current draft is dated August 2011 and includes the MODEL_COMPS table and comments from Lorant Sjowerman, Bill Cotton, and others. Please e-mail egreisen@nrao.edu with comments.

AIPS MANAGER/INSTALLER FAQ

Help for the weary AIPS Installer. Very much experimental and incomplete, and probably needs updating, but better than nothing.

BENCHMARKS (the "DDT" and now the "Y2K")

The AIPS Benchmark was known as the "DDT" (Dirty Dozen Tasks); we have results for a variety of systems. A more modern test called "Y2K" has been developed in 2000 to test faster computers and to use IMAGR rather than MX. For 31DEC04 we have new master images and data sets. In addition, we have a new HUGE test using multi-resolution IMAGR on Cygnus A. Changes to CALIB and VTESS in particular forced this update, while faster computers make Y2K LARGE run in as little as 5 minutes.

DESIGNATED AIP

The former designated-AIP-of-the-week program; rotation has been halted due to the lack of personnel. The e-mail address (daip@nrao.edu) was the best method of reaching all the members of the AIPS Group and will still work. However, we ask you to use the new NRAO-wide helpdesk instead. Go to https://help.nrao.edu and login with your my.nrao.edu username and password. There you may peruse the "knowledge base" (currently empty), enter new tickets in any of several departments including AIPS, and review and update any tickets you have submitted.

AIPS COPYRIGHT

AIPS is copyrighted by Associated Universities, Inc. using the GNU copright form. The full text is available with every AIPS release and may be read here.

RELATED SOFTWARE


FITS

This is the Flexible Image Transport System, and is the standard data interchange format used by most Astronomical Software, including AIPS. Follow the above reference for a very extensive set of documentataion, including a large resource list for Astronomy on the internet.

HELP!

The primary AIPS address is the daip one referenced on our main Contact page, and general queries, requests for distribution tapes, documents, ftp access, and so on should be directed there. Application-oriented questions should be directed to the Designated AIP.


Modified on $Date: 2017/12/12 17:34:23 $
Eric W. Greisen