Chapter 9: Adding Packages
      1. Introduction
      2. Where To Put Them
      3. Package Ownership and Protection
      4. Package setups
      5. Package Startups
      6. License Managers
Introduction: The NERC Package Philosophy

In order to ensure that all users are using the same version of a package, and to minimise the disk space requirement, we recommend the use of a central software server, or at least a single copy of each application in a location which can be served to all users on a site. The packages are made available to all users by means of the automounter tables auto.packages<arch>, found in /nerc/<domain>/yp.

Back to contents

Where To Put Them

Any application installed by local staff or the TSS Applications supporters should be installed, if possible, in its own physical directory in the style /local[n]/packages/<arch>/<package name>. Access to that package directory, from all machines on the network, will then be through the 'logical' pathname /nerc/packages/<package name> which triggers the automount daemon to do the appropriate NFS mounts if necessary. If you are about to install the first package for a new architecture and find that no auto.packages map exists yet for that architecture, contact Systems Group as you may need a new version of the Makefile to include instructions for the new map.

Note that some products., such as PV-Wave, are designed to be almost architecture-independent; they have only a small amount of specific code while the rest of the package is shareable. Products like this should be installed once in a directory /local[n]/packages/unix/<productname> and used for all architectures.

An exception to this is where a proprietary product such as a compiler wants to be installed in a particular place such as /opt or /usr/. We have found over the years that putting it somewhere else causes more trouble in the long run so its wishes should be acceded to.

It will be clear that the work involved in the following is non-trivial and, normally, much of it is done 'back at base' by the TSS Applications and Systems supporters before a package is "released" to users. Indeed, for any production software supported by the central groups they require that it be installed this way, to make support easier and more uniform across all sites. Even if local staff wish to install software for the site's exclusive use and support it themselves, there are still benefits to be gained from the use of the /nerc/packages/scheme. The package is visible from all workstations on the network, and with a properly written setup script the correct user environment is provided with no effort required by the users themselves beyond typing 'setup <packagename>' before invoking it.

Back to contents

Package Ownership and Protection

The majority of packages installed by the Applications Groups are owned by a specific user id and are nearly all in group 'packages' ( GID 1009). This list of uids and gids is defined by Duncan Collins and Robert Sanderson, whom see for the latest additions. See also Appendix 1 of this guide. It is important to note that there are three levels of read-write protection for a package. Firstly, the physical directory must have r-x access all the way down for the software to be visible from across the network. Write access is usually only given to the UID owning the package. Secondly, the /etc/exports file ( or equivalent) on the machine hosting the software must have an entry either for that directory or for one somewhere above it. Unless this file specifies that the package directory is to be exported read-write ( for some or all hosts) it will not be writable from across the network. The third level of protection is the automounter map; unless this specifies read-write mounting the package directory will not be writable from across the network. Any one of these levels can prevent writing; all three must be in agreement for writing to be possible. Ask the package supporter to specify the access requirements both for normal users and for the supporter's own work. The most common arrangement is for the package to be writable from the supporter's workstation ( if any) but read-only elsewhere; however some packages such as ORACLE require write access from all hosts ( but only for the ORACLE ID itself). Sometimes it is necessary for the supporter's machine to have root access and this is added to the /etc/exports file for the machine hosting the package.

With NFS, there is no reason for package software to reside on the architecture it will run on. Our normal scheme is to have a single NFS file server for all vendors and architectures. This eases system management, backup etc.

Package setups

Packages are accessed in the NERC filesystem by means of scripts known collectively as 'setups'. These define the necessary environment variables and add the package executable directories (and possibly others) to the user's PATH. If there are man files the directory containing these is added to MANPATH. They may also check for the presence of other packages and even for the correct versions of these, such as the UNIRAS requirement for the correct version of Fortran on SUNs.

The 'setup' facility relies on the presence of two files called setup and unsetup visible under /nerc/packages/<packagename>/current if the request is 'setup <packagename>'. These files create an environment for the user and put any necessary directories in his path so that he can access the package executables. If an 'old', 'test ' or 'new' version is specified this replaces 'current' in the path.

In recent installations, 'current','old' and so on are simply links to directories containing a few files including setup and unsetup. The packages themselves are in separate directories, pointed to in the paths constructed by 'setup'. Conventionally the directories containing setup and unsetup start with a small 'v'; this is recognised by the software which deals with setup and allows a list of available packages to be presented if 'setup' is run with no arguments. It also allows software version information to be gathered for the configuration database.

Another advantage of this system is that local staff can move the links at will, making a new version of software current, without touching the directories set up by applications supporters.

Older versions of packages still exist which have variations on this structure; where possible these are being brought into line.

SAS software for example is stored in the modern manner as follows

/nerc/packages/sas

Directory v611
Directory v609
Directory sas611
Directory sas609

Symbolic link current -> v611
Symbolic link old -> v609

Thus 'setup sas' will invoke sas 6.11, but 'setup old sas' will find sas609,
and if the user particularly wants 6.11 whether it is current or not he can say

'setup v611 sas'.

The setup script used without an argument gives a list of available packages, an extract from which at Wallingford is shown below. Note that some packages do not have version numbers, such as oracle and status, since they have not been set up with 'v' directories yet.

setup: The following is a list of valid setup commands...
.
.
setup [current] oracle
setup old oracle
setup test oracle
setup [current] pvwave
setup vol_V.wave pvwave
setup vol_VI.wave pvwave
setup [current] sas
setup old sas
setup v609 sas
setup v611 sas
setup [current] status
setup [current] sw
setup [current] tex
.

 

For examples of setup scripts, cd to any of the supported /nerc/packages/<packagename>/current and look at setup and unsetup.

The env_clean utility is a fundamental component of 'setup'; it manipulates colon-separated paths and is used to add or delete components of PATH, MANPATH and other environment variables. It can also remove duplicates from the given pathname.

The 'setup' command is in fact an alias which is defined (among many other things) in /nerc/etc/cshrc, and thus is available to all users except root. If you cannot work as the package owner and need to be root you will need to start a csh and "source /nerc/etc/cshrc" to get setup to work. (Root working is discouraged unless really necessary).
 

Back to contents

Using Xresources

For packages requiring Xresources to be set up, we recommend the following;

Have the Xresources ( XKeysymDB or whatever) as part of the /nerc/packages/<packagename>/current directory, say in ../current/X11/resource_files/XKeysymDB.

Include in the setup a line of the form:

(cat /nerc/packages/<packagename>/current/X11/resource_files/XKeysymDB; xrdb -query) | xrdb -load -quiet -

This will add the required Xresources to the ones the user already has - but it doesn't work for all packages, unfortunately!

Unsetup scripts should unpick as much as possible of what was put in by setup, including the X resources if possible.

Package Startups

Packages may require a startup script run each time a system boots (e.g. to start a license manager daemon) and as part of the NERC structure we provide mechanisms for this. . Here we note only that all such scripts are to be supplied by the package supporter, and put by Systems Group into the directory /nerc/bin/rc.d if they are architecture-dependent or into /nerc/etc/rc.d if they are common to all architectures. These startup files should execute a file within the package directory itself, so that the details of this may be changed by the package supporters; for example the ORACLE startup script ( called S40oracle.csh  ) consists of the single line

su - oracle -c /nerc/packages/oracle/ncs/startora

To indicate which hosts are to run this script a control file is created in the directory /nerc/<domain>/startups, having the same name as the startup file itself, ( i.e. /nerc/<domain>/startups/S40oracle.csh) containing a list of the hosts. If this control file is empty all suitable hosts on the network will run the script at boot time ( as for the script S11xdm for starting xdm on SunOS systems). These control files are maintained by local staff.

Note that the actual scripts do NOT contain any site-specific parameters such as hostnames. The intention is that these be maintained centrally by the package supporter. This is the reason for having hostnames in a separate, locally-maintained file.

Where possible a startup file should write to a log and arrange to retry if it doesn't start first time; the latest version of the ARC/INFO startup (S70arc.sh) is an example of several which now do this:

#!/bin/sh
# Start FLEXlm license manager daemon for ArcInfo software
#
if [ -f /nerc/packages/arc/current/arc.startup ]; then
su arc -c /nerc/packages/arc/current/arc.startup > /var/adm/Arc.license.log 2>&1 &
else
echo " ArcInfo lmgrd failed to start - will retry in 5 minutes"
echo " Normally this problem is because an NFS server is not ready."
echo "sh /nerc/etc/rc.d/$0 " | at -s now + 5 minutes
fi

Note also that this startup is independent of the version of ARC - this makes it possible too to conceal any architecture dependence, as shown in the startups for Networker Client V4.2.3 - see these if they are at your site.

Startups of this kind have also been found useful for other things which a workstation is required to do at boot time - for example on some sites it is necessary to add a default route for network access to the outside world, and this is done with a script S01addroute or something similar.

Back to contents

License Managers

Modern packages may use a license manager daemon which accepts calls on a specified TCP port for requests by remote hosts to invoke a package. Usually there is a limit on the number of simultaneous users of the package. It is recommended that these licence managers be run on production server machines which the site has already committed to keep up as much as possible. These will preferably be in a restricted-access, controlled environment computer room. The software ports are defined in the NIS services table which is built from the ASCII file /etc/services found on the NIS master. The port numbers and their names are currently controlled by Robert Sanderson at Wallingford.

It is a good idea to have a small script which tests that all licence managers are running after any reboot of the licence server - Systems Group can help with this.

Note that licence managers are nearly always host-specific; they cannot be moved to another system simply by editing the license.dat file! Usually the supporter can arrange a transfer with the supplier if this becomes necessary.

Packages and the operating system

In preparation for the installation the installer will need to obtain a list of requirements for the package.These may include patches to the operating system, or configuration changes to add extra shared memory or semaphores (Oracle) or additional swap space (ARC/INFO); to have these done consult Systems Support. Some packages also interface closely with others; SAS and ARC/INFO can talk to ORACLE, provided the versions are compatible - installation or upgrade of one package may have repercussions for others.

Other special requirements for Packages

Some packages also need special procedures for their ids, for example Oracle needs a passwd and group entry in the local /etc/passwd and /etc/group files since it doesn't understand NIS. It needs to be a member of two groups, and requires a LOCAL tape drive or CDROM drive, not an NFS-mounted one, nor indeed NFS-mounted disk space. In many cases the tape drive or CDROM drive will not be available onsite, but will need to be brought by the installer, together with appropriate SCSI cables, thus requiring access to an extra power point at the site and two system reboots to attach and remove the device. All this is to emphasise that installation of a package should be a properly-planned, change-controlled affair, to find any 'gotchas' well before the installer embarks on a long journey to the site.

Back to contents

Example of Typical Package Installation

In this example, we will assume that ARC is being installed on your site for use by Suns. ARC requires a large amount of disk space, the id arc to own the software, a specific 'port' defined in the NIS services table, a huge amount of swap space on each host which will run it and a license manager daemon started every time certain hosts reboot.

12. Discuss with Local Support and the package supporter suitable locations for a) the ARC software and b) the ARC licence manager. A decision on (b) will be needed before the licence can be enabled, as it requires a specific hostid.

13. Check there is enough swap space on each machine which will run arc, if there isn't enough then contact System Support.

# pstat -s Check man pstat to interpret result

 

14.. Check the existence of the arc id; if it doesn't exist then contact the application's supporter for the correct uid, gid etc. and make the entry in the NIS master passwd file (/nerc/<domain>/yp/passwd)

% ypcat passwd | grep arc

arc:xxxxxxx:1017:1009:Martin France:/nerc/packages/arc:/bin/csh

15. Check the existence of the necessary software ports in the NIS services map; if it's not there, again, get the details from the application's supporter and update the NIS master /etc/services file.

% ypcat services | grep arc

arc_license 1700/tcp # arc license manager

16. If you had to add the arc id or services then make the NIS maps.

# cd /var/yp;./make ( or ./ypmake on SG)

17. Create the home directory for arc on the agreed machine in a large enough partition; remember the users will need read and execute access through the /local[n]/packages/<arch>/<package name> directory structure. Remember, too, that there must be a suitable entry in the /etc/exports file. Some packages need read-write access during installation and configuration - check that the exports file and auto.packages<arch> files are providing this. If the package supporter will not log on to the machine where the software is stored, give read-write access in the exports file to the machine where he will log on. This may even need to be root access in some cases.

Login to the host where the physical directory will be and become root

# umask 22 Gives directories drwxr-xr-x
# cd /local/packages/sun4 Create this structure if necessary and note this is for sun4 use.
# mkdir arc
# umask 27 Protect the .cshrc etc from users.
# cp /nerc/scripts/newuser/.* arc/ Even packages may need the standard cshrc
# chown -R arc.packages arc

Note that the rwxr-xr-x privileges must pertain ALL the way down the directory, so check on /local, /local/packages and /local/packages/sun4 as well as the arc directory. Failure to do this will mean the package being invisible as 'setup' won't find it.

18. Now we have to edit the /nerc/<domain>/yp/auto.packagessun4 automounter table back on the NIS master to reflect the addition of the arc package. This should be done when it is convenient to stop and restart the automount daemon on all the Suns in the domain because adding the entry may have no effect until then.

NOTE: No package should be moved without the knowledge and consent of its supporter, as there may be implications for licence managers, hard-coded paths or sundry other things which could stop the package working if not addressed.

When moving the physical whereabouts of an existing package it is best to stop the automount daemons before updating the NIS auto.packages<arch> map.

# cd /nerc/<domain>/yp
# vi auto.packagessun4 And enter a line in the following style..
/nerc/packages/arc <host>:/local[n]/packages/$NERCARCH/arc
# cd /var/yp;./make auto.packagessun4(or ./ypmake on SG) Update the NIS map.

Now make the automount daemon reread its maps; either kill and restart it or type 'automount' as appropriate.

19. At this point the package supporter can install the application!

20. Systems Group will have obtained from the package supporter the name of a file containing instructions for starting up the ARC licence manager - e.g. S70arc.sh, as described earlier in this chapter under Package Startups. This will have been placed in /nerc/etc/rc.d ( i.e. ../master/etc/rc.d on the NERC master).

21.If this is the first ARC/INFO installation on your site you will now need to define a file also called S70arc.sh, which contains only the name of the host which is to run the ARC licence manager, and place this in /nerc/<domain>/startups. Make sure it is world-readable, so that it can be seen from the host which is to act upone it.

22. At a suitable time reboot the machine which is to run the ARC licence manager and check that this restarts correctly.

Back to contents

Nercinstalls

We have found that the installation procedures for third party software frequently require root access, sometimes just to define a few links, sometimes to copy files into the root file system. We tell vendors at every opportunity that such procedures are not good and, among other problems, are likely to lead to files being lost at the next OS upgrade. Another problem is that applications supporters would often prefer not to have the responsibilities of root access! Nonetheless, we have to live with the world as it really is and "nercinstall" is a scheme we have devised to help live with some of these problems.

The basic idea is that each "package" supporter can write one or more scripts, called XXXXX.nercinstall and placed in the package's home directory (strictly the directory which is the first field in the output from ypcat -k auto.packages$NERCARCH ). These scripts will then be run, with superuser privilege, when a workstation is installed or its OS is upgraded. An example of such a script, called nercinstall, for NAG, follows:

#!/bin/csh -f
ln -s /nerc/packages/nag/nagfl16df/libnag.a /usr/lib
 

Points for applications supporters to note when writing nercinstall scripts include:
 

Part of the script for installing a new workstation includes running any nercinstalls for packages on the site.

Back to contents



This page last updated February 18th 1998 by rfcr@itss.nerc.ac.uk