Updating SBGrid software on your mac from your local server

If you run a machine that is frequently disconnected from the network, like a laptop, or don't want to set up an NFS-shared software installation, it is possible to install the software to the hard disk on your local machine.

There are two scripts in the programs tree that can be used to create a copy of the software on your local hard disk. The sbgrid-laptop script allows you to select individual applications from the software distribution for installation on your laptop. By default only the latest version of the selected applications will be installed. The localsync script allows you to copy an entire operating system software branch to your local machine. This script can be used on laptops, but each branch is quite large in terms of disk space usage.

Laptop Script

You can create a customized local installation of the software using the sbgrid-laptop script located at /programs/share/bin/sbgrid-laptop. This script requires you to have an SSH login to a computer that has access to an SBGrid software installation and will let you specify which packages you want to install.

sbgrid-laptop: a tool for creating a local installation of the
  SBGrid software on a laptop.

Usage: sbgrid-laptop [-ituv] [-e list|config]

  -d    Debug mode creates a log file for troubleshooting purposes.
  -e [list|config]
        Edit the software download list.
  -i    Initial set up and installation of the SBGrid software.
  -s    Re-run SSH key set up process.
  -t    Test your current configuration.
  -u    Update an existing installation using the current
        configuration.
  -v    Verbose option for the file transfer.

You need an SSH login to a machine with the SBGrid software installed
to use this script.  The script will create an SSH key and run an SSH
agent process to handle authentication for the file transfers.

Copy the script the machine you want to install the software on, and then run it in install mode: ./sbgrid-laptop -i. Follow the prompts on the screen to create your configuration file, your software installation list and to enable logins using an SSH key for authentication.

Sync Script

There is a script in the programs tree that can be used to copy the appropriate branch from a shared installation to your laptop or workstation.

Copy /programs/share/bin/localsync from the machine that has the software to your home directory on the standalone machine:

scp your.server.name:/programs/share/bin/localsync ~/

Then run the script like this:

chmod 755 ~/localsync
~/localsync [yourusername@your.server.name](mailto:yourusername@your.server.name)

The script will prompt you for your sudo (admin) password in order to create the programs directory, and then it will prompt you for your login password to the server that hosts your programs installation.

Here is an example run on my workstation:

 $ ./localsync [bene@developer.sbgrid.org](mailto:bene@developer.sbgrid.org)
Running 'sudo mkdir /programs && sudo chown bene /programs'
[sudo] password for bene:
Copying the software from the i386-mac branch on your server to your local
hard disk.  Enter your ssh login password below for [bene@developer.sbgrid.org](mailto:bene@developer.sbgrid.org):

receiving file list ...

This will copy the entire branch for your architecture to your local hard drive. For the OS X branch, this will use approximately 100 GB of disk space.

You should use a network cable for the initial download, since the download is fairly large. Subsequent runs of the 'localsync' script are differential; they will only copy new or changed files, so they can be run over a wireless connection. The script can be run at any time to make your local installation match the installation on your local server.

Sharing a Software Installation without NFS

While we recommend that sites export their software installation via NFS to their workstations, it is possible to use rsync to copy the master installation to each workstation. This method has the drawback that it requires manual intervention on the part of the local sysadmin, and updates will not be instantaneous or simultaneous for all machines.

On each client machine, you'll need an 'sbgrid' user. That user will have a passphraseless ssh key that lets it log into the machine that hosts your primary installation. Each client machine will have a cron job for that sbgrid user that runs at 2 AM that will sync the programs from the master installation to the client machine.

Here is a basic command run down. 'server1' holds the programs installation and 'client1' is what will be connecting to it.

  • Login as root/administrator on client1
  • Create the 'sbgrid' user on client1
  • Check the free space on /.
  • If there's enough free space, set up the programs directory:

    mkdir /programs chown -R sbgrid:sbgrid /programs

  • If there is not enough free space on the root volume, you'll need to find space on a local volume.
  • Login as 'sbgrid' on client1.
  • Create the passphraseless ssh key:

    ssh-keygen -d

(hit enter to confirm default file location, no passphrase, etc)

  • Copy the ssh key to server1:

    cat ~/.ssh/id_dsa.pub | ssh server1 'cat - >> ~/.ssh/authorized_keys'

  • Copy the localsync script from server1 to client1. From client1:

File not found: /programs/share/bin/localsync .

(May prompt to accept the host key. If you can't login without a password, the ssh key stuff above has failed, and you'll need to troubleshoot that.) </code> chmod 755 localsync ./localsync sbgrid@server1 </code> (This will take a while depending on the speed of the machines/network.)

  • Test the installation
    • tcsh

    source /programs/sbgrid.cshrc

    • bash

    source /programs/sbgrid.shrc

(You should get a message like this: )

                               Welcome to SBGrid!
********************************************************************************
 Your use of the applications contained in the /programs  directory constitutes
 acceptance of  the terms of the SBGrid License Agreement included  in the file
 /programs/share/LICENSE.  The applications  distributed by SBGrid are licensed
 exclusively to member laboratories of the SBGrid Consortium.

 To hush this license message, run 'touch ~/.agree2sbgrid'.
********************************************************************************
 Please submit bug reports and help requests to:       <[bugs@sbgrid.org](mailto:bugs@sbgrid.org)>  or
                                                       <[http://sbgrid.org/bugs>](http://sbgrid.org/bugs>)
 This installation last updated: 20110411
********************************************************************************
 Total time to initialize: 1 second(s)
  • Add a cronjob for the updates. As the 'sbgrid' user on client1:

    echo "0 2 * * * ~/localsync sbgrid@server1" | crontab

  • Check the cronjob:

    crontab -l

You're done!