SBGrid command line installation client

Installation

The SBGrid software stack can be installed using our command line client for Linux (CentOS/RHEL 6 or 7 recommended) or Apple OSX computers running MacOSX v10.9 - 10.12.

Three software installers are available. Binary releases are available for MacOS and CentOS7 operating systems. A python release is also provided which is compatible with Mac and most Linux (CentOS6 and higher) operating systems.

The SBGrid installation client requires an individual account. To get started, first, set up an account by registering here : SBGrid registration

Check out Jason Key's presentation on SBGrid installation tools

Pre-installation requirements

Admin access required on your computer. The SBGrid installation client will install the software tree to /opt/sbgrid and create a symlink from this directory to /programs. While admin privileges are not required to run the application, you will need admin privileges to create /opt/sbgrid and /programs .

Existing SBGrid installations must be renamed or removed. If you have previously installed SBGrid software on your computer at /programs or at /opt/sbgrid, these directories should be removed or renamed. The client will create these on first run.

Hard Drive Space Depending on what you install, ensure you have enough free hard drive space on your machine.

For large packages (Phenix, Rosetta, CCP4), a wired connection is recommended. Some packages are quite large and can be slow to download, especially over WiFi.

pip (for python whl install only, CentOS6 systems) pip is a package management system used to install and manage software packages written in Python. The software installation client is installable by pip. This may not be on you machine by default, instructions for adding it are included here.

Binary releases

Binary releases of the install client are recommended - there is minimal installation needed for these binaries and they can be run from any location with no dependencies. Untar the archive, make the file executable if necessary, and run from your preferred location.

Mac : 1.0.660, 2017-10-10

Linux : 1.0.660, 2017-10-10

$ tar -zxf sbgrid-1.0.660-Darwin.tgz

$ cd sbgrid-1.0.660-Darwin

$ ./sbgrid
usage:
sbgrid <command> [<command_args>]

Available commands are:

Query packages:
  collections     list available collections
  grids           list available grids
  info            get detailed information on a particular package
  installed       list all installed packages
  list            list available packages
  search          list and search installed packages
  updates         list available updates

Manage packages:
  install             install packages
  install-all
  rebuild             rebuild installation of all installed packages
  remove / uninstall  remove installed packages
  resume              resume interrupted installations
  update              update installed package
  update-all

Environment:
  shell    Load sbgrid environment

Installation management
  activate      activate new installation
  admin         cron mode
  autoclean     removes unused packages
  reactivate    reactivate installation using existing config and package database/list
  save          save current configuration

Additional arguments:
  -h, --help              show usage information and exit current platform
  -l, --info-versions     list available and default versions for <program> on current platform
  -L, --info-all-versions list available and default versions for <program> on all platforms
  --debug                 enable verbose logging
sbgrid: error: too few arguments

Installation using pip

On CentOS6, you can install pip from the EPEL repository (yum -y install python-pip) or as per these directions

Latest build : 1.0.660, 2017-10-10

$ sudo pip install SBGrid_Installer-1.0.660-py2-none-any.whl

Which should produce output similar to :

Unpacking ./SBGrid_Installer-1.0.660-py2-none-any.whl
Requirement already satisfied (use --upgrade to upgrade): pyasn1 in /nfs/programs-dev/i386-mac/python/2.7.5/lib/python2.7/site-packages (from SBGrid-Installer==1.0.488)
Downloading/unpacking requests (from SBGrid-Installer==1.0b2.dev412)
  Downloading requests-2.11.1.tar.gz (485kB): 485kB downloaded
  Running setup.py egg_info for package requests

    warning: no files found matching 'test_requests.py'
Installing collected packages: SBGrid-Installer, requests
  Running setup.py install for requests

    warning: no files found matching 'test_requests.py'
Successfully installed SBGrid-Installer requests
Cleaning up...

Installation on an external drive or USB key

Currently the installer application installs the software tree in /opt/sbgrid and uses a symlink at /programs. This link is required for the software to work, but in principle the software can be located anywhere and linked to /opt/sbgrid. This functionality is not yet included in the SBGrid application, but can be set up manually with a symlink.

An installation can be made on an external drive if you first do an install (into /opt/sbgrid), then move it and symlink to the new location.

Example: 128Gb USB 'key' drives are available on Amazon for 30$ or less and are a nice way to supplement the SSDs that come in Mac laptops. To install on an external drive like this, mount it on your computer. For this example, the mount is /Volumes/sbgrid_client_install. Then, using the SBGrid installer, do an initial installation on the hard drive. To make life easy, just do one application. Then close the installer application.

Next move /opt/sbgrid to the USB key at /Volumes/sbgrid_client_install. You will likely need sudo for this.

$ sudo mv /opt/sbgrid /Volumes/sbgrid_client_install/.

Now we have /Volumes/sbgrid_client_install/sbgrid. Create a symlink from /opt/sbgrid to /Volumes/sbgrid_client_install/sbgrid.

$ sudo ln -s /Volumes/sbgrid_client_install/sbgrid /opt/sbgrid

The symlink at /programs still points to /opt/sbgrid, and that is a link that points to the external location. Open the installer and add the desired applications. These will now install to the external drive.

General Usage

The SBGrid command line installation client takes subcommands and arguments to those commands to perform various tasks. These can be found in the usage info shown above. Each subcommand can also take a -h or --help argument to show more detailed usage info. For example, the 'install' subcommand can take several arguments :

$ ./sbgrid install --help
usage: sbgrid install [-h] [--all-versions] title [title ...]

Install package.

positional arguments:
title           package name or package_name@version

optional arguments:
-h, --help      show this help message and exit
--all-versions  install all versions for a package

Activation

To install the software, you must have valid credentials provided by SBGrid.

Activate with the 'activate' subcommand

usage: sbgrid activate [-h] [--multibranch] site user key

Activate new installation.

positional arguments:
  site           site name
  user           user name
  key            activation key

optional arguments:
  -h, --help     show this help message and exit
  --multibranch  enable installation for all architectures

The --multibranch option allows for installation of Linux or Mac software on server that should host both platforms, usually for access of NFS. This must be set at activation. Once set, Linux and Mac titles will be installed.

Listing available titles

The sbgrid list command will show all available titles colored by their installation status. Installed titles are green, those with available updates are blue, and titles to be removed are red. Software is listed by 'Collection'.

$ ./sbgrid list
-- Crystallography -------------------------------------------------------------------------------------------------------------------------
3dna                          eden                          pdb_redo                      solveresolve
adxv                          eiger2cbf                     pdb-validation-suite          somore
albula                        elves                         
...
-- NMR -------------------------------------------------------------------------------------------------------------------------------------
aqua                          gamma                         nmrview                       remediator
aria                          haddock                       pales                         shiftx2
...

-- Electron Microscopy ---------------------------------------------------------------------------------------------------------------------
2dx                                                      particle
ace2                                                     pft3dr
auto3dem                                                 phoelix
bfactor                                                  powerfit
bsoft                                                    priismive
burnham-brandeis-helical-package                         python-macromolecular-library
chimera                                                  refmac
crop                                                     relion
...

Getting Information about titles

Information about each title, including a description and links to documentation can be show with the sbgrid info command. See example below.

Installing and removing

Install and remove titles with the sbgrid install and sbgrid remove command. Each of these commands can take multiple titles with versions at one time.

  $ ./sbgrid info epmr
Package: EPMR

Available versions:
	* 16.01 (installed)
	* 16.04 (installed)
	* 16.07.1 (installed, default)

Grids(s): SBGrid
Collection(s): Crystallography

Description:
     a program that finds crystallographic molecular replacement solutions
using an evolutionary search algorithm. The program directly optimizes
three rotational and three positional parameters for the search model with
respect to the correlation coefficient between Fo and Fc.

Links:
	Website: http://www.epmr.info/
	Manual: http://www.epmr.info/UsersGuide.html
 $ ./sbgrid info adxv
Package: adxv

Available versions:
	* 1.9.10
	* 1.9.11
	* 1.9.12 (installed, default)
	* 1.9.7

Grids(s): SBGrid
Collection(s): Crystallography

Description:
     displays protein crystallography X-ray diffraction data. The data may
be displayed as a 2-D image, 3-D wire mesh or as integer pixel values. Data
files may be saved in tiff or postscript format.

Technical notes:
     Adxv will display data from most current detectors:    ADSC ccd  Mar
ccd  APS 3x3 ccd  Mar image plate (old and new format)  Raxis II & IV  Fuji
image plate  Crystallographic Binary Format (CBF)  XDS .pck files  Raw
binary 8, 16 & 32 bit integer data

Links:
	Website: http://www.scripps.edu/tainer/arvai/adxv.html
	Manual: http://www.scripps.edu/tainer/arvai/adxv/AdxvUserManual.pdf
  $ ./sbgrid install adxv epmr epmr@16.01
adxv@1.9.12 installed
epmr@16.07.1 installated
epmr@16.01 installed
Updating installation configuration...
When a version string is not specified, the default version is installed or removed.

Updating titles

Show titles with a pending update with the sbgrid updates command. Updates can be applied with the sbgrid update or sbgrid update-all commands.

Example : Installing Albula

$ sbgrid info albula
Package: ALBULA
Name : albula

Available versions:
* 3.2.0-2 (default)
* 3.1.0-13
* 3.0.0-14
* 2.2.12

Collection(s): Crystallography

Description:
   consists of the standalone program ALBULA VIEWER, a fast and easy-to-
use program that allows optimal visualization of PILATUS and EIGER data,
and ALBULA API, a Python programming interface for displaying images and
performing operations and calculations. The API enables you to easily
integrate the viewer functionality into your beamline infrastructure or
experimental setup.

Links:
* Website: https://www.dectris.com/Albula_Overview.html

To install:

$ sbgrid install albula
Installing: albula
Updating installation configuration...
Installation was successfull!

To get started, load the SBGrid environment :

$ source /programs/sbgrid.shrc
                 Software Support by SBGrid (www.sbgrid.org)
********************************************************************************

...

SBGrid installation last updated: 2016-10-21
Please submit bug reports and help requests to:       <bugs@sbgrid.org>  or
                                                      <http://sbgrid.org/bugs>
********************************************************************************

Example : Removing Albula

$ sbgrid remove albula

results in :

Removing: albula
Updating installation configuration...
Package removed!

Version selection

Default versions are installed by default for each application, though individual versions can be selected for each title.

To do so, include the version after the title followed the version. To see what versions are available, use the sbgrid info command. Let;s us the title epmr as an example here:

$ sbgrid info epmr
Package: EPMR
Name : epmr

Available versions:
* 16.04 (default)
* 16.01
* 15.12
* 15.04
* 13.07
* 11.02

Collection(s): Crystallography

Description:
   a program that finds crystallographic molecular replacement solutions
using an evolutionary search algorithm. The program directly optimizes
three rotational and three positional parameters for the search model with
respect to the correlation coefficient between Fo and Fc.

Links:
* Website: http://www.epmr.info/
* Manual: http://www.epmr.info/UsersGuide.html

To install version 11.02, add the version string like so:

$ sbgrid install epmr@11.02

which will produce :

Installing: epmr (11.02)
Updating installation configuration...
Installation was successfull!

Check which versions are installed with the sbgrid info command again

$ sbgrid info epmr
epPackage: EPMR
Name slug: epmr

Available versions:
* 16.04 (default)
* 16.01
* 15.12
* 15.04
* 13.07
* 11.02 (installed)

...

Confirm with the sbwhich command

$ sbwhich epmr
/programs/i386-mac/epmr/11.02/epmr

Reload the sbgrid shell to use the new version ( or set the environment variable EPMR_M=11.02 ).

Software Version Management in the SBGrid environment

If you have a single version of an application installed, explicit version selection in your environment is not needed.

In the case of multiple installed versions, the SBGrid Default version will be the version available at the terminal when it is installed, unless configured otherwise in ~/.sbgrid.conf.

In the case of multiple non-default versions installed, the latest release will be version available in your shell. If you would like to have more than one version of a title installed, and prefer to use an older release or the non-default version when a default version is installed, you must set this explicitly in your ~/.sbgrid.conf file. This is the same as in the standard SBGrid environment.

Version priority in the shell

  1. ~/.sbgrid.conf
  2. Installed single version
  3. Installed Default version
  4. Latest installed release

There is more info on how to do that here --> SBGrid version overrides

Running SBGrid software

To use the software at the terminal in bash, open a new terminal and run

$ source /programs/sbgrid.shrc

or in tcsh

$ source /programs/sbgrid.cshrc

Known Issues

  • rsync required on 873 or 8080. The application will fail if outbound port 873 and port 8080 are blocked by your institution or are otherwise not available. You can check if you can access port 873 here and port 8080 here

For help, email bugs@sbgrid.org.