Using the SBGrid Environment
Support for Site Administrators
Support for Developers
Hardware Support Notes
The SBGrid software stack can be installed using our command line client for Linux (CentOS/RHEL 6 or 7 recommended) or Apple OSX computers running MacOSX v10.11 - 10.14.
Three software installers are available. Binary releases are available for MacOS and CentOS7 operating systems. A python release is also provided which is compatible with Mac and most Linux (CentOS6 and higher) operating systems.
The SBGrid installation client requires an individual account. To get started, first, set up an account by registering here : SBGrid registration
Admin access required on your computer. The SBGrid installation client will install the software tree to /opt/sbgrid and create a symlink from this directory to /programs. While admin privileges are not required to run the application, you will need admin privileges to create /opt/sbgrid and /programs .
Existing SBGrid installations must be renamed or removed. If you have previously installed SBGrid software on your computer at /programs or at /opt/sbgrid, these directories should be removed or renamed. The client will create these on first run.
Hard Drive Space Depending on what you install, ensure you have enough free hard drive space on your machine.
For large packages (Phenix, Rosetta, CCP4), a wired connection is recommended. Some packages are quite large and can be slow to download, especially over WiFi.
pip (for python whl install only, CentOS6 systems) pip is a package management system used to install and manage software packages written in Python. The software installation client is installable by pip. This may not be on you machine by default, instructions for adding it are included here.
Binary releases of the install client are recommended - there is minimal installation needed for these binaries and they can be run from any location with no dependencies. Untar the archive, make the file executable if necessary, and run from your preferred location.
$ tar -zxf sbgrid-1.0.694-Darwin.tgz $ cd sbgrid-1.0.694-Darwin $ ./sbgrid usage: sbgrid <command> [<command_args>] Available commands are: Query packages: collections list available collections grids list available grids info get detailed information on a particular package installed list all installed packages list list available packages search list and search installed packages updates list available updates Manage packages: install install packages install-all rebuild rebuild installation of all installed packages remove / uninstall remove installed packages resume resume interrupted installations update update installed package update-all Environment: shell Load sbgrid environment Installation management activate activate new installation admin cron mode autoclean removes unused packages reactivate reactivate installation using existing config and package database/list save save current configuration Additional arguments: -h, --help show usage information and exit current platform -l, --info-versions list available and default versions for <program> on current platform -L, --info-all-versions list available and default versions for <program> on all platforms --debug enable verbose logging sbgrid: error: too few arguments
On CentOS6, you can install pip from the EPEL repository (yum -y install python-pip) or as per these directions
$ sudo pip install SBGrid_Installer-1.0.694-py2-none-any.whl
Which should produce output similar to :
Unpacking ./SBGrid_Installer-1.0.694-py2-none-any.whl Requirement already satisfied (use --upgrade to upgrade): pyasn1 in /nfs/programs-dev/i386-mac/python/2.7.5/lib/python2.7/site-packages (from SBGrid-Installer==1.0.488) Downloading/unpacking requests (from SBGrid-Installer==1.0b2.dev412) Downloading requests-2.11.1.tar.gz (485kB): 485kB downloaded Running setup.py egg_info for package requests warning: no files found matching 'test_requests.py' Installing collected packages: SBGrid-Installer, requests Running setup.py install for requests warning: no files found matching 'test_requests.py' Successfully installed SBGrid-Installer requests Cleaning up...
Currently the installer application installs the software tree in /opt/sbgrid and uses a symlink at /programs. This link is required for the software to work, but in principle the software can be located anywhere and linked to /opt/sbgrid. This functionality is not yet included in the SBGrid application, but can be set up manually with a symlink.
An installation can be made on an external drive if you first do an install (into /opt/sbgrid), then move it and symlink to the new location.
Example: 128Gb USB 'key' drives are available on Amazon for 30$ or less and are a nice way to supplement the SSDs that come in Mac laptops. To install on an external drive like this, mount it on your computer. For this example, the mount is /Volumes/sbgrid_client_install. Then, using the SBGrid installer, do an initial installation on the hard drive. To make life easy, just do one application. Then close the installer application.
Next move /opt/sbgrid to the USB key at /Volumes/sbgrid_client_install. You will likely need sudo for this.
$ sudo mv /opt/sbgrid /Volumes/sbgrid_client_install/.
Now we have /Volumes/sbgrid_client_install/sbgrid. Create a symlink from /opt/sbgrid to /Volumes/sbgrid_client_install/sbgrid.
$ sudo ln -s /Volumes/sbgrid_client_install/sbgrid /opt/sbgrid
The symlink at /programs still points to /opt/sbgrid, and that is a link that points to the external location. Open the installer and add the desired applications. These will now install to the external drive.
The SBGrid command line installation client takes subcommands and arguments to those commands to perform various tasks. These can be found in the usage info shown above. Each subcommand can also take a -h or --help argument to show more detailed usage info. For example, the 'install' subcommand can take several arguments :
$ ./sbgrid install --help usage: sbgrid install [-h] [--all-versions] title [title ...] Install package. positional arguments: title package name or package_name@version optional arguments: -h, --help show this help message and exit --all-versions install all versions for a package
To install the software, you must have valid credentials provided by SBGrid.
Activate with the 'activate' subcommand
usage: sbgrid activate [-h] [--multibranch] site user key Activate new installation. positional arguments: site site name user user name key activation key optional arguments: -h, --help show this help message and exit --multibranch enable installation for all architectures
The --multibranch option allows for installation of Linux or Mac software on server that should host both platforms, usually for access of NFS. This must be set at activation. Once set, Linux and Mac titles will be installed.
The sbgrid list command will show all available titles colored by their installation status. Installed titles are green, those with available updates are blue, and titles to be removed are red. Software is listed by 'Collection'.
$ ./sbgrid list -- Crystallography ------------------------------------------------------------------------------------------------------------------------- 3dna eden pdb_redo solveresolve adxv eiger2cbf pdb-validation-suite somore albula elves ... -- NMR ------------------------------------------------------------------------------------------------------------------------------------- aqua gamma nmrview remediator aria haddock pales shiftx2 ... -- Electron Microscopy --------------------------------------------------------------------------------------------------------------------- 2dx particle ace2 pft3dr auto3dem phoelix bfactor powerfit bsoft priismive burnham-brandeis-helical-package python-macromolecular-library chimera refmac crop relion ...
Information about each title, including a description and links to documentation can be show with the sbgrid info command. See example below.
Install and remove titles with the sbgrid install and sbgrid remove command. Each of these commands can take multiple titles with versions at one time.
$ ./sbgrid info epmr Package: EPMR Available versions: * 16.01 (installed) * 16.04 (installed) * 16.07.1 (installed, default) Grids(s): SBGrid Collection(s): Crystallography Description: a program that finds crystallographic molecular replacement solutions using an evolutionary search algorithm. The program directly optimizes three rotational and three positional parameters for the search model with respect to the correlation coefficient between Fo and Fc. Links: Website: http://www.epmr.info/ Manual: http://www.epmr.info/UsersGuide.html
$ ./sbgrid info adxv Package: adxv Available versions: * 1.9.10 * 1.9.11 * 1.9.12 (installed, default) * 1.9.7 Grids(s): SBGrid Collection(s): Crystallography Description: displays protein crystallography X-ray diffraction data. The data may be displayed as a 2-D image, 3-D wire mesh or as integer pixel values. Data files may be saved in tiff or postscript format. Technical notes: Adxv will display data from most current detectors: ADSC ccd Mar ccd APS 3x3 ccd Mar image plate (old and new format) Raxis II & IV Fuji image plate Crystallographic Binary Format (CBF) XDS .pck files Raw binary 8, 16 & 32 bit integer data Links: Website: http://www.scripps.edu/tainer/arvai/adxv.html Manual: http://www.scripps.edu/tainer/arvai/adxv/AdxvUserManual.pdf
When a version string is not specified, the default version is installed or removed.
$ ./sbgrid install adxv epmr email@example.com firstname.lastname@example.org installed email@example.com installated firstname.lastname@example.org installed Updating installation configuration...
Show titles with a pending update with the sbgrid updates command. Updates can be applied with the sbgrid update or sbgrid update-all commands.
$ sbgrid info albula
Package: ALBULA Name : albula Available versions: * 3.2.0-2 (default) * 3.1.0-13 * 3.0.0-14 * 2.2.12 Collection(s): Crystallography Description: consists of the standalone program ALBULA VIEWER, a fast and easy-to- use program that allows optimal visualization of PILATUS and EIGER data, and ALBULA API, a Python programming interface for displaying images and performing operations and calculations. The API enables you to easily integrate the viewer functionality into your beamline infrastructure or experimental setup. Links: * Website: https://www.dectris.com/Albula_Overview.html
$ sbgrid install albula
Installing: albula Updating installation configuration... Installation was successfull!
To get started, load the SBGrid environment :
$ source /programs/sbgrid.shrc
Software Support by SBGrid (www.sbgrid.org) ******************************************************************************** ... SBGrid installation last updated: 2016-10-21 Please submit bug reports and help requests to: <email@example.com> or <http://sbgrid.org/bugs> ********************************************************************************
$ sbgrid remove albula
results in :
Removing: albula Updating installation configuration... Package removed!
Default versions are installed by default for each application, though individual versions can be selected for each title.
To do so, include the version after the title followed the version. To see what versions are available, use the sbgrid info command. Let;s us the title epmr as an example here:
$ sbgrid info epmr
Package: EPMR Name : epmr Available versions: * 16.04 (default) * 16.01 * 15.12 * 15.04 * 13.07 * 11.02 Collection(s): Crystallography Description: a program that finds crystallographic molecular replacement solutions using an evolutionary search algorithm. The program directly optimizes three rotational and three positional parameters for the search model with respect to the correlation coefficient between Fo and Fc. Links: * Website: http://www.epmr.info/ * Manual: http://www.epmr.info/UsersGuide.html
To install version 11.02, add the version string like so:
$ sbgrid install firstname.lastname@example.org
which will produce :
Installing: epmr (11.02) Updating installation configuration... Installation was successfull!
Check which versions are installed with the sbgrid info command again
$ sbgrid info epmr
epPackage: EPMR Name slug: epmr Available versions: * 16.04 (default) * 16.01 * 15.12 * 15.04 * 13.07 * 11.02 (installed) ...
Confirm with the sbwhich command
$ sbwhich epmr
Reload the sbgrid shell to use the new version ( or set the environment variable EPMR_M=11.02 ).
If you have a single version of an application installed, explicit version selection in your environment is not needed.
In the case of multiple installed versions, the SBGrid Default version will be the version available at the terminal when it is installed, unless configured otherwise in ~/.sbgrid.conf.
In the case of multiple non-default versions installed, the latest release will be version available in your shell. If you would like to have more than one version of a title installed, and prefer to use an older release or the non-default version when a default version is installed, you must set this explicitly in your ~/.sbgrid.conf file. This is the same as in the standard SBGrid environment.
Version priority in the shell
There is more info on how to do that here --> SBGrid version overrides
To use the software at the terminal in bash, open a new terminal and run
$ source /programs/sbgrid.shrc
or in tcsh
$ source /programs/sbgrid.cshrc
For help, email email@example.com.
Last edited by Jason Key, 2019-02-01 22:18:51