initial pages commit

+[start]

+* [Software Help](help.md)

+* [New Software Requests](requests.md)

+* [Supported Operating Systems](operatingsystems)

+* [Overriding Default Software Versions](overides.md)

+* [Install The Software On Your Laptop](laoptopinstall.md)

+* [Software Installation](installation.md)

+* [Preparing Workstations to run the Software](workstations_setup.md)

+* [Managing your Installation](managing.md)

+* [Administrator Software Version Overrides](admin_overrides.md)

+* [SBGrid CentOS/Fedora Mirrors](mirrors.md)

+* [Admins Mailing List](mailinglist.md)

+* [Resources for Developers](developers.md)

+* [Build and Test Network](build.md)

+* [SBGrid Recommended Hardware](hardware.md)

+* [Setup Stereo 3D](stereo.md)

+====== Administrator Software Blacklist ======

+Site administrators can prevent the shell initialization files from configuring software on a host by creating an empty file at **/programs/local/conf/disabled-hosts/<hostname>**, where **<hostname>** is the output of the //hostname -s// command. This is useful for encouraging users not to run jobs on bastion hosts, file servers, etc.

+

+====== Administrator Software Version Overrides ======

+Most of the software titles installed in the SBGrid Suite have several versions available. Typically, we try to provide at least three versions of each software package, although very old or very new applications may have fewer versions.

+

+There are two methods for site admins to configure version overrides using files placed in a **/programs/local/conf/** directory.

+

+===== Site-wide Version Overrides =====

+A file named **site.conf** will be read when any SBGrid shell is initialized from the installation.

+

+===== Per-host Version Overrides =====

+A file named **<hostname.conf>**, where **<hostname>** is the output of the //hostname -s// command, will be read after the site.conf on the host matching that hostname.

+

+===== Precedence =====

+There are three files that will be examined for version overrides during shell initialization. In order from lowest to highest precedence:

+

+ * /programs/local/conf/site.conf

+ * /programs/local/conf/<hostname.conf>

+ * $HOME/.sbgrid.conf

+

+The overrides are reported during shell initialization:

+

+<code>

+ Configuration options in effect for this shell:

+ Variable Setting Configuration

+ SPIDER_L 18.10 site

+ CNS_L 1.21 machine

+ CCP4_L 6.1.2 personal

+</code>

+

+See [[Usage::Versions#Overriding Default Versions|this page]] for details on the override syntax.

+====== Software Help ======

+===== Getting Software Help =====

+

+If you are having a problem with software included in the SBGrid suite or think you've discovered an application bug, please use the [[http://sbgrid.org/bugs|SBGrid Software Bug Report]] to alert us to the problem. You can also email bugs@sbgrid.org with details on the problem, but the web form is the preferred method.

+

+If the problem with the software started after an update, you can configure your SBGrid environment to [[usage::versions|use an older version of the software]].

+

+===== New Software and Software Updates =====

+To request a new software package or an update to an existing package, please use the [[http://sbgrid.org/req|Software Update/Installation Request form]].

+

+We try to accommodate most requests for new software, but we do occasionally run into difficulties. Despite most of our software being free for academics, some developers insist that each lab download and install their own software. We are happy to provide developers with the list of SBGrid members that have access to their software, but that doesn't always satisfy their requirements.

+

+Even when things go smoothly, license requests and working out licensing details with developers can take some time, so please don't wait until the last minute before requesting software you anticipate you may need for your project.

+

+===== Newsletter =====

+SBGrid sends a monthly newsletter with information concerning software updates, upcoming webinars, talks and other items of interest to our community. The archive of previous newsletters is available online at the [[http://sbgrid.org/news/newsletters|SBGrid Newsletter Archive]].

+======Managing Your Software Installation======

+====='sbgrid-admin' Utility=====

+The //sbgrid-admin// utility used to install the software has a few other uses. There is a download link in the initial site set up email, and it is included in the installation itself at ///programs/share/sbgrid/bin/sbgrid-admin//. It includes built-in help:

+<code>

+ $ sbgrid-admin -h

+sbgrid-admin: a tool for administering an SBGrid Software suite installation

+

+Usage: sbgrid-admin [-chirstu]

+

+ -c Cron mode. Used for automated updates of the software suite.

+ -h Help message.

+ -i Install the software suite on this machine.

+ -p Print a launchd plist for updating the software on OS X machines.

+ -r Regenerate sh/csh shell configuration files.

+ -s Install the software branch for SGI IRIX.

+ -t Test the configuration file.

+ -u Force manual update of an existing installation.

+</code>

+

+The most commonly used option is //sbgrid-admin -u// to force a manual update. This will synchronize your local installation with the SBGrid master installation hosted at Harvard Medical School.

+

+The SBGrid software configuration files use both static and dynamically generated files. The main shell initialization files, **/programs/sbgrid.cshrc** and **/programs/sbgrid.shrc** are static files, but the operating-system specific configurations are dynamically generated and customized for the packages available at your site. //sbgrid-admin -r// will regenerate the branch-specific configuration files should you accidentally delete or alter them.

+

+===== Configuration Options =====

+The //sbgrid-admin// script has some additional configuration options to accommodate the installation requirements at different sites.

+

+The ~sbgrid/.sbgridrc configuration file for the //sbgrid-admin// script must contain a sitename and site key. All other configurations are optional.

+

+Options take the form of //OPTION_NAME="setting"//.

+

+|**Option**|**Example**|**Description**|

+|SB_SITENAME|sbgrid-bene|The site name for your installation provided by SBGrid|

+|SB_SITEKEY|xyxyxyxyxyxyxy|The installation key for your site provided by SBGrid|

+|INSTALL_TARGET|/nfs/programs|The installation directory for the software|

+|RSYNC_ARGS|"%%--%%port 8080"|Advanced arguments to rsync. Can be used to workaround blocked rsync ports.|

+|HTTP_PROXY|host.name:port|Proxy hostname and port for use by rsync and curl|

+|POST_UPDATE_SCRIPT|/absolute/path/to/script|Optional script to be run after a successful software update|

+|CRON_SCHEDULE|5 1 * * *|Set a custom cron schedule|

+\\

+

+===== Rsync Transaction Logging =====

+If you want a date-stamped log of all rsync downloads, deletions and updates, you can add a line like this to the ~sbgrid/.sbgridrc file:

+

+<code>

+RSYNC_ARGS="--log-file=$HOME/logs/sbgrid-admin-`date +%Y%m%d`.log"

+</code>

+

+This will create a date-stamped log file any time an actual change to the software tree happens. In general this should not be necessary, but for troubleshooting purposes it can be useful.

+====== Copying the Software Installation ======

+If you run a machine that is frequently disconnected from the network, like a laptop, or don't want to set up an NFS-shared software installation, it is possible to install the software to the hard disk on your local machine.

+

+There are two scripts in the programs tree that can be used to create a copy of the software on your local hard disk. The ''sbgrid-laptop'' script allows you to select individual applications from the software distribution for installation on your laptop. By default only the latest version of the selected applications will be installed. The ''localsync'' script allows you to copy an entire operating system software branch to your local machine. This script can be used on laptops, but each branch is quite large in terms of disk space usage.

+

+======= Laptop Script ======

+You can create a customized local installation of the software using the ''sbgrid-laptop'' script located at ''/programs/share/bin/sbgrid-laptop''. This script requires you to have an SSH login to a computer that has access to an SBGrid software installation and will let you specify which packages you want to install.

+

+<code>

+sbgrid-laptop: a tool for creating a local installation of the

+ SBGrid software on a laptop.

+

+Usage: sbgrid-laptop [-ituv] [-e list|config]

+

+ -d Debug mode creates a log file for troubleshooting purposes.

+ -e [list|config]

+ Edit the software download list.

+ -i Initial set up and installation of the SBGrid software.

+ -s Re-run SSH key set up process.

+ -t Test your current configuration.

+ -u Update an existing installation using the current

+ configuration.

+ -v Verbose option for the file transfer.

+

+You need an SSH login to a machine with the SBGrid software installed

+to use this script. The script will create an SSH key and run an SSH

+agent process to handle authentication for the file transfers.

+</code>

+

+Copy the script the machine you want to install the software on, and then run it in install mode: ''./sbgrid-laptop -i''. Follow the prompts on the screen to create your configuration file, your software installation list and to enable logins using an SSH key for authentication.

+

+======= Sync Script ======

+There is a script in the programs tree that can be used to copy the appropriate branch from a shared installation to your laptop or workstation.

+

+Copy /programs/share/bin/localsync from the machine that has the software to your home directory on the standalone machine:

+

+<code>scp your.server.name:/programs/share/bin/localsync ~/</code>

+

+Then run the script like this:

+

+<code>

+chmod 755 ~/localsync

+~/localsync yourusername@your.server.name

+</code>

+

+The script will prompt you for your sudo (admin) password in order to create the programs directory, and then it will prompt you for your login password to the server that hosts your programs installation.

+

+Here is an example run on my workstation:

+

+<code>

+ $ ./localsync bene@developer.sbgrid.org

+Running 'sudo mkdir /programs && sudo chown bene /programs'

+[sudo] password for bene:

+Copying the software from the i386-mac branch on your server to your local

+hard disk. Enter your ssh login password below for bene@developer.sbgrid.org:

+

+receiving file list ...

+</code>

+

+This will copy the entire branch for your architecture to your local hard drive. For the OS X branch, this will use approximately 100 GB of disk space.

+

+You should use a network cable for the initial download, since the download is fairly large. Subsequent runs of the 'localsync' script are differential; they will only copy new or changed files, so they can be run over a wireless connection. The script can be run at any time to make your local installation match the installation on your local server.

+

+====== Sharing a Software Installation without NFS ======

+While we recommend that sites export their software installation via NFS to their workstations, it is possible to use rsync to copy the master installation to each workstation. This method has the drawback that it requires manual intervention on the part of the local sysadmin, and updates will not be instantaneous or simultaneous for all machines.

+

+On each client machine, you'll need an 'sbgrid' user. That user will have a passphraseless ssh key that lets it log into the machine that hosts your primary installation. Each client machine will have a cron job for that sbgrid user that runs at 2 AM that will sync the programs from the master installation to the client machine.

+

+Here is a basic command run down. 'server1' holds the programs installation and 'client1' is what will be connecting to it.

+

+ * Login as root/administrator on client1

+ * Create the 'sbgrid' user on client1

+ * Check the free space on /.

+ * If there's enough free space, set up the programs directory:

+<code>

+mkdir /programs

+chown -R sbgrid:sbgrid /programs

+</code>

+ * If there is not enough free space on the root volume, you'll need to find space on a local volume.

+

+ * Login as 'sbgrid' on client1.

+ * Create the passphraseless ssh key:

+<code>

+ssh-keygen -d

+</code>

+

+(hit enter to confirm default file location, no passphrase, etc)

+ * Copy the ssh key to server1:

+<code>

+cat ~/.ssh/id_dsa.pub | ssh server1 'cat - >> ~/.ssh/authorized_keys'

+</code>

+ * Copy the localsync script from server1 to client1. From client1:

+<code>

+scp sbgrid@server1:/programs/share/bin/localsync .

+</code>

+

+(May prompt to accept the host key. If you can't login without a password, the ssh key stuff above has failed, and you'll need to troubleshoot that.)

+</code>

+chmod 755 localsync

+./localsync sbgrid@server1

+</code>

+(This will take a while depending on the speed of the machines/network.)

+

+ * Test the installation:

+<code>

+tcsh

+source /programs/sbgrid.cshrc

+</code>

+ or

+<code>

+bash

+source /programs/sbgrid.shrc

+</code>

+

+(You should get a message like this: )

+<file>

+ Welcome to SBGrid!

+********************************************************************************

+ Your use of the applications contained in the /programs directory constitutes

+ acceptance of the terms of the SBGrid License Agreement included in the file

+ /programs/share/LICENSE. The applications distributed by SBGrid are licensed

+ exclusively to member laboratories of the SBGrid Consortium.

+

+ To hush this license message, run 'touch ~/.agree2sbgrid'.

+********************************************************************************

+ Please submit bug reports and help requests to: <bugs@sbgrid.org> or

+ <http://sbgrid.org/bugs>

+ This installation last updated: 20110411

+********************************************************************************

+ Total time to initialize: 1 second(s)

+</file>

+

+ * Add a cronjob for the updates. As the 'sbgrid' user on client1:

+<code>

+echo "0 2 * * * ~/localsync sbgrid@server1" | crontab

+</code>

+ * Check the cronjob:

+<code>

+crontab -l

+</code>

+

+You're done!

+====== Mailing List for Site Administrators ======

+We have a small mailing list for technical users and site administrators. Visit the [[https://cmcd.hms.harvard.edu/sympa/info/sbgrid-tech|SBGrid-tech mailing list]] page to sign up.

+====== SBGrid Mirrors ======

+

+For the benefit of the local SBGrid community, we recommend the following to access frequently-used open source software. If you would like for us to add a mirror, please [[help@sbgrid.org|contact us]].

+

+\\

+

+^ Mirror Name ^^ Mirror URL ^

+|{{:software:centos_icon_60.png?nolink}}|[[http://www.centos.org|CentOS]]| [[http://mirrors.med.harvard.edu/centos/]]|

+|{{:software:sl6_logo.jpeg?nolink}}|[[http://www.scientificlinux.org/|Scientific Linux]]| [[http://mirrors.med.harvard.edu/scientificlinux/]]|

+====== Supported Operating Systems ======

+The SBGrid Software Suite can run on Linux, OS X and IRIX machines.

+

+Last Update: 2014-02-04

+

+===== Linux =====

+We build and test software under 64-bit Red Hat 6. The majority of our Linux users are using the Red Hat/CentOS/Scientific Linux or Fedora distributions, but we also have labs using the software with Ubuntu, Debian and OpenSuSE. We don't explicitly test the software on these distributions, but it is known to work, and we will work to resolve problems reported by users of these Linux distributions.

+

+We officially support the two most recent releases of Red Hat/CentOS/Scientific Linux (currently 6 and 7) and the two most recent releases of Fedora Linux. At this time, only 64-bit versions of linux are fully supported. As of January 2015, we are no longer updating the 32-bit software branch, though we will work to resolve reported problems.

+

+Currently Supported:

+ * Red Hat Enterprise Linux 6.x and 7.x and the community supported versions: CentOS and Scientific Linux.

+ * Fedora releases actively supported by the Fedora community. Check https://fedoraproject.org/wiki/Releases for the current list.

+ * We are no longer building software for RHEL 5.x, but compatible applications will remain in the software tree. We will respond to support requests for these and we will resolve problems where possible. RHEL5 users may not receive the latest versions of software, but RHEL5 systems will continue to be functional and supported.

+\\

+

+===== OS X Intel =====

+We build and test the programs under the most four most recent OS X releases.

+

+Currently Supported:

+ * OS X Intel 10.7 - 10.10

+

+\\

+

+===== OS X PowerPC =====

+The OS X PowerPC branch is currently frozen. The applications are stable and work well on our PowerPC hardware, but they are not being updated.

+

+Currently Supported:

+ * OS X PowerPC 10.4 - 10.5

+\\

+

+===== IRIX =====

+The IRIX programs branch is currently frozen. The applications are stable and work well on our SGI hardware, but they are not being updated.

+

+Currently Supported:

+ * IRIX 6.5.26 + bugfix releases

+

+\\

+

+===== General =====

+ * You should use a network installation for workstations and servers if at all possible. NFS, SMB and AFP all work for sharing a single installation among multiple machines. The decreased admin overhead, bandwidth and disk savings of sharing a single installation is a significant benefit.

+

+===== Linux =====

+ * All Linux machines need installations of a recent version (6.17+) of csh/tcsh. Red Hat Enterprise Linux 6 and other recent distributions usually have an up to date version. There are RPMs for Red Hat/CentOS/Scientific Linux 5.x in the software tree in the architecture specific directories:

+<code>

+/programs/i386-linux/system/RPMs

+/programs/x86_64-linux/system/RPMs

+</code>

+ * SELinux must be disabled.

+ * 64-bit machines need a number of 32-bit compatibility packages installed **even when using the 64-bit software branch**. This is due to a number of binary-only and legacy packages that are only available as 32-bit applications. Additional fonts for Tcl/Tk, Motif and Xview applications are also encouraged.

+

+Packages for RHEL 6, and 7 and compatible machines:

+<code>

+yum install glibc.i686 libgcc.i686 libX11.i686 libXaw.i686 libXext.i686 ncurses.i686 \

+ redhat-lsb.i686 xorg-x11-fonts-75dpi xorg-x11-fonts-100dpi xorg-x11-fonts-ISO8859-1-75dpi \

+ xorg-x11-fonts-ISO8859-1-100dpi xorg-x11-fonts-misc libXScrnSaver

+</code>

+

+ * The same 32-bit requirement applies for 64-bit Debian and Ubuntu machines. This is supported via the [[https://wiki.debian.org/Multiarch/HOWTO|Multiarch]] system. You'll need at least the following packages to run most of the software. Please email us if you discover we have left some out.

+<code>apt-get install libc6-i386 lsb tcsh gawk libjpeg62 curl libx11-6:i386 libxext6:i386</code>

+

+===== 3D Stereo on RHEL 7 systems with Gnome 3 =====

+

+Nvidia 3D stereo typically requires disabling the Composite extension in X.org. Unfortunately, this extension is required to run Gnome 3 and the gdm login mangager supplied as the default desktop in CentOS and many other linux distributions.

+

+As of version 337.19 of the nvidia driver, some Quadro configurations can operate with the composite extension enabled, but stereo functionality depends on a stereo-aware composite manager. The Composite manager in CentOS7 Gnome3 is not stereo aware and 3D stereo does not work by default.

+

+In order to use 3D stereo on Gnome3 systems, you can switch from Gnome3/gdm to MATE/lightdm.

+The MATE Desktop Environment is the continuation of GNOME 2 which is the default desktop in CentOS 5 and 6.

+

+To change to MATE on CentOS7 systems, first, install the Extra Packages for Enterprise Linux (EPEL) repo:

+<code>

+ yum install epel-release

+</code>

+

+Then install the nvidia drivers from epel

+<code>

+ yum install kmod-nvidia

+</code>

+

+You may find you need these packages as well

+<code>

+ yum install mesa-libGLU-9.0.0-4.el7.x86_64 libXScrnSaver-1.2.2-6.1.el7.x86_64

+</code>

+

+Next, install lightdm which doesn't require the composite extension.

+<code>

+ yum install lightdm

+</code>

+

+From there we do a couple of groupinstalls

+You may have these rpms installed already, but just for good measure

+<code>

+ yum group install "X Window System"

+</code>

+

+then

+<code>

+ yum groupinstall "MATE Desktop"

+</code>

+

+Toggle from gdm to lightdm in systemd using systemctl.

+To see what you are using as a 'display manager', check this link

+ <code>

+ls -l /etc/systemd/system/display-manager.service

+</code>

+

+This is likely pointing to gdm and should be changed to point to lightdm.

+You can change that with systemctl.

+<code>

+ systemctl disable gdm

+ systemctl enable lightdm

+</code>

+

+And then this will reload you changes, similar to changing runlevels with sysinitV

+<code>

+ systemctl isolate graphical.target

+</code>

+

+Make sure to select MATE from the drop menu as your graphical before you login.

+===== Mac OS X ======

+ * Prior to OS X 10.8, the stock X11 has better compatibility than the Xquartz development version available from MacOSForge. With the OS X 10.8 release, X11 is no longer included by Apple, and you'll need to install the latest Xquartz version offered on the MacOSForge site to run X11-based applications.

+

+===== Laptops ======

+ * You can create a local installation to take along with you.

+http://www.sbgrid.org/wiki/software/laptopinstall