QV2016-intro_to_linux.pdf
... ...
Binary files a/QV2016-intro_to_linux.pdf and /dev/null differ
SBGrid-Wiki.md
... ...
@@ -7,15 +7,22 @@
7 7
```
8 8
9 9
## Welcome to the SBGrid Consortium Wiki
10
-
11
-SBGrid is a global consortium of research labs in academia and industry using structural biology (cryoEM, X-ray crystallography and scattering, NMR, molecular dynamics, etc) to address scientific questions. Based at Harvard Medical School in Boston, MA, our team supports research computing for structural biologists with scientific software and computing support.
10
+SBGrid supports research computing for structural biologists with scientific software and computing support. We serve a global consortium of research labs in academia and industry using structural biology (cryoEM, X-ray crystallography and scattering, NMR, molecular dynamics, protein structure prediction and strucure-based drug discovery) to address scientific questions. We are based at Harvard Medical School in Boston, MA.
12 11
13 12
This site provides information on how to install, use, and get help with your SBGrid software installation. Here you can also find information about SBGrid services. Please let us know if you have questions.
14 13
15
-### Recent Wiki Posts
14
+## Frequently accessed
15
+[Installation manager **downloads**](client_downloads)
16
+[Application usage examples](examples)
17
+[Requesting Help](help)
18
+[Managing software versions](versions)
19
+
20
+## Recent Wiki Posts
21
+### 2024
16 22
- [2024.02.28 - Update for macOS support](macos)
17 23
- [2024.02.28 - Installation Manager version 2.9.9](client_downloads)
18 24
25
+## Older
19 26
### 2023
20 27
- [2023.05.04 - Installation Manager version 2.9.7](client_downloads)
21 28
- [2023.05.04 - Installation Manager version 2.9.5](client_downloads)
... ...
@@ -42,3 +49,4 @@ This site provides information on how to install, use, and get help with your SB
42 49
- [2020.11.13 - SBGrid Installation Mangager registration is here](https://sbgrid.org/registration/register/)
43 50
- [2020.11.13 - New SBGrid Installation Mangager downloads](client_downloads)
44 51
- [2020.11.13 - MacOS 10.15 catalina is supported](catalina)
52
+
_Sidebar.md
... ...
@@ -8,7 +8,7 @@ Installing SBGrid Software
8 8
- [Site installations](site_installation)
9 9
- [Graphical installation](sbgrid-gui)
10 10
- [Command line installation](sbgrid-cli)
11
-- [macOS and Apple M1 Silicon](macos)
11
+- [macOS and Apple Silicon](macos)
12 12
13 13
Using the SBGrid Environment
14 14
account_migration.md
... ...
@@ -3,7 +3,7 @@
3 3
The sbgrid-cli *migrate-account* command allows a user to change accounts. This can be useful when moving between SBGrid member labs or to change from legacy SBGrid account to a new account. This is required for installation of BioGrids with SBGrid.
4 4
5 5
The "migrate-account" process does not require reinstallation or extensive changes to the software collection.
6
-A network connection and a recent relatively release of `sbgrid-cli` is required.
6
+A network connection and a the `sbgrid-cli` application is required.
7 7
8 8
<!-- TOC -->
9 9
... ...
@@ -21,7 +21,7 @@ A network connection and a recent relatively release of `sbgrid-cli` is required
21 21
```
22 22
$ ./sbgrid-cli migrate-account --help
23 23
24
- SBGrid Installer 2.2.40 e4c704acc0 —
24
+ SBGrid Installer 2.11. e4c704acc0 —
25 25
_______ _____ _ __
26 26
/ __/ _ )/ ___/___(_)__/ /
27 27
_\ \/ _ / (_ / __/ / _ /
... ...
@@ -152,7 +152,7 @@ error: Credential verification failed for user: jdoe and site: smith_b_123i
152 152
### Permissions and file ownership issue
153 153
154 154
The installation must belong to user account doing the migration.
155
-The error below shows a failure due to ownership of /programs. To fix this, correct the file ownership, typically with the `chown` command.
155
+The error below shows a failure due to file permissions, specifically the incorrect ownership of /programs. To fix this, correct the file ownership, typically with the `chown` command.
156 156
157 157
```
158 158
$ ./sbgrid-cli migrate-account
benchmark_systems.md
... ...
@@ -1,3 +1,2 @@
1 1
# benchmarking hardware and system information
2 2
3
-
cli_v1.md
... ...
@@ -1 +0,0 @@
1
-client_CLI_install-old.md
... ...
\ No newline at end of file
cli_v2-old.md
... ...
@@ -1,431 +0,0 @@
1
-# THIS PAGE IS DEPRECATED
2
-
3
-The information here is likey to be out of date.
4
-
5
-## SBGrid command line installation manager - v2.1.2
6
-
7
-## How do I get it?
8
-
9
-[LINUX : sbgrid-cli_2.1.2_linux.tar.gz](https://sbgrid.org/wiki/downloads/sbgrid-cli_2.1.2_linux.tar.gz)
10
-
11
-[macOS : sbgrid-cli_2.1.2_macos.tar.gz](https://sbgrid.org/wiki/downloads/sbgrid-cli_2.1.2_macos.tar.gz)
12
-
13
-## Supported OSes
14
-
15
-[Linux (CentOS/RHEL 7 or 8 recommended) or Apple OSX computers running MacOSX v10.13 - 10.15.](operatingsystems)
16
-
17
-MacOS 10.15 Catalina can be used, but a work-around is required to add the /programs path.
18
-See [here for more info :](catalina)
19
-
20
-### Pre-installation requirements
21
-
22
-**Admin access required on your computer.**
23
-The SBGrid installation manager will install the software tree to /opt/sbgrid and create a symlink from this directory to /programs.
24
-While admin privileges are not required to run the application, you will likely need admin privileges to create /opt/sbgrid and /programs .
25
-
26
-```
27
- $ ./sbgrid-cli
28
-Usage: sbgrid-cli [command] [options]
29
-
30
-Commands:
31
- help [command] command-specific help
32
-
33
- activate [options] <site user key> | <credential-key> activate a new installation
34
- reactivate [options] <configFilePath> [packagesPath] reactivate installation using config or package list
35
-
36
- install [options] [title] [otherTitles...] install package(s)
37
- remove|uninstall [options] [title] [otherTitles...] remove package(s)
38
- reinstall [options] [title] [otherTitles...] reinstall package(s)
39
- update|upgrade [options] [title] [otherTitles...] update package(s) to the latest release/version
40
-
41
- list [options] [title] list available software titles
42
- installed list installed software titles
43
- updates list available updates
44
- pending list pending installations
45
- obsolete list obsolete software titles
46
- search <query> search availble titles
47
- collections list available package collections
48
- grids list available software trees
49
- info [options] [title] print title information
50
- shell start SBGrid Shell
51
-
52
- clean [options] remove obsolete software versions and titles
53
- admin [options] run periodic installation, update and clean up
54
- rebuild rebuild installation environment
55
- crontab [options] basic script for crontab
56
- save [filename] save current configuration
57
-
58
- check-connection connectivity checks
59
- diag [options] troubleshooting and diagnostics
60
- verify list missing software versions and .rc files
61
-
62
- Options:
63
- -V, --version version
64
- --verbose verbose output
65
- --quiet disable output
66
- -h, --help usage information
67
- --linux specify linux as platform
68
- --darwin specify macOS as platform
69
- --no-color disable output colors
70
-```
71
-
72
-## General Usage
73
-
74
-The SBGrid command line installation manager takes subcommands and arguments to those commands to perform various tasks. These can be found in the usage info shown above.
75
-Each subcommand can also take a -h or --help argument to show usage info.
76
-
77
-### Activation
78
-
79
-To install the software, you must have valid credentials provided by SBGrid.
80
-
81
-Activate with the 'activate' subcommand
82
-
83
-**activate**
84
-
85
-```
86
-Usage: activate [options] [site] [user] [key]
87
-
88
-activate new installation using credentials provided
89
-(credentials are sent via email and can either be a single credential-key, or a site username key triplet)
90
-
91
-Options:
92
- -k, --credential-key <credential-key> activate with activation key
93
- -h, --help output usage information
94
-```
95
-
96
-Trouble activating? See Troubleshooting options further down this page.
97
-
98
-**reactivate**
99
-
100
-_reactivate_ can restore from a saved config file. See below.
101
-
102
-```
103
- $ ./sbgrid-cli reactivate -h
104
-Usage: reactivate [options] <configFilePath> [packagesPath]
105
-
106
-reactivate a new instance of an installation using a “saved” configuration file
107
-to repair an existing installation see “reinstall --all-versions”
108
-
109
-Options:
110
- --skip-folder-checking skip checking if folder with programs already exists
111
- --just-activation skip software titles installation
112
- -h, --help output usage information
113
-```
114
-
115
-### Listing available titles, status
116
-
117
-The _sbgrid list_ command will show all available titles. They are normally colored by their installation status.
118
-Available options :
119
-
120
-**list**
121
-
122
-```
123
-Usage: list [options] [title] [otherTitles...]
124
-
125
-list available software titles
126
-
127
-Options:
128
- -a, --all-versions list all available software versions
129
- -d, --default-versions list default software versions
130
- -c, --collections list available collections
131
- -o, --one-column list software in one column
132
- -l, --long-format list software versions in long format
133
- -h, --help output usage information
134
-```
135
-
136
-**updates**
137
-
138
-Shows available updated version and titles
139
-
140
-```
141
-./sbgrid-cli updates
142
-```
143
-
144
-**obsolete**
145
-
146
-Shows obsolete versions that will be removed.
147
-Obsolete versions are previously installed versions no longer included in the SBGrid installation.
148
-
149
-```
150
-./sbgrid-cli obsolete
151
-```
152
-
153
-### Getting Information about titles
154
-
155
-**info**
156
-
157
-The _info_ flag shows information about each title, including a description and links to documentation can be show with the _sbgrid info <title>_ command. See example below.
158
-
159
-```
160
-Usage: info [options] [title] [otherTitles...]
161
-
162
-print software title(s) metadata
163
-
164
-Options:
165
- -v, --versions list available and default versions for <title> on current platform
166
- -c, --collections list available collections with software count
167
- -h, --help output usage information
168
-```
169
-
170
-For example, info for RELION :
171
-
172
-```
173
- $ ./sbgrid-cli info relion
174
-Fetching requested info...
175
-------------------------------------------------------------------------------------
176
-Package information for relion (i386-mac):
177
-
178
-Package: RELION
179
-Title: relion
180
-Architecture: i386-mac
181
-
182
-Available version(s):
183
- * 1.3 (installed) depends on: openmpi (1.8.4)
184
- * 1.4 (installed) depends on: openmpi (1.8.4)
185
- * 2.1 (installed) depends on: openmpi (2.1.2)
186
- * 3.0.6 (installed) depends on: openmpi (2.1.3)
187
- * 3.0.7 (installed) depends on: openmpi (2.1.3)
188
- * 3.0.8 (installed, default)
189
- * 3.1-beta (installed)
190
-
191
-Grid(s): SBGrid
192
-Collection(s): Electron Microscopy
193
-
194
-Description:
195
-(REgularised LIkelihood OptimisatioN) a stand-alone computer program for Maximum A Posteriori (MAP) refinement of (multiple) 3D reconstructions or 2D class averages in cryo-electron microscopy.
196
-
197
-Technical notes:
198
-**GPU-accelerated RELION : ** RELION versions > 2.0 are GPU-accelerated using Nvidia CUDA on Linux.
199
-For general information on running GPU accelerated applications from SBGrid, please see here : https://sbgrid.org/wiki/gpu</a>
200
-
201
-CUDA builds of RELION are designated by a '_cu<cuda_version>' suffix in the version of the application.
202
-For example, 2.0.4_cu7.5 is linked against CUDA v7.5 libraries and is not recommended for Pascal architecture Nvidia hardware (GTX 1080 and higher). 2.0.4_cu8.0 is built with CUDA 8.0 which supports Pascal architecture GPUs. The '_SP' designation indicates 'single precision' CPU builds. All builds are single precision on the GPU and can be run on 'consumer-grade' GPU hardware.
203
-
204
-**RELION and MPI : **
205
-RELION uses the 'mpirun' executable to manage MPI ranks. SBGrid includes two OpenMPI versions, 2.1.2 (default) and 1.8.4 (legacy).
206
-The OpenMPI mpirun executable must match the version of OpenMPI used to build RELION.
207
-For example, RELION versions 2.1_cu*.0 use OpenMPI version 2.1.2 and must use mpirun from 2.1.2.
208
-
209
-Older RELION versions require an override in $HOME/.sbgrid.conf to set the older OpenMPI version to 1.8.4. Add
210
-
211
-OPENMPI_X=1.8.4"
212
-
213
-to $HOME/.sbgrid.conf and open a new terminal to use the correct mpirun.
214
-You can also use the 'mpirun.relion' executable in place of mpirun to use the MPI that matches the currently configured version of RELION.
215
-
216
-Links:
217
- Website: http://www2.mrc-lmb.cam.ac.uk/relion/index.php/Main_Page
218
- Manual: http://www2.mrc-lmb.cam.ac.uk/relion/index.php/Main_Page
219
- Forumhelp: https://www.jiscmail.ac.uk/cgi-bin/webadmin?A0=CCPEM
220
-```
221
-
222
-### Installing and removing
223
-
224
-Install and remove titles with the _sbgrid install <title>_ and _sbgrid remove <title>_ command. Each of these commands can take multiple titles with versions at one time.
225
-
226
-**install**
227
-
228
-```
229
-install software title(s)
230
-
231
-Options:
232
- -d, --all-default install all default versions of all available software titles
233
- -a, --all-versions install all versions of all available software titles
234
- -c, --collection <collection> install all software titles from collection
235
- title@all install all versions of a software title
236
- title@x.y.z install "x.y.z" version of a software title
237
- -p, --progress-bar show progress bar during installation
238
- -y, --yes continue without confirmation
239
- -h, --help output usage information
240
-```
241
-
242
-**Install examples :**
243
-
244
-Install a single default version of an application :
245
-
246
-```
247
-./sbgrid-cli install relion
248
-```
249
-
250
-Install a specific version of a relion :
251
-
252
-```
253
- ./sbgrid-cli install relion@3.0.8
254
-```
255
-
256
-Install all versions of relion :
257
-
258
-```
259
- ./sbgrid-cli install relion@all
260
-```
261
-
262
-Install the 'Electron Microscopy' collection of titles (see above _list_ command to list _collections_):
263
-
264
-```
265
- $ ./sbgrid-cli install -c 'Electron Microscopy'
266
-```
267
-
268
-Install all default versions of all titles :
269
-
270
-```
271
- ./sbgrid-cli install --all-defaults
272
-```
273
-
274
-Install all versions of all titles ( The full collection):
275
-
276
-```
277
- ./sbgrid-cli install --all-versions
278
-```
279
-
280
-**remove**
281
-
282
-Same options as install flag above.
283
-
284
-```
285
-remove software title(s)
286
-
287
-Options:
288
- -d, --all-default remove all default versions of all installed software titles
289
- -n, --non-default remove all non-default versions of all installed software titles
290
- -a, --all-versions remove all versions of all installed software titles
291
- -c, --collection <collection> remove all software titles from collection
292
- title@all remove all installed versions of a software title
293
- title@x.y.z remove "x.y.z" version of a software title
294
- -y, --yes continue without confirmation
295
- -h, --help output usage information
296
-```
297
-
298
-### Updating titles
299
-
300
-Updates are performed when default titles change to a new version or bugs are fixed in existing verisons. To update, use the update flag.
301
-
302
-**update**
303
-
304
-```
305
-./sbgrid-cli update relion
306
-```
307
-
308
-### Remove obsolete titles
309
-
310
-Use the obsolete flag to list or remove obsolete versions or titles
311
-
312
-**obsolete**
313
-
314
-```
315
- $ ./sbgrid-cli obsolete -h
316
-Usage: obsolete [options]
317
-
318
-list obsolete software versions
319
-
320
-Options:
321
- -r, --remove remove the obsolete software versions
322
- -y, --yes continue without confirmation
323
- -h, --help output usage information
324
-
325
-```
326
-
327
-## Advanced modes
328
-
329
-The advanced modes perform multiple actions and are designed for automated use for a managed 'rolling release' installation.
330
-
331
-**clean**
332
-
333
-Removes all obsolete software versions and titles
334
-
335
-**admin**
336
-
337
-Run periodic installation, update and removal. Installs all new titles, updates where appropriate and cleans up (removes) obsolete versions
338
-
339
-```
340
-Usage: admin [options]
341
-
342
-periodic update and installation
343
-
344
-Options:
345
- -a, --all-versions install all versions of all available software titles
346
- -r, --reinstall reinstall all of installed software titles
347
- -h, --help output usage information
348
-```
349
-
350
-For example running
351
-
352
-```
353
-./sbgrid-cli admin
354
-```
355
-
356
-will install all default, update all installed titles and remove all obsolete versions while
357
-
358
-```
359
-./sbgrid-cli admin -a
360
-```
361
-
362
-will install all versions of all titles, add any new titles, update any pending updates, and remove obsolete versions.
363
-This
364
-
365
-**rebuild**
366
-
367
-_rebuild_ rebuilds the installation environment configuration files.
368
-
369
-**crontab**
370
-
371
-Writes a basic script for crontab for _admin_ functionality
372
-
373
-**save**
374
-
375
-The _save_ flag saves the current configuration so it can be restored at a later stage or replicated on another machine (or in the cloud).
376
-
377
-## Troubleshooting and diagnostics
378
-
379
-**check-connection**
380
-connectivity checks
381
-
382
-**diag**
383
-troubleshooting and diagnostics
384
-
385
-**verify**
386
-list missing software versions and .rc files
387
-
388
-## Installing software for multiple platforms
389
-
390
-On linux, add the --darwin command to install / remove for macOS .
391
-Conversely, use the --linux command to install / remove for Linux while on macOS.
392
-
393
-```
394
- --linux specify linux as platform
395
- --darwin specify macOS as platform
396
-```
397
-
398
-The platform flag is implied for the platform used to install the software.
399
-
400
-### Software Version Management in the SBGrid environment
401
-
402
-If you have a single version of an application installed, explicit version selection in your environment is not needed.
403
-
404
-In the case of multiple installed versions, the _SBGrid Default_ version will be the version available at the terminal when it is installed, unless configured otherwise in _~/.sbgrid.conf_. This is standard for the SBGrid software environment.
405
-
406
-In the case of multiple non-default versions installed, the _latest release_ will be version available in your shell. If you would like to have more than one version of a title installed, and prefer to use an older release or the non-default version when a default version is installed, you must set this explicitly in your _~/.sbgrid.conf_ file. This is the same as in the standard SBGrid environment.
407
-
408
-**Version priority in the shell**
409
-
410
-1. _~/.sbgrid.conf_
411
-2. Installed single version
412
-3. Installed Default version
413
-4. Latest installed release
414
-
415
-There is more info on how to do that here --> [SBGrid version overrides](versions)
416
-
417
-### Running SBGrid software
418
-
419
-To use the software at the terminal in bash, open a new terminal and run
420
-
421
- $ source /programs/sbgrid.shrc
422
-
423
-or in tcsh
424
-
425
- $ source /programs/sbgrid.cshrc
426
-
427
-### Known Issues
428
-
429
-- **rsync required on 873 or 8080**. The application will fail if outbound port 873 and port 8080 are blocked by your institution or are otherwise not available. You can check if you can access [port 873 here](http://portquiz.net:873) and [port 8080 here](http://portquiz.net:8080)
430
-
431
-For help or to report issues, please email *bugs@sbgrid.org.*
client_CLI_install-old.md
... ...
@@ -1,444 +0,0 @@
1
-# THIS PAGE IS DEPRECATED
2
-
3
-The information here is likey to be out of date.
4
-See [sbgrid-cli V2](cli_v2)
5
-
6
-## SBGrid command line installation manager
7
-
8
-- [THIS PAGE IS DEPRECATED](#this-page-is-deprecated)
9
- - [SBGrid command line installation manager](#sbgrid-command-line-installation-manager)
10
- - [Installation](#installation)
11
- - [Pre-installation requirements](#pre-installation-requirements)
12
- - [Binary releases](#binary-releases)
13
- - [Installation on an external drive or USB key](#installation-on-an-external-drive-or-usb-key)
14
- - [General Usage](#general-usage)
15
- - [Activation](#activation)
16
- - [Listing available titles](#listing-available-titles)
17
- - [Getting Information about titles](#getting-information-about-titles)
18
- - [Installing and removing](#installing-and-removing)
19
- - [Updating titles](#updating-titles)
20
- - [Example : Installing Albula](#example--installing-albula)
21
- - [Example : Removing Albula](#example--removing-albula)
22
- - [Version selection](#version-selection)
23
- - [Software Version Management in the SBGrid environment](#software-version-management-in-the-sbgrid-environment)
24
- - [Running SBGrid software](#running-sbgrid-software)
25
- - [Known Issues](#known-issues)
26
-
27
-## Installation
28
-
29
-The SBGrid software stack can be installed using our command line program for [Linux (CentOS/RHEL 7 or 8 recommended) or Apple OSX computers running MacOSX v10.13 - 10.15.](operatingsystems)
30
-CentOS 6 is no longer supported.
31
-
32
-Two software installers are available. Binary releases are available for MacOS and CentOS operating systems.
33
-
34
-The SBGrid installation manager requires an individual account. To get started, first, set up an account by registering here : [SBGrid registration](https://sbgrid.org/registration/register/)
35
-
36
-**Check out [Jason Key's presentation on SBGrid installation tools](https://www.youtube.com/watch?v=sAxIzWhWX5o&t=1s)**
37
-
38
-### Pre-installation requirements
39
-
40
-**Admin access required on your computer.**
41
-The SBGrid installation manager will install the software tree to /opt/sbgrid and create a symlink from this directory to /programs. While admin privileges are not required to run the application, you will need admin privileges to create /opt/sbgrid and /programs. The install manager currently relies on 'sudo' for these tasks and should in installed from an account with sudo privileges. While not strictly required, we recommend a dedicated 'sbgrid' user account for software management. This ensures the software can't be modified by user accounts and that data won't be mistakenly saved into the software stack and possibly inadvertantly deleted by software updates.
42
-
43
-**Existing SBGrid installations must be renamed or removed.**
44
-If you have previously installed SBGrid software on your computer at /programs or at /opt/sbgrid, these directories should be removed or renamed. The manager will create these on first run.
45
-
46
-**Hard Drive Space**
47
-Depending on what you install, ensure you have enough free hard drive space on your machine.
48
-
49
-**For large packages (Phenix, Rosetta, CCP4), a wired connection is recommended.**
50
-Some packages are quite large and can be slow to download, especially over WiFi.
51
-
52
-### Binary releases
53
-
54
-Binary releases of the install manager are recommended - there is minimal installation needed for these binaries and they can be run from any location with no dependencies. Untar the archive, make the file executable if necessary, and run from your preferred location.
55
-
56
-[Mac : 1.0.695, 2020-05-31](downloads/sbgrid-1.0.695-Darwin.tgz)
57
-
58
-[Linux : 1.0.695, 2020-05-31](downloads/sbgrid-1.0.695-Linux.tgz)
59
-
60
-```
61
-$ tar -zxf sbgrid-1.0.695-Darwin.tgz
62
-
63
-$ cd sbgrid-1.0.695-Darwin
64
-
65
-$ ./sbgrid
66
-usage:
67
-sbgrid <command> [<command_args>]
68
-
69
-Available commands are:
70
-
71
-Query packages:
72
- collections list available collections
73
- grids list available grids
74
- info get detailed information on a particular package
75
- installed list all installed packages
76
- list list available packages
77
- search list and search installed packages
78
- updates list available updates
79
-
80
-Manage packages:
81
- install install packages
82
- install-all
83
- rebuild rebuild installation of all installed packages
84
- remove / uninstall remove installed packages
85
- resume resume interrupted installations
86
- update update installed package
87
- update-all
88
-
89
-Environment:
90
- shell Load sbgrid environment
91
-
92
-Installation management
93
- activate activate new installation
94
- admin cron mode
95
- autoclean removes unused packages
96
- reactivate reactivate installation using existing config and package database/list
97
- save save current configuration
98
-
99
-Additional arguments:
100
- -h, --help show usage information and exit current platform
101
- -l, --info-versions list available and default versions for <program> on current platform
102
- -L, --info-all-versions list available and default versions for <program> on all platforms
103
- --debug enable verbose logging
104
-sbgrid: error: too few arguments
105
-```
106
-
107
-### Installation on an external drive or USB key
108
-
109
-Currently the installer application installs the software tree in /opt/sbgrid and uses a symlink at /programs. This link is required for the software to work, but in principle the software can be located anywhere and linked to /opt/sbgrid. This functionality is not yet included in the SBGrid application, but can be set up manually with a symlink.
110
-
111
-An installation can be made on an external drive if you first do an install (into /opt/sbgrid), then move it and symlink to the new location.
112
-
113
-**Example:** 128Gb USB 'key' drives are available on Amazon for 30$ or less and are a nice way to supplement the SSDs that come in Mac laptops. To install on an external drive like this, mount it on your computer. For this example, the mount is _/Volumes/sbgrid_installation_. Then, using the SBGrid installer, do an initial installation on the hard drive. To make life easy, just do one application. Then close the installer application.
114
-
115
-Next move _/opt/sbgrid_ to the USB key at _/Volumes/sbgrid_installation_. You will likely need sudo for this.
116
-
117
- $ sudo mv /opt/sbgrid /Volumes/sbgrid_installation/.
118
-
119
-Now we have _/Volumes/sbgrid_installation/sbgrid_. Create a symlink from _/opt/sbgrid_ to _/Volumes/sbgrid_installation/sbgrid_.
120
-
121
- $ sudo ln -s /Volumes/sbgrid_installation/sbgrid /opt/sbgrid
122
-
123
-The symlink at _/programs_ still points to _/opt/sbgrid_, and that is a link that points to the external location. Open the installer and add the desired applications. These will now install to the external drive.
124
-
125
-## General Usage
126
-
127
-The SBGrid command line installation manager takes subcommands and arguments to those commands to perform various tasks. These can be found in the usage info shown above. Each subcommand can also take a -h or --help argument to show more detailed usage info. For example, the 'install' subcommand can take several arguments :
128
-
129
-```
130
-$ ./sbgrid install --help
131
-usage: sbgrid install [-h] [--all-versions] title [title ...]
132
-
133
-Install package.
134
-
135
-positional arguments:
136
-title package name or package_name@version
137
-
138
-optional arguments:
139
--h, --help show this help message and exit
140
---all-versions install all versions for a package
141
-```
142
-
143
-### Activation
144
-
145
-To install the software, you must have valid credentials provided by SBGrid.
146
-
147
-Activate with the 'activate' subcommand
148
-
149
-```
150
-usage: sbgrid activate [-h] [--multibranch] site user key
151
-
152
-Activate new installation.
153
-
154
-positional arguments:
155
- site site name
156
- user user name
157
- key activation key
158
-
159
-optional arguments:
160
- -h, --help show this help message and exit
161
- --multibranch enable installation for all architectures
162
-
163
-```
164
-
165
-The _--multibranch_ option allows for installation of Linux or Mac software on server that should host both platforms, usually for access of NFS. This must be set at activation. Once set, Linux and Mac titles will be installed.
166
-
167
-### Listing available titles
168
-
169
-The _sbgrid list_ command will show all available titles colored by their installation status. Installed titles are green, those with available updates are blue, and titles to be removed are red. Software is listed by 'Collection'.
170
-
171
-```
172
-$ ./sbgrid list
173
--- Crystallography -------------------------------------------------------------------------------------------------------------------------
174
-3dna eden pdb_redo solveresolve
175
-adxv eiger2cbf pdb-validation-suite somore
176
-albula elves
177
-...
178
--- NMR -------------------------------------------------------------------------------------------------------------------------------------
179
-aqua gamma nmrview remediator
180
-aria haddock pales shiftx2
181
-...
182
-
183
--- Electron Microscopy ---------------------------------------------------------------------------------------------------------------------
184
-2dx particle
185
-ace2 pft3dr
186
-auto3dem phoelix
187
-bfactor powerfit
188
-bsoft priismive
189
-burnham-brandeis-helical-package python-macromolecular-library
190
-chimera refmac
191
-crop relion
192
-...
193
-
194
-```
195
-
196
-### Getting Information about titles
197
-
198
-Information about each title, including a description and links to documentation can be show with the _sbgrid info <title>_ command. See example below.
199
-
200
-### Installing and removing
201
-
202
-Install and remove titles with the _sbgrid install <title>_ and _sbgrid remove <title>_ command. Each of these commands can take multiple titles with versions at one time.
203
-
204
-```
205
- $ ./sbgrid info epmr
206
-Package: EPMR
207
-
208
-Available versions:
209
- * 16.01 (installed)
210
- * 16.04 (installed)
211
- * 16.07.1 (installed, default)
212
-
213
-Grids(s): SBGrid
214
-Collection(s): Crystallography
215
-
216
-Description:
217
- a program that finds crystallographic molecular replacement solutions
218
-using an evolutionary search algorithm. The program directly optimizes
219
-three rotational and three positional parameters for the search model with
220
-respect to the correlation coefficient between Fo and Fc.
221
-
222
-Links:
223
- Website: http://www.epmr.info/
224
- Manual: http://www.epmr.info/UsersGuide.html
225
-
226
-```
227
-
228
-```
229
- $ ./sbgrid info adxv
230
-Package: adxv
231
-
232
-Available versions:
233
- * 1.9.10
234
- * 1.9.11
235
- * 1.9.12 (installed, default)
236
- * 1.9.7
237
-
238
-Grids(s): SBGrid
239
-Collection(s): Crystallography
240
-
241
-Description:
242
- displays protein crystallography X-ray diffraction data. The data may
243
-be displayed as a 2-D image, 3-D wire mesh or as integer pixel values. Data
244
-files may be saved in tiff or postscript format.
245
-
246
-Technical notes:
247
- Adxv will display data from most current detectors: ADSC ccd Mar
248
-ccd APS 3x3 ccd Mar image plate (old and new format) Raxis II & IV Fuji
249
-image plate Crystallographic Binary Format (CBF) XDS .pck files Raw
250
-binary 8, 16 & 32 bit integer data
251
-
252
-Links:
253
- Website: http://www.scripps.edu/tainer/arvai/adxv.html
254
- Manual: http://www.scripps.edu/tainer/arvai/adxv/AdxvUserManual.pdf
255
-```
256
-
257
-```
258
- $ ./sbgrid install adxv epmr epmr@16.01
259
-adxv@1.9.12 installed
260
-epmr@16.07.1 installated
261
-epmr@16.01 installed
262
-Updating installation configuration...
263
-
264
-```
265
-
266
-When a version string is not specified, the default version is installed or removed.
267
-
268
-### Updating titles
269
-
270
-Show titles with a pending update with the _sbgrid updates_ command. Updates can be applied with the _sbgrid update <title>_ or _sbgrid update-all_ commands.
271
-
272
-### Example : Installing Albula
273
-
274
- $ sbgrid info albula
275
-
276
-```
277
-Package: ALBULA
278
-Name : albula
279
-
280
-Available versions:
281
-* 3.2.0-2 (default)
282
-* 3.1.0-13
283
-* 3.0.0-14
284
-* 2.2.12
285
-
286
-Collection(s): Crystallography
287
-
288
-Description:
289
- consists of the standalone program ALBULA VIEWER, a fast and easy-to-
290
-use program that allows optimal visualization of PILATUS and EIGER data,
291
-and ALBULA API, a Python programming interface for displaying images and
292
-performing operations and calculations. The API enables you to easily
293
-integrate the viewer functionality into your beamline infrastructure or
294
-experimental setup.
295
-
296
-Links:
297
-* Website: https://www.dectris.com/Albula_Overview.html
298
-```
299
-
300
-To install:
301
-
302
- $ sbgrid install albula
303
-
304
-```
305
-Installing: albula
306
-Updating installation configuration...
307
-Installation was successfull!
308
-```
309
-
310
-To get started, load the SBGrid environment :
311
-
312
- $ source /programs/sbgrid.shrc
313
-
314
-```
315
- Software Support by SBGrid (www.sbgrid.org)
316
-********************************************************************************
317
-
318
-...
319
-
320
-SBGrid installation last updated: 2016-10-21
321
-Please submit bug reports and help requests to: <bugs@sbgrid.org> or
322
- <http://sbgrid.org/bugs>
323
-********************************************************************************
324
-```
325
-
326
-### Example : Removing Albula
327
-
328
- $ sbgrid remove albula
329
-
330
-results in :
331
-
332
-```
333
-Removing: albula
334
-Updating installation configuration...
335
-Package removed!
336
-```
337
-
338
-### Version selection
339
-
340
-Default versions are installed by default for each application, though individual versions can be selected for each title.
341
-
342
-To do so, include the version after the title followed the version. To see what versions are available, use the _sbgrid info <title>_ command. Let;s us the title epmr as an example here:
343
-
344
- $ sbgrid info epmr
345
-
346
-```
347
-Package: EPMR
348
-Name : epmr
349
-
350
-Available versions:
351
-* 16.04 (default)
352
-* 16.01
353
-* 15.12
354
-* 15.04
355
-* 13.07
356
-* 11.02
357
-
358
-Collection(s): Crystallography
359
-
360
-Description:
361
- a program that finds crystallographic molecular replacement solutions
362
-using an evolutionary search algorithm. The program directly optimizes
363
-three rotational and three positional parameters for the search model with
364
-respect to the correlation coefficient between Fo and Fc.
365
-
366
-Links:
367
-* Website: http://www.epmr.info/
368
-* Manual: http://www.epmr.info/UsersGuide.html
369
-```
370
-
371
-To install version 11.02, add the version string like so:
372
-
373
- $ sbgrid install epmr@11.02
374
-
375
-which will produce :
376
-
377
-```
378
-Installing: epmr (11.02)
379
-Updating installation configuration...
380
-Installation was successfull!
381
-```
382
-
383
-Check which versions are installed with the sbgrid info command again
384
-
385
- $ sbgrid info epmr
386
-
387
-```
388
-epPackage: EPMR
389
-Name slug: epmr
390
-
391
-Available versions:
392
-* 16.04 (default)
393
-* 16.01
394
-* 15.12
395
-* 15.04
396
-* 13.07
397
-* 11.02 (installed)
398
-
399
-...
400
-
401
-```
402
-
403
-Confirm with the _sbwhich_ command
404
-
405
- $ sbwhich epmr
406
-
407
-```
408
-/programs/i386-mac/epmr/11.02/epmr
409
-```
410
-
411
-Reload the sbgrid shell to use the new version ( or set the environment variable EPMR_M=11.02 ).
412
-
413
-### Software Version Management in the SBGrid environment
414
-
415
-If you have a single version of an application installed, explicit version selection in your environment is not needed.
416
-
417
-In the case of multiple installed versions, the _SBGrid Default_ version will be the version available at the terminal when it is installed, unless configured otherwise in _~/.sbgrid.conf_.
418
-
419
-In the case of multiple non-default versions installed, the _latest release_ will be version available in your shell. If you would like to have more than one version of a title installed, and prefer to use an older release or the non-default version when a default version is installed, you must set this explicitly in your _~/.sbgrid.conf_ file. This is the same as in the standard SBGrid environment.
420
-
421
-**Version priority in the shell**
422
-
423
-1. _~/.sbgrid.conf_
424
-2. Installed single version
425
-3. Installed Default version
426
-4. Latest installed release
427
-
428
-There is more info on how to do that here --> [SBGrid version overrides](versions)
429
-
430
-### Running SBGrid software
431
-
432
-To use the software at the terminal in bash, open a new terminal and run
433
-
434
- $ source /programs/sbgrid.shrc
435
-
436
-or in tcsh
437
-
438
- $ source /programs/sbgrid.cshrc
439
-
440
-### Known Issues
441
-
442
-- **rsync required on 873 or 8080**. The application will fail if outbound port 873 and port 8080 are blocked by your institution or are otherwise not available. You can check if you can access [port 873 here](http://portquiz.net:873) and [port 8080 here](http://portquiz.net:8080)
443
-
444
-For help, email *bugs@sbgrid.org.*
client_install_legacy.md
... ...
@@ -1,175 +0,0 @@
1
-# THIS PAGE IS DEPRECATED
2
-
3
-The information here is likey to be out of date.
4
-## SBGrid installation manager
5
-
6
-The SBGrid installation manager is available as command-line tool for macOS and Linux, and as a graphical application for macOS. The command-line and graphical versions are not mutually exclusive - they can be used interchangeably on mac.
7
-
8
-For the latest command-line application, [see here](sbgrid-cli).
9
-For the graphical version, see below.
10
-
11
-**NOTE for BIOGRIDS users** - [ Instructions for BioGrids installation can be found here ](https://biogrids.org/wiki/client_install). BioGrids is a scientific software collection focused on Bioinformatics tools. It is distinct from SBGrid which is focused on structural biology applications. Please see here for more info : [biogrids.org](https://biogrids.org).
12
-
13
-![](images/client_all.png)
14
-
15
-Looking to get started with the SBGrid installation manager? First, set up an account by registering here : [SBGrid registration](https://sbgrid.org/registration/register/)
16
-
17
-- [THIS PAGE IS DEPRECATED](#this-page-is-deprecated)
18
- - [SBGrid installation manager](#sbgrid-installation-manager)
19
- - [Installation](#installation)
20
- - [Pre-installation](#pre-installation)
21
- - [Download, Installation and activation](#download-installation-and-activation)
22
- - [Installation on an external drive or USB key](#installation-on-an-external-drive-or-usb-key)
23
- - [Usage](#usage)
24
- - [General](#general)
25
- - [Example : Installing Albula](#example--installing-albula)
26
- - [Example : Removing Albula](#example--removing-albula)
27
- - [Version selection](#version-selection)
28
- - [Versions and the SBGrid environment](#versions-and-the-sbgrid-environment)
29
- - [Running the software](#running-the-software)
30
- - [Questions, Problems and Known Issues](#questions-problems-and-known-issues)
31
-
32
-## Installation
33
-
34
-### Pre-installation
35
-
36
-**1. Admin access required.**
37
-The SBGrid installation manager will install the software tree to _/opt/sbgrid_ and create a symlink from this directory to _/programs_. While admin privileges are not required to run the application, you will need admin privileges to create _/opt/sbgrid_ and _/programs_ .
38
-
39
-**2. Existing installations must be renamed or removed.**
40
-If you have previously installed SBGrid software on your computer at _/programs_ or at _/opt/sbgrid_, these directories should be removed or renamed. The manager will create these on first run.
41
-
42
-**3. Hard Drive Space**
43
-Depending on what you install, ensure that you have enough free hard drive space on your machine.
44
-
45
-**4. XQuartz is required for many graphical applications on MacOS.**
46
-To use many of the graphical applications (COOT, CCP4, nmrDraw, etc), install the latest Xquartz version offered on the [MacOSForge site](https://www.xquartz.org/).
47
-
48
-**5. For large packages (Phenix, Rosetta, CCP4), a wired connection is recommended.**
49
-Some packages are quite large and can be slow to download, especially over WiFi.
50
-
51
-### Download, Installation and activation
52
-
53
-For macOS Catalina (10.15 and later), a work-around is required for installation. Please see [this page for more info](catalina).
54
-For 10.14 "Mojave" and earlier, this step is not required.
55
-
56
-**1. Download the latest SBGrid installation manager from here.**
57
-
58
-[Latest build : 1.0.695, 2020-05-31](downloads/SBGrid_Installer-1.0.695.dmg)
59
-
60
-**2. Mount the .dmg disk image on your Mac.** Copy the 'SBGrid installer.app' to a convenient location - it does not need to be in /Applications. The .app bundle can be run directly from the mounted dmg image with no installation if desired.
61
-
62
-You may be prompted to that the appication is from an "Unidentified Developer". If you try to open an app that is not signed by an Apple-registered developer, you this a warning dialog. This doesn’t necessarily mean that something’s wrong with the app, some apps were written by developers who are not members of the Apple developers program.
63
-
64
-**3. Enter your site, user name, and key.** You should have received these by email.
65
-
66
-![activation](images/client_activation.png)
67
-
68
-On activation you will be prompted to provide admin credentials for your mac. This is to create the _/opt/sbgrid_ directory and _/programs_ symlink.
69
-
70
-![activation](images/client_permissions.png)
71
-
72
-**4. On successful activation, you should see the manager populate with software titles.** If not, please close the application and try again. If activation continues to fail, you may have an old SBGrid installation at _/programs_ or _/opt/sbgrid_. These should be removed. You may also be blocked from accessing SBGrid servers on port 873 and port 8080 by your institution's firewall. You can check if you can access [port 873 here](http://portquiz.net:873) and [port 8080 here](http://portquiz.net:8080)
73
-For help, email *bugs@sbgrid.org.*
74
-
75
-### Installation on an external drive or USB key
76
-
77
-Currently the installer application installs the software tree in _/opt/sbgrid_ and uses a symlink at _/programs_. The _/programs_ link is required for the software to work, but in principle the software can be anywhere by linking _/opt/sbgrid_ to the desired installation point. This functionality is not yet included in the SBGrid application, but can be set up manually post-install with a symlink. First do an install (into _/opt/sbgrid_), then move it to the desired localtion. Then symlink to the new location from _/opt/sbgrid_.
78
-
79
-**Example:** 128Gb USB 'key' drives are available on Amazon for 30$ or less and are a nice way to supplement the small SSDs that come in Mac laptops. To install on an external drive in this way, first mount it on your computer. For this example, the mount is _/Volumes/sbgrid_programs_. Then, using the SBGrid installer, do an initial installation on the hard drive. To make life easy, just do one application. Then close the installer application.
80
-
81
-Next move **/opt/sbgrid** to the USB key at _/Volumes/sbgrid_programs_. You will likely need sudo for this.
82
-
83
- sudo mv /opt/sbgrid /Volumes/sbgrid_programs/.
84
-
85
-Now we have _/Volumes/sbgrid_programs/sbgrid_. Create a symlink from **/opt/sbgrid** to _/Volumes/sbgrid_programs/sbgrid_.
86
-
87
- sudo ln -s /Volumes/sbgrid_programs/sbgrid /opt/sbgrid
88
-
89
-The symlink at **/programs** still points to **/opt/sbgrid**, and that is a link that points to the external location. Open the installer and add the desired applications. These will now install to the external drive.
90
-
91
-## Usage
92
-
93
-**Instructions for installing the SBGrid software installation client can be found here** --> [Installation Instructions](client_install)
94
-
95
-### General
96
-
97
-The SBGrid software installation manager is a GUI-based installer for the SBGrid software collection. The application allows for selected applications to be installed in the familiar SBGrid environment.
98
-
99
-![installed](images/client_install.png)
100
-
101
-The manager GUI is displays software 'Collections' on the right panel with folder icons under the 'Packages' heading. These Collections control the list of titles displayed in the center 'main' panel.
102
-
103
-The first of two of these 'Collections' are shortcut meta-collections of the local machine. The first is the currently installed titles. The second icon, Updates, shows updates available from SBGrid. 'Pending' is a collection of pending installs and removals. Below these shortcuts are packages are grouped by primary scientific uses. Installation status is also indicated in theses collections.
104
-
105
-Information about each title, including a description and links to documentation are shown in the upper right panel when that title is selected in the main pane.
106
-
107
-Once packages have been configured for installation and/or removal by selecting their checkbox, clicking the 'Apply Changes' button will perform update the titles, adding pending installations and removing pending deletions.
108
-
109
-### Example : Installing Albula
110
-
111
-Check the box to select Albula, then click the 'Apply Changes' button.
112
-Notice the text turns green for pending installations. Pending updates are Orange.
113
-![apply](images/client_apply2.png)
114
-
115
-### Example : Removing Albula
116
-
117
-Uncheck the box for Albula. Removals are shown in red text.
118
-![apply](images/client_remove1.png)
119
-
120
-Click apply changes. Albula will be removed.
121
-![apply](images/client_remove2.png)
122
-
123
-### Version selection
124
-
125
-Default versions are installed by default for each application, though individual versions can be selected for each title. To do so, show the version pane by selecting **"Show all package versions"** from the **"View"** menu. Then, in the bottom right, select a desired version with the check boxes. Multiple versions of a given title can be installed and are indicated in the main panel.
126
-
127
-Enable version panel
128
-![all](images/client_versions1.png)
129
-
130
-Check old/non-default versions
131
-![all](images/client_versions2.png)
132
-
133
-### Versions and the SBGrid environment
134
-
135
-If you have a single version of an application installed, explicitly version selection in your environment will not be needed. In the case of multiple installed versions, the _SBGrid Default_ version will be the version available at the terminal when installed, unless configured otherwise in _~/.sbgrid.conf_.
136
-
137
-If you have multiple non-default versions installed, the _latest release_ will be version available in your shell. If you would like to have more than one version of a title installed, and prefer to use either an older release or the non-default version when a default version is installed, you must set this explicitly in your _~/.sbgrid.conf_ file as in the standard SBGrid environment.
138
-
139
-**Version priority in the shell**
140
-
141
-1. _~/.sbgrid.conf_
142
-2. Installed Default version
143
-3. Latest installed release
144
-
145
-You can do open the configuration file by selecting the _"Edit Configuration File"_ from the _"SBGrid Installer"_ menu.
146
-
147
-![all](images/client_config1.png)
148
-
149
-This will open a text editor with a sample configuration file, or your _~/.sbgrid.conf_ file if you have one already.
150
-
151
-![all](images/client_config2.png)
152
-
153
-There is more info on how to do that here --> [SBGrid version overrides](versions)
154
-
155
-### Running the software
156
-
157
-To run the software, you can click the "SBGrid Shell" button for an initialized bash shell.
158
-
159
-You may also run the software from another terminal on your machine.
160
-To use the software at the terminal in bash, open a new terminal and run
161
-
162
- source /programs/sbgrid.shrc
163
-
164
-or in tcsh
165
-
166
- source /programs/sbgrid.cshrc
167
-
168
-### Questions, Problems and Known Issues
169
-
170
-For questions or problems, the Help menu will direct to the SBGrid help page. Or just email *bugs@sbgrid.org*.
171
-If relevant, please include a screenshot of the GUI if possible (command + shift + 4) and please include the ~/Library/Application Support/SBGrid/sbgrid.log file with your report.
172
-
173
-- **rsync required on 873 or 8080**. The application will fail if outbound port 873 and port 8080 are blocked by your institution or are otherwise not available.
174
-
175
-- **Activation Failed**. In some cases the Authorization dialog for admin privileges does not appear during activation. This causes the manager activation to fail. This is usually due to a previously existing _/programs_ directory or symlink from a prior installation. This must be removed for activation to be successful. Other users have reported successful activation after a restart.
developers.md
... ...
@@ -18,18 +18,14 @@ When applying for grants or financial support please let us know! We can provide
18 18
19 19
You are welcome to use SBGrid laboratories to beta-test your software. Just let us know when the new beta is available, and we will install it for all participating laboratories or a limited subset that you choose.
20 20
21
-### Annual Software Awards
22
-
23
-Members of SBGrid Consortium vote in an annual poll to select their favorite structural biology application. Paul Emsley was the winner of the first poll. He won a Macbook Air laptop for his application Coot.
24
-
25 21
### Stretch Your Resources
26 22
27 23
First level software support is provided by the SBGrid non-profit center at Harvard Medical School. All members of the consortium communicate with the service center for assistance with software installation, upgrades, and basic troubleshooting. For many of the mundane problems involving installation, operating system portability and environmental set up, SBGrid members will never need to contact you for support. Your expertise with the software be best utilized answering questions and working on problems that directly involve the science behind your application.
28 24
29 25
### Broad Exposure for Your Software
30 26
31
-More than 300+ structural biology laboratories from 90 institutions in 19 countries participate in the Consortium including the labs of many HHMI investigators and a few Nobel Prize winners. Deployment of your application can be customized to include only non-profit labs or broadened to include industry labs as well. Please visit the Consortium website for the full list of participating groups that can benefit from rapid access to your software. Please note that most SBGrid Consortium laboratories standardize on hardware and operating systems, which simplifies the installation and support process.
27
+More than 500+ structural biology laboratories from 100+ institutions in 20+ countries participate in the Consortium including the labs of many HHMI investigators and a few Nobel Prize winners. Deployment of your application can be customized to include only non-profit labs or broadened to include industry labs as well. Please visit [https://sbgrid.org](https://sbgrid.org) for the full list of participating groups that can benefit from rapid access to your software. Please note that most SBGrid Consortium laboratories standardize on supported operating systems, which simplifies the installation and support process.
32 28
33 29
### Connect With Users
34 30
35
-Announcing a new version of your software? We can use webex to broadcast your presentation to all laboratories participating in our consortium. We can also disseminate information about your application to our community in our monthly newsletter. Visiting Boston? Please let us know because we'd love to schedule a seminar for members of our local structural biology community.
31
+Announcing a new version of your software? Present in our monthly webinar series to broadcast your presentation to all laboratories participating in our consortium. We can also disseminate information about your application to our community in our monthly newsletter. Visiting Boston? Please let us know because we'd love to schedule a seminar for members of our local structural biology community.
examples.md
... ...
@@ -15,6 +15,3 @@ The following pages provide usage info and examples for select applications in t
15 15
- [PHENIX - phenix.rosetta_refine](examples/phenix.rosetta_refine)
16 16
- [SCIPION - running tutorials without write privileges](examples/running_scipion_tutorials)
17 17
- [DIALS - Version control in DIALS](examples/DIALS_version_control)
18
-
19
-
20
-
getting_started.md
... ...
@@ -1,4 +1,4 @@
1
-## Getting Started
1
+## Getting Started with SBGrid
2 2
3 3
- [Getting Started](#getting-started)
4 4
- [Using the SBGrid software collection](#using-the-sbgrid-software-collection)
... ...
@@ -7,8 +7,7 @@
7 7
8 8
### Using the SBGrid software collection
9 9
10
-The SBGrid software collection can be found installed at **/programs**.
11
-All software in the SBGrid collection is run from the the command line in a terminal. Do set up the shell environment the terminal to run the programs, you will need to initialize the SBGrid environment.
10
+The SBGrid software collection is installed at **/programs** and is run from the the command line in a terminal. To initialize the SBGrid shell environment, source the sbgrid.shrc or sbgrid.cshrc file on your system.
12 11
13 12
Bash users should initialize the SBGrid environment using :
14 13
... ...
@@ -50,4 +49,8 @@ For tcsh in ~/.tcshrc, this would be
50 49
51 50
alias sbg 'source /programs/sbgrid.cshrc'
52 51
52
+### Using SBGrid with LMOD
53
+
54
+It is possible (but not required) to use SBGrid with an existing LMOD deployment. [Please see this page for more info on using module files.](modulefiles). Note that all SBGrid module files are Lua format. LMOD is not included in SBGrid.
55
+
53 56
Questions or problems? Please email **bugs@sbgrid.org**
gpu.md
... ...
@@ -23,7 +23,7 @@ Not all GPUs are created equal - some applications are restricted to GPUs with a
23 23
24 24
You linux workstation must have the proper drivers for your GPU. You can find drivers for your card here : [https://developer.nvidia.com/cuda-zone](https://developer.nvidia.com/cuda-zone)
25 25
26
-We currently support applications that rely on versions of CUDA 7.5 to 10 and these libraries are included with the SBGrid distribution. CUDA libraries do not need to be installed locally - they will not be used by SBGrid software. However, it is important to have a recent driver version if possible.
26
+We currently support applications that rely on versions of CUDA 7.5 to 12.2 and these libraries are included with the SBGrid distribution. CUDA libraries do NOT need to be installed locally - they will not be used by SBGrid software. However, it is important to have a recent driver version if possible.
27 27
28 28
For information about GPU hardware and CUDA / driver version compatibility, [see this table at nvidia.com](https://docs.nvidia.com/deploy/cuda-compatibility/index.html#binary-compatibility__table-toolkit-driver)
29 29
help.md
... ...
@@ -1,14 +1,16 @@
1 1
## Software Help
2 2
3
-If you are having a problem with software included in the SBGrid suite or think you've discovered an application bug, please use the [SBGrid](http://sbgrid.org/bugs) web form to alert us to the problem. You can also email bugs@sbgrid.org with details on the problem, but the web form is the preferred method.
3
+If you are having a problem with software included in the SBGrid suite or think you've discovered an application bug, please use the [SBGrid](http://sbgrid.org/bugs) web form to alert us to the problem. You can also email bugs@sbgrid.org with details on the problem.
4 4
5
-If the problem with the software started after an update, you can configure your SBGrid environment to [use an older version of the software](overrides).
5
+When reporting a problem, more info is better. We will likely require the output of the `sbinfo` command from your SBGrid-enabled terminal. If you are running GPU accelerated software, the output of `nvidia-smi -L` is also helpful.
6
+
7
+If the problem with the software started after a recent update, you can configure your SBGrid environment to [use an older version of the software](overrides).
6 8
7 9
### New Software and Software Updates
8 10
9
-To request a new software package or an update to an existing package, please use the [Software](http://sbgrid.org/req) Update/Installation Request form]].
11
+To request a new software package or an update to an existing package, please use the [Software Update/Installation Request form](http://sbgrid.org/req). You can also email us at softwarerequest@sbgrid.org.
10 12
11
-We try to accommodate most requests for new software, but we do occasionally run into difficulties. Despite most of our software being free for academics, some developers insist that each lab download and install their own software. We are happy to provide developers with the list of SBGrid members that have access to their software, but that doesn't always satisfy their requirements.
13
+We try to accommodate most requests for new software, but we do occasionally run into difficulties. Despite most of our software being free for academics, some developers insist that each lab download and install their own software, usually so that they can track usage. We are happy to provide developers with the list of SBGrid members that have access to their software, but that doesn't always satisfy their requirements.
12 14
13 15
Even when things go smoothly, license requests and working out licensing details with developers can take some time, so please don't wait until the last minute before requesting software you anticipate you may need for your project.
14 16
installation_admin.md
... ...
@@ -1,69 +0,0 @@
1
-## Managing Your SBGrid Site Installation
2
-
3
-<!-- TOC -->
4
-
5
-- [Managing Your SBGrid Site Installation](#managing-your-sbgrid-site-installation)
6
- - [The sbgrid-admin Utility](#the-sbgrid-admin-utility)
7
- - [Usage](#usage)
8
- - [Configuration Options](#configuration-options)
9
- - [Rsync Transaction Logging](#rsync-transaction-logging)
10
-
11
-<!-- /TOC -->
12
-
13
-### The sbgrid-admin Utility
14
-
15
-The sbgrid-admin utility used to install the complete software collection in what we refer to as a "Site installation". This is ideal for large labs, shared and departmental installations that use a shared network file storage for their programs. The installation is updated automatically on a monthly basis. [This installation is large and can take some time to install.](size_graph).
16
-
17
-A download link is provided in the initial site set up email, and the script is included in the installation itself at _/programs/share/sbgrid/bin/sbgrid-admin_.
18
-
19
-### Usage
20
-
21
-sbgrid-admin includes built-in help:
22
-
23
-```
24
- $ sbgrid-admin -h
25
-sbgrid-admin: a tool for administering an SBGrid Software suite installation
26
-
27
-Usage: sbgrid-admin [-chirstu]
28
-
29
- -c Cron mode. Used for automated updates of the software suite.
30
- -h Help message.
31
- -i Install the software suite on this machine.
32
- -p Print a launchd plist for updating the software on OS X machines.
33
- -r Regenerate sh/csh shell configuration files.
34
- -s Install the software branch for SGI IRIX.
35
- -t Test the configuration file.
36
- -u Force manual update of an existing installation.
37
-```
38
-
39
-The most commonly used option is _sbgrid-admin -u_ to force a manual update. This will synchronize your local installation with the SBGrid master installation hosted at Harvard Medical School.
40
-
41
-The SBGrid software configuration files use both static and dynamically generated files. The main shell initialization files, **/programs/sbgrid.cshrc** and **/programs/sbgrid.shrc** are static files, but the operating-system specific configurations are dynamically generated and customized for the packages available at your site. _sbgrid-admin -r_ will regenerate the branch-specific configuration files should you accidentally delete or alter them.
42
-
43
-### Configuration Options
44
-
45
-The _sbgrid-admin_ script has some additional configuration options to accommodate the installation requirements at different sites.
46
-
47
-The ~sbgrid/.sbgridrc configuration file for the _sbgrid-admin_ script must contain a sitename and site key. All other configurations are optional.
48
-
49
-Options take the form of _OPTION_NAME="setting"_.
50
-
51
-| **Option** | ** Example** | **Description** |
52
-| :----------------- | :----------------------: | :-------------------------------------------------------------------------- |
53
-| SB_SITENAME | sbgrid-bene | The site name for your installation provided by SBGrid |
54
-| SB_SITEKEY | xyxyxyxyxyxyxy | The installation key for your site provided by SBGrid |
55
-| INSTALL_TARGET | /nfs/programs | The installation directory for the software |
56
-| RSYNC_ARGS | "--port 8080" | Advanced arguments to rsync. Can be used to workaround blocked rsync ports. |
57
-| HTTP_PROXY | host.name:port | Proxy hostname and port for use by rsync and curl |
58
-| POST_UPDATE_SCRIPT | /absolute/path/to/script | Optional script to be run after a successful software update |
59
-| CRON_SCHEDULE | 5 1 \* \* \* | Set a custom cron schedule |
60
-
61
-### Rsync Transaction Logging
62
-
63
-If you want a date-stamped log of all rsync downloads, deletions and updates, you can add a line like this to the ~sbgrid/.sbgridrc file:
64
-
65
-```
66
-RSYNC_ARGS="--log-file=$HOME/logs/sbgrid-admin-`date +%Y%m%d`.log"
67
-```
68
-
69
-This will create a date-stamped log file any time an actual change to the software tree happens. In general this should not be necessary, but for troubleshooting purposes it can be useful.
installation_admin.md
... ...
@@ -0,0 +1 @@
1
+sbgrid-cli-admin.md
... ...
\ No newline at end of file
installation_overview.md
... ...
@@ -2,12 +2,10 @@
2 2
3 3
The SBGrid Software Suite contains [500+ Scientific software applications](https://sbgrid.org/software/) for [Linux and OS X](operatingsystems).
4 4
5
-The software collection can be installed using either our graphical or command-line installation manager:
5
+The software collection can be installed using either our graphical or command-line installation manager. For more details, see:
6 6
- [SBGrid installation manager graphical interface](sbgrid-gui).
7 7
- [SBGrid installation manager command-line interface](sbgrid-cli)
8 8
9
-We also have a script-based installation for Unix systems that are not compatible with the Linux and macOS installatino managers(BSD, AIX, etc).
10
-
11 9
---
12 10
<!-- TOC -->
13 11
... ...
@@ -22,7 +20,8 @@ We also have a script-based installation for Unix systems that are not compatibl
22 20
23 21
[An SBGrid installation that includes the complete software collection for Linux and/or MacOS.](sbgrid-cli-admin)
24 22
25
-One of the primary benefit of an SBGrid software installation is that it is a curated and actively managed software stack. New titles are added, obsolete versions are removed, and updates happen automatically. New software and bug fixes are only an email request away (bugs@sbgrid.org) without required system administration. While the installation configuration is flexible (CLI/GUI, title/versions, Linux/macOS, etc), this is the recommended mode of installation for most workstations and clusters, large labs, or shared departmental installations using network file storage for their program stack.
23
+One of the primary benefits of an SBGrid software installation is that it is a curated and actively managed software colletion. New titles are added, obsolete versions are removed, and updates happen can happen automatically. That makes new software and bug fixes only an email request away (bugs@sbgrid.org) without required system administration. While the installation configuration is flexible (CLI/GUI, title/versions, Linux/macOS, etc), this is the recommended mode of installation for most workstations and clusters, large labs, or shared departmental installations using network file storage for their program stack.
24
+
26 25
### Graphical installation for MacOS
27 26
28 27
[A Graphical stand-alone installation manager for MacOS](client_install)
laptopinstall.md
... ...
@@ -1,11 +1,9 @@
1 1
## SBGrid Software Installation : Laptops and Desktops without shared network storage
2 2
3
-If you would like to install SBGrid tools on a machine that is frequently disconnected from the network, like a laptop, or don't want to set up an NFS-shared software installation, it is possible to install the software to the hard disk on your local machine.
3
+If you would like to install SBGrid tools on a machine that is frequently disconnected from the network, like a laptop, or don't want to set up an NFS-shared software installation, it is possible to install the software to your local machine.
4 4
5
-We have a GUI-based installation client for MacOS that allows software to be selected, installed, and updated on demand from SBGrid servers. There is also a command-line client for MacOS and Linux that will allow installation and upgrade of individual titles from the command line.
5
+We have a GUI-based installation client for MacOS and Linux that allows software to be selected, installed, and updated on demand from SBGrid servers. There is also a command-line client for MacOS and Linux that will allow installation and upgrade of individual titles from the command line. For more info, see:
6 6
7 7
- [Information on installation of our MacOS GUI.](client_install)
8 8
9 9
- [Information on installation of our command-line tools for MacOS and Linux](client_CLI_install).
10
-
11
-Our sbgrid-laptop installation shell-script is no longer supported.
macos.md
... ...
@@ -1,8 +1,14 @@
1 1
## Using SBGrid with MacOS 11-14
2 2
3
-SBGrid software is compatible with macOS macOS 11 *Big Sur*, macOS 12 *Monterey*, macOS 13 Ventura and macOS 14 Sonoma. Changes to application signing verification in recent versions of macOS may require a work-around for chimerax and possibly other signed app bundles. We are working on a remedy for this.
3
+SBGrid software is compatible with the following versions of macOS:
4
+ - macOS 11 *Big Sur*
5
+ - macOS 12 *Monterey*
6
+ - macOS 13 *Ventura*
7
+ - macOS 14 *Sonoma*
4 8
5
-Initial activation of an SBGrid installation requires a reboot after account activation to accomodate macOS controls in the root partition (/) of the apple filesystem. This additional reboot is only needed after the initial account activation and creation of the _/etc/synthetic.conf_ configuration file. The latest versions for the SBGrid Installation manager (GUI) will handle the creation of the required _/etc/synthetic.conf_ file automatically.
9
+Note that hanges to application signing verification in recent versions of macOS may require a work-around for chimerax and possibly other signed app bundles. We are working on a remedy for this.
10
+
11
+SBGrid is installed at /opt/sbgrid by default and requires a link at /programs to create the proper path. Initial activation of an SBGrid installation requires a admin privleges and a reboot after account activation to accomodate macOS controls in the root partition (/) of the apple filesystem. This additional reboot is **only needed after the initial account activation** and creation of the _/etc/synthetic.conf_ configuration file. The latest versions for the SBGrid Installation manager (GUI) will handle the creation of the required _/etc/synthetic.conf_ file automatically.
6 12
7 13
For users that have upgraded macOS, [the _/etc/synthetic.conf_ file can be created manually.](#manual-creation-of-the-programs-path). This file can be removed during the upgrade process. See below.
8 14
... ...
@@ -48,79 +54,9 @@ They are not mutually exclusive.
48 54
49 55
## SBGrid on Apple M1 Silicon ARM64
50 56
51
-Apple has released ARM64 "apple silicon" macs with the "M1" class CPU. Compatibility and performance of the SBGrid software collection looks good and most applications work with Apple's "Rosetta" compatibility environment. Standards like COOT, PyMOL, Chimera, CCP4, and PHENIX work seamlessly, but there are exceptions (CCPEM, RELION, Schrodinger).
57
+Apple has released ARM64 "apple silicon" macs. Compatibility and performance of the SBGrid software collection looks good and most applications work with Apple's "Rosetta" compatibility environment. Standards like COOT, PyMOL, Chimera, CCP4, and PHENIX work seamlessly, but there are exceptions (CCPEM, Schrodinger).
52 58
53 59
## Known macOS issues
54 60
55
-- Ilastik 1.3.3 and earlier are no supported. Support coming in 1.4.
56 61
- No 32-bit support (dropped in Catalina) means no support for nmrDraw, other 32-bit applications.
57
-- Rosetta fails ( from Baker lab, not Apple Rosetta2 ). Needs to be recompiled with clang.
58 62
- X11 forwarding from Linux is sketchy with XQuartz 2.8 and later. We have had best results by downgrading to either XQuartzx [2.7.11](https://www.xquartz.org/releases/XQuartz-2.7.11.html) or [2.7.8](https://www.xquartz.org/releases/XQuartz-2.7.8.html).
59
-
60
-## Misc tips for macOS
61
-### Disable SIP from Recovery mode (not recommended)
62
-
63
-In rare cases, it may be desirable to disable Apple's System Integrity Protection. For reference, we have included the process to do that here.
64
-
65
-- Restart your Mac.
66
-- Before macOS starts up, hold down Command-R and keep it held down until you see an Apple icon and a progress bar. Release. This boots you into Recovery.
67
-- From the Utilities menu, select Terminal.
68
-- At the prompt run the following and then press Return:
69
-
70
-```
71
-csrutil disable
72
-```
73
-
74
-- Terminal should display a message that SIP was disabled.
75
-- From the apple menu, select Restart.
76
-
77
-### Remount root filesystem as read/write
78
-
79
-If SIP is disabled, it is possible to remount the filesystem as read/write. This is not required to use SBGrid, but can be useful in some circumstances. For reference we have included the instructions here.
80
-
81
-```
82
-sudo mount -uw /
83
-```
84
-
85
-### Re-enable SIP
86
-
87
-- Restart your Mac.
88
-- Before macOS starts up, hold down Command-R and keep it held down until you see an Apple icon and a progress bar. Release. This boots you into Recovery.
89
-- From the Utilities menu, select Terminal.
90
-- At the prompt, re-enbale SIP with :
91
-
92
-```
93
-csrutil enable
94
-```
95
-
96
-- Terminal should display a message that SIP was enabled.
97
-- From the apple menu, select Restart.
98
-
99
-### How to stop nagging macOS update alerts
100
-
101
-It is possible to turn off the nagging pop ups.
102
-
103
-- Open System Preferences.
104
-- Click on Software Update.
105
-- Deselect Automatically keep my Mac up to date
106
-
107
-### How to ignore macOS Sonoma update on your mac
108
-
109
-```
110
- sudo softwareupdate --ignore "macOS Sonoma"
111
-```
112
-
113
-You will see :
114
-
115
-```
116
-Ignored updates:
117
-(
118
- "macOS Sonoma"
119
-)
120
-```
121
-
122
-You can reverse this later by running :
123
-
124
-```
125
- sudo softwareupdate --reset-ignored
126
-```
managing_install.md
... ...
@@ -1,6 +1,6 @@
1 1
## SBGrid installation
2 2
3
-The SBGrid software collection can be installed on Linux and macOS from laptops to compute clusters. Often, in a shared computing environment, it is desirable to install the entire collection as a managed, self-updating software stack. We refer to this installation mode as a site install. Site installations include all the software licensed by a site and can be updated automatically.
3
+The SBGrid software collection can be installed on Linux and macOS computers from laptops to compute clusters. Often, in a shared computing environment, it is desirable to install the entire collection as a managed, self-updating software stack. We refer to this installation mode as a site installation. Site installations include all the software licensed by a site and can be updated automatically.
4 4
5 5
- [SBGrid installation](#sbgrid-installation)
6 6
- [Configuration](#configuration)
... ...
@@ -8,20 +8,18 @@ The SBGrid software collection can be installed on Linux and macOS from laptops
8 8
- [Approximate size per Branch](#approximate-size-per-branch)
9 9
- [Installation](#installation)
10 10
- [Post-Installation Steps](#post-installation-steps)
11
-- [SGI IRIX Software Branch](#sgi-irix-software-branch)
12 11
13 12
### Configuration
14 13
15 14
The SBGrid Software Suite can be installed on a single computer at your site and shared via a network filesystem (NFS) with the workstations or compute nodes on your network.
16 15
We refer to this computer as the site "master" server.
17 16
18
-The master server will require at least 2Tb of free disk space, case-sensitive file system strongly recommended. This machine and must run some variety of UNIX (Linux, macOS, FreeBSD, etc).
19
-The installation script will download the software branches you select, and install a cron job that will run regularly to check for updates.
17
+The master server will require at least 3Tb of free disk space, case-sensitive file system recommended. This machine and must run some variety of UNIX (Linux, macOS, FreeBSD, etc).
18
+The installation application will download the software branches and titles you select, and install a cron job that will run regularly to check for updates.
20 19
21
-Software installation and update are initiated from your local server at your site and downloaded from our servers.
22
-http is used to query our servers to check for update, and rsync is used to download the updates. Outbound HTTP, and RSYNC is necessary.
20
+Software installation and updates are initiated from your site and initiate downloads from our servers. External access is not required to install the software and we do not require access to your computers. The HTTPS protocal is used to query our API to check for updates, and HTTPS or rsync is used to download the software. Thus outbound HTTPS is required.
23 21
24
-NOTE - for Linux and macOS, the `sbgrid-cli` utility should be used to install and manage an installation. This approach has a number of benefits over the script-based install below and is available for Linux and macOS only. [See the sbgrid-cli recommened installation page for more information](sbgrid-cli-admin)
22
+The `sbgrid-cli` utility is used to install and manage an installation and is available for Linux and macOS. We are also using this method successfully on FreeBSD servers. [See the sbgrid-cli recommened installation page for more information](sbgrid-cli-admin)
25 23
26 24
![NFS-based installation](images/site_diagram.png "NFS-based installation")
27 25
... ...
@@ -29,46 +27,41 @@ NOTE - for Linux and macOS, the `sbgrid-cli` utility should be used to install a
29 27
30 28
| Installation Requirements | |
31 29
| ------------------------- | --------------------------------------------------------------------- |
32
-| Hard Drive Space | ~2 Tb recommended for full installation (all titles and versions for mac & linux |
30
+| Hard Drive Space | ~3 Tb recommended for full installation (all titles and versions for mac & linux |
33 31
| Operating System | Linux, macOS / OS X |
34 32
| Privileges | Administrator account needed only to create install path (/programs) |
35
-| Network | Outbound https, rsync (ports 443 and 873 or 8080) |
33
+| Network | Outbound https, optional rsync (ports 443 and 873 or 8080) |
36 34
37 35
### Approximate size per Branch
38 36
39 37
| Branch | size |
40 38
| ------------ | ------- |
41 39
| i386-mac | ~500 Gb |
42
-| x86_64-linux | ~2.0 Tb |
40
+| x86_64-linux | ~2.5 Tb |
43 41
| shared files | ~50 Gb |
44 42
45 43
[Size breakdown of titles in a typical installation](size_graph)
46 44
47
-Email [accounts@sbgrid.org](accounts@sbgrid.org) to request the installation script for the software. Be sure to include the following information in your email:
45
+Email [accounts@sbgrid.org](accounts@sbgrid.org) to request the installation credentials for the software. Be sure to include the following information in your email:
48 46
49 47
* Lab Name
50
- * Types of computers on site: Linux, OS X Intel
48
+ * Types of computers on site: Linux, macOS, GPUs etc.
51 49
* The names and email addresses of lab members that will use the software. One user may be designated to receive the majority of our correspondence if you would prefer not to hear from us too often.
52 50
53
-Once the email has been received and your information has been entered into our system, we will email you a script you will run that will begin the remote installation process. Instructions for running the script will be included in that email, and are reproduced below for reference.
51
+Once the email has been received and your information has been entered into our system, we will email you credentials and instructions to begin the remote installation process. Instructions are also included below for reference.
54 52
55 53
### Installation
56 54
57
-As state above, the `sbgrid-cli` utility should be used to install and manage an installation. This approach has a number of benefits over the script-based install. [See the sbgrid-cli recommened installation page for more information](sbgrid-cli-admin). A site installation can also be performed using the sbgrid-admin script : [sbgrid-admin](installation_admin) utility. This approach is currently supported but will be deprecated in the future.
55
+The `sbgrid-cli` utility is used to install and manage an installation. [See the sbgrid-cli recommened installation page for more information](sbgrid-cli-admin).
58 56
59 57
1. Create an 'sbgrid' user on the machine that will host your software installation. This should be a normal, unprivileged user account.
60 58
2. Log in as this 'sbgrid' user.
61
-3. Download the installation script from the URL provided in the account creation email.
62
-4. Make it executable: `chmod +x sbgrid-admin`
63
-5. Run the script: `./sbgrid-admin -i`
64
-6. The script will walk you through configuring your installation, downloading the software and installing a cron job that will keep the software updated at your site. You will need the **site name** and **install key** provided in the account creation email. The installation may take anywhere from several hours to a day, depending on your network and the number of applications available to your site.
59
+3. [Download the sbgrid-cli installation manager](downloads).
60
+4. Untar and Make the sbgrid-cli file executable with: `chmod +x sbgrid-cli`
61
+5. Run the installation: `./sbgrid-cli activate-site`
62
+6. The applications will walk you through configuring your installation, downloading the software and installing a cron job that will keep the software updated at your site. You will need the **site name** and **install key** provided in the account creation email. The installation may take anywhere from several hours to a day, depending on your network and the number of applications available to your site.
65 63
66 64
### Post-Installation Steps
67 65
68 66
1. Add a symlink in the root directory of each machine that wants to use the software. The installation script will print the suggested syntax for the command.`ln -s /path/to/sbgrid/installation /programs`
69 67
2. Any user that wishes to use the software needs to configure their shell to initialize the SBGrid shell environment. For bash and other sh-compliant shells, add the following line to your ~/.bash_profile or ~/.bashrc: `. /programs/sbgrid.shrc` For csh and tcsh, add the following line to your ~/.cshrc: `source /programs/sbgrid.cshrc`
70
-3. For some linux distributions, the shipped version of tcsh does not support shell variables longer than 4096 characters. Since the current PATH in our default shell initialization is larger than that, you will need to update tcsh. RPMs for Red Hat 4/5 and CentOS 4/5 are provided at /programs/i386-linux/system/RPMS.
71
-
72
-### SGI IRIX Software Branch
73
-
74
-Since the IRIX software branch is frozen and no longer receiving updates, it is not installed and updated in the same manner as the active software branches. If you would like access to SGI IRIX softare, please let us know at bugs@sbgrid.org. Also note that the IRIX branch is only compatible with _tcsh_; there is no sh-compatible configuration for this branch.
mirrors.md
... ...
@@ -1,3 +0,0 @@
1
-# SBGrid Mirrors
2
-
3
-For the benefit of the local SBGrid community, we recommend the following to access frequently-used open source software. If you would like for us to add a mirror, please [contact us](help@sbgrid.org).
modulefiles.md
... ...
@@ -2,18 +2,10 @@
2 2
3 3
## Overview
4 4
5
-Modulefiles for loading and unloading the sbgrid or biogrids environment are available in /programs/share/modulefiles/x86_64-linux and /programs/share/modulefiles/i386-mac. The module files are Lua format and should work with an existing LMOD installation. Loading the sbgrid/biogrids module initializes the entire sbgrid/biogrids environment and is used instead of the "source" command. If you are using modules then you should remove the source lines, such as "source sbgrid.shrc", from your startup rc files.
6
-<!-- TOC -->
7
-
8
-- [SBGrid Module Files](#sbgrid-module-files)
9
- - [Overview](#overview)
10
- - [Accessing Modules](#accessing-modules)
11
- - [Verify](#verify)
12
- - [Load](#load)
13
- - [Unload](#unload)
14
- - [Reporting Issues](#reporting-issues)
15
-
16
-<!-- /TOC -->
5
+Modulefiles for loading and unloading SBGrid applications are available in /programs/share/modulefiles/x86_64-linux and /programs/share/modulefiles/i386-mac. The module files are Lua format and should work with an existing LMOD installation using recent versions of LMOD. We do not include LMOD in the SBGrid collection.
6
+
7
+It is also possible to use modules to load the full sbgrid environment instead of the "source" command. If you are using modules then you should remove the source lines, such as "source sbgrid.shrc", from your startup rc files.
8
+
17 9
## Accessing Modules
18 10
19 11
To access the files you need to update your **MODULEPATH** environment variable.
... ...
@@ -28,15 +20,11 @@ in tcsh
28 20
29 21
The above command may be added to your shell startup files or your site Administrator may update the system wide path.
30 22
31
-## Verify
32
-
33
-**_module avail_** will show you what modules are available for loading. If your MODULEPATH is correct you should see the sbgrid/rc and/or the biogrids/rc module.
23
+## Application only
34 24
35
- $ module avail sbgrid
36
- ---------------------- /programs/share/modulefiles/i386-mac --------------------------
37
- sbgrid/rc
25
+## Full SBGrid environment
38 26
39
-## Load
27
+### Load
40 28
41 29
Loading the sbgrid/biogrids module initializes the entire environment and not individual applications. The module determines if you are using bash or tcsh and loads the shell appropriate environment using the same command for both tcsh and bash.
42 30
... ...
@@ -49,7 +37,7 @@ Loading the sbgrid/biogrids module initializes the entire environment and not in
49 37
... "remainder of startup message"
50 38
$
51 39
52
-## Unload
40
+### Unload
53 41
54 42
You can remove the sbgrid/biogrids environment with the **_module unload_** command. This removes any grid specific environment settings. If after loading sbgrid you load additonal modules, these modules will not be removed. To see what modules you have loaded you can use **_module list_**.
55 43
monolithic.md
... ...
@@ -1,13 +0,0 @@
1
-# SBGrid software environment
2
-
3
-## Overview
4
-
5
-The SBGrid 'Monolithic' software environment provides access to all installed packages using standard shell PATH with minimal required dependencies. [Standard system libraries should be sufficient to run all applications.](workstation_setup) The environment includes added features for users to manage their software version selection.
6
-
7
-You can switch between the legacy SBGrid environment and the default [capsule](capsules) environment with the _sbcap_ command. _sbcap on_ will enable the capsule environment. You can always return to the legacy configuration with the _sbcap off_ command.
8
-
9
-The sbcap setting is persistent across shells - You don't need to do this for every shell.
10
-
11
-[Version Overrides](overrides) can be configured using the sbgrid-overide or by editing .sbgrid.conf. Duplicate executables in path must be handled explicitly by the user. See [here](capsules).
12
-
13
-Please report any bugs to bugs@sbgrid.org
operatingsystems.md
... ...
@@ -15,10 +15,11 @@ SBGrid is built on tested on Red Hat Enterprise Linux 8.x, 9.x and their communi
15 15
16 16
- Rocky Linux 8 and 9
17 17
- Alma Linux 8 and 9
18
- - Oracle Linux
18
+ - Oracle Linux 8 and 9
19 19
- CentOS Stream
20 20
21
-**CentOS 7 is reaches End of Life**
21
+
22
+**CentOS 7 reaches End of Life**
22 23
Ten years ago we announced support for software on CentOS 7 and we’ve been building and running software on that OS ever since. On June 30, 2024, CentOS 7 will reach End of Life and the last of the CentOS 7 security updates.
23 24
24 25
**Rocky Linux and Alma Linux fill the gap left by the discontinued CentOS Linux stable release**
... ...
@@ -27,18 +28,24 @@ Alma and Rocky Linux are both designed to be 100% binary compatible with Red Hat
27 28
**Other Distributions**
28 29
Many labs also use Ubuntu, Mint, Debian and OpenSuSE with SBGrid. We don't explicitly test the software on distributions other than the RHEL-based distros above, but the majority of the software is known to work on Ubuntu 20 and 22. We will work to resolve problems reported by users of these Linux distributions where possible on a best-effort basis.
29 30
31
+SBGrid is supported on Amazon Linux 2 (AL2), though a few titles are known to fail due to older OS libraries used in this distribution (chimeraX).
32
+
30 33
**GPU support**
31
-We support a number of titles that run on nVidia GPU hardware. We include the required libraries for these applications and typically only the latest driver for the hardware is required.
34
+We support a number of titles that run on nVidia GPU hardware. We include the required libraries for these applications and typically only the latest driver for the hardware is required. Due to limitations in a few software titles, versions that rely on older CUDA versions (< CUDA 11.8) may fail on Ampere microarchitecture cards and later. We are working to clarify the compatility for these applications.
32 35
33
-### macOS (x86_64 and ARM64 'apple silicon')
36
+### macOS (x86_64 and ARM64 'Apple Silicon')
34 37
We build and test programs for macOS under the most four most recent macOS releases.
35 38
36 39
Currently Supported:
40
+ - macOS 11 *Big Sur*
41
+ - macOS 12 *Monterey*
42
+ - macOS 13 *Ventura*
43
+ - macOS 14 *Sonoma*
44
+
45
+As above with earlier versions of linux, there are many software applications that are fully functional on earlier versions of MacOSX in the SBGrid tree. We work to maintain an environment compatible with these OS versions, though they may not receive the latest applications.
37 46
38
-- macOS 11 - 14
39
-- As above with earlier versions of linux, there are many software applications that are fully functional on earlier versions of MacOSX in the SBGrid tree. We work to maintain an environment compatible with these OS versions, though they may not receive the latest applications.
40
-- "Apple Silicon" macs can run most of the software in the collection. Few native applications are currently available, but most 64-bit software works through Apple's Rosetta2 compatility environment. Notable exceptions are applications compiled with Intel compilers (some older Rosetta and RELION verions).
47
+"Apple Silicon" macs can run most of the software in the collection. Few native applications are currently available, but most 64-bit software works through Apple's Rosetta2 compatility environment. Notable exceptions are applications compiled with Intel compilers (some older Rosetta and RELION verions).
41 48
42
-### Windows Intel
49
+### Windows via WSL
43 50
44
-Windows provides Linux compatibility environment called "Windows Subsystem for Linux" (WSL), based on Ubuntu. WSL is still under active development. We suspect it is possible to get the SBGrid applications working with WSL2, but we have no experience with Windows can can't provide support. If you have success with SBGrid and WSL, let us know.
51
+Windows provides Linux compatibility environment called "Windows Subsystem for Linux" (WSL2). WSL2 is still under active development. Users have reported success with SBGrid applications working with WSL2, but we have no experience with Windows can can't provide support. If you have success with SBGrid and WSL2, let us know.
overrides.md
... ...
@@ -41,18 +41,35 @@ Running _sbgrid-info -L_ will print the available versions for both of the curre
41 41
42 42
## Overriding Default Versions
43 43
44
-To override the default version of a software title, you need to create an override file. The shell configuration scripts will read this file and override the default version.
44
+The default version of an application can be changed in the SBGrid environment by setting it's corresponding shell variable. To do this, simply specify the version in the correct shell variable (shown in the output above).
45 45
46
-The override file is named .sbgrid.conf and should be placed in your $HOME directory. The syntax for overrides is a simple key=value pair. E.g. it should look like this:
46
+Continuing the example above, setting the version depends on your shell type which can be shown by running `echo $SHELL`. The default shell on Linux is typically bash and macOS is typically zsh.
47 47
48
-`<APPLICATION>_X=1.2`
48
+For bash or zsh, you can set the version for PYMOL on linux using
49 49
50
-where `<APPLICATION>_X` is the name of the application followed by an underscore and the shorthand initial for the architecture. All SBGrid application variables are capitalized. The last field on the line is the software version number you want to use. The initials for each software branch are:
50
+`PYMOL_X=2.2.3`
51 51
52
-| Software Directory | Shorthand Initial |
53
-| ------------------ | ----------------- |
54
-| x86_64-linux | \_X |
55
-| i386-mac | \_M |
52
+tcsh is
53
+
54
+`setenv PYMOL_X 2.2.3`
55
+
56
+then launch the program with the _pymol_ command from the terminal normally. In this case, the 2.2.3 version will run (unless it is not installed).
57
+
58
+On macOS the procedure is the same but uses the PYMOL_M variable.
59
+
60
+
61
+## Making version changes persistent
62
+
63
+To make version changes persistent across shells, a .sbgrid.conf file is required in $HOME. The shell configuration scripts will read the $HOME/.sbgrid.conf file and set the defined version.
64
+
65
+The syntax for .sbgrid.conf is a simple key=value pair. E.g. it should look like this:
66
+
67
+```
68
+# This file can include comments
69
+<APPLICATION>_X=1.2
70
+```
71
+
72
+where `<APPLICATION>_X` is the name of the application followed by an underscore and the shorthand initial for the architecture. All SBGrid application variables are capitalized. The last field on the line is the software version number you want to use. The initials for each software branch are **_X** and **_M** for Linux and macOS respectively:
56 73
57 74
To quickly open the overrides file in your default $EDITOR, you can run _sbgrid-overrides -e_.
58 75
sbgrid-cli-admin.md
... ...
@@ -1,6 +1,6 @@
1
-## SBGrid recommended installation
1
+## SBGrid site installation
2 2
3
-One of the primary benefits of an SBGrid software installation is that it is a managed software stack. New titles are added, obsolete versions are removed, and updates happen automatically. New software and bug fixes are an email request away (bugs@sbgrid.org) without requiring local administrators or system admistration expertise. While there are other installation modes available (GUI, default versions, select titles, etc), this is the recommended mode of installation for most sites.
3
+One of the primary benefits of an SBGrid software installation is that it is a managed software stack. New titles are added, obsolete versions are removed, and updates happen automatically. This makes software updates and bug fixes available to users by an email request (bugs@sbgrid.org), all without requiring local administrators or system admistration expertise. While there are other installation modes available (GUI, default versions, select titles, etc), the site installation is the recommended mode of installation for most sites.
4 4
5 5
## Installation Steps:
6 6
... ...
@@ -45,7 +45,7 @@ tar -zxvf sbgrid-cli_linux.tar.gz
45 45
46 46
**5. Activate the installation**
47 47
48
-Activating the installation will create the required installation directories and core libraries for installing the software. Alternate installation paths can be used with the '--target flag'. The entire software collection will not be installed here - just an initial the core without the titles.
48
+Activating the installation will create the required installation directories and core libraries for installing the software. Alternate installation paths can be used with the '--target flag'. The entire software collection will not be installed here - just an initial the core without the titles. Note that "activation" here is different from other tools, such as conda. For SBGrid, this is only necessary at initial installation.
49 49
50 50
```
51 51
./sbgrid-cli activate-site --target <path to software installation location>
... ...
@@ -133,7 +133,7 @@ $ sbgrid-cli crontab --all-versions --darwin --linux -j 4
133 133
For more info on cron configuration, see [https://www.man7.org/linux/man-pages/man5/crontab.5.html](https://www.man7.org/linux/man-pages/man5/crontab.5.html)
134 134
135 135
136
-## Migrating existing script-based "sbgrid-admin" site installations
136
+## Migrating existing script-based "sbgrid-admin" site installations to sbgrid-cli
137 137
To migrate an existing site-install that uses our sbgrid-admin script, use the `sbgrid-cli migrate-admin` command as your *sbgrid* user. This will convert the legacy sbgrid-admin installation.
138 138
139 139
```
... ...
@@ -182,4 +182,61 @@ sbgrid@computer $ crontab -l
182 182
*/15 * * * * /programs/x86_64-linux/sbgrid_installer/latest/sbgrid-cli admin --all-versions
183 183
```
184 184
185
-Add the relevant `--darwin` or `--linux` flags to the crontab command if your installation should include software for macOS on linux or vice-versa, i.e. for an SBGrid installation shared across Mac and Linux workstations.
... ...
\ No newline at end of file
0
+Add the relevant `--darwin` or `--linux` flags to the crontab command if your installation should include software for macOS on linux or vice-versa, i.e. for an SBGrid installation shared across Mac and Linux workstations.
1
+
2
+
3
+### The sbgrid-admin Utility (legacy method, deprecated)
4
+
5
+The sbgrid-admin utility is our deprecated legacy method to install and manage the software collection. This script can be used to install the complete software collection in what we refer to as a "Site installation", but has a number of limitations. We are no longer using this mode for new installations but older sbgrid-admin installations are supported.
6
+
7
+### Usage
8
+
9
+sbgrid-admin includes built-in help:
10
+
11
+```
12
+ $ sbgrid-admin -h
13
+sbgrid-admin: a tool for administering an SBGrid Software suite installation
14
+
15
+Usage: sbgrid-admin [-chirstu]
16
+
17
+ -c Cron mode. Used for automated updates of the software suite.
18
+ -h Help message.
19
+ -i Install the software suite on this machine.
20
+ -p Print a launchd plist for updating the software on OS X machines.
21
+ -r Regenerate sh/csh shell configuration files.
22
+ -s Install the software branch for SGI IRIX.
23
+ -t Test the configuration file.
24
+ -u Force manual update of an existing installation.
25
+```
26
+
27
+The most commonly used option is _sbgrid-admin -u_ to force a manual update. This will synchronize your local installation with the SBGrid master installation hosted at Harvard Medical School.
28
+
29
+The SBGrid software configuration files use both static and dynamically generated files. The main shell initialization files, **/programs/sbgrid.cshrc** and **/programs/sbgrid.shrc** are static files, but the operating-system specific configurations are dynamically generated and customized for the packages available at your site. _sbgrid-admin -r_ will regenerate the branch-specific configuration files should you accidentally delete or alter them.
30
+
31
+### Configuration Options
32
+
33
+The _sbgrid-admin_ script has some additional configuration options to accommodate the installation requirements at different sites.
34
+
35
+The ~sbgrid/.sbgridrc configuration file for the _sbgrid-admin_ script must contain a sitename and site key. All other configurations are optional.
36
+
37
+Options take the form of _OPTION_NAME="setting"_.
38
+
39
+| **Option** | ** Example** | **Description** |
40
+| :----------------- | :----------------------: | :-------------------------------------------------------------------------- |
41
+| SB_SITENAME | sbgrid-bene | The site name for your installation provided by SBGrid |
42
+| SB_SITEKEY | xyxyxyxyxyxyxy | The installation key for your site provided by SBGrid |
43
+| INSTALL_TARGET | /nfs/programs | The installation directory for the software |
44
+| RSYNC_ARGS | "--port 8080" | Advanced arguments to rsync. Can be used to workaround blocked rsync ports. |
45
+| HTTP_PROXY | host.name:port | Proxy hostname and port for use by rsync and curl |
46
+| POST_UPDATE_SCRIPT | /absolute/path/to/script | Optional script to be run after a successful software update |
47
+| CRON_SCHEDULE | 5 1 \* \* \* | Set a custom cron schedule |
48
+
49
+### Rsync Transaction Logging
50
+
51
+If you want a date-stamped log of all rsync downloads, deletions and updates, you can add a line like this to the ~sbgrid/.sbgridrc file:
52
+
53
+```
54
+RSYNC_ARGS="--log-file=$HOME/logs/sbgrid-admin-`date +%Y%m%d`.log"
55
+```
56
+
57
+This will create a date-stamped log file any time an actual change to the software tree happens. In general this should not be necessary, but for troubleshooting purposes it can be useful.
sbgrid-cli.md
... ...
@@ -624,8 +624,4 @@ For more info on system configuration for SBGrid software, please check here ---
624 624
625 625
**rsync required on 873 or 8080**. The application will fail if outbound port 873 and port 8080 are blocked by your institution or are otherwise not available. You can check if you can access [port 873 here](http://portquiz.net:873) and [port 8080 here](http://portquiz.net:8080)
626 626
627
-## Deprecated releases
628
-
629
-For the unsupported [sbgrid-cli version 1, see here](cli_v1)
630
-
631 627
For help or to report issues, please email *bugs@sbgrid.org.*
sbgrid-cloudinstall.sh
... ...
@@ -1,25 +0,0 @@
1
-#!/bin/bash
2
-
3
-#echo "Downloading sbgrid installation client"
4
-curl -LO https://sbgrid.org/downloads/latest/sbgrid-cli_linux.tar.gz
5
-
6
-# Echo ""
7
-tar -zxvf sbgrid-1.0.695-Linux.tgz
8
-cd sbgrid-1.0.695-Linux
9
-
10
-echo "what is your site name"
11
-read SITE
12
-echo "what is your user name"
13
-read USERNAME
14
-echo "what is your key "
15
-read SBKEY
16
-
17
-echo "your site , user , are : "
18
-echo $SITE $USERNAME $SBKEY
19
-echo "is that correct ?"
20
-
21
-./sbgrid activate $SITE $USERNAME $SBKEY
22
-
23
-./sbgrid install sbgrid_installer
24
-
25
-source /programs/sbgrid.shrc
sbgrid-relion2017.md
... ...
@@ -1,45 +0,0 @@
1
-## SBGrid RELION 2017
2
-Thanks for attending the 2017 RELION workshop at SBGrid / HMS.
3
-Here you can find slides, notes and datasets from the workshop.
4
-
5
-## Slides and Handouts
6
-
7
-* Introduction to Linux [slides as PDF](downloads/20170321--RELION_workshop-intro_to_linux.pdf)
8
-
9
-* Intro Handout: [RELION_Workshop-Intro_handout.pdf](downloads/RELION_Workshop-Intro_handout.pdf)
10
-
11
-* Hardware Handout : [RELION_Workshop-Hardware_handout.pdf](downloads/RELION_Workshop-Hardware_handout.pdf)
12
-
13
-* Hardware Handout2 : [RELION_Workshop-Hardware2_handout.pdf](downloads/RELION_Workshop-Hardware2_handout.pdf)
14
-
15
-## Tutorial dataset and notes for the workshop
16
-
17
-* Notes : [relion_workshop_201703.pdf](downloads/relion_workshop_201703.pdf)
18
-
19
-* Supplementary notes : [supplementary_notes_201703.pdf](downloads/supplementary_notes_201703.pdf)
20
-
21
-* Data (~4Gb): [workshop.tar.gz](https://www.dropbox.com/s/6ra1wwe46rn12ca/workshop.tar.gz?dl=1)
22
-
23
-* Workshop pipeline : [workshop_pipeline.tar.gz](downloads/workshop_pipeline.tar.gz)
24
-
25
-
26
-## Older Content -- Preparing for the workshop
27
-
28
-We will be using RELION and Chimera during the workshop, primarily in Amazon EC2. To access these, you will need a modern web browser and access to WiFi.
29
-
30
-You may also want intall these applications on your laptop or desktop.
31
-SBGrid builds on CentOS 6, though these applications should work on any modern Linux distro.
32
-
33
-For Mac, we support 10.9 and higher. Keep in mind that many of the tools we are using are only available for linux.
34
-
35
-### Register for the SBGrid install client
36
-First, set up an account by registering here : [SBGrid registration](https://sbgrid.org/registration/register/)
37
-
38
-### Installation instructions
39
-A GUI version is also available for MacOS / OSX -
40
-[installation instructions can be found here](client_install).
41
-[GUI usage info is here](client_usage).
42
-
43
-For Mac, install RELION and Chimera.
44
-
45
-For Linux, Install RELION, Chimera, GCTF, CTF, and CUDA.
sharp.md
... ...
@@ -1,8 +0,0 @@
1
-## SBGrid Sharp server
2
-
3
-Due to decreased user demand and increased security considerations, SBGrid no longer maintains a user-accessible server for Global Phasing's application Sharp/AutoSharp. We now include Sharp in the SBGrid programs collection and it can be accessed through the CCP4 interface. A license from Global Phasing is required and must be locally installed.
4
-
5
-Note that Global Phasing offers several Web-based applications for data processing.
6
-Those can be accesses here : [https://www.globalphasing.com/](https://www.globalphasing.com/)
7
-
8
-For more information see [here](https://sbgrid.org/software/titles/sharpautosharp). Documentation for using Sharp can be found [here](http://www.globalphasing.com/sharp/manual/)
versions.md
... ...
@@ -2,6 +2,8 @@
2 2
3 3
Most of the software titles installed in the SBGrid software collection have several versions available. We try to provide at least three versions of each software package, although very old or very new applications may have fewer versions.
4 4
5
+Changing software versions in SBGrid is done using environment variables. Versions can be changed on-the-fly without reconfiguring the shell. Persistent changes can be made using the ~/.sbgrid.conf file.
6
+
5 7
### Display Available Versions
6 8
7 9
To list the available versions of a software title, you can use the _sbgrid_ utility. Running _sbgrid -l_ will list the versions for your current platform. From a Linux machine, a search for CCP4 versions looks like this:
... ...
@@ -48,8 +50,8 @@ APPLICATION_X is the name of the application followed by an underscore and the s
48 50
49 51
| Software Directory | Shorthand Initial |
50 52
| ------------------ | ----------------- |
51
-| x86_64-linux | \_X |
52
-| i386-mac | \_M |
53
+| x86_64-linux | _X |
54
+| i386-mac | _M |
53 55
54 56
To quickly open the overrides file in your default $EDITOR, you can run _sbgrid-overrides -e_.
55 57
workstation_setup.md
... ...
@@ -40,8 +40,6 @@ In addition to the above, on RHEL8, CentOS8 and similar distributions, you will
40 40
41 41
yum install libnsl
42 42
43
-In addition to the above, on Rocky9/AlmaLinux9, you may need to add `libxcrypt-compat` and `glibc-langpack-en` if these have not been installed as dependencies of other packages.
44
-
45 43
- gnome-terminal is required for [hkl2map](https://www.sbgrid.org/software/titles/hkl2map).
46 44
47 45
- The same 32-bit requirement applies for 64-bit Debian and Ubuntu machines. This is supported via the [Multiarch](https://wiki.debian.org/Multiarch/HOWTO) system. You'll need at least the following packages to run most of the software. Please email us if you discover we have left some out.
... ...
@@ -49,6 +47,7 @@ In addition to the above, on Rocky9/AlmaLinux9, you may need to add `libxcrypt-c
49 47
50 48
Information on configuring 3D stereo can be found [here](stereo)
51 49
52
-### Mac OS X
50
+### MacOS
53 51
54 52
- XQuartz is required to provide X11 support on macOS. The latest Xquartz version can be downloaded from the MacOSForge site here: [https://www.xquartz.org ](https://www.xquartz.org/).
53
+