afae37c7749fc68fe1e0585782684f1fa95e720e
sbgrid-cli-admin.md
... | ... | @@ -0,0 +1,115 @@ |
1 | +## SBGrid installation manager admin mode |
|
2 | + |
|
3 | +sbgrid-cli admin mode replaces the sbgrid-admin script installation method used previously to install and automatically update SBGrid installations. `sbgrid-cli admin` offers a some advantages: |
|
4 | + - parallel updates for fast, more efficient transfers |
|
5 | + - default version only for limited disk space |
|
6 | + - rolling updates for immediate releases |
|
7 | + |
|
8 | +The `sbgrid-cli admin` command will: |
|
9 | + |
|
10 | +- Install all new titles added to SBGrid |
|
11 | +- Update existing versions when newer versions are added to SBGrid |
|
12 | +- Remove obsolete verisons that are no longer supported |
|
13 | + |
|
14 | +## Installation Steps: |
|
15 | + |
|
16 | +1. Create an 'sbgrid' user on the machine that will host your software installation. |
|
17 | + This step is not strictly required, but is highly recommended. |
|
18 | + |
|
19 | +``` |
|
20 | +adduser sbgrid |
|
21 | +``` |
|
22 | + |
|
23 | +`sbgrid-cli` will attempt to use sudo to create the install path and symlink at `/programs` that is required to run the software. |
|
24 | +The sudo requirement can be skipped with the `--target` and `--no-link flags` (details below). If you would prefer the sbgrid-cli create `/programs` and the installation path, add the sbgrid user to sudo with this command: |
|
25 | + |
|
26 | +``` |
|
27 | +echo 'sbgrid ALL=(ALL) ALL' > /etc/sudoers.d/sbgrid |
|
28 | +``` |
|
29 | + |
|
30 | +2. Log in as this 'sbgrid' user |
|
31 | + |
|
32 | +``` |
|
33 | +su sbgrid |
|
34 | +``` |
|
35 | + |
|
36 | +3. Download the sbgrid-cli application here: |
|
37 | + [https://sbgrid.org/wiki/client_downloads](client_downloads) |
|
38 | + |
|
39 | +For Linux: |
|
40 | +``` |
|
41 | +curl -LO https://sbgrid-installation-manager.s3.amazonaws.com/2.2.29/sbgrid-cli_2.2.29-linux.tar.gz |
|
42 | +``` |
|
43 | +For Mac: |
|
44 | +``` |
|
45 | +curl -LO https://sbgrid-installation-manager.s3.amazonaws.com/2.2.29/sbgrid-cli_2.2.29-macos.tar.gz |
|
46 | +``` |
|
47 | + |
|
48 | +4. Untar the archive and make it executable by running 'chmod +x sbgrid-cli' |
|
49 | + |
|
50 | +``` |
|
51 | +tar -zxvf sbgrid-cli_2.2.29-linux.tar.gz && chmod +x sbgrid-cli |
|
52 | +``` |
|
53 | + |
|
54 | +5. Activate the installation |
|
55 | + |
|
56 | +Activating the installation will create the required installation paths and directories for installing the software. This is the point where alternate installation paths can be used. Please see XXXX for more info. |
|
57 | + |
|
58 | + |
|
59 | +``` |
|
60 | +./sbgrid-cli activate <site> <user> <key> |
|
61 | +``` |
|
62 | + |
|
63 | +If the sbgrid user does not have sudo access, use |
|
64 | + |
|
65 | +``` |
|
66 | +./sbgrid-cli activate <site> <user> <key> --no-link --target=<path to where the software should be installed> |
|
67 | +``` |
|
68 | +where the target path is writable. This option does not require sudo but will require manual creation of the /programs symlink to the target installation directory. This is not required for installation and updates, only for using the software. It may not be needed on servers that host the programs as a network share. |
|
69 | + |
|
70 | +6. Run the installation using `./sbgrid-cli admin` |
|
71 | + |
|
72 | +7. Add to crontab for periodic updates |
|
73 | + |
|
74 | + |
|
75 | +Post-Install Steps: |
|
76 | + |
|
77 | +1. Once the download has completed, you'll need to add a symlink in |
|
78 | + the root directory of each machine that wants to use the software. |
|
79 | + Typically, this directory is shared via NFS to each of your computers. |
|
80 | + |
|
81 | + `ln -s /path/to/sbgrid/install /programs` |
|
82 | + |
|
83 | +2. Any user that wishes to use SBGrid needs to configure their shell |
|
84 | + to initialize the SBGrid shell environment: |
|
85 | + |
|
86 | + For bash: add `. /programs/sbgrid.shrc` to ~/.bashrc |
|
87 | + For tcsh: add `source /programs/sbgrid.cshrc` to ~/.cshrc |
|
88 | + |
|
89 | +3. Configure your workstations for maximum compatibility: |
|
90 | + |
|
91 | + https://sbgrid.org/wiki/software/workstation_setup |
|
92 | + |
|
93 | + |
|
94 | +### Examples : |
|
95 | + |
|
96 | +To install all default versions of all titles, add new, update existing, remove obsolete, run : |
|
97 | +``` |
|
98 | +$ sbgrid-cli admin |
|
99 | +``` |
|
100 | + |
|
101 | +To do the same for all versions of all titles : |
|
102 | +``` |
|
103 | +$ sbgrid-cli admin --all-versions |
|
104 | +``` |
|
105 | + |
|
106 | +To make this automatic, add a crontab entry. This can be done from `sbgrid-cli` with this command : |
|
107 | + |
|
108 | +``` |
|
109 | +sbgrid-cli crontab |
|
110 | +``` |
|
111 | + |
|
112 | +For more info on cron, see [https://www.man7.org/linux/man-pages/man5/crontab.5.html](https://www.man7.org/linux/man-pages/man5/crontab.5.html) |
|
113 | + |
|
114 | +## Migrating existing site installations |
|
115 | +To migrate an existing site-install, use the `sbgrid-cli migrate-admin` command as the *sbgrid* user. This will convert the legacy sbgrid-admin installation. |
sbgrid-cli.md
... | ... | @@ -23,15 +23,18 @@ Set up an account by registering here : [SBGrid registration](https://sbgrid.org |
23 | 23 | - [Installing and removing](#installing-and-removing) |
24 | 24 | - [Updating titles](#updating-titles) |
25 | 25 | - [Remove obsolete titles](#remove-obsolete-titles) |
26 | - - [Additional subcommands](#additional-subcommands) |
|
27 | - - [Troubleshooting and diagnostics](#troubleshooting-and-diagnostics) |
|
28 | - - [Installing software for multiple platforms](#installing-software-for-multiple-platforms) |
|
29 | - - [Logging](#logging) |
|
30 | 26 | - [Automated installation and updates](#automated-installation-and-updates) |
31 | - - [Software Version Management in the SBGrid environment](#software-version-management-in-the-sbgrid-environment) |
|
32 | - - [Running SBGrid software](#running-sbgrid-software) |
|
33 | - - [Known Issues](#known-issues) |
|
34 | - - [Deprecated releases](#deprecated-releases) |
|
27 | + - [Rebuild an installation](#rebuild-an-installation) |
|
28 | +- [Troubleshooting and diagnostics](#troubleshooting-and-diagnostics) |
|
29 | + - [Check network connectivity](#check-network-connectivity) |
|
30 | + - [diagnostics and tests](#diagnostics-and-tests) |
|
31 | + - [verify configuration files](#verify-configuration-files) |
|
32 | +- [Install software for multiple platforms](#install-software-for-multiple-platforms) |
|
33 | +- [Logging](#logging) |
|
34 | +- [Software Version Management in the SBGrid environment](#software-version-management-in-the-sbgrid-environment) |
|
35 | +- [Running SBGrid software](#running-sbgrid-software) |
|
36 | +- [Known Issues](#known-issues) |
|
37 | +- [Deprecated releases](#deprecated-releases) |
|
35 | 38 | |
36 | 39 | <!-- /TOC --> |
37 | 40 | |
... | ... | @@ -160,7 +163,7 @@ Trouble activating? See Troubleshooting options further down this page. |
160 | 163 | |
161 | 164 | ### Migrating existing accounts |
162 | 165 | |
163 | -The sbgrid-cli feature allows a user to change accounts. This can be useful in a few circumstances: |
|
166 | +The sbgrid-cli feature allows a user to change between SBGrid accounts. This can be useful in a few circumstances: |
|
164 | 167 | |
165 | 168 | - Moving between SBGrid member labs |
166 | 169 | - To change the ownership of an existing installation |
... | ... | @@ -423,11 +426,34 @@ Use the clean subcommand to list or remove obsolete versions or titles |
423 | 426 | $ sbgrid-cli clean [OPTIONS...] |
424 | 427 | ``` |
425 | 428 | |
426 | -### Additional subcommands |
|
429 | +### Automated installation and updates |
|
430 | + |
|
431 | +`sbgrid-cli` can be used in an automated method to periodically install all new titles, all new versions of an updated title, and remove obsolete versions no longer included in the collection. This method is similar to the `sbgrid-admin` script used for SBGrid [*site* installations](managing_install), but with several notable benefits. `sbgrid-cli` allows rolling updates to the software collection so new titles are added continuously. Only updates are installed so data transfers are quick and efficient. |
|
432 | + |
|
433 | +For automated installation of all versions of all titles, this command should be added to crontab for the *sbgrid* software management account: |
|
434 | + |
|
435 | +``` |
|
436 | +sbgrid-cli admin --all-versions -j 4 |
|
437 | +``` |
|
427 | 438 | |
428 | -`sbgrid-cli` provides additional subcommands for automated software colletion maintenance and troubleshooting. |
|
439 | +To only install default versions ( older versions will not be installed but are retained ), use : |
|
440 | + |
|
441 | +``` |
|
442 | +sbgrid-cli admin -j 4 |
|
443 | +``` |
|
429 | 444 | |
430 | -**admin** |
|
445 | +To include software for macOS on a linux host ( ideal for NFS shares ) |
|
446 | +``` |
|
447 | +sbgrid-cli admin --all-versions -j 4 --linux --darwin |
|
448 | +``` |
|
449 | + |
|
450 | +To automate the process the commands above can be added to the crontab of the 'sbgrid' user account. |
|
451 | +This can be done with the `sbgrid-cli crontab` command (see below). |
|
452 | +For example, the line below will check for new versions every 15 minutes. |
|
453 | + |
|
454 | +``` |
|
455 | +*/15 * * * * /programs/x86_64-linux/sbgrid_installer/latest/sbgrid-cli admin --all-versions -j 4 |
|
456 | +``` |
|
431 | 457 | |
432 | 458 | Run periodic installation, update and removal. Installs all new titles, updates where appropriate and removes obsolete versions |
433 | 459 | |
... | ... | @@ -448,7 +474,36 @@ $ sbgrid-cli admin --all-versions |
448 | 474 | ``` |
449 | 475 | |
450 | 476 | will install all versions of all titles, add any new titles, update any pending updates, and remove obsolete versions. |
451 | -This |
|
477 | + |
|
478 | +The `sbgrid-cli admin` functionality replaces the legacy *sbgrid-admin* installation script. [For a tutorial on setup and migration, please see this page.](sbgrid-cli-admin) |
|
479 | + |
|
480 | +**crontab** |
|
481 | + |
|
482 | +Adds a line to the user crontab to configure periodic updates. |
|
483 | + |
|
484 | +The default is to check for new titles every 15 minutes: |
|
485 | + |
|
486 | +``` |
|
487 | +*/15 * * * * /opt/sbgrid/x86_64-linux/sbgrid_installer/latest/sbgrid-cli admin |
|
488 | +``` |
|
489 | + |
|
490 | +but can be modified with the following options. |
|
491 | + |
|
492 | +``` |
|
493 | + --all-versions This will install all the available versions for all |
|
494 | + the available titles |
|
495 | + boolean |
|
496 | + --darwin Specify macOS as a platform |
|
497 | + --linux Specify Linux as a platform |
|
498 | + -j, --num-workers <num-workers> Run installation & update tasks in parallel |
|
499 | + number, default: 4 |
|
500 | +``` |
|
501 | + |
|
502 | + |
|
503 | +For more info on cron, see [https://www.man7.org/linux/man-pages/man5/crontab.5.html](https://www.man7.org/linux/man-pages/man5/crontab.5.html) |
|
504 | + |
|
505 | + |
|
506 | +### Rebuild an installation |
|
452 | 507 | |
453 | 508 | **rebuild** |
454 | 509 | |
... | ... | @@ -458,16 +513,13 @@ This |
458 | 513 | $ sbgrid-cli rebuild |
459 | 514 | ``` |
460 | 515 | |
461 | -**crontab** |
|
462 | - |
|
463 | -Writes a basic script for crontab for _admin_ functionality |
|
464 | - |
|
465 | 516 | **save** |
466 | 517 | |
467 | 518 | The _save_ subcommand saves the current configuration so it can be restored at a later stage or replicated on another machine or in the cloud. |
468 | 519 | |
469 | -### Troubleshooting and diagnostics |
|
520 | +## Troubleshooting and diagnostics |
|
470 | 521 | |
522 | +### Check network connectivity |
|
471 | 523 | **check-connection** |
472 | 524 | |
473 | 525 | Provides network diagnostics for connectivity to SBGrid at Harvard Medical School. |
... | ... | @@ -483,14 +535,15 @@ info: Checking sbgrid.org (HTTPS) ... |
483 | 535 | info: Checking sbgrid.org (HTTP) ... |
484 | 536 | info: Connectivity test succeeded, all remote services are reachable |
485 | 537 | ``` |
486 | - |
|
538 | +### diagnostics and tests |
|
487 | 539 | **diag** |
488 | 540 | Run troubleshooting and diagnostic tests for the software collection. |
489 | 541 | |
542 | +### verify configuration files |
|
490 | 543 | **verify** |
491 | 544 | Lists missing software versions and .rc files. |
492 | 545 | |
493 | -### Installing software for multiple platforms |
|
546 | +## Install software for multiple platforms |
|
494 | 547 | |
495 | 548 | On linux, add the --darwin command to install / remove for macOS . |
496 | 549 | Conversely, use the --linux command to install / remove for Linux while on macOS. |
... | ... | @@ -504,39 +557,11 @@ Useful for servers that share a software stack over NFS. |
504 | 557 | This will list all software available for macOS. |
505 | 558 | The platform flag ( `--linux` or `--darwin`) is implied for the platform used to install the software, i.e. on linux, `--linux` is redundant. |
506 | 559 | |
507 | -### Logging |
|
560 | +## Logging |
|
508 | 561 | |
509 | 562 | `sbgrid-cli` writes log files to `$HOME/.sbgrid/logs` |
510 | 563 | |
511 | -### Automated installation and updates |
|
512 | - |
|
513 | -`sbgrid-cli` can be used in an automated method to periodically install all new titles, all new versions of an updated title, and remove obsolete versions no longer included in the collection. This method is similar to the `sbgrid-admin` script used for SBGrid [*site* installations](managing_install), but with several notable benefits. `sbgrid-cli` allows rolling updates to the software collection so new titles are added continuously. Only updates are installed so data transfers are quick and efficient. |
|
514 | - |
|
515 | -For automated installation of all versions of all titles, this command should be added to crontab for the *sbgrid* software management account: |
|
516 | - |
|
517 | -``` |
|
518 | -sbgrid-cli admin --all-versions -j 4 |
|
519 | -``` |
|
520 | - |
|
521 | -To only install default versions ( older versions will not be installed but are retained ), use : |
|
522 | - |
|
523 | -``` |
|
524 | -sbgrid-cli admin -j 4 |
|
525 | -``` |
|
526 | - |
|
527 | -To include software for macOS on a linux host ( ideal for NFS shares ) |
|
528 | -``` |
|
529 | -sbgrid-cli admin --all-versions -j 4 --linux --darwin |
|
530 | -``` |
|
531 | - |
|
532 | -To automate the process the commands above can be added to the crontab of the 'sbgrid' user account. |
|
533 | -This can be done with the `crontab -e` command. |
|
534 | -The line below will check for new versions every 15 minutes. |
|
535 | - |
|
536 | -``` |
|
537 | -*/15 * * * * /programs/x86_64-linux/sbgrid_installer/latest/sbgrid-cli admin --all-versions -j 4 |
|
538 | -``` |
|
539 | -### Software Version Management in the SBGrid environment |
|
564 | +## Software Version Management in the SBGrid environment |
|
540 | 565 | |
541 | 566 | If you have a single version of an application installed, explicit version selection in your environment is not needed. |
542 | 567 | |
... | ... | @@ -553,7 +578,7 @@ In the case of multiple non-default versions installed, the _latest release_ wil |
553 | 578 | |
554 | 579 | There is more info on how to manage versions here --> [SBGrid version overrides](versions) |
555 | 580 | |
556 | -### Running SBGrid software |
|
581 | +## Running SBGrid software |
|
557 | 582 | |
558 | 583 | To use the software at the terminal in bash, open a new terminal and run |
559 | 584 | |
... | ... | @@ -565,11 +590,11 @@ or in tcsh |
565 | 590 | |
566 | 591 | For more info on system configuration for SBGrid software, please check here ---> [Workstation setup](workstation_setup). |
567 | 592 | |
568 | -### Known Issues |
|
593 | +## Known Issues |
|
569 | 594 | |
570 | 595 | **rsync required on 873 or 8080**. The application will fail if outbound port 873 and port 8080 are blocked by your institution or are otherwise not available. You can check if you can access [port 873 here](http://portquiz.net:873) and [port 8080 here](http://portquiz.net:8080) |
571 | 596 | |
572 | -### Deprecated releases |
|
597 | +## Deprecated releases |
|
573 | 598 | |
574 | 599 | For the unsupported [sbgrid-cli version 1, see here](cli_v1) |
575 | 600 |