Moving data

Basic guidelines and resources

---

/ If you are unsure about any of these
| instructions email help@sbgrid.org for | \ assistence. / —————————————- \ ,, \ (oo)__ (__) )
||–|| *

Internally

Overview

Generally, systems located in the EM Facility and server rooms are interconencted at 10GBe. Desktop systems are connected at 1GBe. Due to many factors, it is unlilkely you will hit 10Gbe speeds, but your transfer will happen much faster on systems connected at 10Gbe.

rsync or mv?

It is safest to use rsync when moving between filesystems.

If moving between folders on the same filesystem it is safe to us the 'mv' command

Externally

transfer.sbgrid.org is a robust system that is set up fer being a high speed transfer node. It can be used as an ssh and rsyncd destination. It can be reached internally by connection withjust the name 'transfer. ie ssh oconnor@transfer

Overview

should already have external-ssh

rsync through ssh

rsync my source to my destination

rsyncd for paswordless deposit location

This is ideal where we may ask outside people to initiate the data transfer. Typically this
is for for directly transfering large data sets to your GPU storage from data collected from umass, janelia or somewhere else via rsycnd.

An example commant would look like - rsync -av mydatafolder rsync://rsyncuser@transfer.sbgrid.org/umass-deposit-mylabname

contact help@sbgrid.org to set up a destination for directly