recommended

Recommended Hardware and Benchmarking

Linux Workstations

The majority of structural biology computing happens on linux; there are several vendors providing linux workstation or server configurations we've had good experience with: ThinkMate, Exxact, Microway. Please feel free to contact us at help@sbgrid.org regarding upcoming hardware, or if you have another hardware vendor to mention.

System configurations can be targeted to particular compute-intenstive workflows (for example, Cryo-EM or small molecule docking). We're in the process of assembling benchmark workflows, datasets and results to help guide hardware decisions.

Apple Workstations

Any Apple machine can be a capable computer for structural biology. The basic Apple educational discount is available through their online web store, and your institution may have negotiated an arrangement with their Apple representative. SBGrid has several labs that run exclusively on Macs and OS X, though these are not advised for CryoEM data processing.

Linux Benchmarking

Benchmarking Systems

System CPU GPU RAM (G) Network (Gb/s)
Single Workstation 8x Xeon E5-1630 v4 @3.7GHz 1x GeForce RTX 2080 128 1
Cryo-EM Cluster Node 1 2x 16 core Xeon(R) Gold 6142 @2.6GHz 7x 1080Ti 384 10
Cryo-EM Cluster Node 2 2x Xeon(R) Gold 6226R @2.9GHz 4x 2080Ti 384 10
Cryo-EM Cluster Node 3 AMD EPYC 7542 @2.9GHz 4x RTX6000 512 10
Cryo-EM Cluster Node 4 Xeon(R) E5-2698 v4 @2.2GHz 8x Tesla V100-SXM2 503 50
AWS AMD EPYC 7R32 NVIDIA A10G Tensor 64 10, burstable
Notes

Core counts determined via nproc output, and include hyperthreading where enabled.

Network capacity on AWS instances may not be directly comparable to physical systems.

Benchmarking Datasets

Dataset Number of Images Storage Space
Beta-Galactosidase 24 3.0G
Cannabinoid Receptor 1-G Protein Complex 2753 476G
Inflammasome 6594 1.6T

Dataset 1

Beta-galactosidase data set from the Namba group at Osaka University, Japan. (EMPIAR-10204, EMD-6840); courtesy of the Relion tutorial. It was collected on a JEOL CRYO ARM 200 microscope.

Dataset 2

Cannabinoid receptor 1-G Protein complex data set from Kumar et al. (EMPIAR-10288, EMD-0339, PDB-6n4b); courtesy of the Skiniotis group at Stanford University via the CryoSPARC tutorial. It was collected on a FEI Titan Krios microscope.

Dataset 3

Inflammasome data set from Sharif et al. (EMPIAR-10597, EMD-22367, PDB-7jkq); courtesy of the Wu group at Harvard University. It was collected on a FEI Titan Krios microscope.

Performance

Beta-galactosidase

Single GPU performance:

Beta-galactosidase single GPU performance

Multi-GPU performance:

Beta-galactosidase multi GPU performance

Multi-GPU Relative Runtime vs Ideal Case:

Relative GPU Runtime: Betagal

Ideal relative speed-up calculated as 1/number of GPUs.

Cannabinoid receptor 1-G protein complex

Single GPU performance:

Cannabinoid single GPU performance

Multi-GPU performance:

Cannabinoid multi-GPU performance

Multi-GPU Relative Runtime vs Ideal Case:

Relative GPU Runtime: Cannabinoid Receptor

Ideal relative speed-up calculated as 1/number of GPUs.

Performance of all three datasets on a single workstation

normalized workstation runtime

Recommendations

  • Total runtime is proportional to the number of images and storage space of the dataset. Beta-galactosidase takes 2820s (0.78h), cannabinoid receptor 67286s (18.7h), and inflammasome 586821s (163h) on a single workstation.
  • Patch Motion Correction (Multi) and Non-Uniform Refinement are the most time-consuming steps. As the number of images and storage space of the dataset increase, the time for Patch Motion Correction increases.
  • As the number of GPUs increase, the runtime for Patch Motion Correction drastically decreases.
  • Cyro-EM cluster Node 3 achieves the best performance overall.

Page edited by: Bojia Cynthia Hu, Grinnell College, Student of the 2022 Summer Scholars Program at Harvard Medical School