We're excited to release a new job to the CryoCloud platform: CryoCloud MotionCorr (MotionCorr CC) - Developed in-house and available as part of CryoCloud v2.11 released in early October
Motion Correction: the mother of all jobs
Motion correction is the first - and often the most computationally demanding - step in the cryo-EM data processing pipeline.
During data acquisition, cryo-EM exposures are not acquired as a single 2D image. Instead, each acquisition consists of multiple raw frames of shorter exposure which are stacked into a movie. Like in photography, the shorter exposures of each frame allow you to account for sample motion without resulting in blurry images. Sample motion arises due to thermal and mechanical stage drift, as well as beam-induced specimen motion. It is these sources of motion that necessitate the alignment of individual frames in the movie before summing up the signal, the so-called motion correction. During this step, patches inside the frame are aligned to each other to estimate the frame shift trajectories, correct for the shifts, and average the signal into one single micrograph with a crisp signal. The accuracy and speed of this step have a direct impact on both data quality and total processing time, which is particularly relevant for large datasets.
During motion correction, the raw data is also significantly reduced in size: movie stacks of 50 or more frames are averaged into a single micrograph, shrinking datasets from several terabytes of raw movies to only a few hundred gigabytes of motion-corrected micrographs. The large size of the raw data is what makes this job computationally demanding - and historically slow.
Up to 4x faster performance using GPU acceleration
Previously, we have used Relion's CPU-based motion correction job on CryoCloud to align raw movies. While this provided decent speed for smaller datasets and movies with fewer frames and smaller dimension, the motion correction of larger datasets would take several hours.
To speed up this process, we have re-engineered the motion correction job from the ground up. By implementing GPU acceleration, we have achieved a 4x performance boost compared to Relion's CPU-based motion correction.
In benchmark tests using three EMPIAR datasets, MotionCorr CC processed up to 5,000 movies per hour, consistently outperforming Relion's implementation on the CryoCloud platform across all tested detectors, file types and image dimensions.
CryoCloud MotionCorr significantly reduces costs
Compute costs scale directly with runtime. As such, faster processing also means substantially lower analysis costs. With 2 to 4x speed-up, motion correction jobs on CryoCloud become 50-75% cheaper compared to the legacy Relion implementation which ran on large CPU instances. Or put differently, using Relion's implementation can be up to 3.5x more expensive. This improvement reduced total cloud usage time, making large-scale or high-throughput projects significantly more cost-efficient.
Notably, for small and medium-sized datasets that are used to obtain initial 2D classes for sample screening, the motion correction step can constitute up to 40% of the end-to-end analysis time. By speeding up this step, total readout and processing time are reduced by up to 30%. While these screening datasets are relatively small and their data analysis is relatively fast, the time and cost savings add up over large screening campaigns/multiple projects.
Modern architecture and full downstream compatibility
MotionCorr CC is fully compatible with downstream polishing and CTF refinement jobs and integrates seamlessly into existing CryoCloud workflows.
It also comes with additional improvements that simplify data handling and improve reliability, such as the detection and exclusion of corrupted image files.
The in-house development of this job also means that we have full understanding and control at the algorithm level, facilitating quality control and the exploration of novel research developments to further improve motion correction.
Key features include:
• Patch-based, dose-weighted alignment
• Quality-of-life updates like direct target exposure settings for EER files (no more calculator needed!)
• Automatic gain orientation detection (coming soon)
Together, these updates make motion correction faster, more reliable, and easier to run, helping you move from raw movies to refined maps faster.
You can read the full CryoCloud v2.11 release notes here