Containerized Serial Crystallography “CrystFEL” VISA workflow
Experiments generate up to 150 TB per day of data saved at the measurement facility. Such large datasets are impractical for users to take home.
Subsequent analysis needs to be performed remotely making it attractive for deployment as a cloud-like use case. Involving EOSC in the analysis and...
Scientific topics: crystallography
Keywords: CrystFEL, Serial crystallography, pulsed X-ray beam, VISA
Resource type: workflow
Containerized Serial Crystallography “CrystFEL” VISA workflow
https://pan-training.eu/workflows/containerized-serial-crystallography-visa-workflow
https://pan-training.eu/materials/containerized-serial-crystallography-crystfel-visa-workflow
Experiments generate up to 150 TB per day of data saved at the measurement facility. Such large datasets are impractical for users to take home.
Subsequent analysis needs to be performed remotely making it attractive for deployment as a cloud-like use case. Involving EOSC in the analysis and re-use of this data is an appropriate use case.
Serial crystallography is a beam-line technique for collecting information on the structure of a protein without growing large protein crystals. Instead, a large number of small protein crystals are held in a pulsed X-ray beam. In a second step, the series of produced images are used to reconstruct a precise 3-D image of the protein structure. Serial crystallography is the preferred technique for obtaining diffraction data of proteins at room temperature, where radiation damage from the X-ray beam starts rapidly. The standard software for analysing serial crystallography is “CrystFEL”.
The proposed workflow was rendered in a standard fashion, which would allow it to be easily adopted by arbitrary systems or also other containerized applications. The only requirements are an Apptainer installation on the system and a Docker or Singularity/Apptainer image of the application, as well as an adjustment of the configuration file for the wrapper script.
crystallography
CrystFEL, Serial crystallography, pulsed X-ray beam, VISA
research data scientist
scientific
Full-field Tomography at PSI
This workflow has some details on the instrument the data is produced from (TOMCAT beamline) and the infrastructure PSI has concerning their data.
If you are more interested in the science and want to reproduce the data and not bother with the surrounding details/context, please refer to the...
Keywords: synchrotron, imaging, Jupyter notebooks, Python, Pulmonary arterial hypertension
Resource type: workflow
Full-field Tomography at PSI
https://pan-training.eu/workflows/backup-fork-of-full-field-tomography-at-psi-wip#workflow
https://pan-training.eu/materials/full-field-tomography-at-psi
This workflow has some details on the instrument the data is produced from (TOMCAT beamline) and the infrastructure PSI has concerning their data.
If you are more interested in the science and want to reproduce the data and not bother with the surrounding details/context, please refer to the Pulmonary arterial hypertension research workflow.
Full-field Tomography at PSI
Tomography datasets often present large volumes (100 GBs - few TBs) difficult to compress and transfer. The tomographic reconstruction is highly demanding on compute (GPU) and storage resources for the intermediate and/or final result. In addition, the optional image segmentation step may be demanding on computer memory.
The offline analysis (after experiment) could be performed remotely by users at home making it attractive for deployment as a cloud-like use case. Finally, this technique is applied at many facilities and in different scientific domains - therefore a portable result is more useful.
This entire process is illustrated with a typical experiment.
synchrotron, imaging, Jupyter notebooks, Python, Pulmonary arterial hypertension
research data scientist
life scientists