Containerization technologies such as Docker and Singularity enable replicable software stacks and compute environments which can be deployed on heterogeneous computational platforms from HPCs to cloud instances. Workflow languages such as Nextflow and Workflow Definition Language (WDL) enable one to automate data cleaning and analysis processes making it easier to test various parameters and tools and enables reproducible science without the computational overhead of virtual machines.
Africa research and training grant projects that will use the tools as part of their analysis and research.
The workshop is aimed at the intermediate/advanced level and will be most useful to data analysts who
will be providing support to their projects and already have some experience analyzing big data on
different computing environments such as HPC.
After this workshop participants should:
● Be familiar with containerization technologies and applications
● Be able to containerize and deploy analysis tools on different environments
● Be familiar with workflow languages and its applications
● Be able to create workflows for their specific analysis use case
● Be familiar with Dockstore and how to submit and pull workflows
This workshop will only provide a foundational basis for continued learning and application of containerization and workflows for research data analysis and will not make one an expert in these technologies, or cover any specific data analysis methods.
● Familiarity with the command line and computational environments such as HPC, OpenStack
● Good knowledge of unix / linux
● Involved in the cleaning, processing and analysis of a DS-I Africa project data
● Utilization of software analysis tools for research / teaching
● Specific use cases for the processing and analysis of data