In Bioinformatics we live at the border of data science and biomedical research. We have seen what open source software and social coding platforms have done to accelerate and define what we now know as software industry. More recently also open data as started to make its way into everyday life even if stronger efforts are currently needed. Our work design is laid on 3 major pillars: sharing, reproducibility, mobility.  

Sharing. Working with public/open data as well as with proprietary data most our systems allow us to fine grade the level or sharing we do on each project. For this we maintain projects on GitHub as well as on a own GitHub Enterprise instance. Data is mostly shared over an OwnCloud instance allowing us to share data with specific users, via password protected links, time limited sharing, and public links.

Reproducibility. As part of good scientific practices we run all our work within containerized environments – Docker, Shifter, Singularity. For this we maintain a public docker image on the Docker Hub as well as private images on a private docker registry – for images containing software with restrictive software licenses – with a Portus authentication layer for fine granularity of access rights.

Mobility. We make sure that our work and data is not sequestered by any platform or software solution. Platforms are chosen taking into account easiness in data retrieval (eg. GitHub’s and

OwnCloud’s API allow fast and efficient retrieval of all code, data, and metadata). Mobility means as well fast deployment and a democratic assess to the same computing resources. Working mostly with SLURM we have extended SLURM’s public repository to include an automated deployment of a Docker image of choice when launching High Performance Computing clusters with SLURM over Gloogle’s Cloud Platform.

Running all our operations through GitHub we systematically use GitHub for all our project management – wiki, issues, projects and respective kanban boards. In addition, we have developed “bit” a tool for seamless integration of local or remote computing, OwnCloud, and GitHub.

See Jorge’s Workflow

 

Jorge’s workflow

Leave a Reply

Your email address will not be published. Required fields are marked *