.. _docker-page: ******************* Docker ******************* Introduction to Docker ======================== .. image:: https://connpass-tokyo.s3.amazonaws.com/thumbs/80/52/80521f18aec0945dfedbb471dad6aa1a.png :width: 400 What is Docker? ------------------- * Platform for developing, shipping and running applications. * Infrastructure as application / code. * First version: 2013. * Company: originally dotCloud (2010), later named Docker. * Established `Open Container Initiative `__. As a software: * `Docker Community Edition `__. * Docker Enterprise Edition. There is an increasing number of alternative container technologies and providers. Many of them are actually based on software components originally from the Docker stack and they normally try to address some specific use cases or weakpoints. As a example, **Singularity**, that we introduce later in this couse, is focused in HPC environments. Another case, **Podman**, keeps a high functional compatibility with Docker but with a different focus on technology (not keeping a daemon) and permissions. Docker components -------------------- .. image:: http://apachebooster.com/kb/wp-content/uploads/2017/09/docker-architecture.png :width: 700 * Read-only templates. * Containers are run from them. * Images are not run. * Images have several layers. .. image:: https://i.stack.imgur.com/vGuay.png :width: 700 Images versus containers ---------------------------- * **Image**: A set of layers, read-only templates, inert. * An instance of an image is called a **container**. When you start an image, you have a running container of this image. You can have many running containers of the same image. *"The image is the recipe, the container is the cake; you can make as many cakes as you like with a given recipe."* https://stackoverflow.com/questions/23735149/what-is-the-difference-between-a-docker-image-and-a-container Docker vocabulary ---------------------------- .. code-block:: console docker .. image:: images/docker_vocab.png :width: 550 Get help: .. code-block:: console docker run --help .. image:: images/docker_run_help.png :width: 550 Using existing images --------------------- Explore Docker hub ****************** Images can be stored locally or shared in a registry. `Docker hub `__ is the main public registry for Docker images. Let's search the keyword **ubuntu**: .. image:: images/dockerhub_ubuntu.png :width: 900 docker pull: import image ************************* * get latest image / latest release .. code-block:: console docker pull ubuntu .. image:: images/docker_pull.png :width: 650 * choose the version of Ubuntu you are fetching: check the different tags .. image:: images/dockerhub_ubuntu_1804.png :width: 850 .. code-block:: console docker pull ubuntu:18.04 Biocontainers ************* https://biocontainers.pro/ Specific directory of Bioinformatics related entries * Entries in `Docker hub `__ and/or `Quay.io `__ (RedHat registry) * Normally created from `Bioconda `__ Example: **FastQC** https://biocontainers.pro/#/tools/fastqc .. code-block:: console docker pull biocontainers/fastqc:v0.11.9_cv7 docker images: list images -------------------------- .. code-block:: console docker images .. image:: images/docker_images_list.png :width: 650 Each image has a unique **IMAGE ID**. docker run: run image, i.e. start a container --------------------------------------------- Now we want to use what is **inside** the image. **docker run** creates a fresh container (active instance of the image) from a **Docker (static) image**, and runs it. The format is: docker run image:tag **command** .. code-block:: console docker run ubuntu:18.04 /bin/ls .. image:: images/docker_run_ls.png :width: 200 Now execute **ls** in your current working directory: is the result the same? You can execute any program/command that is stored inside the image: .. code-block:: console docker run ubuntu:18.04 /bin/whoami docker run ubuntu:18.04 cat /etc/issue You can either execute programs in the image from the command line (see above) or **execute a container interactively**, i.e. **"enter"** the container. .. code-block:: console docker run -it ubuntu:18.04 /bin/bash Run container as daemon (in background) .. code-block:: console docker run -ti --detach ubuntu:18.04 docker run --detach ubuntu:18.04 tail -f /dev/null Run container as daemon (in background) with a given name .. code-block:: console docker run -ti --detach --name myubuntu ubuntu:18.04 docker run --detach --name myubuntu ubuntu:18.04 tail -f /dev/null docker ps: check containers status ---------------------------------- List running containers: .. code-block:: console docker ps List all containers (whether they are running or not): .. code-block:: console docker ps -a Each container has a unique ID. docker exec: execute process in running container ------------------------------------------------- .. code-block:: console docker exec myubuntu uname -a * Interactively .. code-block:: console docker exec -it myubuntu /bin/bash docker stop, start, restart: actions on container ------------------------------------------------- Stop a running container: .. code-block:: console docker stop myubuntu docker ps -a Start a stopped container (does NOT create a new one): .. code-block:: console docker start myubuntu docker ps -a Restart a running container: .. code-block:: console docker restart myubuntu docker ps -a Run with restart enabled .. code-block:: console docker run --restart=unless-stopped --detach --name myubuntu2 ubuntu:18.04 tail -f /dev/null * Restart policies: no (default), always, on-failure, unless-stopped Update restart policy .. code-block:: console docker update --restart unless-stopped myubuntu docker rm, docker rmi: clean up! -------------------------------- .. code-block:: console docker rm myubuntu docker rm -f myubuntu .. code-block:: console docker rmi ubuntu:18.04 Major clean *********** Check used space .. code-block:: console docker system df Remove unused containers (and others) - **DO WITH CARE** .. code-block:: console docker system prune Remove ALL non-running containers, images, etc. - **DO WITH MUCH MORE CARE!!!** .. code-block:: console docker system prune -a * Reference: https://www.digitalocean.com/community/tutorials/how-to-remove-docker-images-containers-and-volumes Volumes ======= Docker containers are fully isolated. It is necessary to mount volumes in order to handle input/output files. Syntax: **--volume/-v** *host:container* .. code-block:: console mkdir datatest touch datatest/test docker run --detach --volume $(pwd)/datatest:/scratch --name fastqc_container biocontainers/fastqc:v0.11.9_cv7 tail -f /dev/null docker exec -ti fastqc_container /bin/bash > ls -l /scratch > exit Ports ===== The same as with volumes, but with ports, to access Internet services. Syntax: **--publish/-p** *host:container* .. code-block:: console docker run --detach --name webserver nginx curl localhost:80 docker exec webserver curl localhost:80 docker rm -f webserver .. code-block:: console docker run --detach --name webserver --publish 80:80 nginx curl localhost:80 docker rm -f webserver .. code-block:: console docker run --detach --name webserver -p 8080:80 nginx curl localhost:80 curl localhost:8080 docker exec webserver curl localhost:80 docker exec webserver curl localhost:8080 docker rm -f webserver Volume exercises ================ 1. Copy the 2 fastq files from available datasets in Github repository and place them in mounted directory 2. Run fastqc interactively (inside container): ```fastqc /scratch/*.gz``` 3. Run fastqc non-interactively (outside the container) docker run --user ----------------- It is possible to run certain containers with a specific user, appending ```run --user```. A convenient command would be: .. code-block:: console docker run --user $(id -u):$(id -g) --detach --volume $(pwd)/datatest:/scratch --name user_test biocontainers/fastqc:v0.11.9_cv7 touch /scratch/userfile