4.8 Exercise 4 - Singularity running and building | Courses @ CRG: Reproducible research and data analysis with Linux containers and Nextflow pipelines

4.8 Exercise 4 - Singularity running and building

Example running BLAST commands in different ways

Answer

Compare with the previous Docker examples

First of all, let’s generate a blast.sif image. We have plenty of ways to do this. One example below:

singularity build blast.sif docker://ncbi/blast:2.10.1

4.8.1 Blast command-line (1)

# If not there create a DB dir
mkdir $HOME/db

cp blast.sif $HOME/db

cd $HOME/db

curl -L https://www.uniprot.org/uniprot/O75976.fasta -o O75976.fasta

curl -L https://www.uniprot.org/uniprot/Q90240.fasta -o Q90240.fasta

singularity exec blast.sif blastp -query O75976.fasta -subject Q90240.fasta

# We can mount if we prefer (as we did with Docker), but it's not strictly necessary
singularity exec -B /home/ec2-user/db:/blastdb blast.sif blastp -query /blastdb/O75976.fasta -subject /blastdb/Q90240.fasta > out.blast

singularity exec -B /home/ec2-user/db:/blastdb blast.sif blastp -query /blastdb/O75976.fasta -subject /blastdb/Q90240.fasta -out /blastdb/output.blast

4.8.2 Blast command-line (2)

# If not there create a DB dir
mkdir $HOME/db

cp blast.sif $HOME/db

cd $HOME/db

# Let's download Swissprot DB
curl -L https://ftp.ncbi.nlm.nih.gov/blast/db/FASTA/swissprot.gz -o swissprot.gz

gunzip swissprot.gz

# Let format the Swissprot DB
singularity exec blast.sif makeblastdb -dbtype prot -parse_seqids -in swissprot

We can retrieve a FASTA sequence by ID

singularity exec blast.sif blastdbcmd -dbtype prot -db swissprot -entry O75976

Example running nginx as a service (NGINX its actually the custom example in Singularity page)
- Build a NGINX image and run as an instance
- Retrieve NGINX version from the instance
- Share a simply HTML page

Answer

# Example recipe from: https://sylabs.io/guides/3.7/user-guide/running_services.html

Bootstrap: docker
From: nginx:1.20
Includecmd: no

%startscript
   nginx

We generate the image:

sudo singularity build nginx.sif nginx.singularity

Then, for using the instance. If you try as normal user:

(base) [ec2-user@ip-172-31-32-16 singularity]$ singularity instance start nginx.sif webserver
2021/04/30 12:23:12 [warn] 15#15: the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:2
2021/04/30 12:23:12 [emerg] 15#15: mkdir() "/var/cache/nginx/client_temp" failed (30: Read-only file system)
nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (30: Read-only file system)

INFO:    instance started successfully

(base) [ec2-user@ip-172-31-32-16 singularity]$ singularity instance list
INSTANCE NAME    PID      IP    IMAGE
webserver        24350          /home/ec2-user/git/CoursesCRG_Containers_Nextflow_May_2021/containers/singularity/nginx.sif

If you try with sudo permissions:

(base) [ec2-user@ip-172-31-32-16 singularity]$ sudo singularity instance start nginx.sif webserver
2021/04/30 12:24:47 [emerg] 15#15: mkdir() "/var/cache/nginx/client_temp" failed (30: Read-only file system)
nginx: [emerg] mkdir() "/var/cache/nginx/client_temp" failed (30: Read-only file system)

INFO:    instance started successfully

(base) [ec2-user@ip-172-31-32-16 singularity]$ singularity instance list
INSTANCE NAME    PID    IP    IMAGE

(base) [ec2-user@ip-172-31-32-16 singularity]$ sudo singularity instance list
INSTANCE NAME    PID      IP    IMAGE
webserver        24409          /home/ec2-user/git/CoursesCRG_Containers_Nextflow_May_2021/containers/singularity/nginx.sif

For this case, it’s necessary to allow writing in the image:

(base) [ec2-user@ip-172-31-32-16 singularity]$ sudo singularity instance start --writable-tmpfs nginx.sif webserver
2021/04/30 12:26:53 [notice] 16#16: using the "epoll" event method
2021/04/30 12:26:53 [notice] 16#16: nginx/1.20.0
2021/04/30 12:26:53 [notice] 16#16: built by gcc 8.3.0 (Debian 8.3.0-6)
2021/04/30 12:26:53 [notice] 16#16: OS: Linux 4.14.225-169.362.amzn2.x86_64
2021/04/30 12:26:53 [notice] 16#16: getrlimit(RLIMIT_NOFILE): 65535:65535
2021/04/30 12:26:53 [notice] 17#17: start worker processes
2021/04/30 12:26:53 [notice] 17#17: start worker process 18

INFO:    instance started successfully

You can check from the web browser if it worked pointing your machine.

We can get the specific NGINX version this way:

sudo singularity exec instance://webserver nginx -version

For a custom HTML file:

mkdir -p $HOME/www

cd $HOME/www

And we place a simply index.html file there:

<html>
        <body>
                Hello world!
        </body>
</html>

We execute now the command mounting where HTMLs are stored.

(base) [ec2-user@ip-172-31-32-16 singularity]$ sudo singularity instance start -B /home/ec2-user/www:/usr/share/nginx/html --writable-tmpfs nginx.sif webserver
2021/04/30 12:31:20 [notice] 15#15: using the "epoll" event method
2021/04/30 12:31:20 [notice] 15#15: nginx/1.20.0
2021/04/30 12:31:20 [notice] 15#15: built by gcc 8.3.0 (Debian 8.3.0-6)
2021/04/30 12:31:20 [notice] 15#15: OS: Linux 4.14.225-169.362.amzn2.x86_64
2021/04/30 12:31:20 [notice] 15#15: getrlimit(RLIMIT_NOFILE): 65535:65535
2021/04/30 12:31:20 [notice] 16#16: start worker processes
2021/04/30 12:31:20 [notice] 16#16: start worker process 17

INFO:    instance started successfully

NGINX is serving web pages by default at the following location: /usr/share/nginx/html