.. _nextflow_4-page:

*******************
Nextflow 4
*******************


Modules and how to re-use the code
==================================

A great advantage of the new DSL2 is to allow the **modularization of the code**.
In particular, you can move a named workflow within a module and keep it aside for being accessed by different pipelines.

The **test4** folder provides an example of using modules.


.. literalinclude:: ../nextflow/test4/test4.nf
   :language: groovy
   

We now include two modules, named **fastqc** and **multiqc**, from ```${baseDir}/modules/fastqc.nf``` and ```${baseDir}/modules/multiqc.nf```.
Let's inspect the **multiQC** module:


.. literalinclude:: ../nextflow/test4/modules/multiqc.nf
   :language: groovy


The module **multiqc** takes as **input** a channel with files containing reads and produces as **output** the files generated by the multiqc program.

The module contains the directive **publishDir**, the tag, the container to be used and has similar input, output and script session as the fastqc process in **test3.nf**.

A module can contain its own parameters that can be used for connecting the main script to some variables inside the module.

In this example we have the declaration of two **parameters** that are defined at the beginning:

.. literalinclude:: ../nextflow/test4/modules/fastqc.nf
   :language: groovy
   :emphasize-lines: 5-6


They can be overridden from the main script that is calling the module:

- The parameter **params.OUTPUT** can be used for connecting the output of this module with one in the main script.
- The parameter **params.CONTAINER** can be used for declaring the image to use for this particular module.

In this example, in our main script we pass only the OUTPUT parameters by writing them as follows:

.. literalinclude:: ../nextflow/test4/test4.nf
   :language: groovy
   :emphasize-lines: 54


While we keep the information of the container inside the module for better reproducibility:

.. literalinclude:: ../nextflow/test4/modules/multiqc.nf
   :language: groovy
   :emphasize-lines: 5


Here you see that we are not using our own image, but rather we use the image provided by **biocontainers** in `quay <https://quay.io/>`__.


Let's have a look at the **fastqc.nf** module:

.. literalinclude:: ../nextflow/test4/modules/fastqc.nf
   :language: groovy


It is very similar to the multiqc one: we just add an extra parameter for connecting the resources defined in the **nextflow.config** file and the label indicated in the process. Also in the script part there is a connection between the fastqc command line and the nnumber of threads defined in the nextflow config file.

To use this module, we have to change the main code as follows:

.. literalinclude:: ../nextflow/test4/test4.nf
   :language: groovy
   :emphasize-lines: 55

The label **twocpus** is specified in the **nextflow.config** file for each profile:

.. literalinclude:: ../nextflow/test4/modules/nextflow.config
   :language: groovy
   :emphasize-lines: 16,32,53

.. note::

	IMPORTANT: You have to specify a default image to run nextflow -with-docker or -with-singularity and you have to have a container(s) defined inside modules.

EXERCISE
===========

Make a module wrapper for the bowtie tool and change the script in test3 accordingly.

.. raw:: html

   <details>
   <summary><a>Solution</a></summary>

**Solution in the folder test5**

.. raw:: html

   </details>
|
|


Reporting and graphical interface
===================================

Nextflow has an embedded function for reporting informations about the resources requested for each job and the timing; to generate a html report, run Nextflow with the `-with-report` parameter :

.. code-block:: console

	nextflow run test5.nf -with-docker -bg -with-report > log


.. image:: images/report.png
  :width: 800


**Nextflow Tower** is an open source monitoring and managing platform for Nextflow workflows. There are two versions:

- Open source for monitoring of single pipelines.
- Commercial one for workflow management, monitoring and resource optimisation.

We will show the open source one.

First, you need to access the `tower.nf <https://tower.nf/>`__ website and login.


.. image:: images/tower.png
  :width: 800


If you selected the email for receiving the instructions and the token to be used.

.. image:: images/tower0.png
  :width: 800

check the email:

.. image:: images/tower1.png
  :width: 800



.. image:: images/tower2.png
  :width: 800


You can generate your token at `https://tower.nf/tokens <https://tower.nf/tokens>`__ and copy paste it in your pipeline using this snippet in the configuration file:

.. code-block:: groovy

	tower {
	  accessToken = '<YOUR TOKEN>'
	  enabled = true
	}


or exporting those environmental variables:

.. code-block:: groovy

	export TOWER_ACCESS_TOKEN=*******YOUR***TOKEN*****HERE*******
	export NXF_VER=21.04.0


Now we can launch the pipeline:

.. code-block:: console

	nextflow run test5.nf -with-singularity -with-tower -bg > log


	CAPSULE: Downloading dependency io.nextflow:nf-tower:jar:20.09.1-edge
	CAPSULE: Downloading dependency org.codehaus.groovy:groovy-nio:jar:3.0.5
	CAPSULE: Downloading dependency io.nextflow:nextflow:jar:20.09.1-edge
	CAPSULE: Downloading dependency io.nextflow:nf-httpfs:jar:20.09.1-edge
	CAPSULE: Downloading dependency org.codehaus.groovy:groovy-json:jar:3.0.5
	CAPSULE: Downloading dependency org.codehaus.groovy:groovy:jar:3.0.5
	CAPSULE: Downloading dependency io.nextflow:nf-amazon:jar:20.09.1-edge
	CAPSULE: Downloading dependency org.codehaus.groovy:groovy-templates:jar:3.0.5
	CAPSULE: Downloading dependency org.codehaus.groovy:groovy-xml:jar:3.0.5


and go to the tower website again:


.. image:: images/tower3.png
  :width: 800


When the pipeline is finished we can also receive a mail.


.. image:: images/tower4.png
  :width: 800