Have you checked the docs?
Description of the bug
when running umitools when process.scratch=true, i found that when the process is completed, the input bam is not only not cleaned up from the work directory but also the input bam is actually saved as a hard link rather than a symlink. the hard link's inode doesn't even match the inode of the original bam that it is linked from! when scratch is false, it is saved as a symlink, but when scratch is true, basically a redundant and unnecessary copy of the original bam is saved and can take up a significant amount of storage.
I think this is because the output channel matches all bams
|
tuple val(meta), path("*.bam") , emit: bam |
and
followLinks is by default set to true (more info here:
https://www.nextflow.io/docs/latest/process.html#output-type-path)
Command used and terminal output
PROFILE=singularity nextflow run ./tests/modules/nf-core/umitools/dedup -entry test_umitools_dedup_paired_end_no_stats -c ./tests/config/nextflow.config -c ./tests/modules/nf-core/umitools/dedup/nextflow.config -process.scratch=true
and the result in the work directory is:
$ ll work/bc/0910497b0c75f374e4ce91f2dd0d57/
total 65K
-rw-r----- 1 myuid mygid 19K Jun 8 12:03 test.bam
-rw-r----- 1 myuid mygid 19K Jun 8 12:03 test.dedup.bam
-rw-r----- 1 myuid mygid 79 Jun 8 12:03 versions.yml
### Relevant files
this is easily reproducible.
### System information
I am running on HPC but still used the local execution mode. I tried with multiple nextflow versions as well.
Have you checked the docs?
Description of the bug
when running umitools when
process.scratch=true, i found that when the process is completed, the input bam is not only not cleaned up from the work directory but also the input bam is actually saved as a hard link rather than a symlink. the hard link's inode doesn't even match the inode of the original bam that it is linked from! when scratch is false, it is saved as a symlink, but when scratch is true, basically a redundant and unnecessary copy of the original bam is saved and can take up a significant amount of storage.I think this is because the output channel matches all bams
modules/modules/nf-core/umitools/dedup/main.nf
Line 15 in fd08c25
and
followLinksis by default set to true (more info here: https://www.nextflow.io/docs/latest/process.html#output-type-path)Command used and terminal output
PROFILE=singularity nextflow run ./tests/modules/nf-core/umitools/dedup -entry test_umitools_dedup_paired_end_no_stats -c ./tests/config/nextflow.config -c ./tests/modules/nf-core/umitools/dedup/nextflow.config -process.scratch=trueand the result in the work directory is: