Skip to content

Commit 0b24358

Browse files
mashehunvnieuwkmribeirodantasJulianFleschkobelavaerts
authored
Merge modules with new topics syntax (#9333)
* topics bismark/align (#9272) * topics bismark/align * fix versions command --------- Co-authored-by: Marcel Ribeiro-Dantas <marcel.ribeirodantas@seqera.io> * Converts FASTQ to use topics for version (#9266) * Converts FASTQ to use topics for version Fixes #9261 * Updates meta.yml file * Updates snapshot * Update modules/nf-core/fastqc/main.nf Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> * Update modules/nf-core/fastqc/meta.yml Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> * Updates snapshot * Removes .mix versions of FASTQ in subworkflows * Updates snapshots of fastqc modules and related subworkflows * Updates snapshots of fastqc * Updates fastqc snapshot * Sanitizes process.out for indexed outputs * Updates fastqc snapshot * Updates fastqc snapshot * Adds topic to the meta.yml file * Fixes syntax usage of topic * Adds new Devcontainer Setup for use with Codespaces (#8983) * Add devcontainer setup from nf-core/tools for testing. Move nf-test install to setup.sh * Add the default user again at the end * Move nf-test install to unprivileged location and add WORKDIR. Add pipefail. Remove copying unnecessary files. * Move Dockerfile * Remove docker-in-docker interfering with docker-outside-docker from parent image. Adapt Mounts to use host paths when starting sibling containers * Fix arm profile * Remove apptainer install * Remove efforts for unprivileged user * Use root user by default * update welcome message. update prompt * Remove explicitly setting NFT_WORKDIR * Add nf-test as local feature * Remove Dockerfile in favor of features * Remove env vars * Add apptainer as local feature * Fix install * Fix install script for apptainer by installing dependencies first * Fix install script for apptainer by installing dependencies first * Update local feature to have version, name, id * ftr: new devcontainer config * Remove redundant tools install * Remove local features. Already installed in parent image * Testing: Change to pulling image from controlled location for testing in codespaces. REVERT THIS * fix: set executable bit flag to run setup.sh * Change to final container image * Bumpy hostRequirements to run in more powerful instance in Codespaces * Switch to using dev tag of our nf-core/devcontainer * Fixes meta.yml file indenting + syntax * Fixes meta.yml with nf-core modules lint fastqc --fix * Removes accidental adding of .devcontainer * Fixes null version in subworkflow snapshot --------- Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> Co-authored-by: JulianFlesch <julianflesch@googlemail.com> # Conflicts: # .devcontainer/devcontainer.json * Multiqc2topics (#9271) * update multiqc to use topics for versions * update nf-test and snapshots * update meta.yml * update main.nf.test after review * removed and re-generated main.nf.test.snap after changes * Fixing remarks * Fix failing linting tests --------- Co-authored-by: Marcel Ribeiro-Dantas <marcel.ribeirodantas@seqera.io> * Add Topic-based tool version capture to Trinity Module (#9273) * add versions topic to trinity module * add missing test for main.nf.test * simplify string replacement for trinity version eval to remove use of colons - avoids potential yml clash * update meta.yml to capture new version topics change * add new test for aggregating versions via topic * update snapshot for new tests * Update modules/nf-core/trinity/meta.yml remove leading space Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> --------- Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> * adding topics channel to tabix_tabix (#9267) * adding topics channel * nf-test * lint meta yml * redo meta.yml * remove dup * remove tabix version * remove tabix versions from subworkflows * fix test * update snapshot * fix snapshots subworkflows * missed one subworkflow * more subworkflows * last (?) subworkflow * harshils alignmnent * too enthusiatic when removing * snaphots because bgzip versions are back * fix topics syntax for linting * Revert "fix topics syntax for linting" This reverts commit 0d25630. * version syntax changed in test * remove newline * Update modules/nf-core/tabix/tabix/main.nf Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> * Update modules/nf-core/tabix/tabix/meta.yml Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> * snapshot for new version_tabix name --------- Co-authored-by: mapo9 <mark.poslter@qbic.uni-tuebingen.de> Co-authored-by: Marcel Ribeiro-Dantas <marcel.ribeirodantas@seqera.io> Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> * Topics/samtools view (#9301) * topics bamstats generalstats * fix version * update samtools/view to use topics * feat: update samtools sort module with version topic support (#9275) * feat: update samtools sort module with version topic support * feat: update samtools sort module with version topic support * feat: update samtools sort module with version topic support * feat: update samtools sort module with version topic support * feat: update samtools sort module with version topic support * feat: update samtools sort module with version topic support * feat: update samtools sort module with version topic support * Update meta.yml and test snapshot * remove .devcontainer * Revert "remove .devcontainer" This reverts commit a97a7a2. --------- Co-authored-by: Marcel Ribeiro-Dantas <marcel@seqera.io> Co-authored-by: Marcel Ribeiro-Dantas <marcel.ribeirodantas@seqera.io> * feat: update samtools stats module with version topic support (#9330) * Adds version topic channel to SAMTOOLS_STATS * Removes version mix in subworkflow * Removes version mix for samtools sort * Updates snapshots for tests * Update modules/nf-core/samtools/stats/main.nf Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com> * Update modules/nf-core/samtools/stats/meta.yml Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com> * Fixes meta.yml * [automated] Update gpu snapshot * [automated] Update gpu snapshot * remove accidentally added file --------- Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com> Co-authored-by: nf-core-bot <core@nf-co.re> Co-authored-by: mashehu <mashehu3@gmail.com> * [automated] Update gpu snapshot * update snapshot * fix new topics structure * fix pre-commit * fix bam_split_by_region linting * fix fastq_align snapshots * update snapshots * remove references for FASTQC.out.versions * fix linting for fastq_align_chromap * update snapshots * remove old version channel for samtools/sort * fix fastq_create_umi_consensus_fgbio snapshot * fix fastq_align_star snapshots * fix id in snapshot * update vcf_gather_bcftools snapshot * update vcf_annotate_ensemblvep snapshot * fix snapshot --------- Co-authored-by: Nicolas Vannieuwkerke <101190534+nvnieuwk@users.noreply.github.com> Co-authored-by: Marcel Ribeiro-Dantas <marcel.ribeirodantas@seqera.io> Co-authored-by: JulianFlesch <julianflesch@googlemail.com> Co-authored-by: Kobe Lavaerts <77338340+kobelavaerts@users.noreply.github.com> Co-authored-by: eit-maxlcummins <mcummins@eit.org> Co-authored-by: Mark Polster <58596435+mapo9@users.noreply.github.com> Co-authored-by: mapo9 <mark.poslter@qbic.uni-tuebingen.de> Co-authored-by: wangyang <38637596+wangdepin@users.noreply.github.com> Co-authored-by: Marcel Ribeiro-Dantas <marcel@seqera.io> Co-authored-by: Simon Pearce <24893913+SPPearce@users.noreply.github.com> Co-authored-by: nf-core-bot <core@nf-co.re>
1 parent f3fb87a commit 0b24358

79 files changed

Lines changed: 1832 additions & 2042 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.pre-commit-config.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,10 @@ repos:
44
hooks:
55
- id: prettier
66
entry: prettier --experimental-cli --write --ignore-unknown --no-cache
7+
exclude: |
8+
(?x)^(
9+
.*\.snap$
10+
)$
711
additional_dependencies:
812
- prettier@3.6.2
913

‎modules/nf-core/bismark/align/main.nf‎

Lines changed: 1 addition & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ process BISMARK_ALIGN {
1616
tuple val(meta), path("*bam") , emit: bam
1717
tuple val(meta), path("*report.txt"), emit: report
1818
tuple val(meta), path("*fq.gz") , emit: unmapped, optional: true
19-
path "versions.yml" , emit: versions
19+
tuple val("${task.process}"), val("bismark"), eval('bismark --version | grep Version | sed -e "s/Bismark Version: v//" | xargs'), topic: versions, emit: versions_bismark
2020

2121
when:
2222
task.ext.when == null || task.ext.when
@@ -61,11 +61,6 @@ process BISMARK_ALIGN {
6161
--genome ${index} \\
6262
--bam \\
6363
${args}
64-
65-
cat <<-END_VERSIONS > versions.yml
66-
"${task.process}":
67-
bismark: \$(echo \$(bismark -v 2>&1) | sed 's/^.*Bismark Version: v//; s/Copyright.*\$//')
68-
END_VERSIONS
6964
"""
7065

7166
stub:
@@ -74,10 +69,5 @@ process BISMARK_ALIGN {
7469
"""
7570
touch ${prefix}.bam
7671
touch ${prefix}.report.txt
77-
78-
cat <<-END_VERSIONS > versions.yml
79-
"${task.process}":
80-
bismark: \$(echo \$(bismark -v 2>&1) | sed 's/^.*Bismark Version: v//; s/Copyright.*\$//')
81-
END_VERSIONS
8272
"""
8373
}

‎modules/nf-core/bismark/align/meta.yml‎

Lines changed: 20 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -85,13 +85,27 @@ output:
8585
description: Output FastQ file(s) containing unmapped reads
8686
pattern: "*.{fq.gz}"
8787
ontologies: []
88+
versions_bismark:
89+
- - ${task.process}:
90+
type: string
91+
description: The process the versions were collected from
92+
- bismark:
93+
type: string
94+
description: The tool name
95+
- 'bismark --version | grep Version | sed -e "s/Bismark Version: v//" | xargs':
96+
type: string
97+
description: The version of the tool
98+
topics:
8899
versions:
89-
- versions.yml:
90-
type: file
91-
description: File containing software versions
92-
pattern: "versions.yml"
93-
ontologies:
94-
- edam: http://edamontology.org/format_3750 # YAML
100+
- - ${task.process}:
101+
type: string
102+
description: The process the versions were collected from
103+
- bismark:
104+
type: string
105+
description: The tool name
106+
- 'bismark --version | grep Version | sed -e "s/Bismark Version: v//" | xargs':
107+
type: string
108+
description: The version of the tool
95109
authors:
96110
- "@phue"
97111
maintainers:

‎modules/nf-core/bismark/align/tests/main.nf.test‎

Lines changed: 5 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -53,8 +53,7 @@ nextflow_process {
5353
bam(process.out.bam[0][1]).getReadsMD5(),
5454
file(process.out.report[0][1]).readLines().contains("Number of alignments with a unique best hit from the different alignments:\t5009"),
5555
process.out.unmapped,
56-
process.out.versions,
57-
path(process.out.versions[0]).yaml
56+
process.out.findAll { key, val -> key.startsWith('versions') },
5857
).match()
5958
}
6059
)
@@ -92,8 +91,7 @@ nextflow_process {
9291
bam(process.out.bam[0][1]).getReadsMD5(),
9392
file(process.out.report[0][1]).readLines().contains("Number of alignments with a unique best hit from the different alignments:\t5009"),
9493
process.out.unmapped,
95-
process.out.versions,
96-
path(process.out.versions[0]).yaml
94+
process.out.findAll { key, val -> key.startsWith('versions') },
9795
).match()
9896
}
9997
)
@@ -128,8 +126,7 @@ nextflow_process {
128126
bam(process.out.bam[0][1]).getReadsMD5(),
129127
file(process.out.report[0][1]).readLines().contains("Number of alignments with a unique best hit from the different alignments:\t5009"),
130128
process.out.unmapped,
131-
process.out.versions,
132-
path(process.out.versions[0]).yaml
129+
process.out.findAll { key, val -> key.startsWith('versions') },
133130
).match()
134131
}
135132
)
@@ -167,8 +164,7 @@ nextflow_process {
167164
bam(process.out.bam[0][1]).getReadsMD5(),
168165
file(process.out.report[0][1]).readLines().contains("Number of alignments with a unique best hit from the different alignments:\t5009"),
169166
process.out.unmapped,
170-
process.out.versions,
171-
path(process.out.versions[0]).yaml
167+
process.out.findAll { key, val -> key.startsWith('versions') },
172168
).match()
173169
}
174170
)
@@ -203,8 +199,7 @@ nextflow_process {
203199
bam(process.out.bam[0][1]).getReadsMD5(),
204200
file(process.out.report[0][1]).readLines().contains("Number of alignments with a unique best hit from the different alignments:\t5047"),
205201
process.out.unmapped,
206-
process.out.versions,
207-
path(process.out.versions[0]).yaml
202+
process.out.findAll { key, val -> key.startsWith('versions') },
208203
).match()
209204
}
210205
)

‎modules/nf-core/bismark/align/tests/main.nf.test.snap‎

Lines changed: 45 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -5,108 +5,113 @@
55
true,
66
[
77

8-
],
9-
[
10-
"versions.yml:md5,72777174718c419301be78c0840f1931"
118
],
129
{
13-
"BISMARK_ALIGN": {
14-
"bismark": "0.25.1"
15-
}
10+
"versions_bismark": [
11+
[
12+
"BISMARK_ALIGN",
13+
"bismark",
14+
"0.25.1"
15+
]
16+
]
1617
}
1718
],
1819
"meta": {
1920
"nf-test": "0.9.2",
20-
"nextflow": "25.04.6"
21+
"nextflow": "25.10.0"
2122
},
22-
"timestamp": "2025-09-26T12:09:08.59445088"
23+
"timestamp": "2025-10-28T15:31:46.994204006"
2324
},
2425
"hisat2 | paired-end | sarscov2 genome [fasta]": {
2526
"content": [
2627
"fb9d284aec4b2c727af3c5ffd77c6381",
2728
false,
2829
[
2930

30-
],
31-
[
32-
"versions.yml:md5,72777174718c419301be78c0840f1931"
3331
],
3432
{
35-
"BISMARK_ALIGN": {
36-
"bismark": "0.25.1"
37-
}
33+
"versions_bismark": [
34+
[
35+
"BISMARK_ALIGN",
36+
"bismark",
37+
"0.25.1"
38+
]
39+
]
3840
}
3941
],
4042
"meta": {
4143
"nf-test": "0.9.2",
42-
"nextflow": "25.04.6"
44+
"nextflow": "25.10.0"
4345
},
44-
"timestamp": "2025-09-26T12:08:47.207193467"
46+
"timestamp": "2025-10-28T15:31:21.888003798"
4547
},
4648
"bowtie2 | paired-end | sarscov2 genome [fasta]": {
4749
"content": [
4850
"458c5350c3f96a8dde710985bfb62827",
4951
false,
5052
[
5153

52-
],
53-
[
54-
"versions.yml:md5,72777174718c419301be78c0840f1931"
5554
],
5655
{
57-
"BISMARK_ALIGN": {
58-
"bismark": "0.25.1"
59-
}
56+
"versions_bismark": [
57+
[
58+
"BISMARK_ALIGN",
59+
"bismark",
60+
"0.25.1"
61+
]
62+
]
6063
}
6164
],
6265
"meta": {
6366
"nf-test": "0.9.2",
64-
"nextflow": "25.04.6"
67+
"nextflow": "25.10.0"
6568
},
66-
"timestamp": "2025-09-26T12:08:02.040747075"
69+
"timestamp": "2025-10-28T15:30:37.313541027"
6770
},
6871
"bowtie2 | single-end | sarscov2 genome [fasta]": {
6972
"content": [
7073
"936c0d5ce713130113e99e09a5b53afd",
7174
true,
7275
[
7376

74-
],
75-
[
76-
"versions.yml:md5,72777174718c419301be78c0840f1931"
7777
],
7878
{
79-
"BISMARK_ALIGN": {
80-
"bismark": "0.25.1"
81-
}
79+
"versions_bismark": [
80+
[
81+
"BISMARK_ALIGN",
82+
"bismark",
83+
"0.25.1"
84+
]
85+
]
8286
}
8387
],
8488
"meta": {
8589
"nf-test": "0.9.2",
86-
"nextflow": "25.04.6"
90+
"nextflow": "25.10.0"
8791
},
88-
"timestamp": "2025-09-26T12:07:37.985126432"
92+
"timestamp": "2025-10-28T15:30:15.340678938"
8993
},
9094
"hisat2 | single-end | sarscov2 genome [fasta]": {
9195
"content": [
9296
"936c0d5ce713130113e99e09a5b53afd",
9397
true,
9498
[
9599

96-
],
97-
[
98-
"versions.yml:md5,72777174718c419301be78c0840f1931"
99100
],
100101
{
101-
"BISMARK_ALIGN": {
102-
"bismark": "0.25.1"
103-
}
102+
"versions_bismark": [
103+
[
104+
"BISMARK_ALIGN",
105+
"bismark",
106+
"0.25.1"
107+
]
108+
]
104109
}
105110
],
106111
"meta": {
107112
"nf-test": "0.9.2",
108-
"nextflow": "25.04.6"
113+
"nextflow": "25.10.0"
109114
},
110-
"timestamp": "2025-09-26T12:08:24.007957857"
115+
"timestamp": "2025-10-28T15:30:58.182901875"
111116
}
112117
}

‎modules/nf-core/fastqc/main.nf‎

Lines changed: 3 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -11,9 +11,9 @@ process FASTQC {
1111
tuple val(meta), path(reads)
1212

1313
output:
14-
tuple val(meta), path("*.html"), emit: html
15-
tuple val(meta), path("*.zip") , emit: zip
16-
path "versions.yml" , emit: versions
14+
tuple val(meta) , path("*.html") , emit: html
15+
tuple val(meta) , path("*.zip") , emit: zip
16+
tuple val("${task.process}"), val('fastqc'), eval('fastqc --version | sed "/FastQC v/!d; s/.*v//"'), emit: versions_fastqc, topic: versions
1717

1818
when:
1919
task.ext.when == null || task.ext.when
@@ -43,22 +43,12 @@ process FASTQC {
4343
--threads ${task.cpus} \\
4444
--memory ${fastqc_memory} \\
4545
${renamed_files}
46-
47-
cat <<-END_VERSIONS > versions.yml
48-
"${task.process}":
49-
fastqc: \$( fastqc --version | sed '/FastQC v/!d; s/.*v//' )
50-
END_VERSIONS
5146
"""
5247

5348
stub:
5449
def prefix = task.ext.prefix ?: "${meta.id}"
5550
"""
5651
touch ${prefix}.html
5752
touch ${prefix}.zip
58-
59-
cat <<-END_VERSIONS > versions.yml
60-
"${task.process}":
61-
fastqc: \$( fastqc --version | sed '/FastQC v/!d; s/.*v//' )
62-
END_VERSIONS
6353
"""
6454
}

‎modules/nf-core/fastqc/meta.yml‎

Lines changed: 20 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -53,13 +53,27 @@ output:
5353
description: FastQC report archive
5454
pattern: "*_{fastqc.zip}"
5555
ontologies: []
56+
versions_fastqc:
57+
- - ${task.process}:
58+
type: string
59+
description: The process the versions were collected from
60+
- fastqc:
61+
type: string
62+
description: The tool name
63+
- fastqc --version | sed "/FastQC v/!d; s/.*v//:
64+
type: string
65+
description: The command used to generate the version of the tool
66+
topics:
5667
versions:
57-
- versions.yml:
58-
type: file
59-
description: File containing software versions
60-
pattern: "versions.yml"
61-
ontologies:
62-
- edam: http://edamontology.org/format_3750 # YAML
68+
- - ${task.process}:
69+
type: string
70+
description: The process the versions were collected from
71+
- fastqc:
72+
type: string
73+
description: The tool name
74+
- fastqc --version | sed "/FastQC v/!d; s/.*v//:
75+
type: string
76+
description: The command used to generate the version of the tool
6377
authors:
6478
- "@drpatelh"
6579
- "@grst"

0 commit comments

Comments
 (0)