ImproperOutputException in line 148 of /crex/proj/sllstore2017093/b2016342/b2016342_nobackup/lts/genome_erosion_pipeline/verena_testing/maintenance/issue
s/GenErode/workflow/rules/12_snpEff.smk:
Outputs of incorrect type (directories when expecting files or vice versa). Output directories must be flagged with directory(). for rule prepare_db_buil
d:
/proj/sllstore2017093/b2016342/b2016342_nobackup/lts/genome_erosion_pipeline/verena_testing/development/testdata/gerp/outgroup_Sc9M7eS_2_HRSCAF_41/all_sc
affolds/snpEff/data/GCF_000283155.1_CerSimSim1.0_genomic.Sc9M7eS_2_HRSCAF_41/genes.gtf

Hi,

In the '8.1_vcf_CpG_filtering.smk' file, the rule 'remove_CpG_vcf' uses bedtools intersect to generate a vcf file without CpG sites, such as:

bedtools intersect -a {input.vcf} -b {input.bed} -header -sorted -g {input.genomefile} > {output.filtered} 2> {log}

However, the {output.filtered} file is not compressed and thus very large, which means that a project directory can very quickly be full.

Would it be possible to compress this vcf file to save space with something like below to generate a *vcf.gz file?

bedtools intersect -a {input.vcf} -b {input.bed} -header -sorted -g {input.genomefile} | bgzip -c > {output.filtered} 2> {log}

Much appreciated, Nic

Docker images currently on private Dockerhub repository. Add them to NBIS Dockerhub and update links to bedtools image in the rules before merging this branch.

For a user, the rule build_snpEff_db failed due to out of memory.

She'll test the following:

java -jar -Xmx64g /usr/local/share/snpeff-4.3.1t-3/snpEff.jar build -gtf22 -c {params.abs_config} -dataDir {params.abs_data_dir} -treatAllAsProteinCoding -v {params.ref_name} 2> {log}

If it works, update all snpEff rules with the java -jar -Xmx flags and automatic calculation of memory from the number of threads (like QualiMap and Picard)

Many of us are now running GenErode with snakemake 7 to avoid issues with Singularity version changes. However, Snakemake has changed their rerun behaviour in Snakemake 7.8 (see https://github.com/snakemake/snakemake/issues/1694). This means that when changing metadata tables for example, snakemake will run everything from the beginning, stating "Set of input files has changed since last execution". To get around this you can use "--rerun-triggers mtime" in the snakemake command. Also applies to any local changes in code or other parameters.

Hello,

I'm trying to set up the slurm profile with cookiecutter (using the latest version of the pipeline v0.4.2), but the options look very different to what is described in GenErode's Wiki.

This is how it looks like:

Apparently it is asking me to manually input all the configuration instead of retrieving it from the config/cluster.yaml.

I used cookiecutter earlier this year both in UPPMAX and another slurm-based cluster and it was working exactly as it is described in the Wiki. I'm not sure if the issue I'm finding now is related to recent changes in UPPMAX or in the Snakemake git profile.

Thanks in advance, I'd appreciate any suggestions on how to proceed in order to set up the Snakemake profile.

Hi,

Since the last maintenance window on uppmax, I am having issues running the pipeline and got the following error:

WorkflowError: Minimum singularity version is 2.4.1. File "/home/nicd/.conda/envs/generode/lib/python3.7/site-packages/snakemake/deployment/singularity.py", line 48, in init

I tried recreating the generode environment, as such:

conda env create -n generode -f environment.yml conda activate generode

But I still have the same issue when launching the snakemake command.

Would you mind helping me with this please?

Thanks!

Snakemake version 6 is not compatible with the latest version of the slurm profile (see https://github.com/NBISweden/GenErode/wiki/8.-FAQ#im-trying-to-run-the-pipeline-with-the-snakemake-slurm-profile-but-im-getting-the-following-error-snakemake-error-unrecognized-arguments---cluster-cancelscancel-how-do-i-solve-this)

resources: mem_mb=lambda wildcards, input, threads, attempt: 6000 * threads - 2000

unset DISPLAY qualimap bamqc -bam {input.bam} --java-mem-size=${resources.mem_mb}M -nt {threads} -outdir {output}

or: def qualimap_mem(wildcards, input, threads, attempt): return 6000 * threads - 2000

resources: mem_mb=qualimap_mem