Failed to pull singularity image

[xhuashen]

Hi，
I encounter a problems
(base) [shenlongxin@qiting-svr1 /qitinglab/software/CADD-scripts-1.7.3]$ ./CADD.sh /qitinglab/sharedata/De_novo_mutation_DB/new_try/Gene4Denovo2/split_for_website/DNM_part01.vcf.gz -o /qitinglab/shenlongxin/test.tsv.gz -c 16 -m
CADD-v1.7 (c) University of Washington, Hudson-Alpha Institute for Biotechnology and Berlin Institute of Health at Charite - Universitatsmedizin Berlin 2013-2024. All rights reserved.
Running snakemake pipeline:
snakemake /qitinglab/shenlongxin/cache/tmp/tmp.lvju9qgLha/DNM_part01.tsv.gz --sdm conda apptainer --apptainer-prefix /qitinglab/software/CADD-scripts-1.7.3/envs/apptainer --singularity-args "--bind /qitinglab/shenlongxin/cache/tmp/tmp.lvju9qgLha " --conda-prefix /qitinglab/software/CADD-scripts-1.7.3/envs/conda --cores 1 --configfile /qitinglab/software/CADD-scripts-1.7.3/config/config_GRCh38_v1.7_noanno.yml --snakefile /qitinglab/software/CADD-scripts-1.7.3/Snakefile -q
host: qiting-svr1
WorkflowError:
Failed to pull singularity image from docker://visze/cadd-scripts-v1_7:0.1.1:
FATAL: While making image from oci registry: error fetching image to cache: failed to get checksum for docker://visze/cadd-scripts-v1_7:0.1.1: pinging container registry registry-1.docker.io: Get "https://registry-1.docker.io/v2/": dial tcp 74.86.151.162:443: i/o timeout
how can I fixed it, why it need container. I want use conda to perform it.

[visze]

Do you have internet access on the machine where you run the command?

[xhuashen]

@visze sorry, I guess I observed a bug, the parameter -m seems is order specific, when I change the order of -m parameter, its ok. unfortunately, new issues comes:
(base) [shenlongxin@qiting-svr1 /qitinglab/software/CADD-scripts-1.7.3]$ ./CADD.sh -m /qitinglab/sharedata/De_novo_mutation_DB/new_try/Gene4Denovo2/split_for_website/DNM_part01.vcf.gz -o /qitinglab/shenlongxin/test.tsv.gz
CADD-v1.7 (c) University of Washington, Hudson-Alpha Institute for Biotechnology and Berlin Institute of Health at Charite - Universitatsmedizin Berlin 2013-2024. All rights reserved.
Running snakemake pipeline:
snakemake /qitinglab/shenlongxin/cache/tmp/tmp.7iPy36v5Th/DNM_part01.tsv.gz --sdm conda --conda-prefix /qitinglab/software/CADD-scripts-1.7.3/envs/conda --cores 1 --configfile /qitinglab/software/CADD-scripts-1.7.3/config/config_GRCh38_v1.7_noanno.yml --snakefile /qitinglab/software/CADD-scripts-1.7.3/Snakefile -q
host: qiting-svr1
CreateCondaEnvironmentException:
Could not create conda environment from /qitinglab/software/CADD-scripts-1.7.3/envs/esm.yml:
Command:
conda env create --quiet --no-default-packages --file "/qitinglab/software/CADD-scripts-1.7.3/envs/conda/4224b1c03dcc7adfb74ac46676a3fdb0_.yaml" --prefix "/qitinglab/software/CADD-scripts-1.7.3/envs/conda/4224b1c03dcc7adfb74ac46676a3fdb0_"
Output:
Channels:

• conda-forge
• pytorch
• nvidia
• bioconda
  Platform: linux-64
  Collecting package metadata (repodata.json): ...working... done
  Solving environment: ...working... failed

LibMambaUnsatisfiableError: Encountered problems while solving:

• package torchaudio-0.13.1-py39_cu117 is excluded by strict repo priority
• package pytorch-cuda-11.6-h867d48c_0 requires cuda 11.6.*, but none of the providers can be installed
• nothing provides __cuda needed by pytorch-1.13.1-cuda112py310he33e0d6_200

[xhuashen]

@visze my server is inner Internet access, may be limited to access some website

[visze]

Okay. now it seems to use conda only.

The issue is that it tries to generate the conda envs but fails.... Unfortunately this is not really on our hand. When we initial realeased it it works. That's why we also provide an docker solution (using apptainer/singuarity) where all dependencies are already packed.

You can try to modifly this file to get it run: envs/esm.yml:

Or try to get the docker approach but it seems you cannot access teh docker registry.

[visze]

Ah. maybe this is the reason whhy it fails. CADD v1.7 needs a GPU tu run on. Do you have a sytsemw ith aGPU and cuda installed?

[xhuashen]

I haven't GPU, is that necessary?

[visze]

One feature is the LLM ESM that requires a GPU for computation. We already managed it to run it on a CPU but is is incredible slow.

So I am not sure about the conda environment if it will only work in a GPU env. You can try to install is by yourself, changing the parameters/versions to see if it builds.

We recommend to use the dockerized environment.

[visze]

BTW: recompilation of the scores is only required for indels. All SNVs are pre-computed and it is much easier to use the pre-computed tabix files than the whole framework

[xhuashen]

@visze thanks so much, I just noticed this, so I can use your online website to calculate very limited number of indel variant, and use the pre-computed whole genome SNV file to extract the CADD info for my query SNV

[visze]

I just saw this docker image where they packed everything into one image with multi thread approach as well as vpu support:

Maybe it helps:

https://hub.docker.com/r/cmgantwerpen/cadd

[xhuashen]

@visze thank so much, I will test it later.