Question

在SGE群集上运行。直到最近我的集群人员告诉我，我才需要启动“这些工作正在使存储中的bossock光纤超载，并且该工作的IO配置文件使我们的存储遇到了麻烦服务请求。我们将需要输出文件，可能还需要输入 snakemake在暂存空间中要读写的文件，然后您将需要复制任何想要返回的内容并从头开始删除文件。”

我一直使用以下bash脚本提交snakemake。

#!/bin/bash
#Submit to the cluster, give it a unique name
#$ -S /bin/bash

#$ -V
#$ -l h_vmem=1.9G,h_rt=20:00:00,tmem=1.9G
#$ -l tscratch=100G
#$ -pe smp 2

# join stdout and stderr output
#$ -j y
#$ -R y


if [ "$1" != "" ]; then
    RUN_NAME=$1
else
    RUN_NAME=$""
fi

#setup scratch space
scratch=/scratch0/annbrown/$JOB_ID



#FOLDER=submissions/$(date +"%Y%m%d%H%M")
FOLDER=${scratch}submissions/$(date +"%Y%m%d%H%M")
mkdir -p $FOLDER
# make /a symlink of my snakemake pipeline in the scratch space
ln -s /home/annbrown/pipelines/rna_seq_snakemake $FOLDER

cd $FOLDER/rna_seq_snakemake


cp config/config.yaml $FOLDER/$RUN_NAMEconfig.yaml


snakemake -s rna_seq.snakefile \
--jobscript cluster_qsub.sh \
--cluster-config config/cluster.yaml \
--cluster-sync "qsub -R y -l h_vmem={cluster.h_vmem},h_rt={cluster.h_rt} -pe {cluster.pe} -o $FOLDER" \
-j 25 \
--nolock \
--rerun-incomplete \
--latency-wait 100

#copy work out of scratch -you may need to change the destination

cp -r $FOLDER ~/annbrown

#delete scratch once you have finished (very important)
rm -rf $scratch

但是当蛇制作开始时，它仍然会将提交的临时文件写入错误的位置

/SAN/vyplab/alb_projects/pipelines/rna_seq_snakemake/.snakemake/tmp.gkxzqok7/snakejob.run_star_pe.139.sh
#$ -S /bin/bash

#$ -cwd
#$ -V
#$ -l h_vmem=4G,h_rt=6:00:00,tmem=4G


# join stdout and stderr output
#$ -j y
#$ -sync y
#$ -R y




cd /SAN/vyplab/alb_projects/pipelines/rna_seq_snakemake && \
/share/apps/python-3.7.2-shared/bin/python3.7 \
-m snakemake /SAN/vyplab/alb_projects/data/muscle/analysis/STAR_aligned/12_9_5_18.Aligned.out.bam --snakefile /SAN/vyplab/alb_projects/pipelines/rna_seq_snakemake/rna_seq.snakefile \
--force -j --keep-target-files --keep-remote \
--wait-for-files /SAN/vyplab/alb_projects/pipelines/rna_seq_snakemake/.snakemake/tmp.gkxzqok7 /SAN/vyplab/vyplab_reference_genomes/STAR/human/GRCh38/star_indices_overhang150/SA /SAN/vyplab/alb_projects/data/muscle/analysis/merged_fastqs/12_9_5_18_1.merged.fastq.gz /SAN/vyplab/alb_projects/data/muscle/analysis/merged_fastqs/12_9_5_18_2.merged.fastq.gz --latency-wait 100 \
 --attempt 1 --force-use-threads \
--wrapper-prefix https://bitbucket.org/snakemake/snakemake-wrappers/raw/ \
   --allowed-rules run_star_pe --nocolor --notemp --no-hooks --nolock \
--mode 2

有人能在临时的临时场地上跑蛇吗？我不确定我是要使用影子指令还是说实话。

让所有snakemake在SGE集群的临时暂存空间中运行

0 个答案: