主页 / user-360874

The69Er's questions

Martin Hope
The69Er
Asked: 2019-07-12 14:57:33 +0800 CST
  • 1

目前，我有以下脚本用于在我创建的可重复环境中的 Unix 系统上使用 HaploTypeCaller 程序：

#!/bin/bash
#parallel call SNPs with chromosomes by GATK
for i in 1 2 3 4 5 6 7
do
  for o in A B D
  do
    for u in _part1 _part2
    do 
      (gatk HaplotypeCaller \
        -R /storage/ppl/wentao/GATK_R_index/genome.fa \
        -I GATK/MarkDuplicates/ApproachBsortedstettler.bam \
        -L chr$i$o$u \
        -O GATK/HaplotypeCaller/HaploSample.chr$i$o$u.raw.vcf &)
    done
  done
done

gatk HaplotypeCaller \
    -R /storage/ppl/wentao/GATK_R_index/genome.fa \
    -I GATK/MarkDuplicates/ApproachBsortedstettler.bam \
    -L chrUn \
    -O GATK/HaplotypeCaller/HaploSample.chrUn.raw.vcf&

如何将这段代码更改为至少部分并行？是否值得做我试图将整个脚本合并到一个不同的脚本中，你可以在这里看到一个不同的问题吗 ？我会在性能上获得相当大的提升吗？

scripting not-root-user
Martin Hope
The69Er
Asked: 2019-07-05 09:41:02 +0800 CST
  • 1

我正在为一组数据构建管道，我的主要部分是这样的

#! /bin/bash

time bwa mem -o bwa/mem/Stettler -M -t 96 -R "@RG\tID:Test\tSM:Stettler\tLB:TestLib\tPL:ILLUMINA" /storage/ppl/wentao/bwa_Index/genome.fa $1 $2
wait
echo "finished mem"
samtools view -Sb -@ 96 -o samtools/Stettler.bam bwa/mem/Stettler
wait
echo  "got stettler"
wait
time samtools sort -@ 96 -O bam -o samtools/sort/approachAsortedstettler.bam samtools/Stettler.bam
wait
echo "sorted"

time samtools index samtools/sort/approachAsortedstettler.bam
wait
echo "finished indexing"

time gatk MarkDuplicates -I samtools/sort/approachAsortedstettler.bam -O GATK/MarkDuplicates/ApproachAsortedstettler.bam -M GATK/MarkDuplicates/metrics/ApproachB
wait
echo "Marked Duplicates"
time samtools index GATK/MarkDuplicates/ApproachAsortedstettler.bam
wait
echo "indexed again ++++++++++++++++++++++++++++++++++++++++"
time bash scripts/Parallelhaplo.sh
wait
echo "Parallelhaplo"

time bash scripts/MergerHAplo.sh
wait
echo "merged"
time vcftools --vcf GATK/MergedSample_gather.raw.vcf --min-meanDP  $3 --recode --out vcftools/MergedGATKdp2.vcf
wait
echo "deep checked"
time gatk IndexFeatureFile --feature-file vcftools/MergedGATKdp2.vcf.recode.vcf
wait
echo "IFF"
time gatk SelectVariants -R /storage/ppl/wentao/GATK_R_index/genome.fa --variant vcftools/MergedGATKdp2.vcf.recode.vcf --concordance vcftools/Mergedmpileupdp2.vcf.recode.vcf -O GATK/SelectVariants/Common$
wait
echo "finished"

并且称为并行Haplo的过程看起来像这样

#!/bin/bash
#parallel call SNPs with chromosomes by GATK

for i in 1 2 3 4 5 6 7;do for o in A B D;do for u in _part1 _part2;do (gatk
 HaplotypeCaller -R /storage/ppl/wentao/GATK_R_index/genome.fa -I 
GATK/MarkDuplicates/ApproachAsortedstettler.bam -L chr$i$o$u -O 
GATK/HaplotypeCaller/HaploSample.chr$i$o$u.raw.vcf &);done;done ; done 

gatk HaplotypeCaller -R /storage/ppl/wentao/GATK_R_index/genome.fa -I 
GATK/MarkDuplicates/ApproachBsortedstettler.bam -L chrUn -O 
GATK/HaplotypeCaller/HaploSample.chrUn.raw.vcf&

wait

echo "parallel call finished"

wait

但是，当我然后执行脚本时，通常会发生 ParallelHaplo 已启动，但由于某种原因，两个脚本中的任何一个都不会等待它完成，因此它进入下一步，因为下一步找不到文件我只是得到错误。那我能做什么？

shell scripting