Error during the finding of norm peaks

[El-Castor]

Hi,

I am trying to find a norm peaks to do ATACqPCR primer using your tool. Concerning my differential region that I have to test the tools work fine. But when I am trying to do find norm peak it doesn't wort using this command :

FindNormPeaks.py -O $output_normControl_DIR -I $BAM_DIR -S $sampleName -G CMiso -narrowpeak -C $config_FindNormPeaks_PATH -R

Here you can see the error from terminal :

FindNormPeaks.py -O $output_normControl_DIR -I $BAM_DIR -S $sampleName -G CMiso -narrowpeak -C $config_FindNormPeaks_PATH -R
Removed existing flag: '/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/FindNormPeaks_failed.flag'
### [Pipeline run code and environment:]

*              Command:  `/opt/share/OTHER_Software/VIRTUALENV/ATACPrimerTool/ATACPrimerTool/pipelines/FindNormPeaks.py -O /NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam -I /NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/genrich_analysis_fullData/sorted_bam_files/sortedByPos -S peaksMale-WIP1-down3UTR -G CMiso -narrowpeak -C /NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/config/FindNormPeaks_config.yaml -R`
*         Compute host:  node15
*          Working dir:  /NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR
*            Outfolder:  /NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/
*  Pipeline started at:   (02-19 13:12:35) elapsed: 0.0 _TIME_

### [Version log:]

*       Python version:  3.7.3
*          Pypiper dir:  `/opt/share/OTHER_Software/VIRTUALENV/ATACPrimerTool/lib/python3.7/site-packages/pypiper`
*      Pypiper version:  0.10.0
*         Pipeline dir:  `/opt/share/OTHER_Software/VIRTUALENV/ATACPrimerTool/ATACPrimerTool/pipelines`
*     Pipeline version:  None
*        Pipeline hash:  b'a7b847da8bc9b239bed81ac41bff3c9795c0644e\n'
*      Pipeline branch:  b'* master\n'
*        Pipeline date:  b'2021-03-17 19:28:58 -0700\n'
*        Pipeline diff:  b''

### [Arguments passed to pipeline:]

*        `config_file`:  `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/config/FindNormPeaks_config.yaml`
*        `sample_name`:  `peaksMale-WIP1-down3UTR`
*            `recover`:  `True`
*          `new_start`:  `False`
*      `output_parent`:  `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam`
*    `genome_assembly`:  `CMiso`
*              `input`:  `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/genrich_analysis_fullData/sorted_bam_files/sortedByPos`
*            `genomeS`:  `hs`
*            `returnN`:  `500`
*              `rmdup`:  `False`
*         `narrowpeak`:  `True`

----------------------------------------


Changed status from initializing to running.

Loading config file: /NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/config/FindNormPeaks_config.yaml

Bam file directory: /NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/genrich_analysis_fullData/sorted_bam_files/sortedByPos

> `Genome`	CMiso	FindNormPeaks	_RES_
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/genrich_analysis_fullData/sorted_bam_files/sortedByPos/trimmed_ATACseq_fem2_rep4_S1_001_filtered__sortedPos.bam.bai`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/rmdup_bams/trimmed_ATACseq_fem2_rep4_S1_001_filtered__sortedPos.rmdup.bam`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/genrich_analysis_fullData/sorted_bam_files/sortedByPos/trimmed_ATACseq_fem3_rep4_S2_001_filtered__sortedPos.bam.bai`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/rmdup_bams/trimmed_ATACseq_fem3_rep4_S2_001_filtered__sortedPos.rmdup.bam`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/genrich_analysis_fullData/sorted_bam_files/sortedByPos/trimmed_ATACseq_fem4_rep4_S2_001_filtered__sortedPos.bam.bai`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/rmdup_bams/trimmed_ATACseq_fem4_rep4_S2_001_filtered__sortedPos.rmdup.bam`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/genrich_analysis_fullData/sorted_bam_files/sortedByPos/trimmed_ATACseq_herma2_rep4_S3_001_filtered__sortedPos.bam.bai`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/rmdup_bams/trimmed_ATACseq_herma2_rep4_S3_001_filtered__sortedPos.rmdup.bam`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/genrich_analysis_fullData/sorted_bam_files/sortedByPos/trimmed_ATACseq_herma3_rep4_S1_001_filtered__sortedPos.bam.bai`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/rmdup_bams/trimmed_ATACseq_herma3_rep4_S1_001_filtered__sortedPos.rmdup.bam`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/genrich_analysis_fullData/sorted_bam_files/sortedByPos/trimmed_ATACseq_herma4_S2_001_filtered__sortedPos.bam.bai`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/rmdup_bams/trimmed_ATACseq_herma4_S2_001_filtered__sortedPos.rmdup.bam`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/genrich_analysis_fullData/sorted_bam_files/sortedByPos/trimmed_ATACseq_mal2_rep4_S3_001_filtered__sortedPos.bam.bai`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/rmdup_bams/trimmed_ATACseq_mal2_rep4_S3_001_filtered__sortedPos.rmdup.bam`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/genrich_analysis_fullData/sorted_bam_files/sortedByPos/trimmed_ATACseq_mal3_rep4_S4_001_filtered__sortedPos.bam.bai`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/rmdup_bams/trimmed_ATACseq_mal3_rep4_S4_001_filtered__sortedPos.rmdup.bam`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/genrich_analysis_fullData/sorted_bam_files/sortedByPos/trimmed_ATACseq_mal4_rep4_S3_001_filtered__sortedPos.bam.bai`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/rmdup_bams/trimmed_ATACseq_mal4_rep4_S3_001_filtered__sortedPos.rmdup.bam`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/MergedPeaks.bed`
Target exists: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/MergedPeaks_counts.bed`
Found lock file: /NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/lock.FindNormPeaks_output__norm_peaks.bed
Overwriting target. . .
Target to produce: `/NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/FindNormPeaks_output/norm_peaks.bed`


> `Rscript /opt/share/OTHER_Software/VIRTUALENV/ATACPrimerTool/ATACPrimerTool/scripts/normpeaks.R /NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/MergedPeaks_counts.bed 500 /NetScratch/cpichot/ATACseq_analysis/ATACseq_cytometrie_dec2019_flowers_FMH_rep4_100K/build-Primer-ATACqPCR/output_normControl_allBam/peaksMale-WIP1-down3UTR/FindNormPeaks_output/norm_peaks.bed` (28311)

<pre>
Le chargement a nécessité le package : DESeq2
Le chargement a nécessité le package : S4Vectors
Le chargement a nécessité le package : stats4
Le chargement a nécessité le package : BiocGenerics
Le chargement a nécessité le package : parallel

Attachement du package : ‘BiocGenerics’

Les objets suivants sont masqués depuis ‘package:parallel’:

    clusterApply, clusterApplyLB, clusterCall, clusterEvalQ,
    clusterExport, clusterMap, parApply, parCapply, parLapply,
    parLapplyLB, parRapply, parSapply, parSapplyLB

Les objets suivants sont masqués depuis ‘package:stats’:

    IQR, mad, sd, var, xtabs

Les objets suivants sont masqués depuis ‘package:base’:

    anyDuplicated, append, as.data.frame, basename, cbind, colnames,
    dirname, do.call, duplicated, eval, evalq, Filter, Find, get, grep,
    grepl, intersect, is.unsorted, lapply, Map, mapply, match, mget,
    order, paste, pmax, pmax.int, pmin, pmin.int, Position, rank,
    rbind, Reduce, rownames, sapply, setdiff, sort, table, tapply,
    union, unique, unsplit, which.max, which.min


Attachement du package : ‘S4Vectors’

Les objets suivants sont masqués depuis ‘package:base’:

    expand.grid, I, unname

Le chargement a nécessité le package : IRanges
Le chargement a nécessité le package : GenomicRanges
Le chargement a nécessité le package : GenomeInfoDb
Le chargement a nécessité le package : SummarizedExperiment
Le chargement a nécessité le package : MatrixGenerics
Le chargement a nécessité le package : matrixStats

Attachement du package : ‘MatrixGenerics’

Les objets suivants sont masqués depuis ‘package:matrixStats’:

    colAlls, colAnyNAs, colAnys, colAvgsPerRowSet, colCollapse,
    colCounts, colCummaxs, colCummins, colCumprods, colCumsums,
    colDiffs, colIQRDiffs, colIQRs, colLogSumExps, colMadDiffs,
    colMads, colMaxs, colMeans2, colMedians, colMins, colOrderStats,
    colProds, colQuantiles, colRanges, colRanks, colSdDiffs, colSds,
    colSums2, colTabulates, colVarDiffs, colVars, colWeightedMads,
    colWeightedMeans, colWeightedMedians, colWeightedSds,
    colWeightedVars, rowAlls, rowAnyNAs, rowAnys, rowAvgsPerColSet,
    rowCollapse, rowCounts, rowCummaxs, rowCummins, rowCumprods,
    rowCumsums, rowDiffs, rowIQRDiffs, rowIQRs, rowLogSumExps,
    rowMadDiffs, rowMads, rowMaxs, rowMeans2, rowMedians, rowMins,
    rowOrderStats, rowProds, rowQuantiles, rowRanges, rowRanks,
    rowSdDiffs, rowSds, rowSums2, rowTabulates, rowVarDiffs, rowVars,
    rowWeightedMads, rowWeightedMeans, rowWeightedMedians,
    rowWeightedSds, rowWeightedVars

Le chargement a nécessité le package : Biobase
Welcome to Bioconductor

    Vignettes contain introductory material; view with
    'browseVignettes()'. To cite Bioconductor, see
    'citation("Biobase")', and for packages 'citation("pkgname")'.


Attachement du package : ‘Biobase’

L'objet suivant est masqué depuis ‘package:MatrixGenerics’:

    rowMedians

Les objets suivants sont masqués depuis ‘package:matrixStats’:

    anyMissing, rowMedians

[1] "Loaded DESeq2 package"
Erreur dans read.table(file = file, header = header, sep = sep, quote = quote,  : 
  no lines available in input
Appels : read.delim -> read.table
Exécution arrêtée
</pre>
Process 28311 returned 1; memory: 0.578GB.  Elapsed: 0:00:15. Peak memory: 0.578GB.

### Pipeline failed at:  (02-19 13:12:51) elapsed: 15.0 _TIME_

Total time: 0:00:15

Changed status from running to failed.
Traceback (most recent call last):
  File "/opt/share/OTHER_Software/VIRTUALENV/ATACPrimerTool/ATACPrimerTool/pipelines/FindNormPeaks.py", line 146, in <module>
    pm.run(cmd, norm_peak)
  File "/opt/share/OTHER_Software/VIRTUALENV/ATACPrimerTool/lib/python3.7/site-packages/pypiper/manager.py", line 756, in run
    self.callprint(cmd, shell, lock_file, nofail, container)  # Run command
  File "/opt/share/OTHER_Software/VIRTUALENV/ATACPrimerTool/lib/python3.7/site-packages/pypiper/manager.py", line 967, in callprint
    self._triage_error(SubprocessError(msg), nofail)
  File "/opt/share/OTHER_Software/VIRTUALENV/ATACPrimerTool/lib/python3.7/site-packages/pypiper/manager.py", line 1970, in _triage_error
    self.fail_pipeline(e)
  File "/opt/share/OTHER_Software/VIRTUALENV/ATACPrimerTool/lib/python3.7/site-packages/pypiper/manager.py", line 1556, in fail_pipeline
    raise e
pypiper.exceptions.SubprocessError: Subprocess returned nonzero result. Check above output for details

Do you have any idea to resolve the issue please ?

Thanks in advance

[katieyost]

Hi - are you able to run FindNormPeaks.py using the test data provided? It also looks like you're using "CMiso" for the genome? Have you provided all the necessary files for a custom genome annotation? The original tool was written for only hg19, hg38, mm10 or mm9 genomes.

[El-Castor]

Hi @katieyost , thanks to respond
I have laready launch ATACPrimerTool using my bamfile with m y CMiso annotation for chr and it's work. It didn't work trying for FindNormPeak. I have already test with the data test and it's work too. I hav egive my custom genome annotation file.

This error cannot due because it didn't find Norm Peaks with the given bam file?

[El-Castor]

up ?

[katieyost]

Apologies for the delay - this may be a bit difficult for me to troubleshoot since you're using a custom genome. If you can provide me with the files you are using (include the custom genome reference files and yaml file) that may help. From what I can tell from the error message it looks like the MergedPeaks_counts.bed files is empty. Can you check if this is the case (as well as the other output files generated before the pipeline fails)?