Core Dumped

[rtemanni]

Hi,
I'm running GenomicsDB 0.9.2 using 9720 files and 5000 partitions and I'm getting the error below in 1000+ jobs where other complete successfully:
terminate called after throwing an instance of 'VCF2BinaryException'
what(): VCF2BinaryException : Could not open file 0200162381_S4_L004.hc.snps.indels.g.vcf.gz : could not load index (VCF/BCF files must be block compressed and indexed)
[hpcgenomicn10:18116] *** Process received signal ***
[hpcgenomicn10:18116] Signal: Aborted (6)
[hpcgenomicn10:18116] Signal code: (-6)
[hpcgenomicn10:18116] [ 0] /lib64/libpthread.so.0(+0xf130)[0x2b1768526130]
[hpcgenomicn10:18116] [ 1] /lib64/libc.so.6(gsignal+0x37)[0x2b176ab355d7]
[hpcgenomicn10:18116] [ 2] /lib64/libc.so.6(abort+0x148)[0x2b176ab36cc8]
[hpcgenomicn10:18116] [ 3] /gpfs/software/tools/gcc-4.9.1/lib64/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x175)[0x2b176a336115]
[hpcgenomicn10:18116] [ 4] /gpfs/software/tools/gcc-4.9.1/lib64/libstdc++.so.6(+0x5e176)[0x2b176a334176]
[hpcgenomicn10:18116] [ 5] /gpfs/software/tools/gcc-4.9.1/lib64/libstdc++.so.6(+0x5e1c1)[0x2b176a3341c1]
[hpcgenomicn10:18116] [ 6] /gpfs/software/tools/gcc-4.9.1/lib64/libstdc++.so.6(+0x5e3d8)[0x2b176a3343d8]
[hpcgenomicn10:18116] [ 7] vcf2tiledb[0x4611d5]
[hpcgenomicn10:18116] [ 8] vcf2tiledb[0x42fbb3]
[hpcgenomicn10:18116] [ 9] vcf2tiledb[0x435b23]
[hpcgenomicn10:18116] [10] vcf2tiledb[0x4398e7]
[hpcgenomicn10:18116] [11] vcf2tiledb[0x43a14b]
[hpcgenomicn10:18116] [12] vcf2tiledb[0x4211e9]
[hpcgenomicn10:18116] [13] /lib64/libc.so.6(__libc_start_main+0xf5)[0x2b176ab21af5]
[hpcgenomicn10:18116] [14] vcf2tiledb[0x424189]
[hpcgenomicn10:18116] *** End of error message ***

I've tested the file (zcat file and running bcftools index --stats) and all seems ok
I removed That file from the list, then the error is coming back with another vcf file.
This is happening only on nodes of a certain queue that have identical settings as the other that are not failing.
Any Idea what might be causing the problem?
Thank you in advance for your help.
Best,
Ramzi

[kgururaj]

Hello Ramzi, could it be that you are running against the limit set on the number of open file descriptors by a single process? The ulimit commands are a good way to check.

[kgururaj]

Ramzi, did raising the file descriptor limit fix this issue?