Unclosed file handles

[saurako]

Hi,
I've been using this module for a project I'm working on and it is a great fit for my needs. Really appreciate the work here.
Over the course of a month of running a process that uses the BigQueue though, I've found that it leaves behind a trail of unclosed file handles. Here's an example:

$ sudo lsof -n | grep -i "DEL"
node       4291       	  saurako  DEL       REG              202,1               394612 /home/saurako/data-processor/data/page-66.dat
node       4291       	  saurako  DEL       REG              202,1               394610 /home/saurako/data-processor/data/page-64.dat
node       4291       	  saurako  DEL       REG              202,1               394611 /home/saurako/data-processor/data/page-65.dat
node       4291       	  saurako  DEL       REG              202,1               394557 /home/saurako/data-processor/data/page-62.dat
node       4291       	  saurako  DEL       REG              202,1               394555 /home/saurako/data-processor/data/page-61.dat
node       4291       	  saurako  DEL       REG              202,1               394558 /home/saurako/data-processor/data/page-63.dat
node       4291       	  saurako  DEL       REG              202,1               394554 /home/saurako/data-processor/data/page-60.dat
node       4291       	  saurako  DEL       REG              202,1               394552 /home/saurako/data-processor/data/page-58.dat

Can you guys please help in fixing this? I'm using the BigQueue data structure, and simply enqueue and dequeue messages from the queue. I do not explicitly close anything, but should I be doing that? If yes, then I'm not sure what should signal that I need to do that?

[dimichgh]

@saurako
Can you please enable debug and see if dispose message is generated below?

mmap-kit/lib/page-factory.js

Lines 20 to 27 in d1bc006

	this.cache = new LRU({
	dispose: function onDispose(key, page) {
	debug('disposing page %s from cache, file: %s', key, page.pageFile);
	page.close();
	},
	max: options.max || 100000, // 400Mb for page size of 4k
	maxAge: options.ttl || 1000 * 60 * 20 // 20 minutes in memory
	});

You can reduce ttl to very small value and see if the file handlers are closed as you add more pages.

[saurako]

Thanks for responding so quickly, @dimichgh. I enabled debug and set the ttl to 30s, and I do see the dispose message being generated:

disposing page 0 from cache, file: meta_data/page-0.dat
disposing page 12 from cache, file: index/page-12.dat
disposing page 0 from cache, file: meta_data/page-0.dat
disposing page 12 from cache, file: index/page-12.dat
disposing page 0 from cache, file: meta_data/page-0.dat
disposing page 12 from cache, file: index/page-12.dat
disposing page 0 from cache, file: meta_data/page-0.dat
disposing page 12 from cache, file: index/page-12.dat

[dimichgh]

@saurako Then, maybe it works correctly?
Are the files that you see still in active state as default ttl is 20 minutes?

[saurako]

@dimichgh - should they show up as deleted via lsof if they were active? Not sure what's going on there.

[dimichgh]

The pages get deleted if they were not used or accessed in LRU cache for 20 minutes.
Upon deletion, the file and its handler should be closed.
The deletion is initiated when a new entry is put into the cache.

[saurako]

@dimichgh so essentially, the expectation is that disk usage should stay constant over time, if the rate of enqueue() is the same as the rate of dequeue(), correct?

[dimichgh]

@saurako right.

[saurako]

Thanks for confirming that, @dimichgh. I'd restarted the processes when I reported this issue. And started on a clean slate. Now I'm again seeing a constant increase in disk usage over the last couple days along with these unclosed file handles. There are only a few of them now since it's just been a couple days. Do you have any suggestions on how do I debug this? One thing I'd thought of is to reduce the ttl, and even if that works, it's only a workaround (but good for now).

[dimichgh]

Do you use default TTL?
Do you have debug enabled to see what gets closed?

[saurako]

Yes, I do.

…

On Thu, Sep 14, 2017 at 6:31 PM Dmytro Semenov ***@***.***> wrote: Do you use default TTL? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#7 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACCsNOSq39cTLDV6jzjw-5XVrOAKbT7cks5sidOLgaJpZM4PVRV-> .

[saurako]

@dimichgh is it necessary to call flush() after enqueuing?