Review and communicate server timeout policies

[jar398]

Sometimes API calls can run for a long time. We don't want to let them run arbitrarily long, because that is potentially forever, consuming server resources uselessly. My current understanding of what happens in this case is:

• nginx has a timeout, I don't know what it is. If a rails call takes longer than that, then I'm not sure what it does - does nginx have a way to abort the rails call, or does rails keep trying to complete the call even though nginx has already delivered a failing HTTP response to the client?
• rails has its own timeout, I don't know what it is.

I think that ideally the nginx timeout is the same as the rails timeout. Perhaps this is already the case - but I would be more relaxed if someone could check this for me.

And if I were to learn what the timeout is, I could put it in the API documentation and/or tell our alpha testers what it is, so I would like to request this as well.

Thanks

[JRice]

Sorry: busy days.

If a rails call takes longer than that, then I'm not sure what it does - does nginx have a way to abort the rails call, or does rails keep trying to complete the call even though nginx has already delivered a failing HTTP response to the client?

...Unicorn will reap the nginx process if carries on for too long. That said, I don't think neo4j recognizes the killed connection: I believe it continues to run long queries even after they are aborted. :\ I am not sure what to do about that, but it could be researched. I'm also not sure this is true, it's just an observation based on load.

I don't think Rails (as we've implemented it) actually has a timeout. You can add a timeout, but that requires work, and I've not done it (most people don't, leaving it to the web server).

The unicorn timeout right now is one of:

proxy_read_timeout 900s;
proxy_send_timeout 300s;

You can read about the specifics of each here.

HTH...

[jar398]

This helps a lot. It inspired me to look into neo4j query timeouts, and there is a way to control them on a per-query basis (doc). I may be able to use that.

I don't think a globally configured timeout on all neo4j queries is a good idea, but a timeout for the web services seems reasonable.

[jar398]

There is a way to configure them globally, which would be much easier to set up. The documentation is here, Table A.230. dbms.transaction.timeout. The default is 0 seconds which I assume means no timeout. So, to match the unicorn timeout, we would put

dbms.transaction.timeout=900s

in neo4j.conf. However: this might have the unfortunate effect of also clobbering rake tasks that take over 15 minutes. Are there any such tasks?