How to use Splash on my side?

[amarynets]

I want to use my own Splash instance on my server and Crawlera service(I use C10).
I try this example and a response is Website crawl ban its mean that Crawlera doesn't deal with it. But if I just use only Crawlera all works well.
Also, I tried this, just deleted request:set_header("X-Crawlera-UA", "desktop").
The result is the same. Is something change in Crawlera API? Or I should use C50 plan or smt bad in my code?
Thanks!

[duhanyul]

I am trying to do a similar thing as well and i keep getting response codes 416 i was wondering if you ever resolved the issue?

[apalala]

@duhanyul 416 may be coming from https://www.distilnetworks.com/

[duhanyul]

@apalala I made ROBOTSTXT_OBEY = False and i dont get the response code anymore so i guess there is something blocking it

[alexf-a]

Not sure if you're still having this issue, but the C50 plan solved it for me. Some websites have defense mechanisms that the C10 won't get around.

For some reason I now only get this problem when I run javascript from the Lua main method. Not sure if there is a proper way to include Javascript with an execute endpoint and the Lua Crawlera script...