IpSpoofAttackError for some users after switching from nginx to Thruster

[viktorsmari]

After deploying Thruster to production, some of our customers (most likely behind a corporate VPN or other setup) started seeing our HTTP ERROR 500 page.

Our error tracking software gave us this error (IP addresses changed)
ActionDispatch::RemoteIp::IpSpoofAttackError: IP spoofing attack?! HTTP_CLIENT_IP="5.6.7.8" HTTP_X_FORWARDED_FOR="1.2.3.4"

Before using Thruster, I was using a simple nginx container in front with minimal config that never had this issue
https://github.com/SteveLTN/https-portal

I like using Thruster, because I just add a gem, and instead I can remove 1 Docker container.

We are using docker-compose.yml, these are the relevant parts:

services:
   app: &app_base
     volumes:
       - thruster:/rails/storage/thruster
     environment:
      - FORWARD_HEADERS=0
 
volumes:
  app-postgres:
  thruster:

I added the FORWARD_HEADERS=0 but that did not solve my spoofing issue.

This is probably not Thruster specific issue, but any suggestions or ideas would be welcome.
Maybe Thruster needs another ENV var to remove the HTTP_CLIENT_IP if that conflict is causing this?

This is how I reproduced the issue:

curl -i https://app.mydomain.com/ \
  -H 'X-Forwarded-For: 1.2.3.4' \
  -H 'Client-IP: 5.6.7.8'

[le0pard]

what hosts in your config.action_dispatch.trusted_proxies ? because you need add hosts, which using truster, like 169.254.0.0/16

[viktorsmari]

what hosts in your config.action_dispatch.trusted_proxies ? because you need add hosts, which using truster, like 169.254.0.0/16

We don't have anything in our trusted_proxies - we have a very simple setup, so I was a bit surprised getting this spoof issue

Rails.application.config.action_dispatch.trusted_proxies
=> nil

[le0pard]

it should be there, if thruster get IP outside known public ranges here - https://github.com/rails/rails/blob/077c3ad60db4c43cccb8f4637b53b8d8eb7a3c19/actionpack/lib/action_dispatch/middleware/remote_ip.rb#L40-L49

[viktorsmari]

If I add thruster to a new Rails 8.1.1 project (in development mode on localhost), and access the rails console and run the command, it is still nil. Is that expected?

Rails.application.config.action_dispatch.trusted_proxies
=> nil

ActionDispatch::RemoteIp::TRUSTED_PROXIES
=>
[#<IPAddr: IPv4:127.0.0.0/255.0.0.0>,
 #<IPAddr: IPv6:0000:0000:0000:0000:0000:0000:0000:0001/ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff>,
 #<IPAddr: IPv6:fc00:0000:0000:0000:0000:0000:0000:0000/fe00:0000:0000:0000:0000:0000:0000:0000>,
 #<IPAddr: IPv4:10.0.0.0/255.0.0.0>,
 #<IPAddr: IPv4:172.16.0.0/255.240.0.0>,
 #<IPAddr: IPv4:192.168.0.0/255.255.0.0>]

[le0pard]

You put there ip of proxy, if it different from this default ranges. Did you check what IP get thruster? If no, check X-Forwarded-For on proxy requests from thruster

[viktorsmari]

I did not put anything or configure anything. I just installed thruster in a new Rails 8.1 project, and got a few IpSpoofAttackError warnings from Honeybadger 😄

We don't have a proxy or anything in front. Just Puma and Thruster handling all the traffic on a VPS.

[le0pard]

Read the code: https://github.com/rails/rails/blob/077c3ad60db4c43cccb8f4637b53b8d8eb7a3c19/actionpack/lib/action_dispatch/middleware/remote_ip.rb#L150-L156 - maybe it will be easier to understand

   # `Client-Ip` and `X-Forwarded-For` should not, generally, both be set. If they
   # are both set, it means that either:
   #
   # 1) This request passed through two proxies with incompatible IP header
   #     conventions.
   #
   # 2) The client passed one of `Client-Ip` or `X-Forwarded-For`
   #     (whichever the proxy servers weren't using) themselves.

This mean if you dont have any additional proxy, nobody should send Client-IP (it is not exists in codebase), thruster send only X-Forwarded-For - https://github.com/basecamp/thruster/blob/main/internal/proxy_handler.go#L49

P.S.

I did not put anything or configure anything

FORWARD_HEADERS=0 is configuration settings in your docker-compose.yml