How to use TLS?

[DazWilkin]

The README.md is lacking in documenting how to use TLS Support:

"If you need TLS support for the server, you'll need to compile with the ssl feature set."

I'm willing to augment the README.md with the solution but I'm unable to get it working myself 😞

I'm unable to curl the TLS endpoint (403):

curl \
--insecure \
--key privatekey.pem \
--cert certificate.pem \
--write-out '%{response_code}' \
https://${HOST}:8888

And I'm unable to discover the device in the gateway.

thing-url-adapter:
Failed to connect to https://${HOST}.local:8888:
FetchError: request to https://${HOST}.local:8888/ failed, reason: self signed certificate

I believe, in the example code, I should replace:

webthing = "0.13.2"

with, e.g.:

[dependencies.webthing]
path = "../webthing-rust"
version = "0.13.2"
features = ["ssl"]

Then:

• I generated a self-signed cert using HOST=localhost
• I generated a self-signed cert using HOST=$(hostname).local
• I referenced the privatekey.pem and certificate.pem from the WebThings Gateway's config's SSL directory

openssl req \
-x509 \
-nodes \
-days 365 \
-newkey rsa:2048 \
-keyout privatekey.pem \
-out certificate.pem \
-subj /CN=${HOST}

And:

let mut server = WebThingServer::new(
    ThingsType::Multiple(things, "Rusty-Device".to_owned()),
    Some(8888),
    None,
    Some(("privatekey.pem".to_string(), "certificate.pem".to_string())),
    Box::new(Generator),
    None,
)

[mrstegeman]

You're doing the right thing. A couple things to note:

1. The Web Thing add-on will not currently accept self-signed certificates. This is why we don't widely advertise the feature. TLS on a private network is tricky.
2. You generally get a 403 status when the hostname doesn't match what's expected. Make sure you set the third parameter of your WebThingServer::new() call appropriately, e.g. Some($HOST) (obviously replacing $HOST with what you're using in your shell commands).

[DazWilkin]

Thank you for the prompt response(s)!

I've been using the Python SDK as a proxy for the Rust SDK.

If I configure the Python example (on 8887):

server = WebThingServer(
    things=MultipleThings([light, sensor], 'Pythonic-Device'),
    port=8887,
    hostname="my-host",
    ssl_options={
        "keyfile": "privatekey.pem",
        "certfile": "certificate.pem",
})

NOTE Good call, thanks... I've replaced my-host in these examples with $(hostname)

I'm (still) unable browse the Python devices using Gateway (because of the self-signed issue so I'll drop trying this) but I am able to curl the endpoint:

curl \
--silent \
--insecure \
--key privatekey.pem \
--cert certificate.pem \
https://${HOST}.local:8887 \
| jq -r '.[].title'
Pythonic Lamp
Pythonic Humidity Sensor

But, using (what I think is) the same configuration with the Rust example (except on 8888), I get 403:

let mut server = WebThingServer::new(
    ThingsType::Multiple(things, "Rusty-Device".to_owned()),
    Some(8888),
    Some("my-host".to_string()),
    Some(("privatekey.pem".to_string(), "certificate.pem".to_string())),
    Box::new(Generator),
    None,
);
server.start(None).await

And:

curl \
--silent \
--insecure \
--key privatekey.pem \
--cert certificate.pem \
https://${HOST}.local:8888 \
--write-out '%{response_code}'
403

I'm either configuring server incorrectly or the code backing it has an issue?

Here are the Gateway log (errors) for completeness but I'll stop pursuing TLS on my private network:

thing-url-adapter:
Failed to connect to https://${HOST}.local:8888:
FetchError: request to https://${HOST}.local:8888/ failed, reason: self signed certificate

thing-url-adapter:
Failed to connect to https://${HOST}.local:8887:
FetchError: request to https://${HOST}.local:8887/ failed, reason: self signed certificate

[mrstegeman]

Try using my-host.local instead of my-host in your Rust code.

[DazWilkin]

Yes, no combination appears to work :-(

Will have another look tomorrow.

[DazWilkin]

No resolution :-(

I used the Actix Web rustls example.

I am able to use the example's certs with the example with both rustls and openssl and both work.

The example works with my cert|key and openssl

However, I am unable to use my cert|key with the example and rustls.

IIUC webthing-rust uses openssl so this may be a red-herring but the rustlsuses ServerConfg and the code fails for me:

config.set_single_cert(cert_chain, keys.remove(0)).unwrap();

I suspect because my cert is self-signed?

However, flipping this around, I'm unable to curl the webthing-rust sample using the actix-web sample's cert|key

IIUC webthing-rust is using actix-web and openssl and so it's still unclear why this doesn't work.

Couple of other perhaps red-herrings but....

webthing-rust file server.rs line ~145 implements HostValidatorMiddelware with call function. When I debug the code, this fails using curl even if I add --header "Host: hades-canyon". If I manually, inject a value into the function, it fails with HTTP/2 stream 0 was not closed cleanly: CANCEL (err 8).

webthing-rust file server.rs line ~942 WebThingsServer::start builds an extensive list of hosts on my machine:

get_addresses() = ["127.0.0.1", "172.17.0.1", "172.18.0.1", "172.19.0.1", "172.20.0.1", "172.21.0.1", "172.22.0.1", "172.23.0.1", "172.24.0.1", "172.25.0.1", "172.26.0.1", "172.27.0.1", "192.168.1.150", "192.168.1.186", "[::1]"]

I'm unsure why this would be necessary to gather these addresses if the service should only accept hostname(.domain):port

Rather flummoxed.

[mrstegeman]

IIUC webthing-rust is using actix-web and openssl

Yes, that's correct.

I'm unsure why this would be necessary to gather these addresses if the service should only accept hostname(.domain):port

Given that these things live on a local network, and mDNS resolution doesn't always work (especially on Windows), we add the local IP addresses to list of allowed hostnames.

This verification is all done to prevent DNS rebinding.

[DazWilkin]

The Host header is being dropped|discarded with TLS and the absence of this header appears to cause a prompt 403 (see below).

non-TLS:

webthing = { path = "../webthing-rust", version = "0.13.2" }

And:

curl  \
--silent \
 --write-out '%{response_code}'
--output /dev/nill
http://hades-canyon.local:8888/

Response:

200

With server.rs after the fn call signature:

println!("{:?}", req.headers());

Yields:

HeaderMap { inner: {"accept": One("*/*"), "host": One("hades-canyon.local:8888"), "user-agent": One("curl/7.68.0")} }

NOTE it includes the host header (and it is correct).

TLS

webthing = { path = "../webthing-rust", version = "0.13.2", features = ["ssl"] }

And:

curl \
--silent \
--insecure \
--key ./secrets/privatekey.pem \
--cert ./secrets/certificate.pem \
--header "dog: Freddie" \
--write-out '%{response_code}' \
 --output /dev/null \
https://hades-canyon.local:8888/

Response:

403

And:

HeaderMap { inner: {"accept": One("*/*"), "user-agent": One("curl/7.68.0"), "dog": One("Freddie")} }

NOTE No header and so:

if host.is_none() {
  return Either::Right(ok(
    req.into_response(HttpResponse::Forbidden().finish().into_body())
 ));
}

Even if I try to manually inject a Host header, it is dropped:

curl \
--silent \
--insecure \
--key ./secrets/privatekey.pem \
--cert ./secrets/certificate.pem \
--header "dog: Freddie" \
--write-out '%{response_code}' \
 --output /dev/null \
https://hades-canyon.local:8888/

Response:

403

[mrstegeman]

Can you try adding --http-1.1 to your TLS curl command?

[DazWilkin]

Hmmm..... As you updated the thread, I was about to add this

Yes, I think it's http/2

[DazWilkin]

Yes, that's it... I now get 200s

[mrstegeman]

Ok, great. Glad we got that figured out!

It looks like we may be able to disable http/2 with this.

[DazWilkin]

curl \
--http1.1 \
--silent \
--insecure \
--key ./secrets/privatekey.pem \
--cert ./secrets/certificate.pem  \
https://hades-canyon.local:8888/

Yields:

200

And:

HeaderMap { inner: {"accept": One("*/*"), "host": One("hades-canyon.local:8888"), "user-agent": One("curl/7.68.0")} }

[DazWilkin]

I tried:

let mut builder = SslAcceptor::mozilla_intermediate(SslMethod::tls()).unwrap();
builder
    .set_private_key_file(o.0.clone(), SslFiletype::PEM)
    .unwrap();
builder.set_certificate_chain_file(o.1.clone()).unwrap();
builder.set_alpn_protos(b"\x06spdy/1\x08http/1.1").unwrap();

NOTE Tried both b"\x06spdy/1\x08http/1.1" and b"\x08http/1.1"

But I get 403s still if I drop the --http1.1

And the Host header isn't received by the call fn:

HeaderMap { inner: {"accept": One("*/*"), "user-agent": One("curl/7.68.0")} }

Is this perhaps because the server is unable to then upgrade the HTTP connection for WebSockets?

[mrstegeman]

Hmm, very strange. I don't have any suggestions right now, other than forcing HTTP/1.1.

[DazWilkin]

Thanks very much for your patience with this.

We have something of a diagnosis.