Running the example gives a Content Encoding Error

[jakewilliami]

Example main.rs from #17:

use awc::Client;

use actix_web::{get, web, HttpResponse, App, HttpServer};

use actix_proxy::{IntoHttpResponse, SendRequestError};

#[get("/{url:.*}")]
async fn proxy(
    path: web::Path<(String,)>,
    client: web::Data<Client>,
) -> Result<HttpResponse, SendRequestError> {
    let (url,) = path.into_inner();

    let url = format!("https://duckduckgo.com/{url}");

    // here we use `IntoHttpResponse` to return the request to
    // duckduckgo back to the client that called this endpoint
    Ok(client.get(&url).send().await?.into_http_response())
}

#[actix_web::main]
async fn main() -> std::io::Result<()> {
    HttpServer::new(|| {
        App::new()
            .app_data(web::Data::new(Client::default())) // This is the important line
            .service(proxy)
    })
    .bind(("127.0.0.1", 8080))?
    .run()
    .await
}

And example Cargo.toml adapted from #2:

[package]
name = "test-server"
version = "0.1.0"
edition = "2021"

[dependencies]
actix-proxy = "0.2.0"
actix-web = { version = "^4.3.0" }
awc = { version = "^3.1.0", features = ["openssl"] }
openssl = { version = "^0.10.32" }

However, I get a Conent Encoding Error:

Is there something I'm doing wrong?

I saw @Sleepful's comment and CORS Anywhere is essentially the functionality I want to emulate, but using Actix. You could entertain the notion that my company blocked Connections so I was hoping to create a proxy to it on an existing website I have whose server is running on Actix. That might bypass the hypothetical block...

P.S., thank you for your work on this package. It looks really promising. I have also seen redirection.io's Actix Proxy, but it does a lot of magic-y things. I like that with this package I can still easily specify the route, for example. It seems to meld well with base Actix.

[jofas]

I need to investigate further why this was okay before but isn't now (maybe some added default feature to awc?), but the problem is that we decompress the body on the proxy, but forward the Content-Encoding header of the proxied request. So say we receive a response with the Content-Encoding: gzip header set. We forward the headers of the proxied request verbatim. So we basically tell our client, "hey, the body will be gzip encoded". But then we decompress the gzip content on the proxy and forward the decompressed body, making the client wonder "you told me I'll get a gzipped body, but this ain't gzip". So this is definitely a bug in the example. You can fix this by telling the client to not decompress:

#[get("/{url:.*}")]
async fn proxy(
    path: web::Path<(String,)>,
    client: web::Data<Client>,
) -> Result<HttpResponse, SendRequestError> {
    let (url,) = path.into_inner();

-   let url = format!("https://duckduckgo.com/{url}");
+   // duckduckgo doesn't like to be proxied anymore it looks like, showing only a blank screen
+   let url = fomrat!("https://users.rust-lang.org/{url}"); 

-   Ok(client.get(&url).send().await?.into_http_response())
+   Ok(client.get(&url).no_decompress().send().await?.into_http_response())
    //                 ^^^^^^^^^^^^^^^^
}

I haven't yet thought about the problem of content encoding during proxying. Or more generally speaking, this crate only does half the proxying, because it ignores any request headers from the client. This might be a good opportunity to provide a second trait FromHttpRequest, that creates the request we forward to our proxied server, instead of constructing a new request.

[jakewilliami]

Thanks @jofas, your updated example is very useful. I understand the problem, too. Nice explanation.