Skip to content

Conversation

@tisonet
Copy link

@tisonet tisonet commented Nov 1, 2024

Update AWS SDK to the latest version to fix issue in EKS containers:

 Ignoring, HTTP credential provider invalid endpoint host, "169.254.170.23", only loopback hosts are allowed.

The support for EKS has been added to SDK by this commit aws/aws-sdk-go@bcc4451#diff-33cf96874b95777ed427cc539ea43d9881ee56a8d98a124dbd6d3b71f7c0e10fR128

@tisonet tisonet requested a review from a team as a code owner November 1, 2024 13:11
@tisonet tisonet requested review from igungor and ilkinulas and removed request for a team November 1, 2024 13:11
@tisonet tisonet force-pushed the feature/update-aws-sdk-to-support-eks-container branch from baad886 to 5f8592d Compare November 1, 2024 13:19
@andrewtapay
Copy link

Thank you for opening this PR. I'm running into the same issue.

@pnowojski
Copy link

pnowojski commented Nov 28, 2024

I have seen the same issue. Could we get this fixed pretty pretty please? @igungor ?

@rkhachatryan
Copy link

Also faced this issue, @ilkinulas can you help with reviewing this PR please?

github-merge-queue bot pushed a commit to neondatabase/neon that referenced this pull request Dec 19, 2024
## Problem

s5cmd doesn't pick up the pod service account

```
2024/12/16 16:26:01 Ignoring, HTTP credential provider invalid endpoint host, "169.254.170.23", only loopback hosts are allowed. <nil>
ERROR "ls s3://neon-dev-bulk-import-us-east-2/import-pgdata/fast-import/v1/br-wandering-hall-w2xobawv": NoCredentialProviders: no valid providers in chain. Deprecated. For verbose messaging see aws.Config.CredentialsChainVerboseErrors
```

## Summary of changes

Switch to offical CLI.


## Testing

Tested the pre-merge image in staging, using `job_image` override in
project settings.


https://neondb.slack.com/archives/C033RQ5SPDH/p1734554944391949?thread_ts=1734368383.258759&cid=C033RQ5SPDH

## Future Work

Switch back to s5cmd once peak/s5cmd#769 gets
merged.

## Refs

- fixes neondatabase/neon-archive-cloud#21876

---------

Co-authored-by: Gleb Novikov <NanoBjorn@users.noreply.github.com>
@vrdn-23
Copy link

vrdn-23 commented Apr 15, 2025

@igungor @ilkinulas Is it possible to get this merged in for a new release? We are using pod-identity credentials for our EKS pods and the minimum version for the AWS Go SDK needs to be greater than v1.47

@vwxyzjn
Copy link

vwxyzjn commented Aug 13, 2025

@tisonet thanks the for PR. Can you share your compiled binaries?

@vwxyzjn
Copy link

vwxyzjn commented Aug 13, 2025

For those in the same boat:

go install github.com/vwxyzjn/s5cmd/v2@8247bb8305a7a9b7ed2c3b1222683765da75b66a


go: downloading github.com/vwxyzjn/s5cmd/v2 v2.2.3-0.20250813212154-8247bb8305a7
go: finding module for package github.com/peak/s5cmd/v2/command
go: found github.com/peak/s5cmd/v2/command in github.com/peak/s5cmd/v2 v2.3.0

 ls -la /home/ubuntu/go/bin/
total 22112
drwxrwxr-x 2 ubuntu ubuntu       19 Aug 13 21:23 .
drwxrwxr-x 4 ubuntu ubuntu       28 Aug 13 21:23 ..
-rwxrwxr-x 1 ubuntu ubuntu 22638599 Aug 13 21:26 s5cmd

Then it works!

@Efrat19
Copy link

Efrat19 commented Sep 14, 2025

Anything we can do to help this PR get merged?

@hustcer
Copy link

hustcer commented Sep 14, 2025

I can not wait anymore, and create a fork release here: https://github.com/hustcer/s5cmd/releases with many PRs merged, including this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants