chore: Update instructions to setup EKS as CE#1101
Conversation
bebosudo
requested review from
gavinelder,
gwright99,
justinegeffen and
mp-seqera
✅ Deploy Preview for seqera-docs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
📝 Documentation Review Summary (PR type: content)
Review powered by Claude Code editorial agents To apply suggestions:
|
bebosudo
left a comment
bebosudo
left a comment
There was a problem hiding this comment.
Reword some comments by Claude
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Alberto Chiusole <1922124+bebosudo@users.noreply.github.com>
gavinelder
left a comment
gavinelder
left a comment
There was a problem hiding this comment.
In general a lot of this content is going to be duplicated between vanilla Kubernetes EKS & GKE.
There may be a need to align items such as the service account configuration language.
I suspect this would benefit from a deeper refactor.
There is currently a design for compute environments where it will be a single Kubernetes form and then you select the cloud provider EKS / GKE / AKS for example
Which we should wait for before making any indepth structural changes.
📝 Documentation Review Summary (PR type: content)
Review powered by Claude Code editorial agents To apply suggestions:
|
📝 Documentation Review Summary (PR type: content)
Review powered by Claude Code editorial agents To apply suggestions:
|
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Justine Geffen <justinegeffen@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Justine Geffen <justinegeffen@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Signed-off-by: Justine Geffen <justinegeffen@users.noreply.github.com>
Signed-off-by: Justine Geffen <justinegeffen@users.noreply.github.com>
justinegeffen
added
2. Reviews complete
justinegeffen
added
the
content-improvement
| You must have an EKS cluster up and running. Follow the [cluster preparation](../compute-envs/k8s#cluster-preparation) instructions to create the resources required by your Seqera instance. In addition to the generic Kubernetes instructions, you must make a number of EKS-specific modifications. | ||
| - Seqera Platform needs an IAM User to obtain details about the EKS cluster, and to fetch log files from the S3 bucket, if one is used as work directory. This user must have the permissions detailed in the [Required Platform IAM permissions](#required-platform-iam-permissions) section. Optionally, permissions can instead be granted to an IAM role that the IAM user can assume when accessing AWS resources. | ||
|
|
||
| - To use Fusion (recommended) to access data hosted on S3, including writing files to the Nextflow work directory, you need an IAM role that allows the EKS Service Account that Seqera pods use to interact with AWS resources. Refer to section [Configure EKS Service Account IAM role for Fusion v2](#configure-eks-service-account-iam-role-for-fusion-v2) for details. |
There was a problem hiding this comment.
Nitpick: change "EKS Service Account" to "Kubernetes Service Account"
| - Seqera Platform needs an IAM User to obtain details about the EKS cluster, and to fetch log files from the S3 bucket, if one is used as work directory. This user must have the permissions detailed in the [Required Platform IAM permissions](#required-platform-iam-permissions) section. Optionally, permissions can instead be granted to an IAM role that the IAM user can assume when accessing AWS resources. | ||
|
|
||
| - To use Fusion (recommended) to access data hosted on S3, including writing files to the Nextflow work directory, you need an IAM role that allows the EKS Service Account that Seqera pods use to interact with AWS resources. Refer to section [Configure EKS Service Account IAM role for Fusion v2](#configure-eks-service-account-iam-role-for-fusion-v2) for details. | ||
| Create a separate IAM role from the optional one assumed by the IAM user to separate the permissions needed by the EKS Service Account from those needed by the IAM user. If you plan to use legacy storage instead of Fusion, you can skip this step. |
There was a problem hiding this comment.
Not sure I understand this. You mean have two IAM Roles?
- One for Platform
- One for Nextflow
I think we should say this is mandatory, not optional.
|
|
||
| ## Required IAM permissions | ||
|
|
||
| Seqera Platform requires an IAM user with specific permissions to launch pipelines, explore buckets with Data Explorer, and run Studio sessions on the AWS EKS compute environment. Some permissions are mandatory for the compute environment to function correctly, while others are optional and enable features like populating dropdown lists in the Platform UI. |
There was a problem hiding this comment.
This is basically repeating the opening of the previous section.
|
|
||
| Seqera Platform requires an IAM user with specific permissions to launch pipelines, explore buckets with Data Explorer, and run Studio sessions on the AWS EKS compute environment. Some permissions are mandatory for the compute environment to function correctly, while others are optional and enable features like populating dropdown lists in the Platform UI. | ||
|
|
||
| Attach permissions directly to an [IAM user](#iam-user-creation), or to an [IAM role](#iam-role-creation-optional) that the IAM user can assume. |
There was a problem hiding this comment.
I would link to the AWS Credential page and let that speak for itself. Focus on the necessary permissions.
| ### S3 access (optional) | ||
|
|
||
| **Assign a service account role to your Seqera IAM user** | ||
| Seqera automatically attempts to fetch a list of S3 buckets available in the AWS account connected to Platform, to provide them in a drop-down menu to be used as Nextflow working directory, and make the compute environment creation smoother. This feature is optional, and users can type the bucket name manually when setting up a compute environment. To allow Seqera to fetch the list of buckets in the account, the `s3:ListAllMyBuckets` action can be added, and it must have the `Resource` field set to `*`. |
There was a problem hiding this comment.
IIRC this only happens when using EKS with Fusion, might be good to qualify. Not relevant to PVC.
|
|
||
| Replace `<YOUR-ACCOUNT-ID>`, `<YOUR-REGION>`, `<YOUR-CLUSTER-ID>`, `<YOUR-EKS-NAMESPACE>`, `<YOUR-EKS-SERVICE-ACCOUNT>` with the corresponding values. | ||
|
|
||
| 1. Annotate the Kubernetes Service Account with the IAM role: |
There was a problem hiding this comment.
Why do CLI annotation as opposed to the putting a placeholder on the upstream manifest?
| 1. Enter a descriptive name for this environment, e.g., `Amazon EKS (eu-west-1)`. | ||
| 1. Select **Amazon EKS** as the target platform. | ||
| 1. Under **Storage**, select either **Fusion storage** (recommended) or **Legacy storage**. The [Fusion v2](https://docs.seqera.io/fusion) virtual distributed file system allows access to your AWS S3-hosted data (`s3://` URLs). This eliminates the need to configure a shared file system in your Kubernetes cluster. See [Configure EKS Service Account IAM role for Fusion v2](#configure-eks-service-account-iam-role-for-fusion-v2) below. | ||
| 1. From the **Credentials** drop-down, select existing AWS credentials, or select **+** to add new credentials. If you're using existing credentials, skip to step 9. The user must have the IAM permissions required to describe and list EKS clusters, per Service Account role requirements. |
There was a problem hiding this comment.
You just spent a whole bunch of lines earlier creating the credential. I'd be more opinionate here and send them back up the earlier in the doc if they haven't done it yet.
| 1. From the **Credentials** drop-down menu, select existing AWS credentials, or select **+** to add new credentials. If you choose to use existing credentials, skip to step 9. | ||
| :::note | ||
| The user must have the IAM permissions required to describe and list EKS clusters, per service account role requirements in the previous section. | ||
| You can create multiple credentials in your Seqera environment. See [Credentials](../credentials/overview). |
There was a problem hiding this comment.
Cut this -- be DRY and use the existing AWS Credentials page.
Note: I recognize that creating credentials inline is technically possible, but this seems to dumb to me since you are now mixing concerns: credential creation within CE creation. IMO better to be opinionated and get them to create a credential as a discrete element and then the CE.
|
|
||
| :::note | ||
| The **Storage claim** isn't needed when Fusion v2 is enabled. | ||
| If you enable Fusion v2 (**Fusion storage** in step 4 above), the head service account must have access to the S3 storage bucket specified as your work directory. In the [Advanced options](#amazon-eks-advanced-options) below, a service account for compute jobs need to also be specified to allow pods to interact with AWS. |
There was a problem hiding this comment.
Qualify compute pod "interact with AWS (S3 minimum)".
| Amazon EKS compute environments include advanced options for storage and work directory paths, resource allocation, and pod customization. | ||
|
|
||
| **Seqera EKS advanced options** | ||
| - The **Storage mount path** is the file system path where Seqera mounts the Storage claim (default: `/scratch`). |
There was a problem hiding this comment.
I'd probably use a common core, then split into Fusion flow and PVC flow.
4 participants
As part of the effort to remove mentions of the old
nf-tower-awsrepo, I noticed that the EKS docs were out of date and not so clear. This PR aims at streamlining the guide and provides some needed clarifications around IAM Roles and Service Accounts needed.I only edited the platform cloud docs for now, once approved I'll backport to v25.x.