diff --git a/platform-cloud/docs/pipeline-optimization/overview.md b/platform-cloud/docs/pipeline-optimization/overview.md index 40ed101bd..ded022d44 100644 --- a/platform-cloud/docs/pipeline-optimization/overview.md +++ b/platform-cloud/docs/pipeline-optimization/overview.md @@ -1,5 +1,5 @@ --- -title: "Pipeline resource optimization" +title: "Pipeline optimization" description: "Optimize the resource usage of your pipelines to save time and money." date: "12 Feb 2024" tags: [compute, resource, optimization] @@ -7,7 +7,7 @@ tags: [compute, resource, optimization] **Available from version 23.3.0** -Pipeline resource optimization takes the resource usage information from previous workflow runs to optimize subsequent runs. +Pipeline optimization takes the resource usage information from previous workflow runs to optimize subsequent runs. When a run completes successfully, an _optimized profile_ is created. This profile consists of Nextflow configuration settings for each process and each resource directive (where applicable): `cpus`, `memory`, and `time`. The optimized setting for a given process and resource directive is based on the maximum use of that resource across all tasks in that process. diff --git a/platform-enterprise_docs/enterprise/_templates/docker/docker-compose.yml b/platform-enterprise_docs/enterprise/_templates/docker/docker-compose.yml index db991b521..3e96b2b2d 100644 --- a/platform-enterprise_docs/enterprise/_templates/docker/docker-compose.yml +++ b/platform-enterprise_docs/enterprise/_templates/docker/docker-compose.yml @@ -18,7 +18,7 @@ services: retries: 10 restart: always volumes: - # To use the pipeline resource optimization service, uncomment the lines below to mount the initialization script + # To use the pipeline optimization service, uncomment the lines below to mount the initialization script # - type: bind # source: ./init.sql # target: /docker-entrypoint-initdb.d/init.sql @@ -141,7 +141,7 @@ services: # - 7070:7070 # restart: always - # Uncomment the following section to enable the pipeline resource optimization service. Add TOWER_ENABLE_GROUNDSWELL + # Uncomment the following section to enable the pipeline optimization service. Add TOWER_ENABLE_GROUNDSWELL # or GROUNDSWELL_SERVER_URL to tower.env environment variables # groundswell: # image: cr.seqera.io/private/nf-tower-enterprise/groundswell:0.3.3 diff --git a/platform-enterprise_docs/enterprise/_templates/docker/tower.env b/platform-enterprise_docs/enterprise/_templates/docker/tower.env index b745c3f20..29983637a 100644 --- a/platform-enterprise_docs/enterprise/_templates/docker/tower.env +++ b/platform-enterprise_docs/enterprise/_templates/docker/tower.env @@ -20,7 +20,7 @@ TOWER_SMTP_PORT=587 TOWER_SMTP_USER=foo TOWER_SMTP_PASSWORD=foo -# Pipeline resource optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) +# Pipeline optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) # Use GROUNDSWELL_SERVER_URL to specify a custom service URL (enables the service by default) TOWER_ENABLE_GROUNDSWELL=false GROUNDSWELL_SERVER_URL="http://groundswell:8090" diff --git a/platform-enterprise_docs/enterprise/_templates/k8s/configmap.yml b/platform-enterprise_docs/enterprise/_templates/k8s/configmap.yml index 3bd695adb..cc40656b6 100644 --- a/platform-enterprise_docs/enterprise/_templates/k8s/configmap.yml +++ b/platform-enterprise_docs/enterprise/_templates/k8s/configmap.yml @@ -24,7 +24,7 @@ data: TOWER_ENABLE_PLATFORMS: "altair-platform,awsbatch-platform,awscloud-platform,azbatch-platform,azcloud-platform,eks-platform,gke-platform,googlebatch-platform,googlecloud-platform,k8s-platform,lsf-platform,moab-platform,slurm-platform,uge-platform" TOWER_REDIS_URL: "redis://:6379" # For a managed Redis service, specify the IP address or DNS name of the service endpoint. If you use the containerized Redis service, specify "redis" instead of an IP address. - # Pipeline resource optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) + # Pipeline optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) # Use GROUNDSWELL_SERVER_URL to specify a custom service URL (enables the service by default) TOWER_ENABLE_GROUNDSWELL: "false" GROUNDSWELL_SERVER_URL: http://groundswell:8090 diff --git a/platform-enterprise_docs/enterprise/configuration/pipeline_optimization.md b/platform-enterprise_docs/enterprise/configuration/pipeline_optimization.md index 6b2801586..7246e6529 100644 --- a/platform-enterprise_docs/enterprise/configuration/pipeline_optimization.md +++ b/platform-enterprise_docs/enterprise/configuration/pipeline_optimization.md @@ -1,24 +1,25 @@ --- title: "Pipeline optimization" -description: "Configure pipeline resource optimization in your Seqera Enterprise deployment." +description: "Configure pipeline optimization in your Seqera Enterprise deployment." date: "12 Feb 2024" tags: [compute, resource, optimization, configuration] --- -[Pipeline resource optimization](../../pipeline-optimization/overview) takes the resource usage information from previous workflow runs to optimize subsequent runs. +[Pipeline optimization](../../pipeline-optimization/overview) takes the resource usage information from previous workflow runs to optimize subsequent runs. -The pipeline resource optimization service requires a separate database schema to store its internal data, but also requires access to the Seqera schema. The Seqera and optimization service schemas can coexist on the same database instance. +The pipeline optimization service requires a separate database schema to store its internal data, but also requires access to the Seqera schema. The Seqera and optimization service schemas can coexist on the same database instance. ## Docker Compose deployment -Docker Compose makes use of a separate container to set up the pipeline resource optimization service during initialization. Configuration steps differ for new and existing deployments. +Docker Compose makes use of a separate container to set up the pipeline optimization service during initialization. Configuration steps differ for new and existing deployments. ### New installation -To use the pipeline resource optimization service in a new Docker Compose installation of Seqera Enterprise, use the following steps: +To use the pipeline optimization service in a new Docker Compose installation of Seqera Enterprise, use the following steps: 1. To run the service from a custom URL, declare the URL with the `GROUNDSWELL_SERVER_URL` environment variable in `tower.env`. A non-zero value for this environment variable activates the optimization service automatically, so `TOWER_ENABLE_GROUNDSWELL` does not need to be set when you declare a custom URL. + 2. Set the `TOWER_ENABLE_GROUNDSWELL` environment variable in `tower.env` to `true`. This enables the service at the default service URL `http://groundswell:8090`. 3. In your [docker-compose.yml](../_templates/docker/docker-compose.yml) file, uncomment the `groundswell` section at the bottom. @@ -27,11 +28,11 @@ To use the pipeline resource optimization service in a new Docker Compose instal 4. Download the [init.sql](../_templates/docker/init.sql) file. Store this file in the mount path of your `docker-compose.yml` file, else update the `source: ./init.sql` line in your `docker-compose.yml` with the file path. -5. When the pipeline resource optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. +5. When the pipeline optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. ### Existing installation -To use the pipeline resource optimization service in an existing Docker Compose installation of Seqera Enterprise, use the following steps: +To use the pipeline optimization service in an existing Docker Compose installation of Seqera Enterprise, use the following steps: 1. To run the service from a custom URL, declare the URL with the `GROUNDSWELL_SERVER_URL` environment variable. A non-zero value for this environment variable activates the optimization service automatically, so `TOWER_ENABLE_GROUNDSWELL` does not need to be set when you declare a custom URL. @@ -69,11 +70,11 @@ To use the pipeline resource optimization service in an existing Docker Compose # SWELL_DB_URL=mysql://db1.abcdefghijkl.us-east-1.rds.amazonaws.com:3306/swell ``` -7. When the pipeline resource optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. +7. When the pipeline optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. ## Kubernetes deployment -Kubernetes deployments use an [initContainer](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) that runs during pod initialization to set up the pipeline resource optimization service. To use the service in new or existing Kubernetes installations of Seqera Enterprise, do the following: +Kubernetes deployments use an [initContainer](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) that runs during pod initialization to set up the pipeline optimization service. To use the service in new or existing Kubernetes installations of Seqera Enterprise, do the following: 1. Download the [groundswell manifest](../_templates/k8s/groundswell.yml). @@ -101,9 +102,10 @@ Kubernetes deployments use an [initContainer](https://kubernetes.io/docs/concept FLUSH PRIVILEGES; ``` -The initContainers process will wait until both the Seqera and pipeline resource optimization service databases are ready before starting the migration in the Seqera database and finally starting the optimization container. +The `initContainers` process waits until both the Seqera and pipeline optimization service databases are ready before starting the migration in the Seqera database, and finally starting the optimization container. + -When the pipeline resource optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. +When the pipeline optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. [platform-k8s]: ../platform-kubernetes diff --git a/platform-enterprise_docs/enterprise/groundswell-docker-compose.md b/platform-enterprise_docs/enterprise/groundswell-docker-compose.md index d9c33a29b..14ddc53a4 100644 --- a/platform-enterprise_docs/enterprise/groundswell-docker-compose.md +++ b/platform-enterprise_docs/enterprise/groundswell-docker-compose.md @@ -1,11 +1,11 @@ --- title: "Pipeline optimization: Docker Compose" -description: Deploy pipeline resource optimization with Docker Compose +description: Deploy pipeline optimization with Docker Compose date: "12 Apr 2023" -tags: [docker, compose, groundswell, deployment] +tags: [docker, compose, groundswell, pipeline optimization, deployment] --- -This guide describes how to deploy the pipeline resource optimization service (Groundswell) for Seqera Platform Enterprise with Docker Compose. +This guide describes how to deploy the pipeline optimization service (referred to as `groundswell` in the configuration file) for Seqera Platform Enterprise using Docker Compose. ## Prerequisites @@ -76,4 +76,4 @@ When pipeline optimization is active, pipelines with at least one successful run ## Configuration -See [Pipeline resource optimization](./configuration/pipeline_optimization) for additional configuration options. +See [Pipeline optimization](./configuration/pipeline_optimization) for additional configuration options. diff --git a/platform-enterprise_docs/enterprise/groundswell-kubernetes.md b/platform-enterprise_docs/enterprise/groundswell-kubernetes.md index caf779bcf..9bf919a76 100644 --- a/platform-enterprise_docs/enterprise/groundswell-kubernetes.md +++ b/platform-enterprise_docs/enterprise/groundswell-kubernetes.md @@ -1,11 +1,11 @@ --- title: "Pipeline optimization: Kubernetes" -description: Deploy pipeline resource optimization on Kubernetes +description: Deploy pipeline optimization on Kubernetes date: "12 Apr 2023" -tags: [kubernetes, groundswell, deployment] +tags: [kubernetes, groundswell, pipeline optimization, deployment] --- -This guide describes how to deploy the pipeline resource optimization service (Groundswell) for Seqera Platform Enterprise on Kubernetes. +This guide describes how to deploy the pipeline optimization service (referred to as `groundswell` in the configuration file) for Seqera Platform Enterprise on Kubernetes. ## Prerequisites @@ -62,4 +62,4 @@ When pipeline optimization is active, pipelines with at least one successful run ## Configuration -See [Pipeline resource optimization](./configuration/pipeline_optimization) for additional configuration options. +See [Pipeline optimization](./configuration/pipeline_optimization) for additional configuration options. diff --git a/platform-enterprise_docs/enterprise/install-groundswell.md b/platform-enterprise_docs/enterprise/install-groundswell.md index ce899648e..699afa692 100644 --- a/platform-enterprise_docs/enterprise/install-groundswell.md +++ b/platform-enterprise_docs/enterprise/install-groundswell.md @@ -1,11 +1,11 @@ --- title: "Pipeline optimization" -description: Install pipeline resource optimization for Seqera Platform Enterprise +description: Install pipeline optimization for Seqera Platform Enterprise date: "12 Apr 2023" tags: [groundswell, pipeline, optimization, installation, deployment] --- -Pipeline resource optimization (Groundswell) uses resource usage data from previous workflow runs to optimize subsequent runs. Deploy after your Platform installation is complete. +Pipeline optimization can be deployed after your Platform installation is complete. It uses resource usage data from previous workflow runs to optimize subsequent runs. ## Deployment options diff --git a/platform-enterprise_docs/enterprise/platform-docker-compose.md b/platform-enterprise_docs/enterprise/platform-docker-compose.md index 31b229ca8..644d7f392 100644 --- a/platform-enterprise_docs/enterprise/platform-docker-compose.md +++ b/platform-enterprise_docs/enterprise/platform-docker-compose.md @@ -56,7 +56,7 @@ Use a managed Redis service for production: - The `db` container should be used only for local testing. If you have configured this service elsewhere, you can remove this container. - - To configure the Seqera pipeline resource optimization service (`groundswell`), see [Pipeline resource optimization](./configuration/pipeline_optimization). + - To configure the Seqera pipeline optimization service (`groundswell`), see [Pipeline optimization](./configuration/pipeline_optimization). - To deploy with Studios, see [Studios deployment](./studios). diff --git a/platform-enterprise_docs/pipeline-optimization/overview.md b/platform-enterprise_docs/pipeline-optimization/overview.md index 40ed101bd..ded022d44 100644 --- a/platform-enterprise_docs/pipeline-optimization/overview.md +++ b/platform-enterprise_docs/pipeline-optimization/overview.md @@ -1,5 +1,5 @@ --- -title: "Pipeline resource optimization" +title: "Pipeline optimization" description: "Optimize the resource usage of your pipelines to save time and money." date: "12 Feb 2024" tags: [compute, resource, optimization] @@ -7,7 +7,7 @@ tags: [compute, resource, optimization] **Available from version 23.3.0** -Pipeline resource optimization takes the resource usage information from previous workflow runs to optimize subsequent runs. +Pipeline optimization takes the resource usage information from previous workflow runs to optimize subsequent runs. When a run completes successfully, an _optimized profile_ is created. This profile consists of Nextflow configuration settings for each process and each resource directive (where applicable): `cpus`, `memory`, and `time`. The optimized setting for a given process and resource directive is based on the maximum use of that resource across all tasks in that process. diff --git a/platform-enterprise_versioned_docs/version-25.1/enterprise/_templates/docker/docker-compose.yml b/platform-enterprise_versioned_docs/version-25.1/enterprise/_templates/docker/docker-compose.yml index 78275a600..44472f51c 100644 --- a/platform-enterprise_versioned_docs/version-25.1/enterprise/_templates/docker/docker-compose.yml +++ b/platform-enterprise_versioned_docs/version-25.1/enterprise/_templates/docker/docker-compose.yml @@ -18,7 +18,7 @@ services: retries: 10 restart: always volumes: - # To use the pipeline resource optimization service, uncomment the lines below to mount the initialization script + # To use the pipeline optimization service, uncomment the lines below to mount the initialization script # - type: bind # source: ./init.sql # target: /docker-entrypoint-initdb.d/init.sql @@ -139,7 +139,7 @@ services: # - 7070:7070 # restart: always - # Uncomment the following section to enable the pipeline resource optimization service. Add TOWER_ENABLE_GROUNDSWELL + # Uncomment the following section to enable the pipeline optimization service. Add TOWER_ENABLE_GROUNDSWELL # or GROUNDSWELL_SERVER_URL to tower.env environment variables # groundswell: # image: cr.seqera.io/private/nf-tower-enterprise/groundswell:0.3.3 diff --git a/platform-enterprise_versioned_docs/version-25.1/enterprise/_templates/docker/tower.env b/platform-enterprise_versioned_docs/version-25.1/enterprise/_templates/docker/tower.env index b128efcce..6defde57b 100644 --- a/platform-enterprise_versioned_docs/version-25.1/enterprise/_templates/docker/tower.env +++ b/platform-enterprise_versioned_docs/version-25.1/enterprise/_templates/docker/tower.env @@ -23,7 +23,7 @@ TOWER_SMTP_PORT=587 TOWER_SMTP_USER=foo TOWER_SMTP_PASSWORD=foo -# Pipeline resource optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) +# Pipeline optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) # Use GROUNDSWELL_SERVER_URL to specify a custom service URL (enables the service by default) TOWER_ENABLE_GROUNDSWELL=false GROUNDSWELL_SERVER_URL="http://groundswell:8090" diff --git a/platform-enterprise_versioned_docs/version-25.1/enterprise/_templates/k8s/configmap.yml b/platform-enterprise_versioned_docs/version-25.1/enterprise/_templates/k8s/configmap.yml index 3496f7f32..3512a86f7 100644 --- a/platform-enterprise_versioned_docs/version-25.1/enterprise/_templates/k8s/configmap.yml +++ b/platform-enterprise_versioned_docs/version-25.1/enterprise/_templates/k8s/configmap.yml @@ -24,7 +24,7 @@ data: TOWER_ENABLE_PLATFORMS: "altair-platform,awsbatch-platform,azbatch-platform,eks-platform,gke-platform,googlebatch-platform,k8s-platform,lsf-platform,moab-platform,slurm-platform,uge-platform" TOWER_REDIS_URL: "redis://:6379" # For a managed Redis service, specify the IP address or DNS name of the service endpoint. If you use the containerized Redis service, specify "redis" instead of an IP address. - # Pipeline resource optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) + # Pipeline optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) # Use GROUNDSWELL_SERVER_URL to specify a custom service URL (enables the service by default) TOWER_ENABLE_GROUNDSWELL: "false" GROUNDSWELL_SERVER_URL: http://groundswell:8090 diff --git a/platform-enterprise_versioned_docs/version-25.1/enterprise/configuration/pipeline_optimization.md b/platform-enterprise_versioned_docs/version-25.1/enterprise/configuration/pipeline_optimization.md index 6b2801586..d1f0d754d 100644 --- a/platform-enterprise_versioned_docs/version-25.1/enterprise/configuration/pipeline_optimization.md +++ b/platform-enterprise_versioned_docs/version-25.1/enterprise/configuration/pipeline_optimization.md @@ -1,21 +1,21 @@ --- title: "Pipeline optimization" -description: "Configure pipeline resource optimization in your Seqera Enterprise deployment." +description: "Configure pipeline optimization in your Seqera Enterprise deployment." date: "12 Feb 2024" tags: [compute, resource, optimization, configuration] --- -[Pipeline resource optimization](../../pipeline-optimization/overview) takes the resource usage information from previous workflow runs to optimize subsequent runs. +[Pipeline optimization](../../pipeline-optimization/overview) takes the resource usage information from previous workflow runs to optimize subsequent runs. -The pipeline resource optimization service requires a separate database schema to store its internal data, but also requires access to the Seqera schema. The Seqera and optimization service schemas can coexist on the same database instance. +The pipeline optimization service requires a separate database schema to store its internal data, but also requires access to the Seqera schema. The Seqera and optimization service schemas can coexist on the same database instance. ## Docker Compose deployment -Docker Compose makes use of a separate container to set up the pipeline resource optimization service during initialization. Configuration steps differ for new and existing deployments. +Docker Compose makes use of a separate container to set up the pipeline optimization service during initialization. Configuration steps differ for new and existing deployments. ### New installation -To use the pipeline resource optimization service in a new Docker Compose installation of Seqera Enterprise, use the following steps: +To use the pipeline optimization service in a new Docker Compose installation of Seqera Enterprise, use the following steps: 1. To run the service from a custom URL, declare the URL with the `GROUNDSWELL_SERVER_URL` environment variable in `tower.env`. A non-zero value for this environment variable activates the optimization service automatically, so `TOWER_ENABLE_GROUNDSWELL` does not need to be set when you declare a custom URL. @@ -27,11 +27,11 @@ To use the pipeline resource optimization service in a new Docker Compose instal 4. Download the [init.sql](../_templates/docker/init.sql) file. Store this file in the mount path of your `docker-compose.yml` file, else update the `source: ./init.sql` line in your `docker-compose.yml` with the file path. -5. When the pipeline resource optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. +5. When the pipeline optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. ### Existing installation -To use the pipeline resource optimization service in an existing Docker Compose installation of Seqera Enterprise, use the following steps: +To use the pipeline optimization service in an existing Docker Compose installation of Seqera Enterprise, use the following steps: 1. To run the service from a custom URL, declare the URL with the `GROUNDSWELL_SERVER_URL` environment variable. A non-zero value for this environment variable activates the optimization service automatically, so `TOWER_ENABLE_GROUNDSWELL` does not need to be set when you declare a custom URL. @@ -69,11 +69,11 @@ To use the pipeline resource optimization service in an existing Docker Compose # SWELL_DB_URL=mysql://db1.abcdefghijkl.us-east-1.rds.amazonaws.com:3306/swell ``` -7. When the pipeline resource optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. +7. When the pipeline optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. ## Kubernetes deployment -Kubernetes deployments use an [initContainer](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) that runs during pod initialization to set up the pipeline resource optimization service. To use the service in new or existing Kubernetes installations of Seqera Enterprise, do the following: +Kubernetes deployments use an [initContainer](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) that runs during pod initialization to set up the pipeline optimization service. To use the service in new or existing Kubernetes installations of Seqera Enterprise, do the following: 1. Download the [groundswell manifest](../_templates/k8s/groundswell.yml). @@ -101,9 +101,10 @@ Kubernetes deployments use an [initContainer](https://kubernetes.io/docs/concept FLUSH PRIVILEGES; ``` -The initContainers process will wait until both the Seqera and pipeline resource optimization service databases are ready before starting the migration in the Seqera database and finally starting the optimization container. +The `initContainers` process waits until both the Seqera and pipeline optimization service databases are ready before starting the migration in the Seqera database, and finally starting the optimization container. -When the pipeline resource optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. + +When the pipeline optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. [platform-k8s]: ../platform-kubernetes diff --git a/platform-enterprise_versioned_docs/version-25.1/enterprise/groundswell-docker-compose.md b/platform-enterprise_versioned_docs/version-25.1/enterprise/groundswell-docker-compose.md index d9c33a29b..14ddc53a4 100644 --- a/platform-enterprise_versioned_docs/version-25.1/enterprise/groundswell-docker-compose.md +++ b/platform-enterprise_versioned_docs/version-25.1/enterprise/groundswell-docker-compose.md @@ -1,11 +1,11 @@ --- title: "Pipeline optimization: Docker Compose" -description: Deploy pipeline resource optimization with Docker Compose +description: Deploy pipeline optimization with Docker Compose date: "12 Apr 2023" -tags: [docker, compose, groundswell, deployment] +tags: [docker, compose, groundswell, pipeline optimization, deployment] --- -This guide describes how to deploy the pipeline resource optimization service (Groundswell) for Seqera Platform Enterprise with Docker Compose. +This guide describes how to deploy the pipeline optimization service (referred to as `groundswell` in the configuration file) for Seqera Platform Enterprise using Docker Compose. ## Prerequisites @@ -76,4 +76,4 @@ When pipeline optimization is active, pipelines with at least one successful run ## Configuration -See [Pipeline resource optimization](./configuration/pipeline_optimization) for additional configuration options. +See [Pipeline optimization](./configuration/pipeline_optimization) for additional configuration options. diff --git a/platform-enterprise_versioned_docs/version-25.1/enterprise/groundswell-kubernetes.md b/platform-enterprise_versioned_docs/version-25.1/enterprise/groundswell-kubernetes.md index caf779bcf..9bf919a76 100644 --- a/platform-enterprise_versioned_docs/version-25.1/enterprise/groundswell-kubernetes.md +++ b/platform-enterprise_versioned_docs/version-25.1/enterprise/groundswell-kubernetes.md @@ -1,11 +1,11 @@ --- title: "Pipeline optimization: Kubernetes" -description: Deploy pipeline resource optimization on Kubernetes +description: Deploy pipeline optimization on Kubernetes date: "12 Apr 2023" -tags: [kubernetes, groundswell, deployment] +tags: [kubernetes, groundswell, pipeline optimization, deployment] --- -This guide describes how to deploy the pipeline resource optimization service (Groundswell) for Seqera Platform Enterprise on Kubernetes. +This guide describes how to deploy the pipeline optimization service (referred to as `groundswell` in the configuration file) for Seqera Platform Enterprise on Kubernetes. ## Prerequisites @@ -62,4 +62,4 @@ When pipeline optimization is active, pipelines with at least one successful run ## Configuration -See [Pipeline resource optimization](./configuration/pipeline_optimization) for additional configuration options. +See [Pipeline optimization](./configuration/pipeline_optimization) for additional configuration options. diff --git a/platform-enterprise_versioned_docs/version-25.1/enterprise/install-groundswell.md b/platform-enterprise_versioned_docs/version-25.1/enterprise/install-groundswell.md index ce899648e..699afa692 100644 --- a/platform-enterprise_versioned_docs/version-25.1/enterprise/install-groundswell.md +++ b/platform-enterprise_versioned_docs/version-25.1/enterprise/install-groundswell.md @@ -1,11 +1,11 @@ --- title: "Pipeline optimization" -description: Install pipeline resource optimization for Seqera Platform Enterprise +description: Install pipeline optimization for Seqera Platform Enterprise date: "12 Apr 2023" tags: [groundswell, pipeline, optimization, installation, deployment] --- -Pipeline resource optimization (Groundswell) uses resource usage data from previous workflow runs to optimize subsequent runs. Deploy after your Platform installation is complete. +Pipeline optimization can be deployed after your Platform installation is complete. It uses resource usage data from previous workflow runs to optimize subsequent runs. ## Deployment options diff --git a/platform-enterprise_versioned_docs/version-25.1/enterprise/platform-docker-compose.md b/platform-enterprise_versioned_docs/version-25.1/enterprise/platform-docker-compose.md index 31b229ca8..644d7f392 100644 --- a/platform-enterprise_versioned_docs/version-25.1/enterprise/platform-docker-compose.md +++ b/platform-enterprise_versioned_docs/version-25.1/enterprise/platform-docker-compose.md @@ -56,7 +56,7 @@ Use a managed Redis service for production: - The `db` container should be used only for local testing. If you have configured this service elsewhere, you can remove this container. - - To configure the Seqera pipeline resource optimization service (`groundswell`), see [Pipeline resource optimization](./configuration/pipeline_optimization). + - To configure the Seqera pipeline optimization service (`groundswell`), see [Pipeline optimization](./configuration/pipeline_optimization). - To deploy with Studios, see [Studios deployment](./studios). diff --git a/platform-enterprise_versioned_docs/version-25.1/pipeline-optimization/overview.md b/platform-enterprise_versioned_docs/version-25.1/pipeline-optimization/overview.md index 40ed101bd..ded022d44 100644 --- a/platform-enterprise_versioned_docs/version-25.1/pipeline-optimization/overview.md +++ b/platform-enterprise_versioned_docs/version-25.1/pipeline-optimization/overview.md @@ -1,5 +1,5 @@ --- -title: "Pipeline resource optimization" +title: "Pipeline optimization" description: "Optimize the resource usage of your pipelines to save time and money." date: "12 Feb 2024" tags: [compute, resource, optimization] @@ -7,7 +7,7 @@ tags: [compute, resource, optimization] **Available from version 23.3.0** -Pipeline resource optimization takes the resource usage information from previous workflow runs to optimize subsequent runs. +Pipeline optimization takes the resource usage information from previous workflow runs to optimize subsequent runs. When a run completes successfully, an _optimized profile_ is created. This profile consists of Nextflow configuration settings for each process and each resource directive (where applicable): `cpus`, `memory`, and `time`. The optimized setting for a given process and resource directive is based on the maximum use of that resource across all tasks in that process. diff --git a/platform-enterprise_versioned_docs/version-25.2/enterprise/_templates/docker/docker-compose.yml b/platform-enterprise_versioned_docs/version-25.2/enterprise/_templates/docker/docker-compose.yml index a57517c62..72d28df8d 100644 --- a/platform-enterprise_versioned_docs/version-25.2/enterprise/_templates/docker/docker-compose.yml +++ b/platform-enterprise_versioned_docs/version-25.2/enterprise/_templates/docker/docker-compose.yml @@ -18,7 +18,7 @@ services: retries: 10 restart: always volumes: - # To use the pipeline resource optimization service, uncomment the lines below to mount the initialization script + # To use the pipeline optimization service, uncomment the lines below to mount the initialization script # - type: bind # source: ./init.sql # target: /docker-entrypoint-initdb.d/init.sql @@ -141,7 +141,7 @@ services: # - 7070:7070 # restart: always - # Uncomment the following section to enable the pipeline resource optimization service. Add TOWER_ENABLE_GROUNDSWELL + # Uncomment the following section to enable the pipeline optimization service. Add TOWER_ENABLE_GROUNDSWELL # or GROUNDSWELL_SERVER_URL to tower.env environment variables # groundswell: # image: cr.seqera.io/private/nf-tower-enterprise/groundswell:0.3.3 diff --git a/platform-enterprise_versioned_docs/version-25.2/enterprise/_templates/docker/tower.env b/platform-enterprise_versioned_docs/version-25.2/enterprise/_templates/docker/tower.env index c55fb6c28..aef4830cb 100644 --- a/platform-enterprise_versioned_docs/version-25.2/enterprise/_templates/docker/tower.env +++ b/platform-enterprise_versioned_docs/version-25.2/enterprise/_templates/docker/tower.env @@ -23,7 +23,7 @@ TOWER_SMTP_PORT=587 TOWER_SMTP_USER=foo TOWER_SMTP_PASSWORD=foo -# Pipeline resource optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) +# Pipeline optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) # Use GROUNDSWELL_SERVER_URL to specify a custom service URL (enables the service by default) TOWER_ENABLE_GROUNDSWELL=false GROUNDSWELL_SERVER_URL="http://groundswell:8090" diff --git a/platform-enterprise_versioned_docs/version-25.2/enterprise/_templates/k8s/configmap.yml b/platform-enterprise_versioned_docs/version-25.2/enterprise/_templates/k8s/configmap.yml index 8b6dd2015..0d266a648 100644 --- a/platform-enterprise_versioned_docs/version-25.2/enterprise/_templates/k8s/configmap.yml +++ b/platform-enterprise_versioned_docs/version-25.2/enterprise/_templates/k8s/configmap.yml @@ -24,7 +24,7 @@ data: TOWER_ENABLE_PLATFORMS: "altair-platform,awsbatch-platform,awscloud-platform,azbatch-platform,eks-platform,gke-platform,googlebatch-platform,googlecloud-platform,k8s-platform,lsf-platform,moab-platform,slurm-platform,uge-platform" TOWER_REDIS_URL: "redis://:6379" # For a managed Redis service, specify the IP address or DNS name of the service endpoint. If you use the containerized Redis service, specify "redis" instead of an IP address. - # Pipeline resource optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) + # Pipeline optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) # Use GROUNDSWELL_SERVER_URL to specify a custom service URL (enables the service by default) TOWER_ENABLE_GROUNDSWELL: "false" GROUNDSWELL_SERVER_URL: http://groundswell:8090 diff --git a/platform-enterprise_versioned_docs/version-25.2/enterprise/configuration/pipeline_optimization.md b/platform-enterprise_versioned_docs/version-25.2/enterprise/configuration/pipeline_optimization.md index 6b2801586..187b8e4e1 100644 --- a/platform-enterprise_versioned_docs/version-25.2/enterprise/configuration/pipeline_optimization.md +++ b/platform-enterprise_versioned_docs/version-25.2/enterprise/configuration/pipeline_optimization.md @@ -1,21 +1,21 @@ --- title: "Pipeline optimization" -description: "Configure pipeline resource optimization in your Seqera Enterprise deployment." +description: "Configure pipeline optimization in your Seqera Enterprise deployment." date: "12 Feb 2024" tags: [compute, resource, optimization, configuration] --- -[Pipeline resource optimization](../../pipeline-optimization/overview) takes the resource usage information from previous workflow runs to optimize subsequent runs. +[Pipeline optimization](../../pipeline-optimization/overview) takes the resource usage information from previous workflow runs to optimize subsequent runs. -The pipeline resource optimization service requires a separate database schema to store its internal data, but also requires access to the Seqera schema. The Seqera and optimization service schemas can coexist on the same database instance. +The pipeline optimization service requires a separate database schema to store its internal data, but also requires access to the Seqera schema. The Seqera and optimization service schemas can coexist on the same database instance. ## Docker Compose deployment -Docker Compose makes use of a separate container to set up the pipeline resource optimization service during initialization. Configuration steps differ for new and existing deployments. +Docker Compose makes use of a separate container to set up the pipeline optimization service during initialization. Configuration steps differ for new and existing deployments. ### New installation -To use the pipeline resource optimization service in a new Docker Compose installation of Seqera Enterprise, use the following steps: +To use the pipeline optimization service in a new Docker Compose installation of Seqera Enterprise, use the following steps: 1. To run the service from a custom URL, declare the URL with the `GROUNDSWELL_SERVER_URL` environment variable in `tower.env`. A non-zero value for this environment variable activates the optimization service automatically, so `TOWER_ENABLE_GROUNDSWELL` does not need to be set when you declare a custom URL. @@ -27,11 +27,11 @@ To use the pipeline resource optimization service in a new Docker Compose instal 4. Download the [init.sql](../_templates/docker/init.sql) file. Store this file in the mount path of your `docker-compose.yml` file, else update the `source: ./init.sql` line in your `docker-compose.yml` with the file path. -5. When the pipeline resource optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. +5. When the pipeline optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. ### Existing installation -To use the pipeline resource optimization service in an existing Docker Compose installation of Seqera Enterprise, use the following steps: +To use the pipeline optimization service in an existing Docker Compose installation of Seqera Enterprise, use the following steps: 1. To run the service from a custom URL, declare the URL with the `GROUNDSWELL_SERVER_URL` environment variable. A non-zero value for this environment variable activates the optimization service automatically, so `TOWER_ENABLE_GROUNDSWELL` does not need to be set when you declare a custom URL. @@ -69,11 +69,11 @@ To use the pipeline resource optimization service in an existing Docker Compose # SWELL_DB_URL=mysql://db1.abcdefghijkl.us-east-1.rds.amazonaws.com:3306/swell ``` -7. When the pipeline resource optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. +7. When the pipeline optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. ## Kubernetes deployment -Kubernetes deployments use an [initContainer](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) that runs during pod initialization to set up the pipeline resource optimization service. To use the service in new or existing Kubernetes installations of Seqera Enterprise, do the following: +Kubernetes deployments use an [initContainer](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) that runs during pod initialization to set up the pipeline optimization service. To use the service in new or existing Kubernetes installations of Seqera Enterprise, do the following: 1. Download the [groundswell manifest](../_templates/k8s/groundswell.yml). @@ -101,9 +101,9 @@ Kubernetes deployments use an [initContainer](https://kubernetes.io/docs/concept FLUSH PRIVILEGES; ``` -The initContainers process will wait until both the Seqera and pipeline resource optimization service databases are ready before starting the migration in the Seqera database and finally starting the optimization container. +The initContainers process will wait until both the Seqera and pipeline optimization service databases are ready before starting the migration in the Seqera database and finally starting the optimization container. -When the pipeline resource optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. +When the pipeline optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. [platform-k8s]: ../platform-kubernetes diff --git a/platform-enterprise_versioned_docs/version-25.2/enterprise/groundswell-docker-compose.md b/platform-enterprise_versioned_docs/version-25.2/enterprise/groundswell-docker-compose.md index d9c33a29b..14ddc53a4 100644 --- a/platform-enterprise_versioned_docs/version-25.2/enterprise/groundswell-docker-compose.md +++ b/platform-enterprise_versioned_docs/version-25.2/enterprise/groundswell-docker-compose.md @@ -1,11 +1,11 @@ --- title: "Pipeline optimization: Docker Compose" -description: Deploy pipeline resource optimization with Docker Compose +description: Deploy pipeline optimization with Docker Compose date: "12 Apr 2023" -tags: [docker, compose, groundswell, deployment] +tags: [docker, compose, groundswell, pipeline optimization, deployment] --- -This guide describes how to deploy the pipeline resource optimization service (Groundswell) for Seqera Platform Enterprise with Docker Compose. +This guide describes how to deploy the pipeline optimization service (referred to as `groundswell` in the configuration file) for Seqera Platform Enterprise using Docker Compose. ## Prerequisites @@ -76,4 +76,4 @@ When pipeline optimization is active, pipelines with at least one successful run ## Configuration -See [Pipeline resource optimization](./configuration/pipeline_optimization) for additional configuration options. +See [Pipeline optimization](./configuration/pipeline_optimization) for additional configuration options. diff --git a/platform-enterprise_versioned_docs/version-25.2/enterprise/groundswell-kubernetes.md b/platform-enterprise_versioned_docs/version-25.2/enterprise/groundswell-kubernetes.md index caf779bcf..9bf919a76 100644 --- a/platform-enterprise_versioned_docs/version-25.2/enterprise/groundswell-kubernetes.md +++ b/platform-enterprise_versioned_docs/version-25.2/enterprise/groundswell-kubernetes.md @@ -1,11 +1,11 @@ --- title: "Pipeline optimization: Kubernetes" -description: Deploy pipeline resource optimization on Kubernetes +description: Deploy pipeline optimization on Kubernetes date: "12 Apr 2023" -tags: [kubernetes, groundswell, deployment] +tags: [kubernetes, groundswell, pipeline optimization, deployment] --- -This guide describes how to deploy the pipeline resource optimization service (Groundswell) for Seqera Platform Enterprise on Kubernetes. +This guide describes how to deploy the pipeline optimization service (referred to as `groundswell` in the configuration file) for Seqera Platform Enterprise on Kubernetes. ## Prerequisites @@ -62,4 +62,4 @@ When pipeline optimization is active, pipelines with at least one successful run ## Configuration -See [Pipeline resource optimization](./configuration/pipeline_optimization) for additional configuration options. +See [Pipeline optimization](./configuration/pipeline_optimization) for additional configuration options. diff --git a/platform-enterprise_versioned_docs/version-25.2/enterprise/install-groundswell.md b/platform-enterprise_versioned_docs/version-25.2/enterprise/install-groundswell.md index ce899648e..699afa692 100644 --- a/platform-enterprise_versioned_docs/version-25.2/enterprise/install-groundswell.md +++ b/platform-enterprise_versioned_docs/version-25.2/enterprise/install-groundswell.md @@ -1,11 +1,11 @@ --- title: "Pipeline optimization" -description: Install pipeline resource optimization for Seqera Platform Enterprise +description: Install pipeline optimization for Seqera Platform Enterprise date: "12 Apr 2023" tags: [groundswell, pipeline, optimization, installation, deployment] --- -Pipeline resource optimization (Groundswell) uses resource usage data from previous workflow runs to optimize subsequent runs. Deploy after your Platform installation is complete. +Pipeline optimization can be deployed after your Platform installation is complete. It uses resource usage data from previous workflow runs to optimize subsequent runs. ## Deployment options diff --git a/platform-enterprise_versioned_docs/version-25.2/enterprise/platform-docker-compose.md b/platform-enterprise_versioned_docs/version-25.2/enterprise/platform-docker-compose.md index 31b229ca8..644d7f392 100644 --- a/platform-enterprise_versioned_docs/version-25.2/enterprise/platform-docker-compose.md +++ b/platform-enterprise_versioned_docs/version-25.2/enterprise/platform-docker-compose.md @@ -56,7 +56,7 @@ Use a managed Redis service for production: - The `db` container should be used only for local testing. If you have configured this service elsewhere, you can remove this container. - - To configure the Seqera pipeline resource optimization service (`groundswell`), see [Pipeline resource optimization](./configuration/pipeline_optimization). + - To configure the Seqera pipeline optimization service (`groundswell`), see [Pipeline optimization](./configuration/pipeline_optimization). - To deploy with Studios, see [Studios deployment](./studios). diff --git a/platform-enterprise_versioned_docs/version-25.2/pipeline-optimization/overview.md b/platform-enterprise_versioned_docs/version-25.2/pipeline-optimization/overview.md index 40ed101bd..ded022d44 100644 --- a/platform-enterprise_versioned_docs/version-25.2/pipeline-optimization/overview.md +++ b/platform-enterprise_versioned_docs/version-25.2/pipeline-optimization/overview.md @@ -1,5 +1,5 @@ --- -title: "Pipeline resource optimization" +title: "Pipeline optimization" description: "Optimize the resource usage of your pipelines to save time and money." date: "12 Feb 2024" tags: [compute, resource, optimization] @@ -7,7 +7,7 @@ tags: [compute, resource, optimization] **Available from version 23.3.0** -Pipeline resource optimization takes the resource usage information from previous workflow runs to optimize subsequent runs. +Pipeline optimization takes the resource usage information from previous workflow runs to optimize subsequent runs. When a run completes successfully, an _optimized profile_ is created. This profile consists of Nextflow configuration settings for each process and each resource directive (where applicable): `cpus`, `memory`, and `time`. The optimized setting for a given process and resource directive is based on the maximum use of that resource across all tasks in that process. diff --git a/platform-enterprise_versioned_docs/version-25.3/enterprise/_templates/docker/docker-compose.yml b/platform-enterprise_versioned_docs/version-25.3/enterprise/_templates/docker/docker-compose.yml index 7e2060579..210c187f3 100644 --- a/platform-enterprise_versioned_docs/version-25.3/enterprise/_templates/docker/docker-compose.yml +++ b/platform-enterprise_versioned_docs/version-25.3/enterprise/_templates/docker/docker-compose.yml @@ -18,7 +18,7 @@ services: retries: 10 restart: always volumes: - # To use the pipeline resource optimization service, uncomment the lines below to mount the initialization script + # To use the pipeline optimization service, uncomment the lines below to mount the initialization script # - type: bind # source: ./init.sql # target: /docker-entrypoint-initdb.d/init.sql @@ -141,7 +141,7 @@ services: # - 7070:7070 # restart: always - # Uncomment the following section to enable the pipeline resource optimization service. Add TOWER_ENABLE_GROUNDSWELL + # Uncomment the following section to enable the pipeline optimization service. Add TOWER_ENABLE_GROUNDSWELL # or GROUNDSWELL_SERVER_URL to tower.env environment variables # groundswell: # image: cr.seqera.io/private/nf-tower-enterprise/groundswell:0.3.3 diff --git a/platform-enterprise_versioned_docs/version-25.3/enterprise/_templates/docker/tower.env b/platform-enterprise_versioned_docs/version-25.3/enterprise/_templates/docker/tower.env index f43ab5b31..06a57ea84 100644 --- a/platform-enterprise_versioned_docs/version-25.3/enterprise/_templates/docker/tower.env +++ b/platform-enterprise_versioned_docs/version-25.3/enterprise/_templates/docker/tower.env @@ -24,7 +24,7 @@ TOWER_SMTP_PORT=587 TOWER_SMTP_USER=foo TOWER_SMTP_PASSWORD=foo -# Pipeline resource optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) +# Pipeline optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) # Use GROUNDSWELL_SERVER_URL to specify a custom service URL (enables the service by default) TOWER_ENABLE_GROUNDSWELL=false GROUNDSWELL_SERVER_URL="http://groundswell:8090" diff --git a/platform-enterprise_versioned_docs/version-25.3/enterprise/_templates/k8s/configmap.yml b/platform-enterprise_versioned_docs/version-25.3/enterprise/_templates/k8s/configmap.yml index 3bd695adb..cc40656b6 100644 --- a/platform-enterprise_versioned_docs/version-25.3/enterprise/_templates/k8s/configmap.yml +++ b/platform-enterprise_versioned_docs/version-25.3/enterprise/_templates/k8s/configmap.yml @@ -24,7 +24,7 @@ data: TOWER_ENABLE_PLATFORMS: "altair-platform,awsbatch-platform,awscloud-platform,azbatch-platform,azcloud-platform,eks-platform,gke-platform,googlebatch-platform,googlecloud-platform,k8s-platform,lsf-platform,moab-platform,slurm-platform,uge-platform" TOWER_REDIS_URL: "redis://:6379" # For a managed Redis service, specify the IP address or DNS name of the service endpoint. If you use the containerized Redis service, specify "redis" instead of an IP address. - # Pipeline resource optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) + # Pipeline optimization service. Set TOWER_ENABLE_GROUNDSWELL true to use default service URL (http://groundswell:8090) # Use GROUNDSWELL_SERVER_URL to specify a custom service URL (enables the service by default) TOWER_ENABLE_GROUNDSWELL: "false" GROUNDSWELL_SERVER_URL: http://groundswell:8090 diff --git a/platform-enterprise_versioned_docs/version-25.3/enterprise/configuration/pipeline_optimization.md b/platform-enterprise_versioned_docs/version-25.3/enterprise/configuration/pipeline_optimization.md index 6b2801586..d1f0d754d 100644 --- a/platform-enterprise_versioned_docs/version-25.3/enterprise/configuration/pipeline_optimization.md +++ b/platform-enterprise_versioned_docs/version-25.3/enterprise/configuration/pipeline_optimization.md @@ -1,21 +1,21 @@ --- title: "Pipeline optimization" -description: "Configure pipeline resource optimization in your Seqera Enterprise deployment." +description: "Configure pipeline optimization in your Seqera Enterprise deployment." date: "12 Feb 2024" tags: [compute, resource, optimization, configuration] --- -[Pipeline resource optimization](../../pipeline-optimization/overview) takes the resource usage information from previous workflow runs to optimize subsequent runs. +[Pipeline optimization](../../pipeline-optimization/overview) takes the resource usage information from previous workflow runs to optimize subsequent runs. -The pipeline resource optimization service requires a separate database schema to store its internal data, but also requires access to the Seqera schema. The Seqera and optimization service schemas can coexist on the same database instance. +The pipeline optimization service requires a separate database schema to store its internal data, but also requires access to the Seqera schema. The Seqera and optimization service schemas can coexist on the same database instance. ## Docker Compose deployment -Docker Compose makes use of a separate container to set up the pipeline resource optimization service during initialization. Configuration steps differ for new and existing deployments. +Docker Compose makes use of a separate container to set up the pipeline optimization service during initialization. Configuration steps differ for new and existing deployments. ### New installation -To use the pipeline resource optimization service in a new Docker Compose installation of Seqera Enterprise, use the following steps: +To use the pipeline optimization service in a new Docker Compose installation of Seqera Enterprise, use the following steps: 1. To run the service from a custom URL, declare the URL with the `GROUNDSWELL_SERVER_URL` environment variable in `tower.env`. A non-zero value for this environment variable activates the optimization service automatically, so `TOWER_ENABLE_GROUNDSWELL` does not need to be set when you declare a custom URL. @@ -27,11 +27,11 @@ To use the pipeline resource optimization service in a new Docker Compose instal 4. Download the [init.sql](../_templates/docker/init.sql) file. Store this file in the mount path of your `docker-compose.yml` file, else update the `source: ./init.sql` line in your `docker-compose.yml` with the file path. -5. When the pipeline resource optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. +5. When the pipeline optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. ### Existing installation -To use the pipeline resource optimization service in an existing Docker Compose installation of Seqera Enterprise, use the following steps: +To use the pipeline optimization service in an existing Docker Compose installation of Seqera Enterprise, use the following steps: 1. To run the service from a custom URL, declare the URL with the `GROUNDSWELL_SERVER_URL` environment variable. A non-zero value for this environment variable activates the optimization service automatically, so `TOWER_ENABLE_GROUNDSWELL` does not need to be set when you declare a custom URL. @@ -69,11 +69,11 @@ To use the pipeline resource optimization service in an existing Docker Compose # SWELL_DB_URL=mysql://db1.abcdefghijkl.us-east-1.rds.amazonaws.com:3306/swell ``` -7. When the pipeline resource optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. +7. When the pipeline optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. ## Kubernetes deployment -Kubernetes deployments use an [initContainer](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) that runs during pod initialization to set up the pipeline resource optimization service. To use the service in new or existing Kubernetes installations of Seqera Enterprise, do the following: +Kubernetes deployments use an [initContainer](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) that runs during pod initialization to set up the pipeline optimization service. To use the service in new or existing Kubernetes installations of Seqera Enterprise, do the following: 1. Download the [groundswell manifest](../_templates/k8s/groundswell.yml). @@ -101,9 +101,10 @@ Kubernetes deployments use an [initContainer](https://kubernetes.io/docs/concept FLUSH PRIVILEGES; ``` -The initContainers process will wait until both the Seqera and pipeline resource optimization service databases are ready before starting the migration in the Seqera database and finally starting the optimization container. +The `initContainers` process waits until both the Seqera and pipeline optimization service databases are ready before starting the migration in the Seqera database, and finally starting the optimization container. -When the pipeline resource optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. + +When the pipeline optimization service is active, pipelines that can be optimized display a lightbulb icon in your Launchpad. Any pipeline with at least one successful run can be optimized. [platform-k8s]: ../platform-kubernetes diff --git a/platform-enterprise_versioned_docs/version-25.3/enterprise/groundswell-docker-compose.md b/platform-enterprise_versioned_docs/version-25.3/enterprise/groundswell-docker-compose.md index d9c33a29b..14ddc53a4 100644 --- a/platform-enterprise_versioned_docs/version-25.3/enterprise/groundswell-docker-compose.md +++ b/platform-enterprise_versioned_docs/version-25.3/enterprise/groundswell-docker-compose.md @@ -1,11 +1,11 @@ --- title: "Pipeline optimization: Docker Compose" -description: Deploy pipeline resource optimization with Docker Compose +description: Deploy pipeline optimization with Docker Compose date: "12 Apr 2023" -tags: [docker, compose, groundswell, deployment] +tags: [docker, compose, groundswell, pipeline optimization, deployment] --- -This guide describes how to deploy the pipeline resource optimization service (Groundswell) for Seqera Platform Enterprise with Docker Compose. +This guide describes how to deploy the pipeline optimization service (referred to as `groundswell` in the configuration file) for Seqera Platform Enterprise using Docker Compose. ## Prerequisites @@ -76,4 +76,4 @@ When pipeline optimization is active, pipelines with at least one successful run ## Configuration -See [Pipeline resource optimization](./configuration/pipeline_optimization) for additional configuration options. +See [Pipeline optimization](./configuration/pipeline_optimization) for additional configuration options. diff --git a/platform-enterprise_versioned_docs/version-25.3/enterprise/groundswell-kubernetes.md b/platform-enterprise_versioned_docs/version-25.3/enterprise/groundswell-kubernetes.md index caf779bcf..9bf919a76 100644 --- a/platform-enterprise_versioned_docs/version-25.3/enterprise/groundswell-kubernetes.md +++ b/platform-enterprise_versioned_docs/version-25.3/enterprise/groundswell-kubernetes.md @@ -1,11 +1,11 @@ --- title: "Pipeline optimization: Kubernetes" -description: Deploy pipeline resource optimization on Kubernetes +description: Deploy pipeline optimization on Kubernetes date: "12 Apr 2023" -tags: [kubernetes, groundswell, deployment] +tags: [kubernetes, groundswell, pipeline optimization, deployment] --- -This guide describes how to deploy the pipeline resource optimization service (Groundswell) for Seqera Platform Enterprise on Kubernetes. +This guide describes how to deploy the pipeline optimization service (referred to as `groundswell` in the configuration file) for Seqera Platform Enterprise on Kubernetes. ## Prerequisites @@ -62,4 +62,4 @@ When pipeline optimization is active, pipelines with at least one successful run ## Configuration -See [Pipeline resource optimization](./configuration/pipeline_optimization) for additional configuration options. +See [Pipeline optimization](./configuration/pipeline_optimization) for additional configuration options. diff --git a/platform-enterprise_versioned_docs/version-25.3/enterprise/install-groundswell.md b/platform-enterprise_versioned_docs/version-25.3/enterprise/install-groundswell.md index ce899648e..441606ec5 100644 --- a/platform-enterprise_versioned_docs/version-25.3/enterprise/install-groundswell.md +++ b/platform-enterprise_versioned_docs/version-25.3/enterprise/install-groundswell.md @@ -1,11 +1,11 @@ --- title: "Pipeline optimization" -description: Install pipeline resource optimization for Seqera Platform Enterprise +description: Install pipeline optimization for Seqera Platform Enterprise date: "12 Apr 2023" tags: [groundswell, pipeline, optimization, installation, deployment] --- -Pipeline resource optimization (Groundswell) uses resource usage data from previous workflow runs to optimize subsequent runs. Deploy after your Platform installation is complete. +Pipeline optimization (Groundswell) uses resource usage data from previous workflow runs to optimize subsequent runs. Deploy after your Platform installation is complete. ## Deployment options diff --git a/platform-enterprise_versioned_docs/version-25.3/enterprise/platform-docker-compose.md b/platform-enterprise_versioned_docs/version-25.3/enterprise/platform-docker-compose.md index 31b229ca8..644d7f392 100644 --- a/platform-enterprise_versioned_docs/version-25.3/enterprise/platform-docker-compose.md +++ b/platform-enterprise_versioned_docs/version-25.3/enterprise/platform-docker-compose.md @@ -56,7 +56,7 @@ Use a managed Redis service for production: - The `db` container should be used only for local testing. If you have configured this service elsewhere, you can remove this container. - - To configure the Seqera pipeline resource optimization service (`groundswell`), see [Pipeline resource optimization](./configuration/pipeline_optimization). + - To configure the Seqera pipeline optimization service (`groundswell`), see [Pipeline optimization](./configuration/pipeline_optimization). - To deploy with Studios, see [Studios deployment](./studios). diff --git a/platform-enterprise_versioned_docs/version-25.3/pipeline-optimization/overview.md b/platform-enterprise_versioned_docs/version-25.3/pipeline-optimization/overview.md index 40ed101bd..ded022d44 100644 --- a/platform-enterprise_versioned_docs/version-25.3/pipeline-optimization/overview.md +++ b/platform-enterprise_versioned_docs/version-25.3/pipeline-optimization/overview.md @@ -1,5 +1,5 @@ --- -title: "Pipeline resource optimization" +title: "Pipeline optimization" description: "Optimize the resource usage of your pipelines to save time and money." date: "12 Feb 2024" tags: [compute, resource, optimization] @@ -7,7 +7,7 @@ tags: [compute, resource, optimization] **Available from version 23.3.0** -Pipeline resource optimization takes the resource usage information from previous workflow runs to optimize subsequent runs. +Pipeline optimization takes the resource usage information from previous workflow runs to optimize subsequent runs. When a run completes successfully, an _optimized profile_ is created. This profile consists of Nextflow configuration settings for each process and each resource directive (where applicable): `cpus`, `memory`, and `time`. The optimized setting for a given process and resource directive is based on the maximum use of that resource across all tasks in that process.