diff --git a/rfcs/0001-api-v2/resources/Semaphore API Design Guide.md b/rfcs/0001-api-v2/resources/Semaphore API Design Guide.md new file mode 100644 index 00000000..b13178da --- /dev/null +++ b/rfcs/0001-api-v2/resources/Semaphore API Design Guide.md @@ -0,0 +1,362 @@ +## + +[Introduction](#introduction) + +[Resources](#resources) + +[Resource-Oriented Design](#resource-oriented-design) + +[Nested Resources](#nested-resources) + +[Endpoint Naming](#endpoint-naming) + +[Response](#response) + +[Response Format](#response-format) + +[Object Instead of IDs](#object-instead-of-ids) + +[Standard Methods](#standard-methods) + +[Create a Resource](#create-a-resource) + +[Read a Resource](#read-a-resource) + +[Update a Resource](#update-a-resource) + +[Delete a Resource](#delete-a-resource) + +[Custom Methods](#custom-methods) + +[Behavior of Custom Methods](#behavior-of-custom-methods) + +[Request and Response Structure](#request-and-response-structure) + +[Examples of Custom Methods](#examples-of-custom-methods) + +[Encouraging the Use of Standard Methods](#encouraging-the-use-of-standard-methods) + +[Error Handling](#error-handling) + +[Error Response Format](#error-response-format) + +[Writing Effective Error Messages](#writing-effective-error-messages) + +[Error Codes](#error-codes) + +[HTTP Status Codes](#http-status-codes) + +[Best Practices for Error Handling](#best-practices-for-error-handling) + +[Standard Fields](#standard-fields) + +[Naming Convention: snake\_case](#naming-convention:-snake_case) + +[Timestamps](#timestamps) + +## Introduction {#introduction} + +Welcome to the **Semaphore API Design Guide**. This guide sets out the core principles and best practices for building and evolving APIs within our CI/CD platform. It’s intended for both internal developers and external contributors to help ensure consistency, simplicity, and scalability across the API. + +This guide focuses on **RESTful APIs** and covers critical areas like URL structures, versioning, error handling, and authentication. By adhering to these guidelines, you'll ensure that any proposed updates align with Semaphore’s technical vision and can be smoothly integrated with existing code. + +We update this guide regularly to reflect new patterns and recommendations. Whether you're adding features, fixing issues, or improving performance, we encourage you to consult this document and follow the outlined practices. Feedback and collaboration are always welcome as we continuously improve our API. + +## Resources {#resources} + +### Resource-Oriented Design {#resource-oriented-design} + +At Semaphore, we follow a **resource-oriented design** approach for our API to ensure simplicity, consistency, and ease of use. This design pattern structures the API around resources—such as pipelines, jobs, or secrets—making it intuitive and predictable for developers. + +By adopting resource-oriented design, we aim to: + +* **Simplify interactions**: Each resource in our API is represented by a unique URL and interacts through standard HTTP methods (GET, POST, PUT, DELETE). This makes it easier for developers to understand and work with the API, using clear and consistent endpoints. +* **Maintain consistency**: Structuring the API around resources ensures that similar types of data are handled in a uniform way. Whether you're retrieving information, updating a record, or deleting an entity, the same approach applies across all resources. +* **Enhance usability**: With clearly defined resources and operations, the API remains intuitive for both new and experienced developers. This approach minimizes confusion and speeds up the development process by reducing the learning curve. + +In practice, every resource in the Semaphore API is treated as a first-class entity, with actions mapped to appropriate HTTP methods. For example: + +* `GET /jobs/{id}` retrieves details for a specific job. +* `POST /pipelines` creates a new pipeline. +* `DELETE /secrets/{secret_name_or_id}` removes a secret. + +This structure not only supports clean, organized API design but also makes scaling and maintaining the API much easier as Semaphore continues to grow. + +### Nested Resources {#nested-resources} + +In the Semaphore API, most resources are defined at the root level, such as `/jobs`, `/pipelines`, and `/secrets`, even though some of them are associated with specific projects. However, for other resources—like deployment targets, project secrets, and tasks—we use a nested structure. The primary reason for this is to handle potential **name conflicts** when resources can have the same name across different projects. + +**Why Nest Resources?** + +When a resource can be identified by name (such as tasks or project secrets) and that name might be used in multiple projects, nesting under the project resource becomes necessary. This ensures that resources are scoped within the correct project, preventing ambiguity in cases where the same resource name exists in more than one project. + +For example: + +* `GET /projects/{project_name_or_id}/tasks/{task_name_or_id}` retrieves a task with the given name or ID, scoped to a specific project. +* `POST /projects/{project_name_or_id}/deployment_targets` creates a deployment target for a specific project. +* `DELETE /projects/{project_name_or_id}/secrets/{secret_name_or_id}` removes a named secret within a project. + +By scoping resources like tasks or secrets under the project, we avoid conflicts that would arise if multiple projects used the same resource name. + +**When to use nested resources** + +* **Handling name conflicts**: Resources like tasks, secrets, and deployment targets that may share names across projects must be nested under `project_name_or_id` to properly differentiate between them. + +In all other cases where resource names are unique or where scoping isn't necessary, resources remain at the root level to keep the API simple and easy to use. + +### Endpoint Naming {#endpoint-naming} + +In the Semaphore API, we use **snake\_case** (underscores between words) for naming endpoints. This convention ensures consistency and readability throughout the API, making it easier for developers to work with. + +**Example** + +* `GET /self_hosted_agent_types`: Retrieve available types of self-hosted agents. +* `POST /projects/{project_name_or_id}/deployment_targets`: Create a new deployment target within a specific project. + +By following this naming convention, we maintain uniformity across all API endpoints, helping both internal and external developers quickly understand the purpose and structure of each endpoint. This consistency simplifies development and ensures a smooth experience when interacting with the API. + +## Response {#response} + +### Response Format {#response-format} + +In the Semaphore API, resources follow a consistent structure to facilitate easy identification and parsing. Each resource contains the following key fields: + +**apiVersion** + +The apiVersion field specifies the version of the API that the resource adheres to. This helps maintain compatibility as the API evolves over time. + +**`Example`**`: "apiVersion": "v2"` + +**kind** + +The `kind` field defines the type of the resource. It helps in identifying the nature of the resource (e.g., a Secret, Job, or Project). + +**Example**: `"kind": "Secret"` + +**metadata** + +The `metadata` field contains non-functional information that helps identify and manage the resource. This section typically includes: + +* **id**: The unique identifier of the resource, (can be replaced with **user\_id** in members API). +* **user\_id**: The unique identifier of the user. +* **org\_id**: The unique identifier of the organization. +* **project\_id**: The unique identifier of the project (if nested under project). +* **name**: A human-readable name for the resource. +* **created\_at**: The timestamp when the resource was created. +* **updated\_at**: The timestamp when the resource was last updated. + +**Example**: + +`"metadata": {` + `"id": "b6654945-cf11-43a1-8e3a-f5a22c7d19fa",` + `"org_id": "7a9f674e-3c3f-47e9-b9ef-bc456a7e27b5",` + `"project_id": "be3d2a67-e741-4c1f-882e-8d7ad5461e4b",` + `"created_at": "2024-07-12T11:02:50.108Z",` + `"updated_at": "2024-07-15T10:30:00.000Z"` +`}` + +**spec** + +The `spec` field contains the functional configuration or the desired state of the resource. This is where the actual content of the resource is defined, based on its `kind`. For instance, if the resource is a `Secret`, the `spec` field would include details like environment variables, files, and access configurations. + +**Example**: + +`"spec": {` + `"access_config": {` + `"attach_access": "YES",` + `"debug_access": "YES",` + `"project_access": "WHITELISTED",` + `"projects": [` +`{ "id": "0e7fa31e-f974-4f57-ab99-72ac8b70df41" }` + `]` + `},` + `"data": {` + `"env_vars": [` + `{` + `"name": "MY_SECRET"` + `}` + `],` + `"files": [` + `{` + `"path": "/path/to/file"` + `}` + `]` + `}` +`}` + +### **Object Instead of IDs** {#object-instead-of-ids} + +In Semaphore's API, we return detailed objects with relevant information for entities such as users and roles, rather than providing only raw identifiers. This ensures that clients receive necessary context directly in the response without requiring additional API calls. + +Initially, the object will include only the `id` field, but as needed, it can be expanded to include other attributes such as `name` or `avatar`. This provides flexibility and allows the API to return more detailed information when necessary. + +**Example** + +`"promoted_by": {` + + `"type": "USER",` + + `"id": "b6654945-cf11-43a1-8e3a-f5a22c7d19fa",` + + `"name": "John Doe",` + + `"avatar": "https://example.com/avatar.jpg"` + +`}` + +By returning full objects, we provide more context for interacting with these entities, reducing the need for additional queries and enhancing usability. + +## Standard Methods {#standard-methods} + +Semaphore’s API follows RESTful principles, using standard HTTP methods to interact with resources. The four main actions—**Create**, **Read**, **Update**, and **Delete** (CRUD)—are supported by corresponding HTTP methods: `POST`, `GET`, `PATCH`, and `DELETE`. Each method behaves consistently across different resources, ensuring predictable API interactions. + +### Create a Resource {#create-a-resource} + +* **Behavior**: The `POST` method is used to create a new resource within a project or other parent resource. When creating a resource, the request body must contain `metadata` and `spec` fields that define both the identity and details of the resource being created. +* **Parameters**: + * `metadata`: Includes essential resource identification fields, such as `project_id`. + * `spec`: Contains the functional details of the resource (e.g., name, configuration, etc.). +* **Request Format**: The request body contains `metadata` and `spec`, while `apiVersion` and `kind` are optional and can be inferred from the URL. +* **Response Format**: The response mirrors the request format, including `metadata`, `spec`, and potentially generated fields like `id`, timestamps, and user identifiers. + +### Read a Resource {#read-a-resource} + +* **Behavior**: The `GET` method retrieves details of an existing resource. This could be a single resource or a collection of resources, depending on the endpoint. No body is required in the request. +* **Parameters**: + * Path parameters (e.g., `resource_id`, `project_name_or_id`) to specify the resource being retrieved. +* **Response Format**: The response includes `metadata` and `spec` fields, detailing the resource’s identity and functional information, along with optional fields like `apiVersion` and `kind`. + +### Update a Resource {#update-a-resource} + +* **Behavior**: The `PATCH` method is used to modify an existing resource. Semaphore only uses `PATCH` for updates, and both request and response bodies follow the same structure. Updates modify specific fields in the resource without replacing the entire object. +* **Parameters**: + * Path parameters for identifying the resource (e.g., `resource_id`, `project_name_or_id`). + * Request body containing `metadata` and `spec` with the fields that need updating. +* **Request Format**: The request body must include both `metadata` and `spec`, ensuring that the resource can be properly identified and the necessary changes applied. +* **Response Format**: The response reflects the updated resource with its new state, using the same structure as the request (i.e., `metadata`, `spec`, optional `apiVersion` and `kind`). + +### Delete a Resource {#delete-a-resource} + +* **Behavior**: The `DELETE` method removes an existing resource. The resource is identified via path parameters, and no body is required in the request. +* **Parameters**: + * Path parameters (e.g., `resource_id`, `project_name_or_id`) to identify the resource to be deleted. +* **Response Format**: Typically, a `204 No Content` status is returned to indicate successful deletion, with no body in the response. + +## Custom Methods {#custom-methods} + +Semaphore's API supports **custom methods** for performing actions that extend beyond basic CRUD operations. These methods allow the API to handle resource-specific operations that require more flexibility, such as starting, stopping, or performing actions on resources. + +### Behavior of Custom Methods {#behavior-of-custom-methods} + +Custom methods are categorized based on the type of action: + +* **Triggering an action**: Use the `POST` method when initiating an action that results in a change or side effect. +* **Idempotent actions**: Use the `GET` method for retrieving information or performing actions that do not change the resource’s state (no side effects). + +### Request and Response Structure {#request-and-response-structure} + +* **Request Structure**: + * The request body for custom methods only needs to include the data required to perform the action. It does not follow the standard `metadata` and `spec` structure used for resource manipulation. + * Path parameters are used to identify the resource (e.g., `project_name_or_id`, `resource_id`), and the body may contain additional parameters relevant to the action. +* **Response Structure**: + * If the action is **synchronous** (the action completes immediately), the response should return the updated state of the resource, reflecting any changes made by the action. + * If the action is **asynchronous** (the action is still processing), the response should return either an empty body or an action ID that the client can use to check the status of the action later. + +### Examples of Custom Methods {#examples-of-custom-methods} + +1. **Pausing a Resource (POST)**: + * **Behavior**: A pipeline or task may be paused, altering its state. + * **Method**: `POST /projects/{project_name_or_id}/pipelines/{pipeline_id}/pause` + * **Request Body**: Includes only the relevant data needed to perform the action, such as user confirmation or reason for pausing. + * **Response**: Returns the updated state of the resource if synchronous, or an action ID if asynchronous. +2. **Checking Deployment Status (GET)**: + * **Behavior**: Retrieving the status of a deployment without modifying it. + * **Method**: `GET /projects/{project_name_or_id}/deployments/{deployment_id}/status` + * **Response**: Returns the current status of the deployment, with no side effects. + +### Encouraging the Use of Standard Methods {#encouraging-the-use-of-standard-methods} + +We encourage using **standard methods** (`POST`, `GET`, `PATCH`, `DELETE`) whenever possible to maintain API consistency. Custom methods should only be used when the action cannot be performed using standard methods, ensuring that the API remains simple and predictable. + +## Error Handling {#error-handling} + +Semaphore’s API provides clear, actionable error messages to help developers quickly understand and resolve issues. All error responses follow a structured format to maintain consistency and provide useful feedback. + +### Error Response Format {#error-response-format} + +Error responses use the following structure: + +* **message**: A human-readable description of the error that explains what went wrong and provides some context. +* **documentation\_url**: A link to relevant documentation for further information or troubleshooting. +* **errors**: An array of specific error details, each containing: + * **resource**: The type of resource where the error occurred (e.g., `Project`). + * **field**: The specific field that caused the issue (e.g., `name`). + * **code**: A short code that indicates the nature of the problem (e.g., `missing_field`). + +**Example Error Response** + +`{` + `"message": "Validation failed: The 'name' field is required to create a project.",` + `"documentation_url": "https://example.com/docs/errors",` + `"errors": [` + `{` + `"resource": "Project",` + `"field": "name",` + `"code": "missing_field"` + `}` + `]` +`}` + +### Writing Effective Error Messages {#writing-effective-error-messages} + +* **Audience**: Error messages should be written for a broad technical audience. Users might not be familiar with your API’s implementation, so avoid assuming deep knowledge of your service. +* **Clarity**: Keep error messages brief but informative. Provide clear guidance on what went wrong and how to fix it. Where more information is necessary, link to documentation. +* **Resolution-Oriented**: Whenever possible, offer actionable advice to resolve the error. If a field is missing or invalid, specify which field and why it matters. + +### Error Codes {#error-codes} + +The following error codes are used to indicate specific issues in requests: + +* **missing**: A resource does not exist. +* **missing\_field**: A required field on a resource has not been set. +* **invalid**: The formatting of a field is invalid. +* **already\_exists**: Another resource has the same value as this field. +* **unprocessable**: The inputs provided were invalid. + +### HTTP Status Codes {#http-status-codes} + +Semaphore’s API uses standard HTTP status codes to communicate the result of requests. These codes help users understand whether a request was successful or why it failed: + +* **200 OK**: The request was successful. +* **201 Created**: The resource was successfully created. +* **400 Bad Request**: The request was malformed or contained invalid data. +* **401 Unauthorized**: Authentication is required or has failed. +* **403 Forbidden**: The user is authenticated but does not have permission to perform the action. +* **404 Not Found**: The requested resource could not be found. +* **422 Unprocessable Entity**: The request was well-formed but contained invalid data. +* **500 Internal Server Error**: An unexpected error occurred on the server. + +### Best Practices for Error Handling {#best-practices-for-error-handling} + +Error messages should be clear, concise, and provide users with sufficient information to understand and correct the issue. Always include links to relevant documentation for further context, and avoid exposing sensitive internal details. + +## Standard Fields {#standard-fields} + +To ensure consistency and clarity, the Semaphore API uses standard naming conventions for certain fields across all resources. One key example is how **timestamps** are handled. + +### Naming Convention: snake\_case {#naming-convention:-snake_case} + +All fields in the API will follow the **snake\_case** convention, where words are separated by underscores. This applies to both request and response fields to ensure uniformity across the entire API. + +### Timestamps {#timestamps} + +All timestamps returned by the API will adhere to the following standards: + +* **UTC Time**: Timestamps will always be in **UTC** to avoid confusion across different time zones. +* **ISO 8601 Format**: The API will use ISO 8601 format for all timestamps, ensuring a consistent and widely recognized format. Example: `"2022-07-08T20:18:44Z"`. +* **Suffix**: All fields related to timestamps will end with the `_at` suffix, indicating the action associated with the time. For example: + * `created_at`: The time when the resource was created. + * `queued_at`: The time when the resource was queued for processing. +* **Specified Timestamps**: For API calls that accept timestamps as input, we use the exact timestamp provided by the client. These timestamps may also include an offset, such as `"2022-07-08T16:18:44+04:00"`. + +This standard ensures consistency across all resources and simplifies timestamp management for both clients and the API itself. \ No newline at end of file diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/dashboards.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/dashboards.md new file mode 100644 index 00000000..0c9a9adc --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/dashboards.md @@ -0,0 +1,46 @@ +Dashboards + +Standard Methods: + +https://semaphore.semaphoreci.com/api/v2/dashboards +https://semaphore.semaphoreci.com/api/v2/dashboards/{name_or_id} + +- list +- describe +- create +- update +- delete + +Resource + +```json +{ + "apiVersion": "v2", + "kind": "Dashboard", + "metadata": { + "id": "645d2d5b-603c-49b2-9022-594297b60aa1", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "name": "my-dashboard", + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + "spec": { + "title": "My Dashboard", + "widgets": [ + { + "name": "My Pipelines", + "type": "PIPELINES", # PIPELINES, WORKFLOWS + "filters": { + "reference": "refs/heads/master", # suport only branches for now? + "project": {"id": "a44db728-f627-4560-a548-9f61d61e8780", "name": "Project1"}, # required for PIPELINES + "pipeline_file": ".semaphore/semaphore.yml" + } + } + ] + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/deployments.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/deployments.md new file mode 100644 index 00000000..4fd44fc5 --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/deployments.md @@ -0,0 +1,56 @@ +Deployments + +Standard Methods + +https://semaphore.semaphoreci.com/api/v2/projects/{name_or_id}/deployment_targets/{name_or_id}/deployments +https://semaphore.semaphoreci.com/api/v2/deployment_targets/{id}/deployments + +https://semaphore.semaphoreci.com/api/v2/deployments + +- list filter deployment_target_id =, reference = "refs/heads/master" "refs/heads/*", triggered_by.id = "" + +Resource + +```json +{ + "apiVersion": "v2", + "kind": "Deployment", + "metadata": { + "id": "a0424427-ff28-4f77-a893-0fdf5f50c380", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "organization1"}, + "project": {"id": "726dbefd-b30c-4c80-bc9a-c0613968a2ca", "name": "project1"}, + "environment": {"id": "9c777b65-213f-4fe1-bc6b-727d2cdde97d" }, + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + + "spec": { + "git": { + "reference": "refs/heads/master", + } + "env_vars": [ + { + "name": "NAME", + "value": "VALUE" + } + ], + + "origin_pipeline": {"id": "97ba55d1-90d7-40b9-a2dd-bdbd02389b58" }, + "promotion_name": "Sxmoon" + }, + + "status": { + "state": "PENDING", + "state_message": "", + "deployment_pipeline": {"id": "adc5a26f-4077-47bc-8d4f-cb4467f2c103" }, + "timeline": { + "triggered_at": "2024-07-12T11:02:50.058Z", + "triggered_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "name": "User1" } + } + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/environments.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/environments.md new file mode 100644 index 00000000..7a11855a --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/environments.md @@ -0,0 +1,91 @@ +Environments + +Standard Methods: + +https://semaphore.semaphoreci.com/api/v2/projects/{name_or_id}/environments +https://semaphore.semaphoreci.com/api/v2/projects/{name_or_id}/environments/{name_or_id} + +https://semaphore.semaphoreci.com/api/v2/environments +https://semaphore.semaphoreci.com/api/v2/environments/{id} + +- list +- describe +- update +- delete +- create + +Custom Methods + +- cordon POST on /projects/{name_or_id}/environments/{name_or_id}/cordon +- uncordon POST on /projects/{name_or_id}/environments/{name_or_id}/uncordon + +Resource: + +```json +{ + "apiVersion": "v2", + "kind": "Environment", + "metadata": { + "id": "9c777b65-213f-4fe1-bc6b-727d2cdde97d", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "project": {"id": "726dbefd-b30c-4c80-bc9a-c0613968a2ca", "name": "Project1"}, + "name": "production", + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + "spec": { + "description": "string", + "url": "string", + + "subject_rules": [ + { + "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", + "type": "USER", # USER, ROLE, ANY, AUTO + }, + { + "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", + "type": "ROLE", + "name": "Contributor" + } + ], + "object_rules": [ + { + "type": "BRANCH", # BRANCH, PR, TAG + "match_mode": "ALL", # REGEX + "pattern": "" + } + ], + + "bookmark_parameters": ["string"], + "secrets": {} + "env_vars": [ + { + "name": "MY_SECRET", + "value": "secret" + } + ], + "files": [ + { + "content": "string", + "path": "/path/to/file" + } + ], + } + + } + + "status": { + "state": "SYNCING", + + "last_deployment": { + "id": "a0424427-ff28-4f77-a893-0fdf5f50c380", + "deployment_pipeline": {"id": "adc5a26f-4077-47bc-8d4f-cb4467f2c103" } + }, + + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/init_jobs.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/init_jobs.md new file mode 100644 index 00000000..9e19266b --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/init_jobs.md @@ -0,0 +1,25 @@ +Init Jobs + +Standard Methods + +https://semaphore.semaphoreci.com/api/v2/init_job + +- describe +- update + +Resource + +```json +{ + "apiVersion": "v2", + "kind": "InitJob", + "metadata": { + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + }, + "spec": { + "agent": { + "os_image": "ubuntu2004", + "machine_type": "e1-standard-2" + } + } +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/job_access.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/job_access.md new file mode 100644 index 00000000..20221347 --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/job_access.md @@ -0,0 +1,37 @@ +Job Access + +Standard Methods + +https://semaphore.semaphoreci.com/api/v2/job_access + +- describe +- update + + Resource + + ```json +{ + "apiVersion": "v2", + "kind": "JobAccess", + "metadata": { + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"} + }, + "spec": { + "debug": { + "project": false, + "default_branch": false, + "non_default_branch": false, + "pull_requests": false, + "forked_pull_requests": false, + "tags": false + }, + "attach": { + "default_branch": false, + "non_default_branch": false, + "pull_requests": false, + "forked_pull_requests": false, + "tags": false + } + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/jobs.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/jobs.md new file mode 100644 index 00000000..fc5db53e --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/jobs.md @@ -0,0 +1,114 @@ +Jobs + +Standard Methods + +https://semaphore.semaphoreci.com/api/v2/jobs + +- list filter state = , project_id = +- describe +- create + +Custom Methods + +- terminate POST on /jobs/{id}/terminate +- log_events GET on /jobs/{id}/log_events +- ssh_key GET on /jobs/{id}/ssh_key + +- debug_job POST on /jobs/debug_job + job_id agent and execution_time_limit +- debug_project POST on /jobs/debug_project + project_id agent and execution_time_limit + +Resource + +```json +HTTP status: 200 + +{ + "metadata": { + "id": "bc8826bd-dbb2-4d28-8c90-7f370ce478fe", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "project": {"id": "726dbefd-b30c-4c80-bc9a-c0613968a2ca", "name": "Project1"}, + "workflow": {"id": "52f2847c-1817-48f2-95a8-b7e6b98c65e0" }, + "pipeline": {"id": "f850cdcf-d3de-4d50-91bf-b6190fa5f9c0" }, + "name": "Job #1", + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + "spec": { + "agent": { + "ports": [{ + "name": "ssh", + "number": 30000 + }], + "containers": [], + "machine": { + "type": "e1-standard-2", + "os_image": "ubuntu2004" + } + }, + "env_vars": [ + { + "name": "SEMAPHORE_WORKFLOW_ID", + "value": "59b32e16-3c4a-4940-899e-348c28396884" + }, + { + "name": "SEMAPHORE_WORKFLOW_NUMBER", + "value": "2" + }, + { + "name": "SEMAPHORE_PIPELINE_ARTEFACT_ID", + "value": "abb4fb87-309d-490a-bf0d-84972641b130" + }, + { + "name": "SEMAPHORE_PIPELINE_ID", + "value": "abb4fb87-309d-490a-bf0d-84972641b130" + }, + { + "name": "SEMAPHORE_PIPELINE_0_ARTEFACT_ID", + "value": "abb4fb87-309d-490a-bf0d-84972641b130" + } + ], + "secrets": [ + {"name": "DOCKERHUB_USERNAME"}, + {"name": "DOCKERHUB_PASSWORD"} + ], + "commands": [ + "sleep 3600" + ], + "epilogue_always_commands": [ + "echo 'Job finished'" + ], + "epilogue_on_fail_commands": [ + "echo 'Job failed'" + ], + "epilogue_on_pass_commands": [ + "echo 'Job passed'" + ], + "execution_time_limit": 3600, + + }, + + "status": { + + "agent": { + "name": "", + "ip": "88.212.212.212" + } + + "result": "STOPPED", + "state": "FINISHED" + + "timeline": { + "created_at": "2024-07-12T11:02:50.116Z", + "enqueued_at": "2024-07-12T11:02:50.045Z", + "scheduled_at": "2024-07-12T11:02:50.045Z", + "started_at": "2024-07-12T11:02:50.045Z", + "finished_at": "2024-07-12T11:02:50.045Z" + } + + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/notifiations.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/notifiations.md new file mode 100644 index 00000000..466ef272 --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/notifiations.md @@ -0,0 +1,59 @@ +Notification + +Standard Mehtods: + +https://semaphore.semaphoreci.com/api/v2/notifications + +- list +- describe +- update +- delete +- create + +Custom Methods: + +Resource: + +```json +{ + "apiVersion": "v2", + "kind": "Notification", + "metadata": { + "id": "ee251df7-6946-4544-840d-b8ac7c02c3e3", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "organization1"}, + "name": "notification1", + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + "spec": { + "rules": [ + { + "name": "Rule1", + "filter": { + "projects": ["string"], + "branches": ["string"], + "pipelines": ["string"], + "results": ["PASSED"] + }, + "notify": { + "slack": { + "endpoint": "string", + "channels": ["string"] + }, + "webhook": { + "method": "POST", + "url": "string", + "retries": 3, + "timeout": 500, + "secret": "string" + } + } + } + ] + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/pipelines.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/pipelines.md new file mode 100644 index 00000000..113fccda --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/pipelines.md @@ -0,0 +1,98 @@ +Pipelines + +Standard Methods: + +https://semaphore.semaphoreci.com/api/v2/pipelines + +- list + filter project_id = , workflow_id = , created_at > <, done_at > <, pipeline_file = , reference = + pr_head_branch = , pr_target_branch = +- describe + +Custom Methods: + +- partial_rebuild POST on /pipelines/{id}/partial_rebuild + request_token in body +- topology GET on /pipelines/{id}/topology +- terminate POST on /pipelines/{id}/terminate (stop or cancel)? + +- validate POST on /pipelines/validate + yaml in body and content-type set to application/yaml + what we need to fully validate a pipeline file? + +Resource: + +```json +{ + "apiVersion": "v2", + "kind": "Pipeline", + "metadata": { + "id": "f850cdcf-d3de-4d50-91bf-b6190fa5f9c0", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "organization1"}, + "project": {"id": "726dbefd-b30c-4c80-bc9a-c0613968a2ca", "name": "project1"}, + "workflow":{ "id": "52f2847c-1817-48f2-95a8-b7e6b98c65e0" }, + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + + "spec": { + "name": "Pipeline", + + "git or payload or something": { + "reference": "refs/heads/master", + "commit_sha": "ac3f9796df42db976814e3fee670e11e3fd4b98a", + "pr_head_branch": "refs/heads/feature-branch", + "pr_target_branch": "refs/heads/master", + "pipeline_file": ".semaphore/semaphore.yml" + } + + "queue": { + "type": 0, # this looks like enum + "scope": "project", + "name": "master-.semaphore/repohub-prod.yml" + }, + + # Optional + "pipeline_yml": "YAML or JSON of pipeline" + }, + + "status": { + + "timeline": { + "pending_at": "2024-07-12T11:02:50.045Z", + "queuing_at": "2024-07-12T11:02:50.045Z", + "running_at": "2024-07-12T11:02:50.045Z", + "stopping_at": "2024-07-12T11:02:50.045Z", + "done_at": "2024-07-12T11:02:50.045Z", + "terminated_by": { + "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b" + } + } + + "state": "INITIALIZING", + "result": "PASSED", + "result_reason": "TEST", + "error_description": "", + "terminate_request": "", + + "blocks": [{ + "id": "484e263a-424a-4820-bff0-bba436c54042", + "error_description": "string", + "name": "string", + "result": "PASSED", + "result_reason": "TEST", + "state": "WAITING", + "jobs": [ + { + "index": 0, + "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", + "name": "string", + "result": "string", + "status": "string" + } + ] + }], + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/pre_flight_checks.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/pre_flight_checks.md new file mode 100644 index 00000000..af3802e2 --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/pre_flight_checks.md @@ -0,0 +1,23 @@ +Pre Fligh Checks + +Standard Methods: + +https://semaphore.semaphoreci.com/api/v2/pre_flight_checks + +- describe +- update + +Resource + +```json +{ + "apiVersion": "v2", + "kind": "PreFlightChecks", + "metadata": { + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"} + }, + "spec": { + "commands": [], + "secrets": [] + } +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/project_artifacts.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/project_artifacts.md new file mode 100644 index 00000000..bb166671 --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/project_artifacts.md @@ -0,0 +1,38 @@ +Project Artifacts + +Standard Mehtods: + +https://semaphore.semaphoreci.com/api/v2/projects/{name_or_id}/artifacts + +- describe +- update + +Resouce: + +```json +{ + "apiVersion": "v2", + "kind": "ProjectArtifacts", + "metadata": { + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "project": {"id": "726dbefd-b30c-4c80-bc9a-c0613968a2ca", "name": "Project1", "url": "https://semaphore.semaphoreci.com/api/v2/projects/project1"}, + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + "spec": { + "project_retention_policies": [ + { "selector": "/example/path/**/*", "age": "7d" } + ], + "workflow_retention_policies": [ + { "selector": "/example/path/**/*", "age": "1d" } + ], + "job_retention_policies": [ + { "selector": "/example/path/**/*", "age": "1w" } + ] + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/project_init_jobs.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/project_init_jobs.md new file mode 100644 index 00000000..588d86e7 --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/project_init_jobs.md @@ -0,0 +1,27 @@ +Project Init Jobs + +Standard Methods + +https://semaphore.semaphoreci.com/api/v2/projects/{name_or_id}/init_job + +- describe +- update + +Resource + +```json +{ + "apiVersion": "v2", + "kind": "ProjectInitJob", + "metadata": { + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "project": {"id": "726dbefd-b30c-4c80-bc9a-c0613968a2ca", "name": "Project1", "url": "https://semaphore.semaphoreci.com/api/v2/projects/project1"}, + }, + "spec": { + "overwritten": true, + "agent": { + "os_image": "ubuntu2004", + "machine_type": "e1-standard-2" + } + } +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/project_job_access.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/project_job_access.md new file mode 100644 index 00000000..56c2adb2 --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/project_job_access.md @@ -0,0 +1,39 @@ +Project Job Access + +Standard Methods + +https://semaphore.semaphoreci.com/api/v2/projects/{name_or_id}/job_access + +- describe +- update + + Resource + + ```json +{ + "apiVersion": "v2", + "kind": "ProjectJobAccess", + "metadata": { + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "project": {"id": "726dbefd-b30c-4c80-bc9a-c0613968a2ca", "name": "Project1", "url": "https://semaphore.semaphoreci.com/api/v2/projects/project1"}, + }, + "spec": { + "overwritten": true, + "debug": { + "project": false, + "default_branch": false, + "non_default_branch": false, + "pull_requests": false, + "forked_pull_requests": false, + "tags": false + }, + "attach": { + "default_branch": false, + "non_default_branch": false, + "pull_requests": false, + "forked_pull_requests": false, + "tags": false + } + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/project_pre_flight_checks.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/project_pre_flight_checks.md new file mode 100644 index 00000000..876b712e --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/project_pre_flight_checks.md @@ -0,0 +1,25 @@ +Project Pre Fligh Checks + +Standard Methods: + +https://semaphore.semaphoreci.com/api/v2/projects/{name_or_id}/pre_flight_checks + +- describe +- update + +Resource + +```json +{ + "apiVersion": "v2", + "kind": "ProjectPreFlightChecks", + "metadata": { + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "project": {"id": "726dbefd-b30c-4c80-bc9a-c0613968a2ca", "name": "Project1", "url": "https://semaphore.semaphoreci.com/api/v2/projects/project1"}, + }, + "spec": { + "overwritten": true, + "commands": [], + "secrets": [] + } +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/project_secrets.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/project_secrets.md new file mode 100644 index 00000000..a6fa88e2 --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/project_secrets.md @@ -0,0 +1,57 @@ +ProjectSecrets + +Standard Methods + +https://semaphore.semaphoreci.com/api/v2/projects/{name_or_id}/secrets + +- list +- describe +- update +- delete +- create + +Resource + +```json +{ + "apiVersion": "v2", + "kind": "ProjectSecret", + "metadata": { + "id": "abbb5683-9310-49bb-9fd2-ac69486a4c6d", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "project": {"id": "726dbefd-b30c-4c80-bc9a-c0613968a2ca", "name": "Project1"}, + "name": "my-secret", + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + "spec": { + "description": "string", + "data": { + "env_vars": [ + { + "name": "ENV_VAR_NAME", + "value": "string" + } + ], + "files": [ + { + "content": "string", + "path": "/path/to/file" + } + ] + } + }, + + # not sure about this + "status": { + "last_used_at": "2024-07-12T11:02:50.119Z", + "last_used_by": { + "id": "7ce0901c-84b0-4bb4-ab96-f0a4b15ae52d", "type": "JOB" + } + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/project_triggers.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/project_triggers.md new file mode 100644 index 00000000..0b9f0249 --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/project_triggers.md @@ -0,0 +1,174 @@ +ProjectTriggers + +Standard Methods: + +https://semaphore.semaphoreci.com/api/v2/projects/{name_or_id}/triggers + +- list required type = + +- describe +- update +- delete +- create + +Resoruce + +```json +{ + "apiVersion": "v2", + "kind": "ProjectTrigger", + "metadata": { + "id": "49f06438-9c54-4d1d-9e4a-76faa292b150", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "project": {"id": "726dbefd-b30c-4c80-bc9a-c0613968a2ca", "name": "project1", "type": "PROJECT", "url": "https://semaphore.semaphoreci.com/api/v2/projects/project1"}, + "name": "project-trigger-1", + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + "spec": { + # API type would be update only (you can pause it, but not delete or create) + "type": "API", # TASK, HOOK #, PULL, S3, PROJECT + "paused": false, + + "display_name": "Project Trigger", + "description": "Project trigger description", + + # CUSTOM GITHUB HOOK - we would like to sign the payload + +# repository_slug: "semaphoreci/toolbox" +settings + + + # HOOK + # URL + + "when": { + "hook": "HOOK_ID", + "branch": "master", + "result": "passed" + } + + "what": { + "pipeline_file": ".semaphore/pipeline.yml", + "reference": "refs/heads/master", + "parameters": [ + {"key": "ENV_VAR_X", "value": "yq. $lookup"}, + {"key": "ENV_VAR_Y", "value": "blablay"} + ], + }, + + + "OIDC" for custom hooks? + + # TASK + "suspended": false, # STATE, can it be for all types? + + "when": { + "scheduled": true, # do we need it? schedule can be null + "cron_schedule": "0 0 * * *", + } + + "what": { + "pipeline_file": ".semaphore/pipeline.yml", + "reference": "refs/heads/master", + "parameters": [ + {"key": "ENV_VAR_X", "value": "blabla"}, + {"key": "ENV_VAR_Y", "value": "blablay"} + ] + } + + # POOL + + "when": { + "cron_schedule": "0 0 * * *", # or interval? + "branch": "master" + } + + "what": { + "pipeline_file": ".semaphore/pipeline.yml" + "reference": "refs/heads/master", # optional + "parameters": [ + {"key": "ENV_VAR_X", "value": "blabla"}, + {"key": "ENV_VAR_Y", "value": "blablay"} + ] + } + + # PROJECT + + "when": { + "project": {"id": "", "name:" "alles"}, + "condition": "pipeline_file = './s/x.yml' AND branch = 'master' AND result ='passed'" + } + + "what": { + "pipeline_definition": ".semaphore/pipeline.yml" + "reference": "refs/heads/master", + + "parameters": [ + {"key": "ENV_VAR_X", "value": "blabla"}, + {"key": "ENV_VAR_Y", "value": "blablay"} + ] + } + + # S3 + + "when": { + "secret": "SECRET_WITH_S3_ACCESS", + "cron_schedule": "0 0 * * *", + "path": "/data/*.zip", + } + + } +} +``` + +# Notes + +- replace "pipeline_file" with "pipeline_definition" +- define triggers options for github triggers: + + "triggers": [ + "BRANCHES", "TAGS", "FORKED_PULL_REQUESTS", "PULL_REQUESTS" + ], + "branches": { + "filter": true, # TODO we will add this field later + "only_new_branches": true, # TODO we will add this field later + "allowed": ["string"], + "pull_requests": true, # TODO we will add this field later + "draft_pull_requests": false, # TODO we will add this field later + } + "tags": { + "skip_ci": true, # TODO we will add this field later + "filter": true, # TODO we will add this field later + "allowed": ["string"], + }, + "pull_requests": { # TODO we will add this field later + "filter": true, + "draft": false, + "target": ["string"], + "base": ["string"], + }, + "forked_pull_requests": { + "filter": true, # TODO we will add this field later + "target": ["string"], # TODO we will add this field later + "base": ["string"], # TODO we will add this field later + "allowed_contributors": [], + "allowed_secrets": ["my-secret"] + }, + "pipeline_file": ".semaphore/semaphore.yml", + "status": { + "enabled": true, # TODO we will add this field later + "pipeline_files": [ + { + "level": "PIPELINE", + "path": ".semaphore/semaphore.yml" + } + ] + } + }, + + } diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/projects.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/projects.md new file mode 100644 index 00000000..454f0b0b --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/projects.md @@ -0,0 +1,77 @@ +Project + +Standard Mehtods: + +https://semaphore.semaphoreci.com/api/v2/projects/{name_or_id} + +- list +- describe +- update +- delete +- create + +Custom Methods: # TODO we will add this field later +- regenerate_deploy_key POST on /projects/{name_or_id}/regenerate_deploy_key + request_token in body +- regenerate_webhook POST on /projects/{name_or_id}/regenerate_webhook + request_token in body + +Resource: + +```json +{ + "apiVersion": "v2", + "kind": "Project", + "metadata": { + "id": "ed12ab59-9224-45c8-82d6-79bf158aaae6", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "organization1"}, + "name": "my-project", + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + "spec": { + "display_name": "My Project", + "description": "My project description", + "visibility": "PUBLIC", + + # we can have here different types of object with different fields? GitHubRepository, GitRepository, BitbucketRepository, GitlabRepository, etc + "repository": { + "integration_type": "GITHUB_APP", # GITHUB, GITLAB, BITBUCKET, GIT + "url": "git@github.com:semaphoreci/toolbox.git", + "default_branch": "master", + + # One can add deploy key to the repository in settings, not managed by Semaphore + # it is also possible to add it as a project/org secret and provide the name? + + "deploy_key": { + "private_key": NAME_OF_THE_SECRET_THAT_SEMAPHORE_MANAGE?, + "fingerprint": "SHA256:CNz2IRGey4Pum8dE1q1MyducXOQCoNM/Vdcc9NecjMg", + "public_key": "yyy" + } + + ## Managed is github/bitbucket/gitlab or any other known provider is used + + # managed deploy key + "deploy_key": { # TODO we will add this field later + "name": "semaphore-semaphoreci-toolbox", + "fingerprint": "SHA256:CNz2IRGey4Pum8dE1q1MyducXOQCoNM/Vdcc9NecjMg", + "message": "" + } + # managed webhook + "webhook": { # TODO we will add this field later + "enabled": true, + "url": "https://semaphore.semaphoreci.com/webhooks/ed12ab59-9224-45c8-82d6-79bf158aaae6", + "message": "" + } + + # state of managed connection + "connected": true, + "connection_owner": { + "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", type: "USER" + }, + } + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/promotion_triggers.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/promotion_triggers.md new file mode 100644 index 00000000..30c4ae12 --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/promotion_triggers.md @@ -0,0 +1,47 @@ +PromotionTriggers + +Standard Methods + +https://semaphore.semaphoreci.com/api/v2/promotion_triggers + +- list +- create + +Resource: + +```json +{ + "apiVersion": "v2", + "kind": "PromotionTrigger", + "metadata": { + "id": "d605c1ed-5664-4ce3-8419-14d3d7337c35", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "project": {"id": "726dbefd-b30c-4c80-bc9a-c0613968a2ca", "name": "Project1"}, + "pipeline": { "id": "f850cdcf-d3de-4d50-91bf-b6190fa5f9c0" }, + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + + "spec": { + "promotion_name": "Sxmoon", + "override": false, + "parameters": [ + { + "name": "Parameter name", + "value": "Parameter value" + } + ] + }, + "status": { + "status": "PASSED", + "triggered_pipeline": { "id": "d605c1ed-5664-4ce3-8419-14d3d7337c35" }, + "timeline": { + "triggered_at": "2024-07-12T11:02:50.116Z" + } + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/secrets.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/secrets.md new file mode 100644 index 00000000..601596ad --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/secrets.md @@ -0,0 +1,62 @@ +Secret + +Standard Methods: + +https://semaphore.semaphoreci.com/api/v2/secrets/{name_or_id} + +- list +- describe +- update +- delete +- create + +Resource + +```json +{ + "apiVersion": "v2", + "kind": "Secret", + "metadata": { + "id": "b6654945-cf11-43a1-8e3a-f5a22c7d19fa", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "name": "my-secret", + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + "spec": { + "description": "string", + "access_config": { + "attach_access": "YES", + "debug_access": "YES", + "project_access": "WHITELIST", + "projects": [ + {"id": "0e7fa31e-f974-4f57-ab99-72ac8b70df41", "name": "project-1"}, + {"id": "0e7fa31e-f974-4f57-ab99-72ac8b70df42", "name": "project-2"} + ] + }, + "data": { + "env_vars": [ + { + "name": "MY_SECRET", + "value": "secret" + } + ], + "files": [ + { + "content": "string", + "path": "/path/to/file" + } + ] + }, + }, + + "status": { + "last_used_at": "2024-07-12T11:02:50.108Z", + "last_used_by": { "id": "7ce0901c-84b0-4bb4-ab96-f0a4b15ae52d", "type": "JOB" } + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/self_hosted_agent_types.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/self_hosted_agent_types.md new file mode 100644 index 00000000..80ccc9a5 --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/self_hosted_agent_types.md @@ -0,0 +1,45 @@ +SelfHostedAgentType + +Standard Methods: + +https://semaphore.semaphoreci.com/api/v2/self_hosted_agent_types/{name_or_id} + +- list +- describe +- update +- delete +- create + +Custom Methods + +- reset token POST on /self_hosted_agent_types/{name_or_id}/reset_token + +Resource: + +```json +{ + "apiVersion": "v2", + "kind": "SelfHostedAgentType", + "metadata": { + "id": "ed12ab59-9224-45c8-82d6-79bf158aaae6", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "name": "my-agent-type" + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + "spec": { + "agent_name_settings": { + "assignment_origin": "AWS_STS", + "aws": { + "account_id": "123456789012", + "role_name_patterns": "my-role-name" + }, + "release_after": 0 + }, + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/self_hosted_agents.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/self_hosted_agents.md new file mode 100644 index 00000000..e15e4310 --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/self_hosted_agents.md @@ -0,0 +1,48 @@ +SelfHostedAgent + +Standard Methods: + +https://semaphore.semaphoreci.com/api/v2/self_hosted_agents/{name_or_id} + +- list + filter status = "WAITING_FOR_JOB" AND type = "s1-my-type" +- describe + +Custom Methods + +- disconnect POST on /self_hosted_agents/{name_or_id}/disconnect +- disconnect POST on /self_hosted_agents/disconnect + filter (status = "WAITING_FOR_JOB" AND type = "s1-my-type") + +Resource: + +```json +{ + "apiVersion": "v2", + "kind": "SelfHostedAgent", + "metadata": { + "id": "ed12ab59-9224-45c8-82d6-79bf158aaae6", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "name": "string", + "timeline": { + "created_at": "2024-07-12T11:02:50.092Z", + } + }, + + "spec": { + "arch": "string", + "hostname": "string", + "os": "Ubuntu 20.04.6 LTS", + "type": "s1-my-type", + "version": "v2.2.6" + } + + # Not sure what goes to spec and what goes to status. Lucas should help + "status": { + "timeline": { + "connected_at": "2024-07-12T11:02:50.092Z" + } + "status": "WAITING_FOR_JOB", + "ip_address": "string", + "pid": 0, + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/task_triggers.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/task_triggers.md new file mode 100644 index 00000000..c986eaf0 --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/task_triggers.md @@ -0,0 +1,49 @@ +TaskTriggers + +# Will be replaces with project triggers calls + +Standard Methods + +https://semaphore.semaphoreci.com/api/v2/projects/{name_or_id}/tasks/{name_or_id}/triggers +https://semaphore.semaphoreci.com/api/v2/tasks/{id}/triggers + +- create + +Resource + +```json +{ + "apiVersion": "v2", + "kind": "TaskTrigger", + "metadata": { + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "project": {"id": "726dbefd-b30c-4c80-bc9a-c0613968a2ca", "name": "project1", "type": "PROJECT", "url": "https://semaphore.semaphoreci.com/api/v2/projects/project1"}, + "task": {"id": "a3fa7cf5-0fee-4a76-ab75-f1a57f1316c1" }, + "workflow": {"id": "33c93e7e-5239-4b80-8d86-1cdcc0a15087" } + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + "spec": { + "reference": "refs/heads/master", # or keep it branch? + "parameters": [ + { + "name": "Parameter name", + "value": "Parameter value" + } + ], + "pipeline_file": ".semaphore/semaphore.yml", + + }, + + "status": { + "status": "PASSED" + "timeline": { + "scheduled_at": "2024-07-12T11:02:50.106Z", + } + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/tasks.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/tasks.md new file mode 100644 index 00000000..035ac18f --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/tasks.md @@ -0,0 +1,66 @@ +Tasks + +# Will be replaces with project triggers + +Standard Methods + +https://semaphore.semaphoreci.com/api/v2/projects/{name_or_id}/tasks/{name_or_id} +https://semaphore.semaphoreci.com/api/v2/tasks/{id} + +- list +- describe +- update +- delete +- create + +Resource + +```json +{ + "apiVersion": "v2", + "kind": "Task", + "metadata": { + "id": "49f06438-9c54-4d1d-9e4a-76faa292b150", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "Organization1"}, + "project": {"id": "726dbefd-b30c-4c80-bc9a-c0613968a2ca", "name": "project1", "type": "PROJECT", "url": "https://semaphore.semaphoreci.com/api/v2/projects/project1"}, + "name": "periodic-task", + "timeline": { + "created_at": "2024-07-12T11:02:50.108Z", + "created_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + "updated_at": "2024-07-12T11:02:50.108Z", + "updated_by": { "id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER" }, + } + }, + "spec": { + "display_name": "Periodic task", + "description": "Periodic task description", + + "cron_schedule": "0 0 * * *", + "scheduled": false, + "suspended": false, + "paused": false, + + "branch": "master", + "pipeline_file": ".semaphore/pipeline.yml", + + "parameters": [ + { + "name": "PARAM_NAME", + "description": "Parameter description", + "required": true + "default_value": "Default value", + "options": [ + "string" + ], + } + ] + } + + "status": { + "timeline": { + "paused_at": "2024-07-12T11:02:50.100Z", + "paused_by": {"id": "18b40eaf-ce10-49c3-93f2-e4fe72d5160b", "type": "USER"} + } + } +} +``` diff --git a/rfcs/0001-api-v2/resources/endpoint-descriptions/workflows.md b/rfcs/0001-api-v2/resources/endpoint-descriptions/workflows.md new file mode 100644 index 00000000..4cd52145 --- /dev/null +++ b/rfcs/0001-api-v2/resources/endpoint-descriptions/workflows.md @@ -0,0 +1,90 @@ +Workflows + +Standard Methods: + +https://semaphore.semaphoreci.com/api/v2/workflows/{id} + +- list +- describe +- create project_id reference + (commit_sha pipeline_file) # or maybe remove it? and add it as a trigger? + +Custom Methods + +- rerun POST on /workflows/{id}/rerun + request_token in body +- terminate POST on /workflows/{id}/terminate + + Resource: + + ```json +{ + "apiVersion": "v2", + "kind": "Workflow", + "metadata": { + "id": "f850cdcf-d3de-4d50-91bf-b6190fa5f9c0", + "organization": {"id": "02b7c528-b0f1-4266-8e5e-10b7b984e76a", "name": "organization1"}, + "project": {"id": "726dbefd-b30c-4c80-bc9a-c0613968a2ca", "name": "project1", "url": "https://semaphore.semaphoreci.com/api/v2/projects/project1"}, + "timeline": { + "created_at": "2024-07-12T11:02:50.116Z", + "created_by": { "id": "5df2cadb-310f-4c14-8784-e27289133a78", "type": "USER" } + } + }, + "spec": { + "initial_pipeline": { "id": "f850cdcf-d3de-4d50-91bf-b6190fa5f9c0" }, + + + # should rerun be a trigger as well? + "rerun_of": {"id": "f850cdcf-d3de-4d50-91bf-b6190fa5f9c0" } + + # better name for this? + "trigger_call": { + "type": "HOOK", + #"id": "f850cdcf-d3de-4d50-91bf-b6190fa5f9c0", + + "pipeline_file": ".semaphore/semaphore.yml", + # user connected to SCM account + "triggered_by": {"id": "f850cdcf-d3de-4d50-91bf-b6190fa5f9c0" } + } + + "trigger_call": { + "type": "TASK", + #"id": "f850cdcf-d3de-4d50-91bf-b6190fa5f9c0", + + + "task": {"id": "f850cdcf-d3de-4d50-91bf-b6190fa5f9c0" } + "triggered_by": {"id": "f850cdcf-d3de-4d50-91bf-b6190fa5f9c0" } + + + + "pipeline_file": ".semaphore/semaphore.yml", + + "parameters": [ + {"key": "ENV_VAR_X", "value": "blabla"}, + {"key": "ENV_VAR_Y", "value": "blablay"} + ], + "reference": "refs/heads/master" + # User/Service Account that manually triggered the task (UI or API) + } + + "trigger_call": { + "type": "API", + "id": "f850cdcf-d3de-4d50-91bf-b6190fa5f9c0", + "pipeline_file": ".semaphore/semaphore.yml" + + "reference": "refs/heads/master", + "commit_sha": "60a2df0a7e8f27d044350cbfe12c040f36f042f2", + + # User/Service Account that manually triggered the API + "triggered_by": {"id": "f850cdcf-d3de-4d50-91bf-b6190fa5f9c0" } + } + + "repository": { + "reference": "refs/heads/master", + "commit_sha": "60a2df0a7e8f27d044350cbfe12c040f36f042f2", + + # github specific stuff + "pr_head_branch": "feature-branch", + "pr_target_branch": "master" + } + } +} + ``` diff --git a/rfcs/0001-api-v2/rfc.md b/rfcs/0001-api-v2/rfc.md new file mode 100644 index 00000000..71289383 --- /dev/null +++ b/rfcs/0001-api-v2/rfc.md @@ -0,0 +1,64 @@ +# RFC 0000 - API-v2: Standardized and Fully Documented API + +## What + +This RFC proposes the development of API-v2 for Semaphore, which aims to standardize our API structure, migrate existing services, and implement comprehensive, sustainable documentation for all endpoints. + +## Why + +Our current API is in an alpha version, and many services lack complete documentation. This situation hinders user adoption, makes integration difficult, and complicates future development. By creating API-v2, we aim to: + +1. Improve the developer experience for our users. +2. Ensure consistency across all our API services. +3. Establish a sustainable process for maintaining up-to-date documentation. +4. Facilitate easier onboarding for new services in the future. + +## Solution details + +### Functionality + +1. Migration of existing services: + - Audit all current API services and endpoints. + - Define a standard format for endpoint structure and response schemas. + - Migrate each existing service to the new standard, ensuring backward compatibility where possible. + +2. Documentation system: + - Implement an OpenAPI specification for all endpoints. + - Set up automated documentation generation from code comments and OpenAPI specs. + - Create a user-friendly documentation portal with interactive examples. + +3. Versioning strategy: + - Implement clear versioning for the API (e.g., `/v2/` prefix for all new endpoints). + - Establish a deprecation policy for old endpoints. + +4. Authentication and security: + - Review and potentially update authentication methods. + - Implement rate limiting and other security measures consistently across all endpoints. + +### Architecture & Design + +The proposed updates are currently split into two groups: + +#### Endpoint updates + +The proposal for the updates of all of the existing endpoints. +They can be found in the [resources/endpoint-descriptions](./resources/endpoint-descriptions/) + +**Please share your feedback as a direct comment on those file.** + +#### API design guide + +This guide defines the core principles and best practices for building and evolving APIs within our CI/CD platform. + +It can be found in the [resources/Semaphore API Design Guide.md](./resources/Semaphore%20API%20Design%20Guide.md) + +**Please share your feedback as a direct comment on the design guide file.** + +## Open questions + +- What are the aditional changes we should make in the existing endpoints? +- Should we make any changes to the API Design Guide? +- How do we maintain parallel support for both v1 (alpha) and v2 APIs during the transition, and set a hard cutoff date? +- How do we handle breaking changes in the future once v2 is established? +- How can we encourage internal teams to keep the documentation up-to-date as they make changes to the API? +- Should we implement a developer feedback system within the new documentation portal?