From 1d63af2e6392f565c63c5b7cc8f425decf679b71 Mon Sep 17 00:00:00 2001 From: pablomendezroyo Date: Fri, 6 Feb 2026 10:48:48 +0100 Subject: [PATCH 1/2] Add GitHub Actions documentation for staker and standard packages, including workflows and CI/CD overview --- docs/dev/github-actions/overview.md | 51 ++++ .../staker-packages/consensus-clients.md | 238 +++++++++++++++ .../staker-packages/execution-clients.md | 277 ++++++++++++++++++ .../staker-packages/overview.md | 101 +++++++ .../staker-packages/web3signer.md | 46 +++ .../standard-packages.md} | 19 +- .../multi-configuration.md | 2 +- .../single-configuration.md | 2 +- sidebars.js | 49 +++- 9 files changed, 766 insertions(+), 19 deletions(-) create mode 100644 docs/dev/github-actions/overview.md create mode 100644 docs/dev/github-actions/staker-packages/consensus-clients.md create mode 100644 docs/dev/github-actions/staker-packages/execution-clients.md create mode 100644 docs/dev/github-actions/staker-packages/overview.md create mode 100644 docs/dev/github-actions/staker-packages/web3signer.md rename docs/dev/{github-actions.md => github-actions/standard-packages.md} (82%) diff --git a/docs/dev/github-actions/overview.md b/docs/dev/github-actions/overview.md new file mode 100644 index 000000000..dab0de1d0 --- /dev/null +++ b/docs/dev/github-actions/overview.md @@ -0,0 +1,51 @@ +--- +title: "GitHub Actions for Dappnode Packages" +sidebar_label: Overview +llm_description: "Overview of GitHub Actions workflows for automating upstream updates, testing, and releases in Dappnode packages." +--- + +# GitHub Actions for Dappnode Packages + +GitHub Actions help automate workflows around your projects on GitHub. For Dappnode packages, we utilize specific GitHub Actions to manage updates, testing, and releases. + +## Types of Dappnode Packages + +Dappnode packages fall into two main categories, each with different CI/CD requirements: + +### Standard Packages + +Standard packages (e.g., Rotki, IPFS, etc.) use a simpler CI workflow that includes: + +- **Upstream version checking**: Automatically detects new upstream releases +- **Build and pre-release**: Builds packages and creates pre-releases on PRs and pushes + +See [Standard Packages CI](./standard-packages.md) for detailed workflow configurations. + +### Staker Packages + +Staker packages require more complex CI that includes integration testing on real DAppNode hardware. These packages include: + +- **Execution clients** (Geth, Reth, Nethermind, Besu, Erigon) +- **Consensus clients** (Lodestar, Teku, Prysm, Nimbus, Lighthouse) +- **Web3Signer** +- **MEV-Boost** + +The staker package CI runs on a GitHub self-hosted runner with DAppNode pre-installed and running with `DEV=true`, which enables the WebSocket frontend-backend RPC as an API. This allows the CI to use the same RPC calls a user would make to configure a staker setup. + +See [Staker Packages CI](./staker-packages/overview.md) for detailed information about: + +- [Execution Client Workflows](./staker-packages/execution-clients.md) +- [Consensus Client Workflows](./staker-packages/consensus-clients.md) +- [Web3Signer Workflows](./staker-packages/web3signer.md) + +## Common Requirements + +All DAppNode package workflows require: + +- **`GITHUB_TOKEN`**: Automatically provided by GitHub for authentication +- **`PINATA_API_KEY`** and **`PINATA_SECRET_API_KEY`**: For interfacing with the IPFS pinning service Pinata (standard packages) + +Staker packages additionally require: + +- Access to a self-hosted runner with DAppNode installed +- Pre-synced execution client volumes for faster test execution diff --git a/docs/dev/github-actions/staker-packages/consensus-clients.md b/docs/dev/github-actions/staker-packages/consensus-clients.md new file mode 100644 index 000000000..bc7536dbe --- /dev/null +++ b/docs/dev/github-actions/staker-packages/consensus-clients.md @@ -0,0 +1,238 @@ +--- +title: "Consensus Client Workflows" +sidebar_label: Consensus Clients +llm_description: "GitHub Actions workflows for DAppNode consensus client packages including sync-test and release workflows." +--- + +# Consensus Client Workflows + +Consensus client packages (Lodestar, Teku, Prysm, Nimbus, Lighthouse) use three GitHub Actions workflows. Unlike execution clients, consensus clients **do not need a `sync.yml` workflow** because they use checkpoint sync, which doesn't require pre-synced volumes. + +## Workflow Overview + +| Workflow | Trigger | Purpose | +| ---------------- | ---------------------- | ----------------------------- | +| `auto_check.yml` | Scheduled (every 4h) | Check for upstream updates | +| `sync-test.yml` | Pull requests + manual | Test package changes | +| `release.yml` | Push to main + manual | Release with attestation test | + +:::note +Consensus clients don't have a `sync.yml` workflow because they use **checkpoint sync**. This allows them to sync quickly from a trusted checkpoint rather than from genesis, eliminating the need for pre-synced volumes. +::: + +## 1. Auto Check for Upstream Updates (`auto_check.yml`) + +Same as standard packages - checks for upstream updates and creates PRs when new versions are available. + +```yaml +name: Bump upstream version + +on: + schedule: + - cron: "00 */4 * * *" + push: + branches: + - "master" + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - run: npx @dappnode/dappnodesdk github-action bump-upstream + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PINATA_API_KEY: ${{ secrets.PINATA_API_KEY }} + PINATA_SECRET_API_KEY: ${{ secrets.PINATA_SECRET_API_KEY }} +``` + +## 2. Sync Test (`sync-test.yml`) + +Triggered on PRs to test package changes. Builds the consensus client package, uploads it to the local IPFS node, and runs a sync test paired with an execution client. + +### Workflow File + +```yaml +name: Consensus Client Sync Test + +on: + workflow_dispatch: + inputs: + execution_client: + description: "Execution Client" + required: true + type: choice + options: [geth, reth, nethermind, besu, erigon] + pull_request: + branches: + - "main" + paths-ignore: + - "README.md" + +jobs: + build: + runs-on: staking-test-hoodi + name: Build + outputs: + ipfs_hash: ${{ steps.extract_hash.outputs.ipfs_hash }} + steps: + - uses: actions/checkout@v6 + - name: Build and upload + run: npx @dappnode/dappnodesdk build --provider=http://$(docker inspect DAppNodeCore-ipfs.dnp.dappnode.eth --format '{{.NetworkSettings.Networks.dncore_network.IPAddress}}'):5001 --variant=hoodi + - name: Extract IPFS hash from releases.json + id: extract_hash + run: | + # Get the last key's hash from releases.json + IPFS_HASH=$(jq -r 'to_entries | last | .value.hash' package_variants/hoodi/releases.json) + echo "ipfs_hash=$IPFS_HASH" >> $GITHUB_OUTPUT + echo "Extracted IPFS hash: $IPFS_HASH" + + sync-test: + runs-on: staking-test-hoodi + name: Consensus Client Sync Test + needs: [build] + steps: + - name: Run sync + run: | + docker run --rm --pull=always --network dncore_network \ + -v /var/run/docker.sock:/var/run/docker.sock \ + -e MODE=sync \ + -e CONSENSUS_CLIENT='teku' \ + -e IPFS_HASH=${{ needs.build.outputs.ipfs_hash }} \ + -e EXECUTION_CLIENT=${{ github.event.inputs.execution_client }} \ + -e GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }} \ + -e GITHUB_REPOSITORY=${{ github.repository }} \ + -e GITHUB_PR_NUMBER=${{ github.event.pull_request.number }} \ + -e GITHUB_RUN_ID=${{ github.run_id }} \ + -e GITHUB_SERVER_URL=${{ github.server_url }} \ + ghcr.io/dappnode/staker-test-util/test-runner:latest +``` + +### Key Differences from Execution Client Workflows + +- **`CONSENSUS_CLIENT`** is set to the package being tested (e.g., `teku`) +- **`EXECUTION_CLIENT`** is the input parameter for selecting which execution client to pair with +- The IPFS hash refers to the consensus client package being tested + +### PR Report + +After the test completes, a report is posted to the PR including: + +- Clients used and versions (before/after install) +- Timing measurements for each operation +- Container error logs (if any) +- Link to full CI logs + +## 3. Release (`release.yml`) + +Triggered on push to main. Runs a full attestation test before releasing the package. + +### Workflow File + +```yaml +name: "Release" + +on: + workflow_dispatch: + inputs: + execution_client: + description: "Execution Client" + required: true + type: choice + options: [geth, reth, nethermind, besu, erigon] + push: + branches: + - "main" + paths-ignore: + - "README.md" + +jobs: + build: + runs-on: staking-test-hoodi + name: Build + outputs: + ipfs_hash: ${{ steps.extract_hash.outputs.ipfs_hash }} + steps: + - uses: actions/checkout@v6 + - name: Build and upload + run: npx @dappnode/dappnodesdk build --provider=http://$(docker inspect DAppNodeCore-ipfs.dnp.dappnode.eth --format '{{.NetworkSettings.Networks.dncore_network.IPAddress}}'):5001 --variant=hoodi + - name: Extract IPFS hash from releases.json + id: extract_hash + run: | + # Get the last key's hash from releases.json + IPFS_HASH=$(jq -r 'to_entries | last | .value.hash' package_variants/hoodi/releases.json) + echo "ipfs_hash=$IPFS_HASH" >> $GITHUB_OUTPUT + echo "Extracted IPFS hash: $IPFS_HASH" + + test: + name: Test + runs-on: staking-test-hoodi + needs: [build] + steps: + - uses: actions/checkout@v6 + - name: Run staker test runner + run: | + docker run --rm --pull=always \ + --network dncore_network \ + -v /var/run/docker.sock:/var/run/docker.sock \ + -e MODE=test \ + -e IPFS_HASH=${{ needs.build.outputs.ipfs_hash }} \ + -e CONSENSUS_CLIENT='teku' \ + -e EXECUTION_CLIENT=${{ github.event.inputs.execution_client }} \ + -e GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }} \ + -e GITHUB_REPOSITORY=${{ github.repository }} \ + -e GITHUB_PR_NUMBER=${{ github.event.pull_request.number }} \ + -e GITHUB_RUN_ID=${{ github.run_id }} \ + -e GITHUB_SERVER_URL=${{ github.server_url }} \ + ghcr.io/dappnode/staker-test-util/test-runner:latest + + release: + name: Release + runs-on: ipfs-dev-gateway + needs: [test] + steps: + - uses: actions/checkout@v6 + - uses: actions/setup-node@v6 + with: + node-version: "22" + - name: Publish + run: npx @dappnode/dappnodesdk publish patch --content_provider=http://10.200.200.7:5001 --eth_provider=https://web3.dappnode.net --timeout 2h --all-variants + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + DEVELOPER_ADDRESS: "0xf35960302a07022aba880dffaec2fdd64d5bf1c1" +``` + +### Features + +- **Proof of Attestation Test**: The test job uses `MODE=test` which: + 1. Syncs execution and consensus clients (consensus uses checkpoint sync) + 2. Imports validators into web3signer + 3. Waits for validators to become live on the beacon chain + 4. Confirms successful attestation +- **Three-stage pipeline**: Build → Test → Release +- **Different runners**: Test runs on `staking-test-hoodi`, release on `ipfs-dev-gateway` + +## Checkpoint Sync + +Consensus clients benefit from **checkpoint sync**, which allows them to: + +1. Start syncing from a recent finalized checkpoint instead of genesis +2. Sync in minutes rather than hours or days +3. Avoid the need for pre-synced volumes on the self-hosted runner + +This is why consensus client packages don't need a dedicated `sync.yml` workflow to maintain a synced state. + +## Environment Variables + +| Variable | Description | +| ------------------- | ------------------------------------------------------------------- | +| `MODE` | `sync` for sync only, `test` for full attestation test | +| `CONSENSUS_CLIENT` | The consensus client name being tested (e.g., `teku`, `lighthouse`) | +| `EXECUTION_CLIENT` | The execution client to pair with | +| `IPFS_HASH` | The IPFS hash of the built package to test | +| `NETWORK` | The network to use (e.g., `hoodi`) | +| `GITHUB_TOKEN` | For PR commenting and GitHub API access | +| `GITHUB_REPOSITORY` | Repository name for PR comments | +| `GITHUB_PR_NUMBER` | PR number for posting reports | +| `GITHUB_RUN_ID` | Run ID for linking to CI logs | +| `GITHUB_SERVER_URL` | GitHub server URL for links | diff --git a/docs/dev/github-actions/staker-packages/execution-clients.md b/docs/dev/github-actions/staker-packages/execution-clients.md new file mode 100644 index 000000000..37e4f36e1 --- /dev/null +++ b/docs/dev/github-actions/staker-packages/execution-clients.md @@ -0,0 +1,277 @@ +--- +title: "Execution Client Workflows" +sidebar_label: Execution Clients +llm_description: "GitHub Actions workflows for DAppNode execution client packages including sync, sync-test, and release workflows." +--- + +# Execution Client Workflows + +Execution client packages (Geth, Reth, Nethermind, Besu, Erigon) use four GitHub Actions workflows to ensure proper testing and release automation. + +## Workflow Overview + +| Workflow | Trigger | Purpose | +| ---------------- | ------------------------ | ----------------------------- | +| `auto_check.yml` | Scheduled (every 4h) | Check for upstream updates | +| `sync.yml` | Scheduled daily + manual | Keep execution client synced | +| `sync-test.yml` | Pull requests + manual | Test package changes | +| `release.yml` | Push to main + manual | Release with attestation test | + +## 1. Auto Check for Upstream Updates (`auto_check.yml`) + +Same as standard packages - checks for upstream updates and creates PRs when new versions are available. + +```yaml +name: Bump upstream version + +on: + schedule: + - cron: "00 */4 * * *" + push: + branches: + - "master" + +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v3 + - run: npx @dappnode/dappnodesdk github-action bump-upstream + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + PINATA_API_KEY: ${{ secrets.PINATA_API_KEY }} + PINATA_SECRET_API_KEY: ${{ secrets.PINATA_SECRET_API_KEY }} +``` + +## 2. Sync Production (`sync.yml`) + +This workflow keeps the execution client synced on the self-hosted runner. By maintaining a synced state, other workflows (sync-test, release) complete much faster. + +### Workflow File + +```yaml +name: Execution Client Sync Production + +on: + schedule: + - cron: "0 4 * * *" + workflow_dispatch: + inputs: + consensus_client: + description: "Consensus Client" + required: true + type: choice + options: [lodestar, teku, prysm, nimbus, lighthouse] + +jobs: + sync: + runs-on: staking-test-hoodi + steps: + - name: Run sync + run: | + docker run --rm --pull=always --network dncore_network \ + -v /var/run/docker.sock:/var/run/docker.sock \ + -e MODE=sync \ + -e EXECUTION_CLIENT='geth' \ + -e CONSENSUS_CLIENT=${{ github.event.inputs.consensus_client }} \ + -e NETWORK=hoodi \ + ghcr.io/dappnode/staker-test-util/test-runner:latest +``` + +### Features + +- **Scheduled daily**: Runs at 4 AM UTC to keep the client synced +- **Manual trigger**: Can be triggered manually with any consensus client +- **Uses test-runner image**: The `ghcr.io/dappnode/staker-test-util/test-runner` container handles the sync logic +- **Network access**: Connects to the DAppNode's `dncore_network` to communicate with other services + +## 3. Sync Test (`sync-test.yml`) + +Triggered on PRs to test package changes. Builds the package, uploads it to the local IPFS node, and runs a sync test with the new version. + +### Workflow File + +```yaml +name: Execution Client Sync Test + +on: + workflow_dispatch: + inputs: + consensus_client: + description: "Consensus Client" + required: true + type: choice + options: [lodestar, teku, prysm, nimbus, lighthouse] + pull_request: + branches: + - "main" + paths-ignore: + - "README.md" + +jobs: + build: + runs-on: staking-test-hoodi + name: Build + outputs: + ipfs_hash: ${{ steps.extract_hash.outputs.ipfs_hash }} + steps: + - uses: actions/checkout@v6 + - name: Build and upload + run: npx @dappnode/dappnodesdk build --provider=http://$(docker inspect DAppNodeCore-ipfs.dnp.dappnode.eth --format '{{.NetworkSettings.Networks.dncore_network.IPAddress}}'):5001 --variant=hoodi + - name: Extract IPFS hash from releases.json + id: extract_hash + run: | + # Get the last key's hash from releases.json + IPFS_HASH=$(jq -r 'to_entries | last | .value.hash' package_variants/hoodi/releases.json) + echo "ipfs_hash=$IPFS_HASH" >> $GITHUB_OUTPUT + echo "Extracted IPFS hash: $IPFS_HASH" + + sync-test: + runs-on: staking-test-hoodi + name: Execution Client Sync Test + needs: [build] + steps: + - name: Run sync + run: | + docker run --rm --pull=always --network dncore_network \ + -v /var/run/docker.sock:/var/run/docker.sock \ + -e MODE=sync \ + -e EXECUTION_CLIENT='geth' \ + -e IPFS_HASH=${{ needs.build.outputs.ipfs_hash }} \ + -e CONSENSUS_CLIENT=${{ github.event.inputs.consensus_client }} \ + -e GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }} \ + -e GITHUB_REPOSITORY=${{ github.repository }} \ + -e GITHUB_PR_NUMBER=${{ github.event.pull_request.number }} \ + -e GITHUB_RUN_ID=${{ github.run_id }} \ + -e GITHUB_SERVER_URL=${{ github.server_url }} \ + ghcr.io/dappnode/staker-test-util/test-runner:latest +``` + +### Features + +- **Builds to local IPFS**: Uses the DAppNode's IPFS node as the provider +- **IPFS hash extraction**: Extracts the hash from `releases.json` for the test +- **PR integration**: Posts a test report to the PR with results +- **GitHub context**: Passes GitHub environment variables for PR commenting + +### PR Report + +After the test completes, a report is posted to the PR including: + +- Clients used and versions (before/after install) +- Timing measurements for each operation +- Container error logs (if any) +- Link to full CI logs + +## 4. Release (`release.yml`) + +Triggered on push to main. Runs a full attestation test before releasing the package. + +### Workflow File + +```yaml +name: "Release" + +on: + workflow_dispatch: + inputs: + consensus_client: + description: "Consensus Client" + required: true + type: choice + options: [lodestar, teku, prysm, nimbus, lighthouse] + push: + branches: + - "main" + paths-ignore: + - "README.md" + +jobs: + build: + runs-on: staking-test-hoodi + name: Build + outputs: + ipfs_hash: ${{ steps.extract_hash.outputs.ipfs_hash }} + steps: + - uses: actions/checkout@v6 + - name: Build and upload + run: npx @dappnode/dappnodesdk build --provider=http://$(docker inspect DAppNodeCore-ipfs.dnp.dappnode.eth --format '{{.NetworkSettings.Networks.dncore_network.IPAddress}}'):5001 --variant=hoodi + - name: Extract IPFS hash from releases.json + id: extract_hash + run: | + # Get the last key's hash from releases.json + IPFS_HASH=$(jq -r 'to_entries | last | .value.hash' package_variants/hoodi/releases.json) + echo "ipfs_hash=$IPFS_HASH" >> $GITHUB_OUTPUT + echo "Extracted IPFS hash: $IPFS_HASH" + + test: + name: Test + runs-on: staking-test-hoodi + needs: [build] + steps: + - uses: actions/checkout@v6 + - name: Run staker test runner + run: | + docker run --rm --pull=always \ + --network dncore_network \ + -v /var/run/docker.sock:/var/run/docker.sock \ + -e MODE=test \ + -e IPFS_HASH=${{ needs.build.outputs.ipfs_hash }} \ + -e CONSENSUS_CLIENT=${{ github.event.inputs.consensus_client }} \ + -e GITHUB_TOKEN=${{ secrets.GITHUB_TOKEN }} \ + -e GITHUB_REPOSITORY=${{ github.repository }} \ + -e GITHUB_PR_NUMBER=${{ github.event.pull_request.number }} \ + -e GITHUB_RUN_ID=${{ github.run_id }} \ + -e GITHUB_SERVER_URL=${{ github.server_url }} \ + ghcr.io/dappnode/staker-test-util/test-runner:latest + + release: + name: Release + runs-on: ipfs-dev-gateway + needs: [test] + steps: + - uses: actions/checkout@v6 + - uses: actions/setup-node@v6 + with: + node-version: "22" + - name: Publish + run: npx @dappnode/dappnodesdk publish patch --content_provider=http://10.200.200.7:5001 --eth_provider=https://web3.dappnode.net --timeout 2h --all-variants + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + DEVELOPER_ADDRESS: "0xf35960302a07022aba880dffaec2fdd64d5bf1c1" +``` + +### Features + +- **Proof of Attestation Test**: The test job uses `MODE=test` which: + 1. Syncs execution and consensus clients + 2. Imports validators into web3signer + 3. Waits for validators to become live on the beacon chain + 4. Confirms successful attestation +- **Three-stage pipeline**: Build → Test → Release +- **Different runners**: Test runs on `staking-test-hoodi`, release on `ipfs-dev-gateway` +- **All variants**: Publishes all package variants with `--all-variants` + +### Test Logs + +The proof of attestation test logs include: + +- Validator beaconcha.in URL +- Clients used and versions +- Sync timing measurements +- Attestation confirmation + +## Environment Variables + +| Variable | Description | +| ------------------- | ------------------------------------------------------ | +| `MODE` | `sync` for sync only, `test` for full attestation test | +| `EXECUTION_CLIENT` | The execution client name (e.g., `geth`, `reth`) | +| `CONSENSUS_CLIENT` | The consensus client to pair with | +| `IPFS_HASH` | The IPFS hash of the built package to test | +| `NETWORK` | The network to use (e.g., `hoodi`) | +| `GITHUB_TOKEN` | For PR commenting and GitHub API access | +| `GITHUB_REPOSITORY` | Repository name for PR comments | +| `GITHUB_PR_NUMBER` | PR number for posting reports | +| `GITHUB_RUN_ID` | Run ID for linking to CI logs | +| `GITHUB_SERVER_URL` | GitHub server URL for links | diff --git a/docs/dev/github-actions/staker-packages/overview.md b/docs/dev/github-actions/staker-packages/overview.md new file mode 100644 index 000000000..04f6a15b8 --- /dev/null +++ b/docs/dev/github-actions/staker-packages/overview.md @@ -0,0 +1,101 @@ +--- +title: "Staker Packages CI Overview" +sidebar_label: Overview +llm_description: "Overview of CI/CD workflows for DAppNode staker packages including execution clients, consensus clients, and web3signer." +--- + +# Staker Packages CI Overview + +Staker packages require more complex CI than standard packages because they need integration testing on real DAppNode hardware. This ensures that execution clients, consensus clients, web3signer, and MEV-boost work correctly together in a real staking environment. + +## Architecture + +The staker package CI runs on a **GitHub self-hosted runner** with the following characteristics: + +- **DAppNode pre-installed** and running with `DEV=true` +- The `DEV=true` environment variable enables the **WebSocket frontend-backend RPC as an API** +- The CI uses the **same RPC calls a user would make** to configure a staker setup +- **Pre-synced execution client volumes** are loaded on the runner, enabling fast sync and attestation tests +- Consensus clients use **checkpoint sync**, so no pre-synced volume is required + +## Test Types + +### Sync Test + +Verifies that both execution and consensus clients reach a synced state. This test ensures: + +- The staker configuration is valid +- Clients can synchronize with the network +- The package builds correctly and can be installed + +A report is posted to the PR with test details including: + +- Clients used and their versions +- Timing measurements +- Container error logs (if any) + +### Proof of Attestation Test + +This is the most comprehensive test, executed during releases: + +1. **Sync phase**: Same as the sync test - ensures clients are synchronized +2. **Attestation phase**: Imports validators into web3signer and waits for them to become live on the beacon chain + +A successful attestation proves the **full staker stack is operational end-to-end**. + +## Package-Specific Workflows + +Different staker packages have different workflow requirements: + +| Package Type | auto_check.yml | sync.yml | sync-test.yml | release.yml | +| ----------------- | -------------- | -------- | ------------- | ----------- | +| Execution Clients | ✅ | ✅ | ✅ | ✅ | +| Consensus Clients | ✅ | ❌ | ✅ | ✅ | +| Web3Signer | 🔄 TBD | 🔄 TBD | 🔄 TBD | 🔄 TBD | + +See the specific workflow documentation: + +- [Execution Client Workflows](./execution-clients.md) +- [Consensus Client Workflows](./consensus-clients.md) +- [Web3Signer Workflows](./web3signer.md) + +## Manual Triggering + +All staker package workflows can be **triggered manually** via `workflow_dispatch`, allowing you to: + +- Test any client combination (execution + consensus) +- Debug issues with specific client pairings +- Run tests outside of the normal CI triggers + +When triggering manually, you can select the consensus client to pair with the execution client (or vice versa). + +## Self-Hosted Runners + +The staker tests run on specific self-hosted runners: + +- **`staking-test-hoodi`**: Main runner for sync and attestation tests on Hoodi testnet +- **`ipfs-dev-gateway`**: Used for the final release step to publish packages + +## Test Report Example + +When a sync test completes, a report like this is posted to the PR: + +```markdown +## ✅ SYNC TEST REPORT - PASSED + +### 📦 Clients Used + +| Component | DNP Name | DNP Version | +| --------- | ----------------------------- | ----------- | +| Execution | `hoodi-reth.dnp.dappnode.eth` | `0.1.4` | +| Consensus | `teku-hoodi.dnp.dappnode.eth` | `0.1.3` | + +### ⏱️ Timing Measurements + +| Operation | Duration | Status | +| ---------------------- | -------- | ------ | +| WaitForBeaconchainSync | 3ms | ✅ | +| WaitForExecutionSync | 24.015s | ✅ | +``` + +The report includes version tracking (before/after install), timing measurements, and any container error logs for debugging. diff --git a/docs/dev/github-actions/staker-packages/web3signer.md b/docs/dev/github-actions/staker-packages/web3signer.md new file mode 100644 index 000000000..29fab5b7b --- /dev/null +++ b/docs/dev/github-actions/staker-packages/web3signer.md @@ -0,0 +1,46 @@ +--- +title: "Web3Signer Workflows" +sidebar_label: Web3Signer +llm_description: "GitHub Actions workflows for the DAppNode Web3Signer package (to be designed)." +--- + +# Web3Signer Workflows + +:::caution Work in Progress +The CI workflows for Web3Signer packages are currently being designed. This documentation will be updated once the workflows are finalized. +::: + +## Overview + +Web3Signer is a critical component in the staking stack that handles validator key management and signing. The CI workflows for Web3Signer will need to verify: + +- Proper key import functionality +- Signing operations work correctly +- Integration with both execution and consensus client pairs +- Slashing protection database handling + +## Planned Workflows + +The following workflows are being considered for Web3Signer packages: + +| Workflow | Status | Purpose | +| ---------------- | ------ | ----------------------------- | +| `auto_check.yml` | 🔄 TBD | Check for upstream updates | +| `sync-test.yml` | 🔄 TBD | Test package changes | +| `release.yml` | 🔄 TBD | Release with integration test | + +## Testing Considerations + +Web3Signer testing presents unique challenges: + +1. **Key Management**: Tests need to handle validator keys securely +2. **Slashing Protection**: Must verify slashing protection DB is working correctly +3. **Client Compatibility**: Should test with multiple execution/consensus client combinations +4. **API Verification**: Web3Signer's remote signing API should be validated + +## Contributing + +If you have ideas or want to contribute to the Web3Signer CI workflow design, please: + +- Join the discussion on [Discord](https://discord.com/invite/dappnode) +- Open an issue on [GitHub](https://github.com/dappnode) diff --git a/docs/dev/github-actions.md b/docs/dev/github-actions/standard-packages.md similarity index 82% rename from docs/dev/github-actions.md rename to docs/dev/github-actions/standard-packages.md index b8e55bb98..5cf46384e 100644 --- a/docs/dev/github-actions.md +++ b/docs/dev/github-actions/standard-packages.md @@ -1,17 +1,18 @@ --- -title: "GitHub Actions for Dappnode Packages" -llm_description: "GitHub Actions workflows for automating upstream updates and package releases." +title: "Standard Packages CI" +sidebar_label: Standard Packages +llm_description: "GitHub Actions workflows for standard Dappnode packages including upstream version checking and release automation." --- -# GitHub Actions for Dappnode Packages +# Standard Packages CI -GitHub Actions help automate workflows around your projects on GitHub. For Dappnode packages, we utilize specific GitHub Actions to manage updates and releases. +Standard Dappnode packages (e.g., Rotki, IPFS) use a straightforward CI workflow consisting of two main GitHub Actions: upstream version checking and build/release automation. ## 1. Auto Check for Upstream Updates (`auto_check.yml`) This action checks if there's an available update from the upstream repository. If an update is found, it creates a Pull Request with the new version. -### Workflow File: +### Workflow File **Filename:** `auto_check.yml` @@ -37,7 +38,7 @@ jobs: PINATA_SECRET_API_KEY: ${{ secrets.PINATA_SECRET_API_KEY }} ``` -**Features:** +### Features - **Scheduled Runs**: The action runs every 4 hours (as per the cron setting). - **Environment Variables**: Uses `GITHUB_TOKEN` for authentication, and `PINATA_API_KEY` and `PINATA_SECRET_API_KEY` for interfacing with the IPFS pinning service Pinata. @@ -46,7 +47,7 @@ jobs: This action builds the Dappnode package and creates a pre-release that's ready to be published. -### Workflow File: +### Workflow File **Filename:** `main.yml` @@ -84,8 +85,8 @@ jobs: DEVELOPER_ADDRESS: "0xf35960302a07022aba880dffaec2fdd64d5bf1c1" ``` -**Features:** +### Features - **Branch Filters**: The action only triggers on the `main`, `master`, and version tags (e.g., `v1.0.0`). - **Paths Ignore**: Ignores changes solely related to `README.md`. -- **Conditional Runs**: Different steps are executed based on the event that triggered the workflow (e.g., push event or pull request). \ No newline at end of file +- **Conditional Runs**: Different steps are executed based on the event that triggered the workflow (e.g., push event or pull request). diff --git a/docs/dev/package-development/multi-configuration.md b/docs/dev/package-development/multi-configuration.md index b49f5d9fd..422c221e4 100644 --- a/docs/dev/package-development/multi-configuration.md +++ b/docs/dev/package-development/multi-configuration.md @@ -195,6 +195,6 @@ After running the publish command, you will receive a link to sign the transacti ### Automating Publication with GitHub Actions -If your package source code is hosted on GitHub, you can automate the publishing process by setting up GitHub Actions. Use the workflows [here](/docs/dev/github-actions.md) to integrate publishing workflows, ensuring your package builds and publishes automatically. +If your package source code is hosted on GitHub, you can automate the publishing process by setting up GitHub Actions. Use the workflows [here](/docs/dev/github-actions/overview.md) to integrate publishing workflows, ensuring your package builds and publishes automatically. By following these steps, you'll be able to develop, build, and publish multi-variant packages on Dappnode, making it easier to support multiple configurations from a single codebase. diff --git a/docs/dev/package-development/single-configuration.md b/docs/dev/package-development/single-configuration.md index 300e96576..ed3508e86 100644 --- a/docs/dev/package-development/single-configuration.md +++ b/docs/dev/package-development/single-configuration.md @@ -118,4 +118,4 @@ All package names must follow this convention: ### Automating Publication with GitHub Actions -If the package source code is hosted on GitHub, you can automate the publishing process by setting up GitHub Actions. Use the workflows [here](/docs/dev/github-actions.md) to integrate publishing workflows. +If the package source code is hosted on GitHub, you can automate the publishing process by setting up GitHub Actions. Use the workflows [here](/docs/dev/github-actions/overview.md) to integrate publishing workflows. diff --git a/sidebars.js b/sidebars.js index 7fb3efc67..ddde2dab0 100644 --- a/sidebars.js +++ b/sidebars.js @@ -295,7 +295,7 @@ const sidebars = { type: "doc", label: "Aztec", id: "user/rollups/aztec", - } + }, ], }, { @@ -477,8 +477,6 @@ const sidebars = { label: "How To Install Dappnode App", id: "user/dappnode-app/how-to-install", }, - - ], }, { @@ -495,8 +493,6 @@ const sidebars = { label: "Premium services", id: "user/dappnode-premium/premium-services", }, - - ], }, { @@ -719,9 +715,46 @@ const sidebars = { id: "dev/dns", }, { - type: "doc", - label: "Github Actions", - id: "dev/github-actions", + type: "category", + label: "GitHub Actions", + items: [ + { + type: "doc", + label: "Overview", + id: "dev/github-actions/overview", + }, + { + type: "doc", + label: "Standard Packages", + id: "dev/github-actions/standard-packages", + }, + { + type: "category", + label: "Staker Packages", + items: [ + { + type: "doc", + label: "Overview", + id: "dev/github-actions/staker-packages/overview", + }, + { + type: "doc", + label: "Execution Clients", + id: "dev/github-actions/staker-packages/execution-clients", + }, + { + type: "doc", + label: "Consensus Clients", + id: "dev/github-actions/staker-packages/consensus-clients", + }, + { + type: "doc", + label: "Web3Signer", + id: "dev/github-actions/staker-packages/web3signer", + }, + ], + }, + ], }, ], smoothSidebar: [ From fdf55da0b700bf47b5ba9c993cd0b95264171597 Mon Sep 17 00:00:00 2001 From: Marc Font <36164126+Marketen@users.noreply.github.com> Date: Fri, 6 Feb 2026 11:39:37 +0100 Subject: [PATCH 2/2] changes to intro (#508) --- docs/dev/github-actions/overview.md | 8 +++---- .../staker-packages/consensus-clients.md | 2 +- .../staker-packages/execution-clients.md | 6 +++--- .../staker-packages/overview.md | 21 +++++++++++++------ .../staker-packages/web3signer.md | 2 +- 5 files changed, 24 insertions(+), 15 deletions(-) diff --git a/docs/dev/github-actions/overview.md b/docs/dev/github-actions/overview.md index dab0de1d0..d0c93b6d3 100644 --- a/docs/dev/github-actions/overview.md +++ b/docs/dev/github-actions/overview.md @@ -23,14 +23,14 @@ See [Standard Packages CI](./standard-packages.md) for detailed workflow configu ### Staker Packages -Staker packages require more complex CI that includes integration testing on real DAppNode hardware. These packages include: +Staker packages require more complex CI that includes integration testing on real Dappnode hardware. These packages include: - **Execution clients** (Geth, Reth, Nethermind, Besu, Erigon) - **Consensus clients** (Lodestar, Teku, Prysm, Nimbus, Lighthouse) - **Web3Signer** - **MEV-Boost** -The staker package CI runs on a GitHub self-hosted runner with DAppNode pre-installed and running with `DEV=true`, which enables the WebSocket frontend-backend RPC as an API. This allows the CI to use the same RPC calls a user would make to configure a staker setup. +The staker package CI runs on a GitHub self-hosted runner with Dappnode pre-installed and running with `DEV=true`, which enables the WebSocket frontend-backend RPC as an API. This allows the CI to use the same RPC calls a user would make to configure a staker setup. See [Staker Packages CI](./staker-packages/overview.md) for detailed information about: @@ -40,12 +40,12 @@ See [Staker Packages CI](./staker-packages/overview.md) for detailed information ## Common Requirements -All DAppNode package workflows require: +All Dappnode package workflows require: - **`GITHUB_TOKEN`**: Automatically provided by GitHub for authentication - **`PINATA_API_KEY`** and **`PINATA_SECRET_API_KEY`**: For interfacing with the IPFS pinning service Pinata (standard packages) Staker packages additionally require: -- Access to a self-hosted runner with DAppNode installed +- Access to a self-hosted runner with Dappnode installed - Pre-synced execution client volumes for faster test execution diff --git a/docs/dev/github-actions/staker-packages/consensus-clients.md b/docs/dev/github-actions/staker-packages/consensus-clients.md index bc7536dbe..de8ad5d78 100644 --- a/docs/dev/github-actions/staker-packages/consensus-clients.md +++ b/docs/dev/github-actions/staker-packages/consensus-clients.md @@ -1,7 +1,7 @@ --- title: "Consensus Client Workflows" sidebar_label: Consensus Clients -llm_description: "GitHub Actions workflows for DAppNode consensus client packages including sync-test and release workflows." +llm_description: "GitHub Actions workflows for Dappnode consensus client packages including sync-test and release workflows." --- # Consensus Client Workflows diff --git a/docs/dev/github-actions/staker-packages/execution-clients.md b/docs/dev/github-actions/staker-packages/execution-clients.md index 37e4f36e1..4ea8b5853 100644 --- a/docs/dev/github-actions/staker-packages/execution-clients.md +++ b/docs/dev/github-actions/staker-packages/execution-clients.md @@ -1,7 +1,7 @@ --- title: "Execution Client Workflows" sidebar_label: Execution Clients -llm_description: "GitHub Actions workflows for DAppNode execution client packages including sync, sync-test, and release workflows." +llm_description: "GitHub Actions workflows for Dappnode execution client packages including sync, sync-test, and release workflows." --- # Execution Client Workflows @@ -83,7 +83,7 @@ jobs: - **Scheduled daily**: Runs at 4 AM UTC to keep the client synced - **Manual trigger**: Can be triggered manually with any consensus client - **Uses test-runner image**: The `ghcr.io/dappnode/staker-test-util/test-runner` container handles the sync logic -- **Network access**: Connects to the DAppNode's `dncore_network` to communicate with other services +- **Network access**: Connects to the Dappnode's `dncore_network` to communicate with other services ## 3. Sync Test (`sync-test.yml`) @@ -149,7 +149,7 @@ jobs: ### Features -- **Builds to local IPFS**: Uses the DAppNode's IPFS node as the provider +- **Builds to local IPFS**: Uses the Dappnode's IPFS node as the provider - **IPFS hash extraction**: Extracts the hash from `releases.json` for the test - **PR integration**: Posts a test report to the PR with results - **GitHub context**: Passes GitHub environment variables for PR commenting diff --git a/docs/dev/github-actions/staker-packages/overview.md b/docs/dev/github-actions/staker-packages/overview.md index 04f6a15b8..537b188db 100644 --- a/docs/dev/github-actions/staker-packages/overview.md +++ b/docs/dev/github-actions/staker-packages/overview.md @@ -1,21 +1,30 @@ --- title: "Staker Packages CI Overview" sidebar_label: Overview -llm_description: "Overview of CI/CD workflows for DAppNode staker packages including execution clients, consensus clients, and web3signer." +llm_description: "Overview of CI/CD workflows for Dappnode staker packages including execution clients, consensus clients, and web3signer." --- # Staker Packages CI Overview -Staker packages require more complex CI than standard packages because they need integration testing on real DAppNode hardware. This ensures that execution clients, consensus clients, web3signer, and MEV-boost work correctly together in a real staking environment. +Staker packages require more complex CI than standard packages because they need integration testing on real Dappnode hardware. This ensures that execution clients, consensus clients, web3signer, and MEV-boost work correctly together in a real staking environment. ## Architecture The staker package CI runs on a **GitHub self-hosted runner** with the following characteristics: -- **DAppNode pre-installed** and running with `DEV=true` -- The `DEV=true` environment variable enables the **WebSocket frontend-backend RPC as an API** -- The CI uses the **same RPC calls a user would make** to configure a staker setup -- **Pre-synced execution client volumes** are loaded on the runner, enabling fast sync and attestation tests +### Fresh Dappnode Instance + +Each test runs on a **fresh Dappnode instance** that is always kept up to date with the **latest versions of all core packages**. When a test starts, **no staker packages are installed** - the environment is completely clean, ensuring consistent and reproducible test results. + +### API-Driven Testing + +- The Dappmanager runs with `DEV=true`, which **enables the WebSocket frontend-backend RPC as an API** +- The CI uses the **same RPC calls a user would make** through the UI to configure their staker setup (installing execution clients, consensus clients, web3signer, etc.) +- This ensures tests are as close to real user interactions as possible + +### Pre-Synced Volumes + +- **Pre-synced execution client databases** are stored in their respective volumes on the runner, enabling fast sync without waiting hours for a full sync from scratch - Consensus clients use **checkpoint sync**, so no pre-synced volume is required ## Test Types diff --git a/docs/dev/github-actions/staker-packages/web3signer.md b/docs/dev/github-actions/staker-packages/web3signer.md index 29fab5b7b..50db978e7 100644 --- a/docs/dev/github-actions/staker-packages/web3signer.md +++ b/docs/dev/github-actions/staker-packages/web3signer.md @@ -1,7 +1,7 @@ --- title: "Web3Signer Workflows" sidebar_label: Web3Signer -llm_description: "GitHub Actions workflows for the DAppNode Web3Signer package (to be designed)." +llm_description: "GitHub Actions workflows for the Dappnode Web3Signer package (to be designed)." --- # Web3Signer Workflows