Add blog strengthen longhorn with automated end-to-end tests #1295
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Chris <chris.chien@suse.com>
chriscchien
force-pushed
the
official/automate_manual_to_e2e
branch
from
11702ff to
cf648c9
Compare
sushant-suse
requested changes
Jan 19, 2026
Contributor
sushant-suse
left a comment
sushant-suse
left a comment
There was a problem hiding this comment.
Hi, added some minor suggestions for your consideration.
Thanks.
|
|
||
| ### Why E2E Testing Matters for Longhorn | ||
|
|
||
| E2E testing is a core part of ensuring Longhorn remains stable, resilient, and production ready. While integration tests validate individual components, E2E tests validate the entire system under real-world conditions, especially automated tests from user-reported issues, disruptive operations such as node shutdowns or reboots, and scenarios that may be repeated hundred times to find stability issues. Automating these scenarios helps the QA team quickly validate every Longhorn release, reducing manual workload and ensuring issues are found early. |
Contributor
There was a problem hiding this comment.
Suggested change
| E2E testing is a core part of ensuring Longhorn remains stable, resilient, and production ready. While integration tests validate individual components, E2E tests validate the entire system under real-world conditions, especially automated tests from user-reported issues, disruptive operations such as node shutdowns or reboots, and scenarios that may be repeated hundred times to find stability issues. Automating these scenarios helps the QA team quickly validate every Longhorn release, reducing manual workload and ensuring issues are found early. | |
| E2E testing is a core part of ensuring Longhorn remains stable, resilient, and production-ready. While integration tests validate individual components, E2E tests validate the entire system under real-world conditions; especially automated tests derived from user-reported issues, disruptive operations such as node shutdowns or reboots, and scenarios that may be repeated hundreds of times to uncover stability issues. Automating these scenarios helps the QA team quickly validate every Longhorn release, reducing manual workload and ensuring issues are found early. |
| - Longhorn upgrades | ||
| - Engine version validate | ||
|
|
||
| The integration test suite is gradually being deprecated as we plan to move all test cases into the E2E test framework. |
Contributor
There was a problem hiding this comment.
Suggested change
| The integration test suite is gradually being deprecated as we plan to move all test cases into the E2E test framework. | |
| The integration test suite is gradually being deprecated in favor of moving all test cases into the E2E test framework. |
|
|
||
| The E2E test suite is written in **Robot Framework** using a BDD-style structure. It focuses on: | ||
|
|
||
| - Converting user reported issues into automated test cases |
Contributor
There was a problem hiding this comment.
Suggested change
| - Converting user reported issues into automated test cases | |
| - Converting user-reported issues into automated test cases |
| - Node reboots | ||
| - Node power-down | ||
| - Network disconnect | ||
| - Automating [manual test cases](https://longhorn.github.io/longhorn-tests/manual/) that executed in release testing |
Contributor
There was a problem hiding this comment.
Suggested change
| - Automating [manual test cases](https://longhorn.github.io/longhorn-tests/manual/) that executed in release testing | |
| - Automating [manual test cases](https://longhorn.github.io/longhorn-tests/manual/) that are executed during release testing |
| - Network disconnect | ||
| - Automating [manual test cases](https://longhorn.github.io/longhorn-tests/manual/) that executed in release testing | ||
|
|
||
| As part of ongoing improvements, integration tests are actively being **migrated into the E2E suite**. The idea is to eventually use one unified test framework. Since E2E is written in BDD style, it’s much easier to understand the test steps from the logs and see exactly why a test failed, which helps a lot when debugging. |
Contributor
There was a problem hiding this comment.
Suggested change
| As part of ongoing improvements, integration tests are actively being **migrated into the E2E suite**. The idea is to eventually use one unified test framework. Since E2E is written in BDD style, it’s much easier to understand the test steps from the logs and see exactly why a test failed, which helps a lot when debugging. | |
| As part of ongoing improvements, integration tests are actively being **migrated into the E2E suite**. The goal is to eventually use one unified test framework. Since E2E tests are written in a BDD-style format, it’s much easier to understand the test steps from the logs and see exactly why a test failed, which greatly helps with debugging. |
| e2e/keywords/*.resource | ||
| ``` | ||
|
|
||
| #### 5. Robot → Resource → Python Flow |
Contributor
There was a problem hiding this comment.
Suggested change
| #### 5. Robot → Resource → Python Flow | |
| #### 5. Robot → Resource → Python Flow | |
| (Python function that waits for the recurring job pod to start) | ||
| ``` | ||
|
|
||
| #### 6. Complete the test case |
Contributor
There was a problem hiding this comment.
Suggested change
| #### 6. Complete the test case | |
| #### 6. Complete the test case | |
| #### 6. Complete the test case | ||
| Following the above steps, convert each step in the [manual test case](https://longhorn.github.io/longhorn-tests/manual/release-specific/v1.1.0/recurring-backup-job-interruptions/) into clear, human-readable Robot Framework test case. | ||
|
|
||
| ## Run test |
Contributor
There was a problem hiding this comment.
Suggested change
| ## Run test | |
| ## Run a test | |
| Following the above steps, convert each step in the [manual test case](https://longhorn.github.io/longhorn-tests/manual/release-specific/v1.1.0/recurring-backup-job-interruptions/) into clear, human-readable Robot Framework test case. | ||
|
|
||
| ## Run test | ||
| From the `longhorn-tests/e2e` folder, execute the test with: |
Contributor
There was a problem hiding this comment.
Suggested change
| From the `longhorn-tests/e2e` folder, execute the test with: | |
| From the `longhorn-tests/e2e` folder, execute the test with: | |
| /run.sh -t "Recurring backup job interruptions when Allow Recurring Job While Volume Is Detached is disabled" | ||
| ``` | ||
|
|
||
| Robot Framework will generate a standard test report. |
Contributor
There was a problem hiding this comment.
Suggested change
| Robot Framework will generate a standard test report. | |
| Robot Framework will generate a standard test report. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Which issue(s) this PR fixes:
Issue #
What this PR does / why we need it:
Add blog strengthen longhorn with automated end-to-end tests
Special notes for your reviewer:
Additional documentation or context