HIVE-1939: Add local e2e test support #2818

huangmingxia · 2025-12-17T15:40:22Z

Add Makefile target test-e2e-local and refactor local-e2e-test.sh to use common setup functions with interactive prompts and auto-detection for easier local testing.

openshift-ci · 2025-12-17T15:40:26Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

openshift-ci-robot · 2025-12-17T15:40:27Z

@huangmingxia: This pull request references HIVE-1939 which is a valid jira issue.

Details

In response to this:

Add Makefile target test-e2e-local and refactor local-e2e-test.sh to use common setup functions with interactive prompts and auto-detection for easier local testing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2025-12-17T15:40:39Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: huangmingxia
Once this PR has been reviewed and has the lgtm label, please assign 2uasimojo for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Add Makefile target test-e2e-local and refactor local-e2e-test.sh to auto-detect configuration and provide sensible defaults.

2uasimojo

I got partway through this before realizing it's a Draft and you probably don't want me looking at it yet :)

...But it was a useful exercise because I think I realized what I want the user experience to be. Rather than a whole separate script that sets up all the env vars and runs e2e in a subprocess, what if the UX was more like:

$ source hack/e2e_env_setup.sh
$ make test-e2e  # or `make test-e2e-pool`

Lots of the cool interactive discovery you've done here can be reused. But then the other advantage is that I can run e2e twice without having to redo all of that.

As part of this work, e2e-common.sh could start with an env check to make sure all the vars it expects are set.

2uasimojo · 2025-12-18T22:09:08Z

hack/local-e2e-test.sh

+# Local e2e test script - easier to run e2e tests locally
+# This script provides sensible defaults and can be customized via environment variables
+#
+# Usage:


This is nice, but can we make it the output of the command if help is requested? Normally I would say that would be the case if an argument like -?, -h, --help, or help was provided; but in this case it seems like this script normally accepts no arguments, so we could just print it if any argument is given.

2uasimojo · 2025-12-18T22:12:20Z

hack/local-e2e-common.sh

+                if command -v oc &> /dev/null; then
+                    DETECTED_RELEASE_IMAGE=$(KUBECONFIG="$KUBECONFIG" oc get clusterversion version -o=jsonpath='{.status.desired.image}' 2>/dev/null || true)
+                elif command -v kubectl &> /dev/null; then
+                    DETECTED_RELEASE_IMAGE=$(KUBECONFIG="$KUBECONFIG" kubectl get clusterversion version -o=jsonpath='{.status.desired.image}' 2>/dev/null || true)
+                fi


We're going to need oc in the actual test suite no matter what, so we may as well check for it once, at the top level of this library (or at least via a func that's called from local-e2e-test.sh), and fail if it's not found.

(It may be possible to run e2e with kubectl standing in for oc -- I haven't tried. If that's the case, then the suggested func could be a little cleverer and, in the absence of oc but the presence of kubectl, run something like alias oc=kubectl.)

Thank you! If oc is unavailable in the local environment, kubectl can also be used for some queries. However, the ideal situation is still to use oc.

2uasimojo · 2025-12-18T22:16:52Z

hack/local-e2e-common.sh

+        fi
+    else
+        # Auto-detect: use default kubeconfig location if KUBECONFIG is not set
+        DEFAULT_KUBECONFIG="${HOME}/.kube/config"


I appreciate that we're trying to make this as seamless as possible, but since this test is not guaranteed to be nondestructive, I don't think we should try so hard to discover a default.

In fact, I'm not super convinced we should even use whatever $KUBECONFIG is currently set in the env -- we should consider requiring it to be provided explicitly, either via a specially-named env var ($HIVE_E2E_HUB_KUBECONFIG?) or a CLI argument (--hub-kubeconfig). If they want to use their $KUBECONFIG they can always do so (--hub-kubeconfig $KUBECONFIG).

Yes, I can pass the kubeconfig by adding the --hub-kubeconfig parameter. I actually added this option at first, but then removed it later. The reason I removed it was that I was planning for maximum automation, similar to how it's done in CI, pass the cloud parameter to automatically trigger the subsequent e2e tests.
However, after thinking about it again, I now see your point. It makes sense to include these parameters, as well as --help. Thanks a lot!

2uasimojo · 2025-12-18T22:17:54Z

hack/local-e2e-common.sh

+
+# Check cluster connectivity using oc or kubectl
+# Requires KUBECONFIG to be set (should be called after setup_kubeconfig)
+function check_cluster_connectivity() {


Nit: cluster is ambiguous; suggest using hub or spoke, as appropriate, throughout.

Thanks! Here should verify whether the provided hub cluster is available.

2uasimojo · 2025-12-18T22:22:52Z

hack/local-e2e-common.sh

+# Setup HIVE_IMAGE: auto-detect, prompt, or use existing value
+# Must export the variable so e2e-common.sh and child processes can access it
+function setup_hive_image() {
+    if [ -z "$HIVE_IMAGE" ]; then


Here's where I think we could default to $IMG since that's what the rest of the Makefile uses.

(While I'm here, I noticed that Makefile var defaults to something that's almost certainly obsolete for everyone. Can we get rid of that?)

(Later) I see that e2e-common.sh is doing this the other way around... but I think it would still work to chicken-egg-chicken the var from here, if set.

Got it - I will check this.

2uasimojo · 2025-12-18T22:28:59Z

hack/local-e2e-common.sh

+function setup_hive_image() {
+    if [ -z "$HIVE_IMAGE" ]; then
+        echo "HIVE_IMAGE not set, attempting to get latest tag from quay.io..."
+        if ! command -v curl &> /dev/null || ! command -v jq &> /dev/null; then


Again, jq is required by e2e-test.sh, so let's add it to the "things to check for up front" function mentioned elsewhere.

2uasimojo · 2025-12-18T22:32:48Z

hack/local-e2e-common.sh

+                exit 1
+            fi
+        else
+            HIVE_IMAGE_REPO="${HIVE_IMAGE_REPO:-redhat-user-workloads/crt-redhat-acm-tenant/hive-operator/hive}"


This is kind of cool -- if I set $HIVE_IMAGE_REPO to point to my personal one, it'll still go out and try to discover the latest image...

...but only if I built an index, which I normally don't. I can't think of a great way to discover that the most recent image in my personal repo is a legit hive container image... but again, we may be trying too hard here. Perfect is the enemy of done :)

in my personal repo is a legit hive container image.

Thank you -
I'm not sure if the information retrieved by this command can be used for verification, Entrypoint is /opt/services/manager.
I'll go learn how to verify whether the provided Hive image is a legit Hive container image.

podman inspect quay.io/redhat-user-workloads/crt-redhat-acm-tenant/hive-operator/hive:hive-on-push-ml47k-build-image-index | jq '.[0].Config.Entrypoint' [ "/opt/services/manager" ]

% podman run --rm quay.io/skopeo/stable:latest inspect --override-arch amd64 --override-os linux docker://quay.io/redhat-user-workloads/crt-redhat-acm-tenant/hive-operator/hive:hive-on-push-ml47k-build-image-index' ... "Labels": { "architecture": "x86_64", "build-date": "2025-12-18T23:04:10Z", "com.redhat.component": "hive-rhel9", "com.redhat.license_terms": "https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI", "cpe": "cpe:/a:redhat:enterprise_linux:9::appstream", "description": "Hive is an operator which runs as a service on top of Kubernetes/OpenShift. The Hive service can be used to provision and perform initial configuration of OpenShift clusters", "distribution-scope": "public", "io.buildah.version": "1.41.4", "io.k8s.description": "Hive is an operator which runs as a service on top of Kubernetes/OpenShift. The Hive service can be used to provision and perform initial configuration of OpenShift clusters", "io.k8s.display-name": "hive-operator", "io.openshift.expose-services": "", "io.openshift.tags": "cluster,management,provision", "maintainer": "Red Hat, Inc.", "name": "hive", "org.opencontainers.image.created": "2025-12-18T23:04:10Z", "org.opencontainers.image.revision": "b061ef0a98a594fa2099647cf228495566317c5f", "release": "1", "summary": "API driven OpenShift 4 cluster provisioning and management", "url": "https://github.com/openshift/hive", "vcs-ref": "b061ef0a98a594fa2099647cf228495566317c5f", "vcs-type": "git", "vendor": "Red Hat, Inc.", "version": "1" }, "Architecture": "amd64", "Os": "linux", "Layers": [ "sha256:46a9484471e55e0e501c08ff903616330af0505ba749ef70e8c87e103e07844a", "sha256:b8daaea900e82c4617ef1a3b0a24c2a1597570ec225d03bf40feeff2173dddbb" ], "LayersData": [ { "MIMEType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "Digest": "sha256:46a9484471e55e0e501c08ff903616330af0505ba749ef70e8c87e103e07844a", "Size": 40040759, "Annotations": null }, { "MIMEType": "application/vnd.docker.image.rootfs.diff.tar.gzip", "Digest": "sha256:b8daaea900e82c4617ef1a3b0a24c2a1597570ec225d03bf40feeff2173dddbb", "Size": 563249031, "Annotations": null } ], "Env": [ "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "container=oci", "SMDEV_CONTAINER_OFF=1", "HOME=/home/hive" ] }

2uasimojo · 2025-12-18T22:39:17Z

hack/local-e2e-common.sh

+            fi
+        fi
+    fi
+    export HIVE_IMAGE="$HIVE_IMAGE"


nit: I believe this can just be spelled

Suggested change

export HIVE_IMAGE="$HIVE_IMAGE"

export HIVE_IMAGE

2uasimojo · 2025-12-18T22:39:37Z

hack/local-e2e-common.sh

+                fi
+            fi
+        fi
+    fi


This may be more of a stylistic preference, but generally when I see this many layers of indent it's a signal that the code could be simplified to reduce branching and improve readability/maintainability. Applies throughout. (Did an AI help you write this?)

Yes - I provided the AI with the command to extract the image, and it generated the corresponding function :)

2uasimojo · 2025-12-18T22:41:28Z

hack/local-e2e-common.sh

+                echo "Error: HIVE_IMAGE cannot be empty" >&2
+                exit 1
+            fi
+        else


If we're accepting an input or using a default, do we want to go check that the thing exists in quay, and that we have access to it?

(Note: If I had been writing this, I probably wouldn't have gone to such great lengths to cover every possible case. But since you've got a path that queries the quay API anyway...)

Thank you, that makes a lot of sense - I didn't think to check whether we have access to this image.
I found the validate_image function logic from bundle-gen.py, maybe we can use similar logic, and I'll check it again.

huangmingxia · 2025-12-19T18:30:41Z

@2uasimojo I really appreciate your help with the review!! It's been incredibly helpful. I will update this draft according to your comments and add the e2e-pool test configuration.
When I ran the tests locally, I set up a simple environment configuration and had AI help me quickly organize it:) I haven't had the chance to review it thoroughly, so I left it as a draft and saved it here.

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 17, 2025

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Dec 17, 2025

Add local e2e test support

8bf5e8d

Add Makefile target test-e2e-local and refactor local-e2e-test.sh to auto-detect configuration and provide sensible defaults.

huangmingxia force-pushed the HIVE-1939 branch from ca0c0f7 to 8bf5e8d Compare December 17, 2025 15:46

huangmingxia changed the title ~~HIVE-1939: Add interactive local e2e test support~~ HIVE-1939: Add local e2e test support Dec 17, 2025

2uasimojo reviewed Dec 18, 2025

View reviewed changes

HIVE-1939: Add local e2e test support #2818

Are you sure you want to change the base?

HIVE-1939: Add local e2e test support #2818

Uh oh!

Conversation

huangmingxia commented Dec 17, 2025

Uh oh!

openshift-ci bot commented Dec 17, 2025

Uh oh!

openshift-ci-robot commented Dec 17, 2025 • edited by openshift-ci bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci bot commented Dec 17, 2025

Uh oh!

2uasimojo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

huangmingxia commented Dec 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

openshift-ci-robot commented Dec 17, 2025 •

edited by openshift-ci bot

Loading