Skip to content

CSPL-4358 Split BusConfiguration CR into 2 CRs - Queue and ObjectStorage #1643

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kasiakoziol wants to merge 27 commits into
base: CSPL-3551-ingestion-cr
Choose a base branch
from
+3,457 −2,283
Open

CSPL-4358 Split BusConfiguration CR into 2 CRs - Queue and ObjectStorage #1643

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
c1ad439
CSPL-4358 Splitting BusConfiguration into Bus and LargeMessageStore
kasiakoziol Dec 12, 2025
cb8daf2
CSPL-4358 Update docs
kasiakoziol Dec 12, 2025
61c0387
CSPL-4358 Addressing comments
kasiakoziol Dec 12, 2025
3eb98f7
CSPL-4358 Adding more validations
kasiakoziol Dec 15, 2025
254cbf0
CSPL-4360 Secret reference added for Bus CR
kasiakoziol Dec 16, 2025
f992c40
CSPL-4360 Fix failing tests
kasiakoziol Dec 16, 2025
143dbe0
CSPL-4360 Add Splunk restart
kasiakoziol Dec 16, 2025
3c7b2d7
CSPL-4360 Fix failing tests
kasiakoziol Dec 16, 2025
e4e083a
CSPL-4360 Fix failing tests
kasiakoziol Dec 17, 2025
3cb9148
CSPL-4360 Fix errors with failing validation on status
kasiakoziol Dec 17, 2025
ba73a87
CSPL-4358 Rename Bus to Queue
kasiakoziol Dec 19, 2025
b6f5b0b
CSPL-4358 Rename LargeMessageStore to ObjectStorage
kasiakoziol Dec 19, 2025
607632f
CSPL-4358 Making region authRegion and optional, simplifying endpoint
kasiakoziol Dec 19, 2025
30a82c6
Merge branch 'feature/CSPL-4358-split-bus-into-2-crs' into feature/CS…
kasiakoziol Dec 19, 2025
fafed27
CSPL-4360 Fixing tests after merge
kasiakoziol Dec 19, 2025
e0a10ba
CSPL-4360 Fix validation that fails for status
kasiakoziol Dec 19, 2025
155b21a
CSPL-4360 Fix failing to get k8s secret
kasiakoziol Dec 19, 2025
f8afd5a
CSPL-4360 Fix failing integ and helm tests
kasiakoziol Dec 22, 2025
47d1a35
CSPL-4360 Fixing failing tests due to incorrect secret ref
kasiakoziol Jan 8, 2026
532ca28
CSPL-4360 Addressing comments
kasiakoziol Jan 13, 2026
5c6e786
CSPL-4360 Addressing secret value change and removing redundant contr…
kasiakoziol Jan 13, 2026
42dc8e8
CSPL-4360 Update of docs, helm tests and validations
kasiakoziol Jan 13, 2026
c50984b
CSPL-4360 Add secret watch and fix controller tests
kasiakoziol Jan 13, 2026
e81280c
CSPL-4360 Update docs
kasiakoziol Jan 28, 2026
d29aed1
CSPL-4360 Restart Splunk when SA changed
kasiakoziol Jan 29, 2026
52ff612
Merge pull request #1649 from splunk/feature/CSPL-4360-secret-ref-ing…
kasiakoziol Feb 9, 2026
4ebe9d8
Addressing comments
kasiakoziol Feb 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/arm-AL2023-build-test-push-workflow-AL2023.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,8 @@ jobs:
DEPLOYMENT_TYPE: ""
ARM64: "true"
GRAVITON_TESTING: "true"
AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID: ${{ secrets.AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID }}
AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY: ${{ secrets.AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY }}
steps:
- name: Chekcout code
uses: actions/checkout@v2
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/arm-AL2023-int-test-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,8 @@ jobs:
DEPLOYMENT_TYPE: ""
ARM64: "true"
GRAVITON_TESTING: "true"
AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID: ${{ secrets.AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID }}
AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY: ${{ secrets.AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY }}
steps:
- name: Set Test Cluster Nodes and Parallel Runs
run: >-
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/arm-RHEL-build-test-push-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,8 @@ jobs:
DEPLOYMENT_TYPE: ""
ARM64: "true"
GRAVITON_TESTING: "true"
AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID: ${{ secrets.AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID }}
AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY: ${{ secrets.AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY }}
steps:
- name: Set Test Cluster Nodes and Parallel Runs
run: >-
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/arm-RHEL-int-test-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,8 @@ jobs:
DEPLOYMENT_TYPE: ""
ARM64: "true"
GRAVITON_TESTING: "true"
AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID: ${{ secrets.AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID }}
AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY: ${{ secrets.AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY }}
steps:
- name: Set Test Cluster Nodes and Parallel Runs
run: >-
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/arm-Ubuntu-build-test-push-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,8 @@ jobs:
DEPLOYMENT_TYPE: ""
ARM64: "true"
GRAVITON_TESTING: "true"
AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID: ${{ secrets.AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID }}
AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY: ${{ secrets.AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY }}
steps:
- name: Chekcout code
uses: actions/checkout@v2
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/arm-Ubuntu-int-test-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,8 @@ jobs:
DEPLOYMENT_TYPE: ""
ARM64: "true"
GRAVITON_TESTING: "true"
AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID: ${{ secrets.AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID }}
AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY: ${{ secrets.AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY }}
steps:
- name: Set Test Cluster Nodes and Parallel Runs
run: >-
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/build-test-push-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,8 @@ jobs:
EKS_SSH_PUBLIC_KEY: ${{ secrets.EKS_SSH_PUBLIC_KEY }}
CLUSTER_WIDE: "true"
DEPLOYMENT_TYPE: ""
AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID: ${{ secrets.AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID }}
AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY: ${{ secrets.AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY }}
steps:
- name: Chekcout code
uses: actions/checkout@v2
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/distroless-build-test-push-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,8 @@ jobs:
EKS_SSH_PUBLIC_KEY: ${{ secrets.EKS_SSH_PUBLIC_KEY }}
CLUSTER_WIDE: "true"
DEPLOYMENT_TYPE: ""
AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID: ${{ secrets.AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID }}
AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY: ${{ secrets.AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY }}
steps:
- name: Chekcout code
uses: actions/checkout@v2
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/distroless-int-test-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ jobs:
S3_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
CLUSTER_WIDE: "true"
DEPLOYMENT_TYPE: ""
AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID: ${{ secrets.AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID }}
AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY: ${{ secrets.AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY }}
steps:
- name: Set Test Cluster Nodes and Parallel Runs
run: >-
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/helm-test-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,8 @@ jobs:
HELM_REPO_PATH: "../../../../helm-chart"
INSTALL_OPERATOR: "true"
TEST_VPC_ENDPOINT_URL: ${{ secrets.TEST_VPC_ENDPOINT_URL }}
AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID: ${{ secrets.AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID }}
AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY: ${{ secrets.AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY }}
steps:
- uses: chrisdickinson/setup-yq@3d931309f27270ebbafd53f2daee773a82ea1822
- name: Checking YQ installation
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/int-test-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,8 @@ jobs:
S3_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
CLUSTER_WIDE: "true"
DEPLOYMENT_TYPE: ""
AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID: ${{ secrets.AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID }}
AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY: ${{ secrets.AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY }}
steps:
- name: Set Test Cluster Nodes and Parallel Runs
run: >-
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/manual-int-test-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ jobs:
PRIVATE_REGISTRY: ${{ secrets.ECR_REPOSITORY }}
S3_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
CLUSTER_WIDE: ${{ github.event.inputs.CLUSTER_WIDE }}
AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID: ${{ secrets.AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID }}
AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY: ${{ secrets.AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY }}
steps:
- name: Set Test Cluster Nodes and Parallel Runs
run: >-
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/namespace-scope-int-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,8 @@ jobs:
PRIVATE_REGISTRY: ${{ secrets.ECR_REPOSITORY }}
S3_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
CLUSTER_WIDE: "false"
AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID: ${{ secrets.AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID }}
AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY: ${{ secrets.AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY }}
steps:
- name: Set Test Cluster Nodes and Parallel Runs
run: >-
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/nightly-int-test-workflow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,8 @@ jobs:
PRIVATE_REGISTRY: ${{ secrets.ECR_REPOSITORY }}
S3_REGION: ${{ secrets.AWS_DEFAULT_REGION }}
CLUSTER_WIDE: "true"
AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID: ${{ secrets.AWS_INDEX_INGEST_SEP_ACCESS_KEY_ID }}
AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY: ${{ secrets.AWS_INDEX_INGEST_SEP_SECRET_ACCESS_KEY }}
steps:
- name: Set Test Cluster Nodes and Parallel Runs
run: >-
Expand Down
11 changes: 10 additions & 1 deletion PROJECT
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,16 @@ resources:
controller: true
domain: splunk.com
group: enterprise
kind: BusConfiguration
kind: Queue
path: github.com/splunk/splunk-operator/api/v4
version: v4
- api:
crdVersion: v1
namespaced: true
controller: true
domain: splunk.com
group: enterprise
kind: ObjectStorage
path: github.com/splunk/splunk-operator/api/v4
version: v4
version: "3"
134 changes: 0 additions & 134 deletions api/v4/busconfiguration_types.go
View file
Open in desktop

This file was deleted.

19 changes: 15 additions & 4 deletions api/v4/indexercluster_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,20 @@ const (
IndexerClusterPausedAnnotation = "indexercluster.enterprise.splunk.com/paused"
)

// +kubebuilder:validation:XValidation:rule="has(self.queueRef) == has(self.objectStorageRef)",message="queueRef and objectStorageRef must both be set or both be empty"
// +kubebuilder:validation:XValidation:rule="!has(oldSelf.queueRef) || self.queueRef == oldSelf.queueRef",message="queueRef is immutable once created"
// +kubebuilder:validation:XValidation:rule="!has(oldSelf.objectStorageRef) || self.objectStorageRef == oldSelf.objectStorageRef",message="objectStorageRef is immutable once created"
// IndexerClusterSpec defines the desired state of a Splunk Enterprise indexer cluster
type IndexerClusterSpec struct {
CommonSplunkSpec `json:",inline"`

// Bus configuration reference
BusConfigurationRef corev1.ObjectReference `json:"busConfigurationRef,omitempty"`
// +optional
// Queue reference
QueueRef corev1.ObjectReference `json:"queueRef"`

// +optional
// Object Storage reference
ObjectStorageRef corev1.ObjectReference `json:"objectStorageRef"`

// Number of search head pods; a search head cluster will be created if > 1
Replicas int32 `json:"replicas"`
Expand Down Expand Up @@ -115,8 +123,11 @@ type IndexerClusterStatus struct {
// Auxillary message describing CR status
Message string `json:"message"`

// Bus configuration
BusConfiguration BusConfigurationSpec `json:"busConfiguration,omitempty"`
// Credential secret version to track changes to the secret and trigger rolling restart of indexer cluster peers when the secret is updated
CredentialSecretVersion string `json:"credentialSecretVersion,omitempty"`

// Service account to track changes to the service account and trigger rolling restart of indexer cluster peers when the service account is updated
ServiceAccount string `json:"serviceAccount,omitempty"`
}

// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
Expand Down
20 changes: 16 additions & 4 deletions api/v4/ingestorcluster_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,19 +28,28 @@ const (
IngestorClusterPausedAnnotation = "ingestorcluster.enterprise.splunk.com/paused"
)

// +kubebuilder:validation:XValidation:rule="self.queueRef == oldSelf.queueRef",message="queueRef is immutable once created"
// +kubebuilder:validation:XValidation:rule="self.objectStorageRef == oldSelf.objectStorageRef",message="objectStorageRef is immutable once created"
// IngestorClusterSpec defines the spec of Ingestor Cluster
type IngestorClusterSpec struct {
// Common Splunk spec
CommonSplunkSpec `json:",inline"`

// Number of ingestor pods
// +kubebuilder:validation:Minimum=3
// +kubebuilder:default=3
Replicas int32 `json:"replicas"`

// Splunk Enterprise app repository that specifies remote app location and scope for Splunk app management
AppFrameworkConfig AppFrameworkSpec `json:"appRepo,omitempty"`

// Bus configuration reference
BusConfigurationRef corev1.ObjectReference `json:"busConfigurationRef"`
// +kubebuilder:validation:Required
// Queue reference
QueueRef corev1.ObjectReference `json:"queueRef"`

// +kubebuilder:validation:Required
// Object Storage reference
ObjectStorageRef corev1.ObjectReference `json:"objectStorageRef"`
}

// IngestorClusterStatus defines the observed state of Ingestor Cluster
Expand Down Expand Up @@ -69,8 +78,11 @@ type IngestorClusterStatus struct {
// Auxillary message describing CR status
Message string `json:"message"`

// Bus configuration
BusConfiguration BusConfigurationSpec `json:"busConfiguration,omitempty"`
// Credential secret version to track changes to the secret and trigger rolling restart of indexer cluster peers when the secret is updated
CredentialSecretVersion string `json:"credentialSecretVersion,omitempty"`

// Service account to track changes to the service account and trigger rolling restart of indexer cluster peers when the service account is updated
ServiceAccount string `json:"serviceAccount,omitempty"`
}

// +kubebuilder:object:root=true
Expand Down
Loading
Loading