Update CI workflow matrix #72

joshuay03 · 2025-06-17T00:20:46Z

~~WIP until green.~~ Looks like I need workflow approval so will mark as ready and iterate.

justinhoward

Thanks @joshuay03 ! Can you talk a bit more about the changes you're making and some of those decisions?

justinhoward · 2025-06-17T20:47:06Z

.github/workflows/ci.yml

    services:
      redis:
-        image: redis
+        image: redis:${{ matrix.redis }}


matrix.redis is actually the gem version, not the server version.

Ah yes I see that now, that simplifies things a bunch if we just want to use the latest server version.

we just want to use the latest server version

Should we do the same for opensearch and elasticsearch?

The reason I added the different OpenSearch/ElasticSearch versions was because there were some backwards-compatibility breaks introduced by Elastic around the time they switched their licensing. I had to work around those issues.

justinhoward · 2025-06-17T20:48:09Z

.github/workflows/ci.yml

+        ruby: ['2.3', '2.4', '2.5', '2.6', '2.7', '3.0', '3.1', '3.2', '3.3', '3.4', head, jruby-head, truffleruby-head]
+        redis: ['3', '4', '5', '6', '7', '8']
+        search: [
+          ['opensearch-ruby:1', 'opensearchproject/opensearch:1'],
+          ['opensearch-ruby:2', 'opensearchproject/opensearch:2'],
+          ['opensearch-ruby:3', 'opensearchproject/opensearch:3'],
+          ['elasticsearch:7', 'elasticsearch:7.17.28'],
+          ['elasticsearch:8', 'elasticsearch:8.18.2'],
+          ['elasticsearch:9', 'elasticsearch:9.0.2']
+        ]


This matrix is much too large to actually run. We'll want to pair it down to just a sampling of the things we want to support.

the things we want to support

Yes this is what I don't have context on. So we've got rubies 2.3 - 3.4 (as per the gemspec) and head versions just to catch incompatibilities early, and redis gem majors 3 and 4. Would you like support the last three major versions of opensearch and elasticsearch? Or just two like redis?

FWIW the matrix should be good now, I'm testing it in my fork: joshuay03#1

and redis gem majors 3 and 4

If this is what we're going with, I can update:

faulty/faulty.gemspec

Line 32 in b3921b3

spec.add_development_dependency 'redis', '>= 3.0'

Would you like support the last three major versions of opensearch and elasticsearch? Or just two like redis?

Went with two in my fork to leave capacity in the matrix for cluster enabled runs: joshuay03#4

justinhoward · 2025-06-17T20:49:07Z

.github/workflows/ci.yml

      - uses: ruby/setup-ruby@v1
        with:
-          ruby-version: '2.7'
+          ruby-version: '3.4'


I'd rather keep these on the lower supported versions I think. Is there a reason you want to change it?

Just thought I'd use the latest ruby, happy to revert back to 2.7.

My thinking is that unless it's unreasonable, we should support dev tooling on our supported ruby versions. One way to check if that's the case is by using the older ruby to run it in CI. I think there's a limitation of Rubocop that prevents using the oldest version, and that's how I ended up with 2.7.

Of course, we might consider dropping support for some older ones, but hasn't been important yet since older support has been trivial.

joshuay03 · 2025-06-17T23:55:22Z

.github/workflows/ci.yml

        env:
          discovery.type: single-node
          plugins.security.disabled: ${{ contains(matrix.search[1], 'opensearch') && 'true' || '' }}
+          OPENSEARCH_INITIAL_ADMIN_PASSWORD: ${{secrets.OPENSEARCH_INITIAL_ADMIN_PASSWORD}}


https://opensearch.org/blog/replacing-default-admin-credentials/

Something to keep in mind when setting the secret:

Please re-try with a minimum 8 character password and must contain at least one uppercase letter, one lowercase letter, one digit, and one special character that is strong. Password strength can be tested here: https://lowe.github.io/tryzxcvbn

We can just choose something and stick it in the file directly right? There's no reason it needs to be secret.

joshuay03 · 2025-06-18T03:46:55Z

Can you talk a bit more about the changes you're making and some of those decisions?

My main goal here is to have a matrix that's explicit about all supported dependencies, and excludes incompatibilities as needed, so all permutations are covered.

Edit: I've got everything except for the ruby head runs green in joshuay03#1. Not sure what's going on there but my guess is it's an upstream issue.

justinhoward · 2025-06-18T15:16:17Z

.github/workflows/ci.yml

      - uses: ruby/setup-ruby@v1
        with:
-          ruby-version: '2.7'
+          ruby-version: '3.4'


My thinking is that unless it's unreasonable, we should support dev tooling on our supported ruby versions. One way to check if that's the case is by using the older ruby to run it in CI. I think there's a limitation of Rubocop that prevents using the oldest version, and that's how I ended up with 2.7.

Of course, we might consider dropping support for some older ones, but hasn't been important yet since older support has been trivial.

justinhoward · 2025-06-18T15:17:19Z

faulty.gemspec

  spec.add_development_dependency 'connection_pool', '~> 2.0'
  spec.add_development_dependency 'json'
-  spec.add_development_dependency 'redis', '>= 3.0'
+  spec.add_development_dependency 'redis', '>= 4.0'


Do we need to drop support for 3? That's going to be a breaking change.

3 wasn't running in CI, so I opted to drop it instead of adding it there. But you're right, that would warrant a separate PR.

justinhoward · 2025-06-18T15:22:44Z

spec/patch/elasticsearch_spec.rb


  def build_client(**options)
-    patched_module::Client.new(options)
+    if Gem.loaded_specs['opensearch-ruby']


This looks reasonable, but patched_module should already do the right thing here right?

justinhoward · 2025-06-18T15:27:33Z

.github/workflows/ci.yml

+          ['elasticsearch:8', 'elasticsearch:8.18.2'],
+          ['elasticsearch:9', 'elasticsearch:9.0.2']
+        ]
+        exclude:


This is still going to be a pretty big matrix with a lot of CI time, as opposed to the include approach, that just takes a sample of the combinations of supported versions. I'm going to run CI and see what it looks like.

joshuay03 · 2025-06-30T05:43:01Z

@justinhoward Thank you for reviewing, but I think I'm going to close this and leave CI changes to you, and only focus on redis cluster support (#74). At the very least though, I think it would be good to have the latest rubies in CI.

justinhoward · 2025-06-30T17:59:18Z

Okay, thanks for your time!

joshuay03 force-pushed the update-ci-workflow branch from 588fadf to 7435e18 Compare June 17, 2025 00:23

joshuay03 marked this pull request as ready for review June 17, 2025 00:25

joshuay03 moved this to In Progress / Pending Review in Open Source Jun 17, 2025

joshuay03 added this to Open Source Jun 17, 2025

joshuay03 force-pushed the update-ci-workflow branch from 7435e18 to 6ce1e90 Compare June 17, 2025 08:44

justinhoward reviewed Jun 17, 2025

View reviewed changes

joshuay03 force-pushed the update-ci-workflow branch 2 times, most recently from 98b8633 to 2c25553 Compare June 17, 2025 23:53

joshuay03 commented Jun 17, 2025

View reviewed changes

joshuay03 force-pushed the update-ci-workflow branch 11 times, most recently from 5b39156 to 8cb392e Compare June 18, 2025 03:44

joshuay03 force-pushed the update-ci-workflow branch 9 times, most recently from 2757c7f to cd9f8f1 Compare June 18, 2025 07:19

joshuay03 force-pushed the update-ci-workflow branch 3 times, most recently from 65a752c to 24187fa Compare June 18, 2025 07:31

Update CI workflow matrix

15700fd

joshuay03 force-pushed the update-ci-workflow branch 2 times, most recently from 02ad5f1 to 15700fd Compare June 18, 2025 10:30

justinhoward reviewed Jun 18, 2025

View reviewed changes

joshuay03 mentioned this pull request Jun 19, 2025

redis-clustering support buildkite/faulty#1

Merged

joshuay03 mentioned this pull request Jun 30, 2025

redis-clustering support #74

Open

joshuay03 closed this Jun 30, 2025

joshuay03 moved this from In Progress / Pending Review to Noped in Open Source Jun 30, 2025

Update CI workflow matrix #72

Update CI workflow matrix #72

Uh oh!

Conversation

joshuay03 commented Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

justinhoward left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joshuay03 Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joshuay03 Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joshuay03 Jun 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joshuay03 commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joshuay03 commented Jun 30, 2025

Uh oh!

justinhoward commented Jun 30, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

joshuay03 commented Jun 17, 2025 •

edited

Loading

joshuay03 Jun 17, 2025 •

edited

Loading

joshuay03 Jun 17, 2025 •

edited

Loading

joshuay03 Jun 17, 2025 •

edited

Loading

joshuay03 commented Jun 18, 2025 •

edited

Loading