Oracle perf #4187

akostadinov · 2025-12-10T21:08:53Z

Improve oracle performance by

disabling array fetch for queries with lobs
fetch lobs using piecewise fetch LONG interface
clean-up prepared statements when connection returns to the pool to free memory

Also added a couple of small fixes.

Supersedes #4176

akostadinov · 2025-12-10T21:10:27Z

config/initializers/oracle.rb

 ActiveSupport.on_load(:active_record) do
  if System::Database.oracle?
-    require 'arel/visitors/oracle12_hack'
+    require 'arel/visitors/oracle12_hack' || next # once done, we can skip setup


I believe on application reload we don't need any of this code to be executed again. Using this require statement as a state tracking tool know whether we ever run or not. If your local development environment breaks on code reload, this would be the culprit. Be my guests any time!

akostadinov · 2025-12-10T21:10:43Z

features/support/hooks.rb


 After do
-  ActiveRecord::Base.clear_active_connections!
+  ActiveRecord::Base.connection_handler.clear_active_connections!


avoid DEPRECATION warning

akostadinov · 2025-12-10T21:13:06Z

Gemfile

 # NOTE: Use ENV['DB'] only to install oracle dependencies
 group :oracle do
  oracle = -> { (ENV['ORACLE'] == '1') || ENV.fetch('DATABASE_URL', ENV['DB'])&.start_with?('oracle') }
+  ENV['NLS_LANG'] ||= 'AMERICAN_AMERICA.UTF8' if oracle


Another attempt at setting reliably NLS_LANG before oracle connection is established and only if using oracle. Might be a hack but I'm not sure any other place would be less of a hack and I'm tired of trying to figure out where that other place might be. So here I'm sure it's gonna be set prior loading the oracle gems.

Did we have encoding problems before this?

You can see always a complaint that we fallback to some 7bit ASCII encoding if you don't set this variable separately in your environment. But we have always tried to set a default in the initializer. But that has been too late. So this one should be early enough always.

you can see that line removed from the initializer

qltysh · 2025-12-15T18:55:09Z

❌ 8 blocking issues (8 total)

Tool	Category	Rule	Count
reek	Lint	OracleLobLargeUpdateTest has the variable name 'large_policy_data_v2'	3	❌
reek	Lint	OracleLobLargeUpdateTest#generate_large_policies_config calls '[test_policy].to_json' 2 times	2	❌
rubocop	Style	Incorrect formatting, autoformat by running `qlty fmt`.	1	❌
rubocop	Lint	Models should subclass `ApplicationRecord`.	1	❌
rubocop	Style	Surrounding space missing in default value assignment.	1	❌

@qltysh one-click actions:

Auto-fix formatting (qlty fmt && git push)

jlledom

I would ask you to add better description to complicated PRs like this. The amount of investigation required to understand any single line in that oracle initializer is big enough for the PR to not be reviewable in practice. Now you have claude, you can tell it to explain what you did, and verify it's correct before pasting it in the PR description.

Also, creating one single commit for each conceptually different work would make reviewer's life easier as well.

jlledom · 2025-12-17T09:15:11Z

Gemfile

 # NOTE: Use ENV['DB'] only to install oracle dependencies
 group :oracle do
  oracle = -> { (ENV['ORACLE'] == '1') || ENV.fetch('DATABASE_URL', ENV['DB'])&.start_with?('oracle') }
+  ENV['NLS_LANG'] ||= 'AMERICAN_AMERICA.UTF8' if oracle


Did we have encoding problems before this?

config/initializers/oracle.rb

jlledom · 2025-12-19T13:31:08Z

See the errors in the Oracle cucumbers:

Message:could not obtain a connection from the pool within 5.000 seconds (waited 5.000 seconds); all pooled connections were in use (ActiveRecord::ConnectionTimeoutError)

They might be related to the changes 🤕.

akostadinov · 2025-12-19T14:25:56Z

They might be related to the changes 🤕.

Could be, although I have no idea what might be the case as we don't change the connection management. I wonder if this closing of cursors might take too long sometimes. But then, that can also explain issues we have previously seen with random slowdowns. Will need to try with and without it and then whateever else needed. This is a nightmare.

jlledom · 2026-01-07T09:32:22Z

They might be related to the changes 🤕.

Could be, although I have no idea what might be the case as we don't change the connection management. I wonder if this closing of cursors might take too long sometimes. But then, that can also explain issues we have previously seen with random slowdowns. Will need to try with and without it and then whateever else needed. This is a nightmare.

My thoughts are with you

akostadinov · 2026-01-08T11:58:36Z

config/initializers/oracle.rb

+      def close_and_clear_statements
+        start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        statement_count = @statements&.length || 0
+        @statements&.clear


Some AI explanation why clearing cursors from app end may not be much of a performance penalty because oracle server still caches these for reuse.

Why Oracle Keeps "Closed" Cursors

Oracle's V$OPEN_CURSOR view is somewhat misleadingly named - it shows cursors Oracle is tracking for the session, not just actively open ones. Oracle keeps them for performance:

Session Cursor Cache: When you "close" a cursor from the application side, Oracle often keeps it cached in case you run the same SQL again. This is controlled by the SESSION_CACHED_CURSORS parameter.

Soft vs Hard Close:
- Soft close: Application closes the cursor, but Oracle keeps it in its cache
- Hard close: Oracle actually deallocates it

akostadinov · 2026-01-08T17:19:01Z

@jlledom , now I'm not sure the changes are related to the failures. I can't reproduce the exact errors but other failure types randomly appear.

jlledom · 2026-01-13T13:54:13Z

@jlledom , now I'm not sure the changes are related to the failures. I can't reproduce the exact errors but other failure types randomly appear.

Currently there's only one cuke failing and it's the usual Couldn't find element we see all around. That's not what I saw, I think I saw this one: https://app.circleci.com/pipelines/github/3scale/porta/33121/workflows/ad596262-de41-42cd-a5fa-4461611dd883/jobs/381695/tests but I don't remember well. Anyway, it's gone now, so fine for me.

akostadinov · 2026-01-29T18:25:38Z

Message:could not obtain a connection from the pool within 5.000 seconds

Idk why I didn't check that earlier but I see this error also in master so not related to any of the changes. I think it is more related to not using performance plan maybe.

akostadinov · 2026-01-29T18:38:19Z

@jlledom thank you for the review, I believe besides rebasing I only added the last two commits.

As I said earlier, I see same transient issues without my changes in circleci.

jlledom

I can't understand everything here, but if tests pass I approve.

jlledom · 2026-01-30T09:27:46Z

config/initializers/oracle.rb

+      end
+    end
+
+    ActiveRecord::Base.skip_callback(:update, :after, :enhanced_write_lobs)


Please explain this.

We skip the redundant enhanced_write_lobs callback. It is something never needed by our usage and mot needed for the wider project either especially with the quoting fixes.

jlledom · 2026-01-30T09:28:15Z

config/initializers/oracle.rb

+      SQL_UTF8_CHUNK_CHARS = 8191 # (32767÷4), 4 bytes max character; 1000 without MAX_STRING_SIZE=EXTENDED
+      BLOB_INLINE_LIMIT = 16383 # (32767÷2) 2000 without MAX_STRING_SIZE=EXTENDED
+      PLSQL_BASE64_CHUNK_SIZE = 24_573


Why this values?

we can fit only so many real bytes encoded in base64 within the 32767 bytes value limit

Now I don't remember if I tested this empirically that any more bytes can be processed, but too tired now to go back to it anymore. 24k is enough of a chunk size to not impose a significant overhead. Moreover this is something people shouldn't rely on. This is only for huge values within inline queries. And I see no practical reason one to use it except in dev mode, to see their SQL being run. In which case using huge values will not be productive anyway.

Hope it makes sense.

jlledom · 2026-01-30T09:40:26Z

config/initializers/oracle.rb

+
+      # Generate a scalar subquery with PL/SQL function to build large BLOBs.
+      # Uses DBMS_LOB.WRITEAPPEND with base64-encoded chunks for efficiency.
+      # Testing showed hextoraw() unusable for being more than 100x slower.


Then why are you using hextoraw() instead of this in other scenarios?

For a single value, it doesn't show noticeable delay. If used with big data within PL/SQL then it becomes a problem. Since it was already working, I didn't find value in checking whether the base64 functions can be used outside the PL/SQL context. And it is better to avoid the complicated PL/SQL for small values using either function. If not for anything else, just because one uses to_sql for readability. And seeing a very popular standard Oracle SQL function is much more readable than the ugly PL/SQL insertion.

akostadinov · 2026-01-30T14:19:49Z

if tests pass I approve.

In fact tests passed without rerun this time.

I'm gonna merge the PR and update with the even better implementation if oracle support provides a solution later.

akostadinov commented Dec 10, 2025

View reviewed changes

akostadinov mentioned this pull request Dec 10, 2025

THREESCALE-12061: Monkey patch ruby-oci8 to fix memory bloat (master) #4176

Closed

jlledom reviewed Dec 17, 2025

View reviewed changes

jlledom previously approved these changes Dec 18, 2025

View reviewed changes

akostadinov dismissed jlledom’s stale review via c4f6939 January 8, 2026 10:23

akostadinov force-pushed the oracle-perf branch from c4f6939 to 6768407 Compare January 8, 2026 10:37

akostadinov commented Jan 8, 2026

View reviewed changes

akostadinov force-pushed the oracle-perf branch from 6768407 to 0044b2f Compare January 8, 2026 14:57

jlledom previously approved these changes Jan 13, 2026

View reviewed changes

akostadinov dismissed jlledom’s stale review via 547356e January 23, 2026 18:20

akostadinov force-pushed the oracle-perf branch 2 times, most recently from 547356e to ec0ab9e Compare January 26, 2026 17:52

akostadinov added 3 commits January 29, 2026 11:08

remove DEPRECATION

290f8cc

oracle performance

9297bb6

proper fix for writing lobs

0f1b5da

akostadinov force-pushed the oracle-perf branch from ec0ab9e to ca93f51 Compare January 29, 2026 09:09

akostadinov added 2 commits January 29, 2026 20:34

reliable fetch, inline quoting and tests

8172bc1

installing oracle oci on Fedora 43

c698c64

akostadinov force-pushed the oracle-perf branch from 5abae26 to c698c64 Compare January 29, 2026 18:35

akostadinov requested a review from jlledom January 29, 2026 18:36

jlledom approved these changes Jan 30, 2026

View reviewed changes

akostadinov merged commit b9f3332 into master Jan 30, 2026
12 of 18 checks passed

akostadinov deleted the oracle-perf branch January 30, 2026 14:21

Oracle perf #4187

Oracle perf #4187

Uh oh!

Conversation

akostadinov commented Dec 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

qltysh bot commented Dec 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jlledom left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jlledom commented Dec 19, 2025

Uh oh!

akostadinov commented Dec 19, 2025

Uh oh!

jlledom commented Jan 7, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akostadinov commented Jan 8, 2026

Uh oh!

jlledom commented Jan 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

akostadinov commented Jan 29, 2026

Uh oh!

akostadinov commented Jan 29, 2026

Uh oh!

jlledom left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akostadinov commented Jan 30, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

akostadinov commented Dec 10, 2025 •

edited

Loading

qltysh bot commented Dec 15, 2025 •

edited

Loading

jlledom commented Jan 13, 2026 •

edited

Loading