Skip to content

Conversation

@JimFuller-RedHat
Copy link
Contributor

@JimFuller-RedHat JimFuller-RedHat commented Dec 16, 2025

This PR moves latest filter effectively from pure SQL to performing ranking in rust code (with some sql queries).

This is required mainly to find cpes (on top level externally linked sboms).

Summary by Sourcery

Improve selection of latest SBOM graphs by resolving ancestor SBOMs and ranking results in-memory for latest queries.

Bug Fixes:

  • Correct latest graph resolution for component-based queries by accounting for externally linked ancestor SBOMs and proper per-CPE ranking.

Enhancements:

  • Require analysis service database connections to be Send + Sync and introduce recursive ancestor resolution for SBOM relationships.

Build:

  • Add async-recursion crate to support recursive async SBOM ancestor resolution.

Tests:

  • Expand latest filter test coverage with additional exact/partial name cases and add corresponding CycloneDX SBOM fixtures for TC-3278 scenarios.

@sourcery-ai
Copy link
Contributor

sourcery-ai bot commented Dec 16, 2025

Reviewer's Guide

Refactors how latest SBOM graphs are resolved and ranked by moving the ranking logic from a SQL window function into Rust, adding recursive ancestor resolution, and extending tests/fixtures for latest filter behavior, while also tightening async trait bounds and wiring in new test data for TC-3278.

Sequence diagram for the new latest SBOM graph retrieval flow

sequenceDiagram
    actor Client
    participant AnalysisService
    participant InnerService
    participant Database

    Client->>AnalysisService: retrieve_latest(query, options)
    AnalysisService->>InnerService: load_latest_graphs_query(connection, GraphQuery)

    rect rgb(230,230,250)
        InnerService->>Database: query sbom_node to find matched_sbom_ids
        Database-->>InnerService: matched_sbom_ids (sbom_id, node_id, name, published)

        loop for each matched_sbom
            InnerService->>Database: query package_relates_to_package for top_package_of_sbom
            Database-->>InnerService: top_package_of_sbom

            InnerService->>InnerService: resolve_all_ancestors(sbom_id, node_ref, visited)
            activate InnerService
                loop recursive ancestor resolution
                    InnerService->>Database: resolve_rh_external_sbom_ancestors
                    Database-->>InnerService: direct_ancestors
                    alt new_ancestor_not_visited
                        InnerService->>Database: query package_relates_to_package for ancestor packages
                        Database-->>InnerService: ancestor packages
                        InnerService->>InnerService: recursive resolve_all_ancestors(...)
                    else cycle_detected
                        InnerService-->>InnerService: stop recursion
                    end
                end
            deactivate InnerService

            InnerService->>Database: query sbom_package_cpe_ref for CPEs of top_ancestor_sbom
            Database-->>InnerService: cpe_list
            InnerService->>InnerService: build RankedSbom entries
        end

        InnerService->>InnerService: apply_rank(matched_sboms)
        InnerService-->>InnerService: ranked_sboms with rank
        InnerService->>InnerService: filter rank == 1 and dedup to latest_ids
    end

    InnerService->>Database: load_graphs(connection, latest_ids)
    Database-->>InnerService: graphs
    InnerService-->>AnalysisService: Vec<(Uuid, Arc<PackageGraph>)>
    AnalysisService-->>Client: response with latest graphs
Loading

Class diagram for InnerService latest graph loading and helpers

classDiagram
    class InnerService {
        +load_latest_graphs_query(connection, query) Result~Vec<(Uuid, Arc_PackageGraph)>~, Error
        +load_graphs(connection, sbom_ids) Result~Vec<(Uuid, Arc_PackageGraph)>~, Error
    }

    class RankedSbom {
        +Uuid matched_sbom_id
        +String matched_name
        +Uuid ancestor_sbom_id
        +Uuid cpe_id
        +DateTimeWithTimeZone sbom_date
        +Option~usize~ rank
    }

    class resolve_all_ancestors_fn {
        +resolve_all_ancestors(sbom_sbom_id, sbom_node_ref, connection, visited) async Vec~ResolvedSbom~
    }

    class apply_rank_fn {
        +apply_rank(items Vec~RankedSbom~) Vec~RankedSbom~
    }

    class ResolvedSbom {
        +Uuid sbom_id
        +String node_id
    }

    InnerService --> RankedSbom : creates
    InnerService --> resolve_all_ancestors_fn : calls
    InnerService --> apply_rank_fn : calls
    resolve_all_ancestors_fn --> ResolvedSbom : returns
Loading

File-Level Changes

Change Details Files
Reimplemented load_latest_graphs_query to resolve ancestor SBOMs recursively in Rust and to perform CPE-based ranking client-side instead of via SQL window functions.
  • Added RankedSbom struct and apply_rank helper to emulate SQL RANK partitioned by cpe_id and ordered by sbom_date descending.
  • Introduced resolve_all_ancestors async-recursive helper using resolve_rh_external_sbom_ancestors and package_relates_to_package to walk SBOM ancestry while avoiding cycles via a HashSet of visited IDs.
  • Replaced the previous SeaORM-based window-function query builders (build_ranked_query, find, find_rank_name, JoinCpe, query_all) and GraphQuery pattern-matching paths with a simpler flow: find matching SBOM nodes, compute their top ancestor SBOMs and associated CPEs, rank them in Rust, dedupe rank-1 SBOM IDs, then call load_graphs.
  • Temporarily hard-coded node-name filtering with contains("openssl") for all GraphQuery variants, pending proper wiring of the GraphQuery into the new logic.
  • Added Send + Sync bounds on the generic ConnectionTrait for load_latest_graphs_query to support the async-recursive ancestor resolution.
modules/analysis/src/service/load.rs
Expanded latest-filter integration tests and fixtures for TC-3278 to cover new name and rpm cases tied to newly added SBOM test data.
  • Added new parameterized test cases covering latest name-exact search, additional name-partial searches (including libsoup3), and ensured latest: true paths are exercised.
  • Extended the list of expected CycloneDX fixture files in resolve_rh_variant_latest_filter_tc_3278 to include datagrid and libsoup3 latest/older container and rpm SBOMs.
  • Added corresponding JSON SBOM fixtures for datagrid and libsoup3 under etc/test-data/cyclonedx/rh/latest_filters/TC-3278.
  • Updated Cargo.lock to include the new dependency and test data entries.
modules/analysis/src/endpoints/tests/latest_filters.rs
etc/test-data/cyclonedx/rh/latest_filters/TC-3278/container/datagrid-datagrid-8/latest/binary-2025-12-04-62F0D268C8094C2.json
etc/test-data/cyclonedx/rh/latest_filters/TC-3278/container/datagrid-datagrid-8/latest/image-index-2025-12-04-2BE8E55FCB8946B.json
etc/test-data/cyclonedx/rh/latest_filters/TC-3278/container/datagrid-datagrid-8/latest/product-2025-12-08-6483D4F2E4B1469.json
etc/test-data/cyclonedx/rh/latest_filters/TC-3278/container/datagrid-datagrid-8/older/binary-2025-10-06-85E079C4EC034F1.json
etc/test-data/cyclonedx/rh/latest_filters/TC-3278/container/datagrid-datagrid-8/older/image-index-2025-10-06-BD74B271CC444BA.json
etc/test-data/cyclonedx/rh/latest_filters/TC-3278/container/datagrid-datagrid-8/older/product-2025-10-07-D792D72A114A47A.json
etc/test-data/cyclonedx/rh/latest_filters/TC-3278/rpm/libsoup3/latest/product-2025-12-11-E2251709C91242C.json
etc/test-data/cyclonedx/rh/latest_filters/TC-3278/rpm/libsoup3/latest/rpm-2025-12-10-5609FCE0067D4F6.json
etc/test-data/cyclonedx/rh/latest_filters/TC-3278/rpm/libsoup3/older/product-2025-11-11-CD32282B963F42C.json
etc/test-data/cyclonedx/rh/latest_filters/TC-3278/rpm/libsoup3/older/rpm-2025-10-27-B38A3A44DA644A4.json
Cargo.lock
Adjusted crate configuration and service trait bounds to support the new async-recursive implementation.
  • Added async-recursion dependency to the analysis crate to allow recursive async ancestor resolution.
  • Updated AnalysisService::retrieve_latest to require C: ConnectionTrait + Send + Sync, mirroring the new bounds on load_latest_graphs_query.
  • Added extern crate core; to modules/analysis/src/lib.rs to satisfy new uses coming from refactored code or dependencies.
modules/analysis/Cargo.toml
modules/analysis/src/service/mod.rs
modules/analysis/src/lib.rs

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

sourcery-ai[bot]

This comment was marked as outdated.

@JimFuller-RedHat JimFuller-RedHat changed the title Tc 3278 fix deux Tc 3278 fix latest filtering Dec 16, 2025
@codecov
Copy link

codecov bot commented Dec 19, 2025

Codecov Report

❌ Patch coverage is 90.74733% with 26 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.51%. Comparing base (648d488) to head (d125666).

Files with missing lines Patch % Lines
modules/analysis/src/service/load/rank.rs 87.61% 10 Missing and 16 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2185      +/-   ##
==========================================
+ Coverage   68.24%   68.51%   +0.27%     
==========================================
  Files         376      378       +2     
  Lines       21208    21435     +227     
  Branches    21208    21435     +227     
==========================================
+ Hits        14473    14687     +214     
+ Misses       5868     5863       -5     
- Partials      867      885      +18     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants