Skip to content

Introduce a common pagination utility for raw API calls #4731

@pritishpai

Description

@pritishpai

Is there an existing issue for this?

  • I have searched the existing issues

Problem statement

Several places in UCX perform manual pagination over raw API responses, each implementing their own loop:
workspace_access/groups.py — SCIM startIndex/count offset pagination for account groups ([introduced in #PR_NUMBER])
workspace_access/generic.py — page_token/next_page_token cursor pagination for feature store listing
assessment/dashboards.py — iterator-based pagination for Redash queries
Each re-implements the same boilerplate (loop, accumulate, terminate). This makes it easy to introduce bugs (e.g., missing pagination was the root cause of not fetching all Account Groups at a customer) and increases maintenance burden.

Proposed Solution

Create a shared pagination utility and refactor all separate pagination attempts to use the utility.

Additional Context

No response

Metadata

Metadata

Assignees

No one assigned

    Projects

    Status

    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions