Sync with Upstream, Enhance Cloudberry Compatibility, and Pass All Tests

[robertmu]

Summary [sync-with-upstream]

This issue outlines the necessary work to update our gpbackup tool. The primary goals are to synchronize our codebase with the latest upstream gpbackup, resolve critical compatibility issues with Cloudberry, which is based on PostgreSQL 14, and ensure the tool's stability by passing all integration and end-to-end tests. This effort is crucial for meeting customer requirements, particularly for backup from Greenplum and restore to Cloudberry.

Key Problems

1. Database Version Incompatibility: The current main branch is based on legacy gpbackup code, which only supports up to Greenplum 7 (based on PostgreSQL 12). This creates compatibility issues with Cloudberry, which is based on PostgreSQL 14, due to differences in their underlying system tables.
2. Incorrect Cloudberry Handling: The main branch improperly treats Cloudberry as a standard Greenplum 7 instance, failing to account for its unique characteristics.
3. Unsupported Cross-Platform Restore: The main branch does not support the customer requirement of backing up a Greenplum database and successfully restoring it to a Cloudberry instance.
4. Failing Test Suites: The current main branch fails the full integration suite (552 tests) and the end-to-end suite (189 tests), indicating underlying instability.
5. Reported Bugs: Internal documentation has noted several bugs affecting backup and restore operations in customer environments.

Actions Completed

To address the issues above, we have completed the following steps in this phase:

1. Synced with Upstream: We have successfully updated our codebase to the latest version from the official upstream repository greenplum-db/gpbackup-archive.
2. Achieved Full Test Coverage: In coordination with the Cloudberry 2.0 release, we have fixed all related bugs and successfully passed all integration and end-to-end tests. This process was critical for discovering and fixing issues caused by system catalog differences between PostgreSQL 12 and 14, as well as addressing incompatibilities between Greenplum 7 and Cloudberry.
3. Verified Bug Fixes: The bugs identified in our internal documents are confirmed to be resolved in the latest branch.

Next Steps

1. Promote Branch and Release New Version: Rename the sync-with-upstream branch to main. After that, publish a new release (version 1.30.0 or higher) to meet the version requirement for the end-to-end restore tests.
2. Enable CI/CD: Implement a CI/CD pipeline to automate testing and deployment, ensuring the long-term stability and quality of the project.

[tuhaihe]

Thank you for your excellent work! Additionally, we need to cherry-pick some updates for Cloudberry from the main branch. I'll take care of this later.