feat: Add models for rest scan planning

[geruh]

related to #2775 and #2792

Rationale for this change

This PR adds the Pydantic models for the REST catalog server-side scan planning API focusing on the synchronous use cases first.

There's some redundancy here with things like RESTDataFile vs the existing DataFile in the manifest module.
As mentioned in #2792 the manifest logic with encoding/decoding is dependency on Avro. Rather than trying to solve that unification problem upfront and blocking ourselves, I went with separate REST pydantic types for now. The plan is to eventually add conversion methods or a common interface so these can work together with our existing scan task and content file types.

The models and tests here align with the Java implementation.

Are these changes tested?

Yes, added tests and works against POC

Are there any user-facing changes?

No

[kevinjqliu]

LGTM! Agreed with the DataFile serialization problem. We can refactor later on and unify the representation.

For now lets move this forward. Excited to see scan planning in action for pyiceberg 😄

[kevinjqliu]

Suggested change

referenced_data_file: str | None = Field(alias="referenced-data-file", default=None)

PositionDeleteFile doesnt have this
https://github.com/apache/iceberg/blob/0651b8913d27c3b1c9aca4a9609bec521905fb36/open-api/rest-catalog-open-api.yaml#L4450-L4466

[geruh]

Ahh yeah it's another gap in the spec. This was for v3 puffin file.

curl -s -X POST http://localhost:8181/v1/namespaces/default/tables/test_positional_mor_deletes_v3/plan \
    -H "Content-Type: application/json" \
    -d '{"snapshot-id":null,"filter":true,"case-sensitive":true}' | jq

{
  "status": "completed",
  "plan-id": "sync-e1f10f90-b25e-41f4-a089-dcd66a0a2b33",
  "delete-files": [
    {
      "spec-id": 0,
      "content": "position-deletes",
      "file-path": "s3://warehouse/default/test_positional_mor_deletes_v3/data/00000-24-214b8ae2-af1b-4caf-88e9-08481c4161d1-00001-deletes.puffin",
      "file-format": "puffin",
      "partition": [],
      "file-size-in-bytes": 467,
      "record-count": 1,
      "referenced-data-file": "s3://warehouse/default/test_positional_mor_deletes_v3/data/00000-20-309d44b6-a7ee-4309-b629-292375bf84c5-0-00001.parquet",
      "content-offset": 4,
      "content-size-in-bytes": 42
    }
  ],
  "file-scan-tasks": [
    {
      "data-file": {
        "spec-id": 0,
        "content": "data",
        "file-path": "s3://warehouse/default/test_positional_mor_deletes_v3/data/00000-20-309d44b6-a7ee-4309-b629-292375bf84c5-0-00001.parquet",
        "file-format": "parquet",
        "partition": [],
        "file-size-in-bytes": 1009,
        "record-count": 12,
        "split-offsets": [
          4
        ],
        "sort-order-id": 0,
        "first-row-id": 0
      },
      "delete-file-references": [
        0
      ],
      "residual-filter": true
    }
  ]
}

[geruh]

let's omit this for now, as its redundant with the delete-file-references.

[kevinjqliu]

LGTM! Since the changes is isolated to scan_planning.py, lets move this forward to keep experimenting with rest scan plan

[kevinjqliu]

Thanks for the PR @geruh