Skip to content

Conversation

@manuzhang
Copy link
Member

@manuzhang manuzhang commented Jan 29, 2026

Closes #12414

After Fix:
Google Chrome 2026-01-29 22 58 48

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes an issue where partition information was not properly displayed in job descriptions for the RewritePositionDeleteFilesSparkAction in Spark 4.1. The fix adds a description field to StructProjection to carry partition information through the projection process.

Changes:

  • Modified StructProjection to store and return a description string
  • Updated PartitionUtil.coercePartition() to pass partition description when wrapping PartitionData
  • Modified RewritePositionDeleteFilesSparkAction to extract and use the partition description in job descriptions

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
api/src/main/java/org/apache/iceberg/util/StructProjection.java Added description field and getter method, plus overloaded wrap method to accept description
core/src/main/java/org/apache/iceberg/util/PartitionUtil.java Updated to pass partition description when wrapping PartitionData instances
spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/actions/RewritePositionDeleteFilesSparkAction.java Added helper method to extract partition description for job descriptions

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@manuzhang manuzhang force-pushed the partition-desc branch 2 times, most recently from be1b165 to 5beb72d Compare January 29, 2026 02:55
@manuzhang manuzhang requested a review from Copilot January 29, 2026 02:55
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

this.description = description;
return this;
}

Copy link

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The public method getDescription() lacks documentation explaining what the description represents, when it may be null, and its intended use case. Consider adding a Javadoc comment.

Suggested change
/**
* Returns the optional description associated with the currently wrapped {@link StructLike}.
*
* <p>The description is a human-readable identifier or context string supplied when calling
* {@link #wrap(StructLike, String)}. It may be {@code null}, for example when
* {@link #wrap(StructLike)} is used or when {@code null} is explicitly passed as the
* description.
*
* <p>The description, if present, is propagated to projections created via
* {@link #copyFor(StructLike)}.
*
* @return the description for the wrapped struct, or {@code null} if no description was set
*/

Copilot uses AI. Check for mistakes.
public StructProjection wrap(StructLike newStruct) {
return wrap(newStruct, null);
}

Copy link

Copilot AI Jan 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The overloaded wrap method with description parameter lacks documentation explaining the purpose of the description parameter and when it should be provided versus using the no-argument version.

Suggested change
/**
* Wraps a new {@link StructLike} instance in this projection.
*
* <p>The {@code description} is an optional, human-readable label for the wrapped struct,
* which may be used in diagnostics such as error messages or logging. Callers that do not
* need to associate a description with the struct can use {@link #wrap(StructLike)} instead,
* which is equivalent to passing {@code null} for this parameter.
*
* @param newStruct the struct to wrap with this projection, or {@code null}
* @param description an optional description of the wrapped struct, or {@code null}
* @return this projection, wrapping the given struct
*/

Copilot uses AI. Check for mistakes.
@manuzhang manuzhang marked this pull request as ready for review January 29, 2026 15:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Partition info is not displayed in job description for RewritePositionDeleteFilesSparkAction

1 participant