Skip to content

Comments

SNOW-3006577 Support Remote File Path#2779

Closed
sfc-gh-bli wants to merge 1 commit intoscls-pocfrom
snow-3006577
Closed

SNOW-3006577 Support Remote File Path#2779
sfc-gh-bli wants to merge 1 commit intoscls-pocfrom
snow-3006577

Conversation

@sfc-gh-bli
Copy link

@sfc-gh-bli sfc-gh-bli commented Feb 23, 2026

Pre-review checklist

  • I've confirmed that instructions included in README.md are still correct after my changes in the codebase.
  • I've added or updated automated unit tests to verify correctness of my new code.
  • I've added or updated integration tests to verify correctness of my new code.
  • I've confirmed that my changes are working by executing CLI's commands manually on MacOS.
  • I've confirmed that my changes are working by executing CLI's commands manually on Windows.
  • I've confirmed that my changes are up-to-date with the target branch.
  • I've described my changes in the release notes.
  • I've described my changes in the section below.
  • I've described my changes in the documentation.

Changes description

This change adds support for remote file paths in Spark application submissions. The CLI now accepts files with protocol prefixes (s3://, https://, hdfs://, gs://, local://, file://) and handles them appropriately:

  • Remote files (s3://, https://, hdfs://, gs://, local://) are used directly without uploading to a stage
  • The --snow-file-stage parameter is now only required when uploading local files
  • File paths with protocol prefixes are preserved as-is in the generated Spark query
  • Local files without protocol prefixes continue to be uploaded to the specified stage

The implementation includes:

  • Detection logic to differentiate between local and remote files
  • Path handling that preserves remote URLs while prepending stage paths for local files
  • Validation to prevent absolute local paths (starting with "/")
  • Support for file:// protocol files which are treated as local and uploaded to stages

Copy link
Author

sfc-gh-bli commented Feb 23, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@sfc-gh-bli sfc-gh-bli marked this pull request as ready for review February 23, 2026 22:36
@sfc-gh-bli sfc-gh-bli requested a review from a team as a code owner February 23, 2026 22:36
@sfc-gh-bli sfc-gh-bli marked this pull request as draft February 23, 2026 22:38
@sfc-gh-bli sfc-gh-bli closed this Feb 23, 2026
@sfc-gh-bli sfc-gh-bli deleted the snow-3006577 branch February 23, 2026 22:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant