Skip to content

Add WinSCP documentation for Kubeflow file transfers#50

Draft
Copilot wants to merge 2 commits intomainfrom
copilot/update-kubeflow-docs-winscp
Draft

Add WinSCP documentation for Kubeflow file transfers#50
Copilot wants to merge 2 commits intomainfrom
copilot/update-kubeflow-docs-winscp

Conversation

Copy link

Copilot AI commented Feb 3, 2026

Adds WinSCP as a documented file transfer solution for moving datasets, model artifacts, and checkpoints between local machines and Kubernetes persistent volumes in Kubeflow environments. Addresses common Windows enterprise use case for data transfer to PVCs.

Changes

  • Data Preparation tools - Added WinSCP to tools list alongside Spark, Dask, Feast
  • New Section 8: File Transfer & Data Management Tools - Comprehensive WinSCP documentation including:
    • Kubeflow-specific use cases (dataset uploads, model artifact transfers, notebook storage access)
    • Setup workflow for K8s environments (SSH → bastion → PVC mount points)
    • Integration diagram showing local machine → bastion → PVC → pod data flow
    • Alternatives (kubectl cp, rsync, cloud CLIs)
  • Resources - WinSCP official docs and Kubernetes file transfer guide links
  • Section renumbering - Previous sections 8-9 renumbered to 9-10

Context

Enterprise Kubeflow users on Windows frequently need secure file transfer to Kubernetes volumes. WinSCP provides GUI-based SFTP/SCP access for transferring training data and retrieving results without kubectl or cloud CLI expertise.

Original prompt

Objective

Update the Kubeflow Ecosystem documentation to include WinSCP as a file transfer and data management tool.

Requirements

Update the KUBEFLOW_ECOSYSTEM_OVERVIEW.md file to integrate WinSCP in the following areas:

1. Add WinSCP to Data Preparation Section

In the "Detailed AI Lifecycle Stages" > "Data Preparation" section, add WinSCP to the tools list:

### Data Preparation
- **Purpose:** Ingest raw data, perform feature engineering to extract ML features for the offline feature store, and prepare training data for model development.
- **Tools:** Spark, Dask, Flink, Ray, Kubeflow Spark Operator, Feast, **WinSCP (for secure file transfers)**
- **Kubeflow Integration:** Spark Operator, Feast

2. Add New Section: File Transfer & Data Management Tools

Add this new section after "## 7. Extensibility & Flexibility":

## 8. File Transfer & Data Management Tools

### WinSCP
**WinSCP** is a popular SFTP/SCP client for Windows that enables secure file transfer between your local machine and Kubernetes persistent volumes or remote storage.

**Use Cases in Kubeflow:**
- Transfer training datasets to Kubernetes persistent volumes
- Upload/download model artifacts and checkpoints
- Access and manage files in Kubeflow Notebook persistent storage
- Move feature data to/from offline feature stores
- Backup and retrieve experiment results

**Key Features:**
- Graphical and command-line interface
- Support for SFTP, SCP, FTP, and WebDAV protocols
- Synchronization and scripting capabilities
- Integration with PuTTY for SSH key management

**Setup for Kubeflow:**
1. Configure SSH access to your Kubernetes nodes or bastion host
2. Mount persistent volumes to accessible paths
3. Use WinSCP to connect via SFTP/SCP
4. Transfer data files to PVC mount points

**Integration Diagram (ASCII):**

[Local Machine] <--WinSCP/SFTP--> [K8s Node/Bastion] --> [PVC] --> [Kubeflow Pods]
|
[Training Data]
[Model Artifacts]
[Notebooks]


**Alternatives:**
- `kubectl cp` - Native Kubernetes file copy
- `rsync` - Command-line synchronization
- Cloud storage CLIs (gsutil, aws s3, az storage)
- Web-based file managers in Jupyter notebooks

3. Update Resources Section

Update the "Resources & References" section (now renumbered to ## 10) to include:

### File Transfer Tools
- [WinSCP Official Site](https://winscp.net/)
- [WinSCP Documentation](https://winscp.net/eng/docs/start)
- [Kubernetes File Transfer Guide](https://kubernetes.io/docs/reference/kubectl/cheatsheet/#copy-files-and-directories-to-and-from-containers)

4. Update Table of Contents Flow

Since we're adding a new section, update the section numbering:

  • Section 8: File Transfer & Data Management Tools (NEW)
  • Section 9: Production vs Development Phases (previously 8)
  • Section 10: Resources & References (previously 9)

Acceptance Criteria

  • WinSCP is added to the Data Preparation tools list
  • New dedicated section for "File Transfer & Data Management Tools" is created
  • WinSCP use cases specific to Kubeflow are documented
  • ASCII diagram showing WinSCP integration is included
  • Setup instructions for Kubeflow environment are provided
  • Alternative tools are mentioned
  • Resources section includes WinSCP links
  • Section numbering is updated correctly
  • All Markdown formatting is proper and consistent

Additional Context

WinSCP is commonly used in enterprise environments where users need to transfer large datasets or model files to Kubernetes persistent volumes. This addition makes the documentation more practical for Windows users working with Kubeflow.

This pull request was created from Copilot chat.


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Co-authored-by: Stacey77 <54900383+Stacey77@users.noreply.github.com>
Copilot AI changed the title [WIP] Update Kubeflow Ecosystem documentation for WinSCP integration Add WinSCP documentation for Kubeflow file transfers Feb 3, 2026
Copilot AI requested a review from Stacey77 February 3, 2026 01:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants