chore: Trainer: Specialized Trainers

[szaher]

Proposing framework-aware trainer classes (TorchTrainer, MPITrainer, JAXTrainer, XGBoostTrainer) with automatic runtime discovery via the trainer.kubeflow.org/framework label, and a RuntimeConfig dataclass to separate per-job environment settings from training logic.

Issue: #285

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, in Fixes #<issue number>, #<issue number>, ... format, will close the issue(s) when PR gets merged):

Fixes #

Checklist:

• Docs included if any changes are user facing

[google-oss-prow]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign astefanutti for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coveralls]

Pull Request Test Coverage Report for Build 21919758470

Details

• 0 of 0 changed or added relevant lines in 0 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage remained the same at 67.948%

---

Totals
Change from base Build 21919105330:	0.0%
Covered Lines:	2828
Relevant Lines:	4162

---

💛 - Coveralls

[Copilot]

Pull request overview

This PR introduces a design proposal for specialized trainer abstractions in the Kubeflow SDK. The proposal addresses the gap between the overly generic CustomTrainer and the overly narrow BuiltinTrainer by introducing framework-aware trainer classes with automatic runtime discovery.

Changes:

• Proposes a BaseTrainer abstract interface for all specialized trainers
• Defines four Tier 1 framework-specific trainers: TorchTrainer, MPITrainer, JAXTrainer, and XGBoostTrainer
• Introduces RuntimeConfig dataclass to separate runtime environment configuration from training logic
• Describes runtime auto-discovery using the trainer.kubeflow.org/framework label
• Maintains 100% backward compatibility with existing trainer APIs

[astefanutti]

Should we exclude / enforce there can be only one training runtime per trainer.kubeflow.org/framework label value or should we design a more flexible mechanism. I'm thinking about having multiple Torch runtimes with different PyTorch versions, or different accelerators.

[astefanutti]

It would be useful to have an example of tier 2 trainer that are configuration driven, e.g., compare current and after situation for TorchTune trainer.

[szaher]

/reopen

[google-oss-prow]

@szaher: Failed to re-open PR: state cannot be changed. The repository that submitted this pull request has been deleted.

Details

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[szaher]

for some reason this pr was deleted even before I delete the repo, so I created a fresh clone and push it again in #308