feat: Validation Split & MLflow Tracking

[SK8-infi]

Fixes

Closes: #22

Type of Change

• Bug fix
• New feature
• Documentation / Refactor
• Math / Logic correction

Description

Implemented validation split and MLflow tracking for monitoring model generalization performance.

Changes:

• Added validation_split: float = 0.2 parameter to Model.train() method
• Split data into training and validation sets after preprocessing
• Modified training loop to train epoch-by-epoch and calculate validation loss after each epoch
• Added _calculate_validation_loss() method supporting both classification (cross-entropy) and regression (MSE)
• Extended Rust bindings with forward() method to expose raw model outputs for validation loss calculation
• Updated save_model() to log both loss and val_loss metrics to MLflow with epoch steps
• Added error handling for edge cases (empty predictions, shape mismatches, small validation sets)

Result: MLflow dashboard now displays overlapping train/validation loss curves for monitoring overfitting and generalization.

How Has This Been Tested?

• Unit Tests: Created and ran test_validation_split.py covering classification, regression, and edge cases
• Manual Testing: Verified with dummy datasets and confirmed loss curves in MLflow
• Integration: Confirmed Rust core forward() method works correctly with Python API
• Existing Tests: All existing pytest tests pass

Screenshots / Logs

📊 Data split: 80 training samples, 20 validation samples
Epoch 0/10 - Train Loss: 0.8602, Val Loss: 0.2851

📈 Logging 10 training metrics points...
📈 Logging 10 validation metrics points...

Contribution Context

• I am contributing through the SWOC program.

[github-actions]

Manual Action Required!

This pull request is not yet linked to an issue. Please update the description to include 'Closes: #issue_number' in the appropriate section. This is required to pass validation and initiate metadata synchronization.

[github-actions]

Thank you for opening this PR! Our automated system is currently verifying the PR requirements.
Internal Discussion: Discord

[github-actions]

Validation Successful!

This pull request has been verified and linked to issue #22. The system is now synchronizing metadata from the referenced issue. Kindly await maintainer review of your changes.

[Satyamgupta2365]

heyy. CI is failing on test_model_save_load_preserves_preprocessing.
The validation step is producing no predictions, causing a shape mismatch (y_true (1,2) vs y_pred (0,)).
Please ensure validation data uses the same preprocessing and that predict() is correctly called for X_val, especially for small validation sizes.

[SK8-infi]

heyy. CI is failing on test_model_save_load_preserves_preprocessing. The validation step is producing no predictions, causing a shape mismatch (y_true (1,2) vs y_pred (0,)). Please ensure validation data uses the same preprocessing and that predict() is correctly called for X_val, especially for small validation sizes.

Thanks @Satyamgupta2365 for review. Will keep this in mind. currently the work is in progress.

[github-actions]

Validation Successful!

This pull request has been verified and linked to issue #22. The system is now synchronizing metadata from the referenced issue. Kindly await maintainer review of your changes.

[github-actions]

Validation Successful!

This pull request has been verified and linked to issue #22. The system is now synchronizing metadata from the referenced issue. Kindly await maintainer review of your changes.

[SK8-infi]

Hii @Satyamgupta2365 finished the PR from my side and is ready for review. Thanks

[github-actions]

Validation Successful!

This pull request has been verified and linked to issue #22. The system is now synchronizing metadata from the referenced issue. Kindly await maintainer review of your changes.

[github-actions]

Validation Successful!

This pull request has been verified and linked to issue #22. The system is now synchronizing metadata from the referenced issue. Kindly await maintainer review of your changes.

[debug-soham]

@SK8-infi Thank you for the effort! While validation tracking is a great addition, this PR introduces critical architectural issues that must be addressed:

• Optimizer State Reset: Moving the training loop to Python causes a new Adam/SGD instance to be created every epoch. This resets Adam's moment estimates and time-step to zero, breaking its adaptive logic.
• Performance Overhead: Calling the Rust backend once per epoch significantly increases FFI marshaling and context-switching overhead.
• Regressions: The code lacks the batch_size parameter recently merged in PR #63, which will cause build and runtime failures.

Requested Changes:

1. Revert the Python Loop: Move the validation logic into the Rust train method so the optimizer remains persistent.
2. Sync with Main: Update signatures to include the mandatory batch_size parameter.
3. Preserve forward(): Keep this new method in lib.rs as it is a valuable utility.

[debug-soham]

@SK8-infi Please update us on the status. You have 24 hours to address the issues or we’ll close this PR and reassign the task to keep development on track.

[github-actions]

Validation Successful!

This pull request has been verified and linked to issue #22. The system is now synchronizing metadata from the referenced issue. Kindly await maintainer review of your changes.

[SK8-infi]

had to do some changes in tests as the new signature is (train_losses, val_losses)

[github-actions]

Validation Successful!

This pull request has been verified and linked to issue #22. The system is now synchronizing metadata from the referenced issue. Kindly await maintainer review of your changes.

[debug-soham]

@SK8-infi Build tests are failing.

[SK8-infi]

Seems to be an issue happened during resolving conflicts. Looking into it

[debug-soham]

@SK8-infi There are currently no merge conflicts. However, all build tests must pass prior to review. Please address the issues.

[SK8-infi]

Okay. Ensuring to be completed by EOD

[Aamod007]

@SK8-infi any updates for PR ?

[github-actions]

Validation Successful!

This pull request has been verified and linked to issue #22. The system is now synchronizing metadata from the referenced issue. Kindly await maintainer review of your changes.

[SK8-infi]

Hii
Please check the fixes
Thanks

[Aamod007]

@SK8-infi Critical Fixes (MUST DO):

1. Fix parameter order in Python API call
2. Fix hidden layer type mismatch
3. Add integration tests with real Rust extension

Recommended Improvements (SHOULD DO):
4. Add data shuffling before split
5. Add random seed parameter for reproducibility
6. Add stratification option for classification

Future Enhancements (NICE TO HAVE):

• K-fold cross-validation support
• Early stopping based on validation loss
• Validation metrics beyond loss
• Learning rate scheduling

[debug-soham]

@SK8-infi, please update us on the status. You have 24 hours to address the issues or we’ll close this PR and reassign the task to keep development on track.

[SK8-infi]

Hii @debug-soham
Due to prior commitments, I will not be able to work on this issue anymore. You may close and reassign.