Skip to content

Refactor processing and modality retrieval#2

Merged
MichelDucartier merged 15 commits intomasterfrom
processor-rework
Oct 1, 2025
Merged

Refactor processing and modality retrieval#2
MichelDucartier merged 15 commits intomasterfrom
processor-rework

Conversation

@BoyeGuillaume
Copy link
Contributor

No description provided.

@BoyeGuillaume BoyeGuillaume marked this pull request as ready for review October 1, 2025 09:21
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the processing and modality retrieval system by introducing a new modular architecture for handling different modality types. The refactoring moves from the old modality_imp system to a new modalities module with base classes and auto-registration functionality.

Key changes include:

  • Introduction of new base classes (BaseModality, BaseModalityConfig, BaseModalityProcessor) for consistent modality handling
  • Implementation of an AutoModality registry system for dynamic modality registration and instantiation
  • Addition of a new dataset/loader system to replace the old registry pattern
  • Creation of an MLP projector module for standardized projection layers

Reviewed Changes

Copilot reviewed 30 out of 31 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/multimeditron/utils/init.py Adds new imports and global cache functionality with warning system
src/multimeditron/train/trainer.py Removes unnecessary blank lines and uncomments code
src/multimeditron/model/prompt_tokenizers.py Removes unused import and cleans up whitespace
src/multimeditron/model/projectors/mlp.py New MLP projector module with comprehensive documentation
src/multimeditron/model/model.py Major refactoring to use new modality system with AutoModality
src/multimeditron/model/modalities/ New modality system with base classes and image modality implementations
src/multimeditron/model/data_loader.py Updates to use new modality system and adds comprehensive documentation
src/multimeditron/dataset/ New loader system replacing old registry pattern
config/config_alignment.yaml Configuration updates to support new loader system

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

MichelDucartier and others added 4 commits October 1, 2025 11:39
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copy link
Contributor Author

@BoyeGuillaume BoyeGuillaume left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK

@MichelDucartier MichelDucartier merged commit 781114f into master Oct 1, 2025
@MichelDucartier MichelDucartier deleted the processor-rework branch October 1, 2025 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants