[ENHANCEMENT] Choose Model Dynamically Based on Request

[farazoman]

Problem (one or two sentences)

When doing something like reading or writing a file, or other less intensive tasks it is useful to use cheaper models. Now we can set profiles, but to be able to dynamically choose based on some constraints could reduce costs by a lot.

Context (who is affected and when)

Everyone. Right now simple mode / profile choices reduces costs, esp with orchestrator. But there can be a lot of other sub-tasks which can be done by an even cheaper model.

Desired behavior (conceptual, not technical)

Auto routing for the model.

Constraints / preferences (optional)

Configurable to some extent so that we can decide what model/temp we want in which use case.

For example we could have some dimensions, like:

• ambiguity of request ( is the ask clear, does it require clarification or investigation)
• size of request (does it need to be broken down to complete)
• determinism of request (debugging, refactoring, might need higher temp; info gathering may be lower size)
• more?

Request checklist

• I've searched existing Issues and Discussions for duplicates
• This describes a specific problem with clear context and impact

Roo Code Task Links (optional)

No response

Acceptance criteria (optional)

GIVEN an agent requires to do a simple task which can easily be done with less thinking
WHEN executing the task
THEN the agent will choose an apt model for the task, thereby reducing costs and improving the end result

Proposed approach (optional)

No response

Trade-offs / risks (optional)

No response

[roomote]

I'd like to help with this issue, but I need some clarification to ensure I propose the right solution. After reviewing the codebase, I have several questions about the desired scope and approach:

Scope Questions:

1. Routing level: Should the dynamic model selection happen within a single task (e.g., different tool calls like read_file vs. complex code generation use different models), or at the task level (e.g., routing the entire conversation to a cheaper model for simple requests)?
2. Interaction with existing modes/profiles: Currently, each mode (code, architect, ask, etc.) can already be bound to a specific provider profile. Should this feature replace that system, extend it (e.g., add an "auto" profile option), or work alongside it at a more granular sub-task level?

Approach Questions:
3. Routing mechanism: How should routing decisions be made?

• Rule-based: User-configurable rules mapping task characteristics to models (e.g., "if tool is read_file, use cheap model")
• LLM-based classifier: A small/fast model classifies each request before routing (adds latency + cost for the classifier call itself)
• Provider-level routing: Leverage existing provider routing features (e.g., OpenRouter's auto-routing, Requesty's routing) rather than implementing routing in Roo Code itself
• Heuristic-based: Built-in heuristics based on observable signals (tool type, message length, mode, etc.)

4. Configurability: The issue mentions dimensions like ambiguity, size, and determinism. These are inherently subjective and hard to measure without an LLM classifier. Would a simpler approach work initially - such as configuring model tiers (e.g., "heavy" vs. "light") and mapping specific operations to tiers?

Technical Constraints:
5. API key management: Dynamic model switching may require switching between providers (e.g., Anthropic for complex tasks, a cheaper provider for simple ones). Should this be limited to a single provider, or support cross-provider routing (which would require multiple API keys to be configured)?
6. Token budget awareness: Should the router consider remaining token budget or cost-per-token when making decisions?

Could you share your thoughts on these questions? In particular, knowing the intended routing level (#1) and mechanism (#3) would help me propose a concrete implementation plan.

Please tag @roomote in your response with the answers, and I'll be happy to propose a detailed implementation plan.

[farazoman]

@roomote

1. potentially within a single task, or rather, a sub-task can spin up sub-agents or commandline calls to run an agent in parallel with a cheaper model for example.
2. I think this should replace the existing model, or it can be an option first so that users can choose as they prefer
3. A hybrid approach, rule/heuristic based for some standard tools, llm-based as an option which runs if rule/hueristic based doesnt work, if provider level routing is available then that is used instead of llm-based. To keep it simple, we can start with rule based.
4. we could use some heuristics based on those dimensions but i do think a simpler approach would make sense for the first iteration
5. multi-provider can be out of scope for now
6. not at all, it should use the ideas from point 3

[roomote]

Thanks for the detailed answers @farazoman! I've seen your comment and will work on an initial implementation focusing on:

1. Rule/heuristic-based routing as the first iteration (starting simple)
2. Within a single provider (multi-provider out of scope for now)
3. Tool-level routing - mapping specific tool operations (like read_file, list_files) to a configurable "light" model tier
4. Optional feature - users can opt-in via settings, keeping existing behavior as default

I'll explore the codebase and open a draft PR to share progress. Feedback and guidance welcome!

[roomote]

Opened a draft PR with my attempt at implementing the heuristic-based model routing: #11272

The implementation follows the approach discussed:

• Rule/heuristic-based routing as a first iteration
• Experimental feature behind modelRouting flag (disabled by default)
• Tool-group classification: If previous turn only used read tools (read_file, list_files, search_files, codebase_search), next turn uses the configured light model
• Same provider: Light model must be from the same provider
• 35 tests covering the router logic

Feedback welcome!