Unify query routing through MCP tools with early tool call decision

[Copilot]

Problem

Tool calls were being returned as text to users instead of being executed. The architecture had dual routing paths (paper search vs clustering tools) with tool decisions made during response generation, after papers were already retrieved.

Solution

Unified Routing Architecture

Moved tool call decision from response generation to query analysis phase. All queries now route through MCP tools:

Before:

query() → rewrite_query() → search_papers() → generate_response() → [decide tools] → execute

After:

query() → analyze_and_route_query() → [decide tools] → execute_tools() → generate_response()

Key Changes

• Tool call decision during query analysis: _analyze_and_route_query() always returns tool calls (JSON format), eliminating dual execution paths

• General-purpose paper search tool: Extended get_recent_developments with optional year filtering (n_years: Optional[int]). When n_years=None, searches all years instead of just recent papers

• JSON tool call detection: Added parse_json_tool_call() to handle models that return tool calls as JSON text instead of OpenAI's native format. Supports multiple formats gracefully:

  # Single tool
  {"name": "analyze_topic_relevance", "arguments": {...}}
  
  # Array of tools  
  [{"name": "get_cluster_topics", "arguments": {...}}, ...]
  
  # OpenAI-like nested format
  {"tool_calls": [{"function": {"name": "...", "arguments": {...}}}]}

• Improved tool schemas: Removed non-standard default fields, added specific example queries for each tool, updated system prompts to guide LLMs on proper tool usage

• Test architecture: Ensured MCP tools use same database paths as RAG via proper config singleton behavior. Unit tests use mocked EM/DB, integration tests use real fixtures with set_test_db()/set_test_embedding_db()

Benefits

• Single unified interface eliminates conditional routing logic
• Easier to extend (new tools don't require routing changes)
• Better performance (don't search papers if tools will be used)
• Proper tool execution for models with varying levels of tool support

Original prompt

---

This section details on the original issue you should resolve

<issue_title>RAG: tool calls are returned as answers to the user</issue_title>
<issue_description>example:
Q: how many papers about uncertainty quantification were presented at neurips 2025?
A: [TOOL_CALLS]analyze_topic_relevance{"query": "Uncertainty quantification", "distance_threshold": 1.1, "conferences": ["NeurIPS"], "years": [2025]}

Instead, when a tool call is detected ([TOOL_CALLS]) the respective tool should be called and the results should be added to the context of the chat and the chat model should then formulate an answer to the user question based on the tool call results.</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes RAG: tool calls are returned as answers to the user #110

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[thawn]

Suggested change

	# Not valid JSON or doesn't match expected format
	# Not valid JSON or doesn't match expected format
	logger.debug(f"Error parsing JSON tool calls. Make sure the model supports tool calling and returns proper JSON formatted responses. Got response from model: {response_text.strip()}")

[Copilot]

Done in commit 6bc0fe0. Added descriptive debug message that includes the response text.

[thawn]

Suggested change

	return None
	logger.debug(f"No JSON tool call(s) found in response {response_text.strip()}")
	return None

[Copilot]

Done in commit 6bc0fe0. Added debug message before return.

[thawn]

be more specific, give one example for each tool. For example:

Specific questions about a topic. ('what is the latest research on ...?', 'Which are the most relevant papers about ...'): get_recent_developments()

Identify relevant topics ('what were the hot topics this year?', 'which research areas were covered most this year?'): get_cluster_topics()

Identify how relevant a specific topic was this year ('how many papers about ... were published this year?', 'how important was ... at this conference?): analyze_topic_relevance()

Identify trends for specific topics ('how has ... evolved over the years?', 'has .. become more or less relevant?'): get_topic_evolution()

Cluster visualization requests ('show me papers clustered by topic.', 'plot an overview of papers with similar paper grouped together'): get_cluster_visualization()

[Copilot]

Done in commit 6bc0fe0. Updated system prompt with specific examples for each tool as suggested.

[thawn]

remove part 2 of the prompt, it refers to the redundant rewrite_and_search_papers tool

[Copilot]

Done in commit 6bc0fe0. Removed part 2 referencing rewrite_and_search_papers.

[thawn]

only recommend standard openAI tool calling format

[Copilot]

Done in commit 6bc0fe0. Now only recommends standard OpenAI format: {"name": "tool_name", "arguments": {...}}

[thawn]

avoid code duplication: this code duplicates lines 359 - 367

[Copilot]

Done in commit 6bc0fe0. Created _get_fallback_route_info() helper function to eliminate duplication.

[thawn]

the tool call is effectively rewriting the query, therefore, this tool effectively does the same as get_recent_developments() remove rewrite_and_search_papers and use get_recent_developments() instead

[Copilot]

Partially implemented in commit 6bc0fe0. I've removed rewrite_and_search_papers from mcp_server.py and mcp_tools.py, and updated fallbacks to use get_recent_developments. However, complete migration requires discussion:

Challenge: rewrite_and_search_papers searches all papers while get_recent_developments filters by recent years (last n_years). Different semantics.

Current state:

• Tool removed from codebase
• Fallbacks route to get_recent_developments
• 11 tests failing (need mocks updated)

Questions:

1. Should get_recent_developments be made more general (remove/make optional the year filter)?
2. Or should we add a new general search tool?
3. How should tests be structured when tools create their own EM/DB instances?

Would appreciate guidance on preferred approach before completing the migration.