Add tool use compression for long running research tasks #434
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Introduced a new pathway for compressing chat history while preserving critical information, citations, and context. - Added functionality to intelligently summarize tool calls and results, ensuring that essential data such as URLs, file names, and exact figures are retained. - Implemented error handling to provide fallback summaries in case of compression failures. - Created integration tests to validate compression quality and retention of key information. - Developed a comprehensive guide for testing compression quality and metrics tracking.
…summaries - Introduced a new pathway to synthesize tool calls and results into structured research summaries, preserving citations and key findings. - Enhanced chat history management by validating and reconstructing messages to ensure no orphaned tool results remain. - Implemented pre-request token checks to optimize chat history compression before model requests. - Added comprehensive integration tests to validate the functionality and accuracy of the summarization process, ensuring proper extraction and preservation of URLs and citations. - Improved error handling to provide fallback summaries in case of summarization failures.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces a new chat history compression system and improves memory and tool call management across the entity agent system. The most significant change is the addition of a robust chat history compression pathway that preserves citations, URLs, and critical context, ensuring downstream tasks can continue seamlessly. Additionally, it introduces safety checks for memory usage and enforces tool call limits to prevent resource overuse.
Chat history compression:
sys_compress_chat_history.js, a new pathway that compresses chat history while preserving essential information, including user requests, tool calls/results, exact data (numbers, URLs, citations), and conversation context. The implementation extracts and validates citations/URLs, formats chat history for LLM prompts, and logs preservation rates.Tool call management:
sys_entity_agent.jsto enforce a limit on the number of tool calls executed per run, preventing the system from exceeding the maximum allowed tool calls and logging a warning if the limit is reached.Memory management:
sys_memory_manager.jsto immediately skip memory operations ifuseMemoryis explicitly set tofalse, improving efficiency and configurability.