Feat: Chunking large inputs

[rihp]

There are some cases where the result of an invokation is very long and exceeds the token window limit

In those cases we can implement some type of chunking mechanism that processes the input in batches that the LLM can handle.

For some reference see this issue and specifically this comment

The idea would be to apply the following new steps when sending a message to the agent:

0. Create an env variable to set the chunking size
1. When the agent is about to process any input, check the length of the input with tiktoken
2. If the input is larger than the context window, chunk it based on env config length.
3. Save the output in a variable
4. Process the next chunk and append the result to the previous variable to update its contents with the new information.
5. Repeat the process until you´ve processed all the chunks
6. Return the combination of all the responses to the chunked bits

[rihp]

This one is being implemented and works as a Proof of Concept in #29

[rihp]

The summarization module was refactored,

today, I think we should remove the summarization of permanent messages as it messages with the initialization prompt, the user goal and the loaded wraps.

Essentially removing this line or implementing it differently

PolyGPT/src/chat.ts

Line 111 in 4acd195

await this._summarize("persistent");