LLMABDA

🐑 Run Quantized Agents on AWS Lambda for Cheap 🐥

What is this?

LLMabda is a simple proxy for llama.cpp server that runs on AWS Lambda.

Deployment

Information about deployment is in deploy/

Demo

1. Agent (Using ggml-org/Qwen3-1.7B-GGUF)

• source code
• "What is 5 + (5 * 6)? Please shout the final answer."

2. Summarize text (Using ggml-org/Qwen3-1.7B-GGUF)

• source code
• "What happened to the Crystal Extractor?"