diff --git a/docs/MACRO_SUPPORT.md b/docs/MACRO_SUPPORT.md new file mode 100644 index 0000000..cda4515 --- /dev/null +++ b/docs/MACRO_SUPPORT.md @@ -0,0 +1,89 @@ +# Macro Support in Agent CLI + +## Overview + +The agent CLI uses [Bun macros](https://bun.sh/docs/bundler/macros) to optionally optimize the loading of model provider information from [models.dev](https://models.dev/api.json). + +## How It Works + +In `src/provider/models.ts`, we import the `data` function from `models-macro.ts` with the macro directive: + +```typescript +import { data } from "./models-macro" with { type: "macro" } +``` + +When macros are supported, this function executes at **bundle-time** rather than runtime, potentially improving startup performance. + +## Limitations + +### Node Modules Restriction + +**Important:** Due to Bun's security model, macros cannot execute from code installed in `node_modules`. This means: + +- ✅ **Works**: When running the agent from source (development mode) +- ❌ **Does not work**: When installed globally via `bun install -g @deep-assistant/agent` +- ❌ **Does not work**: When installed as a dependency in another project + +This is a fundamental security restriction in Bun, as documented in the [official Bun macro documentation](https://bun.sh/docs/bundler/macros): + +> "For security reasons, macros cannot be run from node_modules." + +### Fallback Behavior + +The code is designed to gracefully fallback when macros are unavailable: + +1. If the macro fails (e.g., when running from node_modules), the `data()` function is called normally at runtime +2. The runtime version fetches from `https://models.dev/api.json` or reads from `MODELS_DEV_API_JSON` environment variable +3. Results are cached in the user's cache directory for future use + +## Testing + +### Development Mode (Macros Enabled) + +```bash +# Run from source - macros work +bun run src/index.js +``` + +### Production Mode (Macros Disabled) + +```bash +# Simulate global install behavior +cd /tmp +bun install -g @deep-assistant/agent +echo "hi" | agent +``` + +When installed globally, the macro directive is ignored and runtime fetching is used automatically. + +## Why Keep The Macro? + +Even though macros don't work in production installs, we keep the macro implementation because: + +1. **Development Performance**: Provides faster startup when developing the CLI +2. **Future Compatibility**: If Bun adds ways to bundle/compile packages, macros could work +3. **Graceful Degradation**: The fallback works seamlessly, so there's no downside + +## Alternative Approaches Considered + +### Pre-bundling + +We investigated pre-bundling the application with `bun build` to inline macro results. However: + +- Async macros with network fetching caused the bundler to hang +- This appears to be a limitation or bug in Bun's current macro implementation + +### Cached JSON + +We considered pre-fetching and bundling the models.dev JSON: + +- This would add ~600KB to the package size +- The runtime fetch + cache approach works well and keeps package small +- Users only fetch the data once per cache refresh interval + +## Conclusion + +The current implementation provides the best of both worlds: +- Macro optimization for development +- Reliable runtime fetching for production +- Automatic fallback with no user configuration needed diff --git a/experiments/test-agent-simple.sh b/experiments/test-agent-simple.sh new file mode 100755 index 0000000..0422d53 --- /dev/null +++ b/experiments/test-agent-simple.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +# Test that the agent works with the macro restoration + +echo "Testing agent in development mode..." +echo '{"message": "What is 2+2? Answer in one word."}' | timeout 30 bun run src/index.js --no-server 2>&1 | head -50 diff --git a/experiments/test-macro.ts b/experiments/test-macro.ts new file mode 100755 index 0000000..47b3bc3 --- /dev/null +++ b/experiments/test-macro.ts @@ -0,0 +1,12 @@ +#!/usr/bin/env bun + +/** + * Experiment to test if the macro works in development mode + */ + +import { data } from "../src/provider/models-macro" with { type: "macro" } + +console.log("Testing macro...") +const result = await data() +console.log("Macro result length:", result.length) +console.log("First 200 chars:", result.substring(0, 200)) diff --git a/scripts/build.ts b/scripts/build.ts new file mode 100755 index 0000000..bb3d415 --- /dev/null +++ b/scripts/build.ts @@ -0,0 +1,35 @@ +#!/usr/bin/env bun + +/** + * Build script to bundle the agent CLI with macros support + * + * This bundles the application so that macros are evaluated at build time + * and their results are inlined into the bundle. This avoids the security + * restriction that prevents macros from running in node_modules. + */ + +const result = await Bun.build({ + entrypoints: ['./src/index.js'], + outdir: './dist', + target: 'bun', + format: 'esm', + minify: false, + sourcemap: 'external', + // Macros are enabled by default in Bun.build + splitting: false, // Keep everything in one file for simplicity + packages: 'bundle', // Bundle all dependencies +}) + +if (!result.success) { + console.error('Build failed:') + for (const message of result.logs) { + console.error(message) + } + process.exit(1) +} + +console.log('✓ Build successful!') +console.log(` Generated ${result.outputs.length} output(s)`) +for (const output of result.outputs) { + console.log(` - ${output.path}`) +} diff --git a/src/provider/models.ts b/src/provider/models.ts index c3eb209..17a2d60 100644 --- a/src/provider/models.ts +++ b/src/provider/models.ts @@ -2,7 +2,7 @@ import { Global } from "../global" import { Log } from "../util/log" import path from "path" import z from "zod" -import { data } from "./models-macro" +import { data } from "./models-macro" with { type: "macro" } export namespace ModelsDev { const log = Log.create({ service: "models.dev" })