Silk

A high-performance domain-specific language (DSL) for building distributed multi-agent systems using the contract-net auction protocol.

Overview

Silk is a specialized programming language designed for implementing intelligent agents that coordinate through auctions and message passing. Built with Zig for performance and type safety, Silk provides first-class support for agent capabilities, policies, actions, and the contract-net protocol.

Key Features

• Agent-First Design: Built-in constructs for capabilities, state, policies, and actions
• Contract-Net Protocol: Native support for auction-based task allocation and coordination
• Type-Safe: Strong typing with inference for reliability and performance
• High Performance: Built with Zig, no garbage collection overhead
• Message Passing: Integrated support for ZeroMQ and other transport protocols
• Policy/Action Separation: Clean separation between decision-making and execution
• Observability: Enterprise-grade tracing and logging support
• Arena Allocation: Efficient memory management for agent runtime

Quick Start

Prerequisites

• Zig 0.15.2 or higher
• mise (optional, for version management)

Installation

1. Clone the repository:

git clone <repository-url>
cd silk

2. Build the project:

zig build

3. Run tests to verify installation:

zig build test

Your First Silk Agent

Create a simple agent that responds to messages:

// Define agent capabilities
capability "example.responder" {
  topics   = ["tasks.simple"]
  latency  <= 1000ms
  quality  >= 0.8
  tools    = ["zmq"]
}

// Agent state
state message_count = 0

// Initialize the agent
handle initialize(ctx) {
  zmq.log("info", "Agent initialized", {agent_id: ctx.agent_id})
}

// Handle incoming messages
handle process_message(msg) {
  message_count = message_count + 1

  zmq.log("info", "Processing message", {
    count: message_count,
    content: msg.content
  })

  return {
    status: "success",
    processed: message_count
  }
}

// Policy for deciding whether to bid on tasks
policy should_bid(stimulus) {
  // Bid if we have capacity and the task matches our capabilities
  if message_count < 100 {
    return {
      bid: true,
      confidence: 0.9,
      estimated_cost: 50ms
    }
  } else {
    return {bid: false}
  }
}

// Action to execute when assigned a task
act execute_task(task) {
  let result = process_task(task)
  yield result
}

// Learn from task outcomes
learn update_model(outcome) {
  if outcome.success {
    // Increase confidence for similar tasks
    zmq.log("info", "Task successful, updating model")
  }
}

Run your agent:

zig build run -- agents/your_agent.silk

Contract-Net runtime (in-process)

Silk ships an in-process Contract-Net runtime for fast local experimentation:

• Builtins: contract_net.announce, submit_bid, close, assign, complete, stats, on_assignment, on_complete.
• Weighting: pass {confidence_weight: 0.6, price_weight: 0.4} as the third arg to announce.
• Example: run the leader/worker sample zig build run -- run agents/cnp_leader.silk to see announce → bid → close → assign → complete with callback logs.
• The close result includes winner, bid_count, confidence, score, and auction_id; each bid is kept for debugging.
• A tiny in-process pub/sub mock (InprocZmq) is available for transport testing while a real ZeroMQ adapter is designed.

Interactive REPL

Silk includes a powerful interactive REPL with enhanced error reporting, command history, and introspection capabilities:

silk  # Start the REPL

REPL Features

Enhanced Error Reporting:

• Color-coded error messages (red for errors, yellow for warnings, green for success)
• Source code snippets with error location markers (^^^)
• "Did you mean?" suggestions for undefined variables
• Stack traces for runtime errors

Command History:

• Persistent history saved to ~/.silk_history
• Navigate with ↑/↓ arrow keys
• Ctrl+R for reverse search (planned)

Introspection Commands:

• :symbols - List all defined variables and functions
• :type <expr> - Show the type of an expression
• :imports - Display loaded modules
• :load <file> - Load and execute a Silk file
• :reset - Clear interpreter state
• :help - Show available commands
• :clear - Clear the screen
• :exit/:quit - Exit the REPL

Example REPL Session

Silk REPL v0.1.0
Type :help for help, :exit to quit

>>> state x = 42
>>> fn greet(name) {
...   return "Hello, " + name
... }
>>> greet("World")
"Hello, World"
>>> :symbols
Defined symbols:
  x: 42
  greet: <function>
>>> :type x
Type: int
>>> :load examples/utils.silk
info: Loaded file 'examples/utils.silk' successfully
>>> :imports
Loaded modules:
  examples/utils.silk
>>>

Language Features

Core Constructs

• Declarations: capability, tool, state, policy, act, handle, learn, import
• Statements: let, return, yield, if/then/else, for, while
• Types: int, float, string, bool, duration, arrays, objects, Result<T>, Bid, Stimulus
• Built-in Functions: Message passing, timers, logging, and more

Agent Components

1. Capability Declarations: Define what your agent can do and its constraints
2. State Management: Persistent and ephemeral state with automatic management
3. Policy Functions: Decision-making logic for bidding and task selection
4. Action Functions: Execution logic for performing tasks
5. Message Handlers: Process incoming messages and events
6. Learning Functions: Adapt based on outcomes and feedback

Contract-Net Protocol

Silk provides native support for the contract-net auction protocol:

// Auction leader announces task
announce_task(task_spec) -> List<Bid>

// Agents submit bids
policy calculate_bid(task) -> Bid

// Leader assigns task to winner
assign_task(agent_id, task) -> Assignment

// Agent executes and returns result
act perform_task(assignment) -> Result

See the example leader agent for a complete auction orchestration implementation.

Project Structure

silk/
├── src/                      # Core language implementation
│   ├── Lexer.zig            # Tokenization and lexical analysis
│   ├── Token.zig            # Token type definitions
│   ├── Parser.zig           # Recursive descent parser with Pratt parsing
│   ├── Ast.zig              # Abstract Syntax Tree definitions
│   ├── Value.zig            # Runtime value types and environment
│   ├── Interpreter.zig      # Tree-walking interpreter
│   ├── main.zig             # CLI entry point
│   └── root.zig             # Public module API
├── agents/                   # Example agent implementations
│   └── leader.silk          # Contract-net auction leader example
├── docs/                     # Documentation
│   └── SILK_REFERENCE.md    # Complete language reference
├── build.zig                # Build configuration
├── build.zig.zon            # Package manifest
└── README.md                # This file

Development

Building

# Build executable and library
zig build

# Build with optimizations
zig build -Doptimize=ReleaseFast

# Install to system (optional)
zig build install

Running

# Run the Silk CLI
zig build run

# Run with an agent file
zig build run -- agents/leader.silk

# Run with arguments
zig build run -- --help

Testing

The project includes comprehensive unit tests for all components:

# Run all tests
zig build test

# Run with verbose output
zig build test -- --summary all

# Run specific test
zig build test -- --test-filter "parser"

Test Coverage:

• Lexer: Token generation, string/number parsing, comments, duration literals
• Parser: All declaration types, statements, expressions, precedence rules
• Interpreter: Expression evaluation, control flow, function calls, state management

Code Organization

The implementation follows a clean pipeline architecture:

1. Lexer (552 lines): Source code → Tokens
2. Parser (1,176 lines): Tokens → Abstract Syntax Tree (AST)
3. Interpreter (704 lines): AST → Runtime execution

Each component is:

• Self-contained with minimal dependencies
• Thoroughly tested with unit tests
• Documented with inline comments
• Memory-safe using Zig's arena allocators

Documentation

• Language Reference: Complete language specification, syntax, and semantics
• Example Agents: Real-world agent implementations
• API Documentation: Generate with zig build docs (coming soon)

Using Silk as a Library

Silk can be used as a library in your Zig projects:

const silk = @import("silk");

pub fn main() !void {
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    defer _ = gpa.deinit();
    const allocator = gpa.allocator();

    // Parse Silk source code
    const source = "state counter = 0";
    var lexer = silk.Lexer.init(source);
    var parser = try silk.Parser.init(allocator, &lexer);
    defer parser.deinit();

    var program = try parser.parseProgram();

    // Execute the program
    var interpreter = try silk.Interpreter.init(allocator);
    defer interpreter.deinit();

    try interpreter.execute(&program);
}

Add to your build.zig.zon:

.dependencies = .{
    .silk = .{
        .url = "https://github.com/your-org/silk/archive/<commit>.tar.gz",
        .hash = "<hash>",
    },
},

Development Status

Version: 0.0.0 (Early Development)

Completed

• ✅ Complete lexer with comprehensive token support
• ✅ Recursive descent parser with Pratt parsing for expressions
• ✅ Tree-walking interpreter with arena allocation
• ✅ Core language constructs (declarations, statements, expressions)
• ✅ Type system foundation (primitives, arrays, objects)
• ✅ Function definitions and calls
• ✅ Control flow (if/else, loops)
• ✅ Variable binding and scoping
• ✅ Comprehensive test suite

In Progress

• 🚧 Standard library implementation
• 🚧 Built-in functions (message passing, timers, etc.)
• 🚧 Contract-net protocol runtime
• 🚧 Transport layer integrations (ZeroMQ, TCP, Unix sockets)
• 🚧 Advanced language features (streams, async/await)

Planned

• 📋 JIT compilation or bytecode VM for performance
• 📋 IDE support (LSP, syntax highlighting)
• 📋 Package manager
• 📋 Formal specification and verification tools
• 📋 Distributed runtime and cluster management
• 📋 Monitoring and observability dashboard

Contributing

Contributions are welcome! Here's how to get started:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Make your changes with tests
4. Ensure all tests pass (zig build test)
5. Commit your changes (git commit -m 'Add amazing feature')
6. Push to the branch (git push origin feature/amazing-feature)
7. Open a Pull Request

Development Guidelines

• Follow Zig style conventions
• Add tests for all new features
• Update documentation as needed
• Keep commits focused and descriptive
• Ensure code compiles with no warnings

Architecture

Silk uses a tree-walking interpreter architecture for simplicity and clarity:

Source Code
    ↓
[Lexer] → Tokens
    ↓
[Parser] → Abstract Syntax Tree (AST)
    ↓
[Interpreter] → Runtime Execution
    ↓
Results

Key design decisions:

• Arena Allocation: All AST nodes and runtime values use arena allocators for efficient bulk deallocation
• Two-phase Execution: Parse phase (compilation) and execution phase are separate
• Immutable AST: Once parsed, the AST is immutable during execution
• Environment Chain: Lexical scoping via parent environment pointers
• Tagged Unions: Type-safe runtime values using Zig's tagged unions

Performance

While currently an interpreter, Silk is designed for performance:

• Written in Zig (no GC, predictable performance)
• Arena allocation reduces allocation overhead
• Zero-copy string handling where possible
• Efficient value representation with tagged unions
• Planned: JIT compilation and bytecode VM

Community

• Issues: GitHub Issues
• Discussions: GitHub Discussions
• Documentation: docs/

Acknowledgments

• Built with Zig
• Inspired by the contract-net protocol from multi-agent systems research
• Parser design influenced by Pratt parsing techniques

Related Projects

• ZeroMQ - High-performance async messaging library
• Zig Language Server (ZLS) - IDE support for Zig

---

Note: Silk is in early development. APIs and language features are subject to change. Feedback and contributions are greatly appreciated!