Implement Symmetric Send/Receive Architecture with Persistent Worker Threads

[sylvain-pierrot]

Summary

This PR implements a symmetric architecture for the BP Socket system, replacing the "thread-per-message" model with persistent worker threads, significantly improving system performance and stability. It also adds comprehensive support for Bundle Protocol flags in sendmsg() operations, enabling fine-grained control over bundle transmission behavior.

Key Features:

• Persistent worker threads instead of thread-per-message
• Userspace send queue with backpressure mechanism
• Complete BP flags support for sendmsg() operations
• Kernel optimizations for high-throughput scenarios
• Enhanced BP client with proper timeout handling

Problems Solved

Before:

• ❌ New thread created for each sent message
• ❌ Rapid system saturation ("failed to create send thread" errors)
• ❌ Asymmetric architecture (receive vs send)
• ❌ Manual endpoint reference management
• ❌ Blocking during endpoint closure
• ❌ Limited flag support in sendmsg()

After:

• ✅ Persistent worker thread per endpoint (like receive)
• ✅ Userspace send queue with backpressure
• ✅ Symmetric and consistent architecture
• ✅ Automatic resource management
• ✅ Clean and fast endpoint closure
• ✅ BP flags support in sendmsg()

Technical Changes

1. Symmetric Architecture

// Before: Single thread per message
pthread_create(&thread, NULL, ion_send_thread, args); // For each message

// After: Persistent worker thread
while (running) {
    // Consumes send queue
    // Processes messages with bp_send()
}

2. Userspace Send Queue

• Structure: send_queue_item with EID, payload, flags
• Management: FIFO with 1000 message limit
• Backpressure: Returns -EAGAIN when queue is full
• Synchronization: Mutex + condition variable

3. Kernel Optimizations

• Memory: GFP_ATOMIC instead of GFP_KERNEL to avoid blocking
• Logs: Reduced verbose logging for better performance
• Flags: Full BP flags support (priority, custody, status reporting)

4. Enhanced BP Client

• Threads: Send and receive in separate threads
• Timeout: Timeout handling to prevent blocking
• Clean exit: Proper shutdown with Ctrl+C

Performance Improvements

• Throughput: Support for thousands of messages per second
• Latency: Reduced send latency
• Stability: No more system saturation errors
• Resources: Optimized memory and CPU usage