Skip to content

Improve bot message filtering to prevent debugging messages #138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
comfy-pr-bot wants to merge 39 commits into
base: main
Choose a base branch
from
Open

Improve bot message filtering to prevent debugging messages #138

comfy-pr-bot wants to merge 39 commits into from

Conversation

@comfy-pr-bot
Copy link
Member

@comfy-pr-bot comfy-pr-bot commented Jan 9, 2026

Summary

Improves the bot message filtering system to prevent debugging messages from being shown to users in Slack. Users will now only see clean, relevant task progress without internal system messages.

Problem

The bot currently shows debugging messages like "retrying the slack upload...", local file paths, timestamps, and other system internals to end users in Slack channels. These messages are confusing and not relevant to users.

Solution

Created a comprehensive message filtering system that automatically removes debugging information before displaying to users:

New Files

  • bot/utils/filterDebugMessages.ts: Core filtering utility with filterInternalThoughts() and getFilteredContent() functions
  • bot/utils/filterDebugMessages.spec.ts: Comprehensive test suite with 18 tests (all passing)

What Gets Filtered

  1. Retry/upload messages ("retrying the slack upload...")
  2. File paths (/repos/, /home/, /tmp/)
  3. Process info (PID: 12345, exit code: 0)
  4. Timestamps ([2024-01-09T10:30:45], etc.)
  5. ANSI escape codes and terminal control characters
  6. System commands (bun run, npm install, node_modules)
  7. Git internals (git fetch, HEAD is now at)
  8. Debug logging ([DEBUG], [TRACE], [VERBOSE])
  9. Stack traces (at functionName (...))

Changes

  • bot/index.ts (lines ~1061-1117): Integrated filtering into Slack update flow

    • Filters terminal output before sending to my_internal_thoughts
    • Logs filtered-out content to BOT-LOGS.md for debugging
    • Enhanced prompt instructions with explicit examples

  • CLAUDE.md: Added comprehensive documentation section on message filtering

    • Usage examples
    • Integration details
    • Benefits and testing instructions

Testing

bun test bot/utils/filterDebugMessages.spec.ts

All 18 tests passing ✅

Benefits

  • Cleaner User Experience: Users only see relevant task progress
  • Debugging Preserved: Filtered content logged to BOT-LOGS.md
  • Reduced Confusion: No more system messages confusing users
  • Consistent Output: Standardized filtering across all bot responses

Stats

  • 4 files changed
  • 554 insertions, 3 deletions
  • 18 comprehensive tests

🤖 Generated with Claude Code

snomiao and others added 30 commits January 8, 2026 01:58
@snomiao @claude
feat: add ComfyPR Bot with Slack/Notion integration and fix event han…
c0bf042 
…dling

## Bot Skills Implementation

Created comprehensive bot skills system with CLI tools:

**Slack Skills** (`bot/slack/`):
- `msg-update.ts`: Update Slack messages programmatically
- `msg-read-thread.ts`: Read entire Slack threads with markdown conversion
- `parseSlackMessageToMarkdown.ts`: Convert Slack formatting to Markdown
- `slackTsToISO.ts`: Convert Slack timestamps to ISO 8601 format
- Full test coverage with `.spec.ts` files (16 tests passing)

**Notion Skills** (`bot/`):
- `notion-search.ts`: Search Notion docs from Comfy-Org workspace

**Documentation**:
- `bot/README.md`: Complete usage guide for all skills
- `bot/BUGFIX.md`: Detailed bug fixes documentation
- Updated `CLAUDE.md` with bot skills section

## Bug Fixes

### Fix 1: App Mention Event Schema Mismatch
- Made `client_msg_id` optional (Slack doesn't always include it)
- Added `attachments` field for message unfurls
- Events with attachments now process correctly

### Fix 2: WebSocket System Messages
- Added handlers for "hello" (connection acknowledgment)
- Added handlers for "disconnect" messages
- Only truly unhandled messages are logged as `MSG_NOT_MATCHED`

## Files Modified
- `bot/index.ts`: Fixed event schema and WebSocket message handling
- `CLAUDE.md`: Added ComfyPR Bot Skills documentation section

## Security
- All environment variables properly referenced
- No tokens or secrets committed
- Log files excluded via `.gitignore`

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
chore: improve .gitignore for security and organization
aa81b39 
Add .env, logs/, repos/, and TODO.md to ignore list.
Add comment about .env vs .env.local usage.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
chore: add configuration placeholders to .env
5734189 
Add CS_ORIGIN, CS_TOKEN, and PRBOT_PORT environment variables
with placeholder values for local override.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
docs: add bot architecture and TODO tracking documentation
69288f2 
- Document master-worker bot architecture pattern
- Add coding sub-agent (pr-bot) usage documentation
- Add comprehensive TODO tracking and resolution workflow
- Update bot skills to clarify READ-ONLY access for master agent
- List priority TODOs across the project

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
feat: rewrite bot Dockerfile for Bun and Node.js runtime
b03c5cc 
Replace PyTorch/ComfyUI setup with Debian-based Bun runtime.
Install Node.js via nvm and Bun for executing bot services.
Configure entrypoint to run bot/index.ts.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
docs: add unified CLI documentation to bot README
263113f 
Document new bot/cli.ts unified CLI with yargs.
Update Notion search path references.
Add common command examples for PR, Slack, and Notion operations.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
refactor: move notion-search.ts to bot/notion/search.ts
ce04d36 
Reorganize Notion utilities into dedicated directory.
Move bot/notion-search.ts to bot/notion/search.ts for better organization.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
feat: add ComfyUI registry search utility
45866ab 
Add bot/registry/search.ts for searching ComfyUI custom nodes
via the registry API.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
feat: add GitHub code utilities (pr-bot and issue-search)
b870e6c 
Add bot/code/ directory with:
- pr-bot.ts: CLI for spawning coding sub-agents on GitHub repos
- issue-search.ts: Search GitHub issues across Comfy-Org
- coding/pr-agent.ts: Core logic for repository cloning and agent spawning
- coding/pr-agent.spec.ts: Test suite for pr-agent
- coding/README.md: Documentation for coding sub-agent system

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
docs: add ComfyUI code search documentation
c03e785 
Add bot/comfy-codesearch/README.md documenting the code search
service integration for searching ComfyUI repositories.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
feat: add Slack file operations and URL parser utilities
9067f21 
Add bot/slack/ utilities:
- file.ts: Upload/download files, get file info, post messages with files
- parseSlackUrl.ts: Parse Slack message URLs to extract channel and timestamp
- stream.ts: Placeholder for Slack chat streaming (TODO)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
feat: add unified bot CLI with yargs
3942df9 
Add bot/cli.ts as consolidated entry point for all bot commands:
- GitHub: pr (spawn sub-agent), issue-search
- Registry: search custom nodes
- Slack: update, read-thread, upload/download files
- Notion: search (placeholder)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
feat: enhance Slack thread reader with YAML output and metadata
c0418ce 
- Add YAML output format instead of JSON
- Include file attachments, message attachments, and reactions in output
- Export readSlackThread function for reuse
- Move import.meta.main block to top of file

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
feat: major refactor of ComfyPR Bot with enhanced capabilities
b6c16f7 
- Add comprehensive logging with metadata and context
- Implement master-worker architecture documentation in agent prompt
- Add skills for code search, registry search, issue search, and pr-bot
- Enhance Slack integration with better error handling
- Add repository context for Comfy-Org projects
- Improve agent spawning with detailed instructions
- Add TODO tracking and progress reporting capabilities
- Refactor message handling and response formatting
- Add support for Claude Code integration

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
chore: update dependencies for bot capabilities
b51bd5d 
Add dependencies:
- @slack/socket-mode for Slack Socket Mode support
- execa for process execution
- faker for test data generation
- terminal-render for console output
- Update @slack/web-api and zod

Add pr-bot bin entry point and dev:bot script.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
fix: use relative path prefix for cache directories
f909c83 
Change cache paths from '.cache/' to './.cache/' for consistency
and clarity in github, notion, and slack cache stores.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
feat: add bot service to docker-compose and remove gh-service
4a663f2 
- Add bot service with Bun runtime and Claude credentials mount
- Remove deprecated gh-service
- Mount project directory and Claude credentials for bot

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
chore: remove deprecated github-webhook-event-type
334f4d7 
Remove unused run/github-webhook-event-type/index.ts file.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
feat: add Claude Code skills for bot integrations
2dd61c7 
Add .claude/skills/ directory with skill definitions:
- github-pr-bot: Spawn coding sub-agents for code changes
- github-issue-search: Search GitHub issues
- code-search: Search ComfyUI codebase
- registry-search: Search custom nodes registry
- slack-msg-update: Update Slack messages
- slack-thread-reader: Read Slack threads
- notion-search: Search Notion documentation
- pr-bot-cli: Unified PR bot CLI

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao @claude
feat: add GitHub webhook API endpoint with event storage
ce4562c 
Add app/api/webhook/github/ Next.js API route:
- route.ts: Handle GitHub webhook events with signature verification
- webhook-events.ts: Store events in SQLite database
- setup-indexes.ts: Create database indexes for efficient querying
- test-webhook.ts: Test utility for webhook endpoint
- route.spec.ts: Test suite for webhook handler
- README.md and USAGE_EXAMPLES.md: Documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao
Update bot configuration and documentation
5f85bb8
@snomiao @claude
fix: replace markdown_text with text in Slack API calls
6ef9a25 
- Use standard 'text' parameter instead of non-standard 'markdown_text'
- Update all slack.chat.postMessage and slack.chat.update calls
- Change CLI from 'codex-yes' to 'claude-yes'
- Fix dev:bot script to watch specific file

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@snomiao
chore: add codes/ and tmp/ to .gitignore
67e3ec3 
- Added codes/ directory to gitignore (temporary cloned repos)
- Added tmp/ directory to gitignore (temporary working files)
- Prevents accidental commit of temporary data
@snomiao
feat: add Smart Restart Manager for bot
0fd09ad 
- Implemented RestartManager class that watches for file changes
- Only restarts when bot is idle (no active tasks)
- Includes debouncing (1s) and idle checking (5s intervals)
- Comprehensive test suite with 6 passing tests
- Interactive example demonstrating restart behavior
- Replaces problematic --watch flag that interrupted tasks
@snomiao
test: add Working Tasks Manager test suite
4d7846c 
- Tests for addWorkingTask() and removeWorkingTask() helpers
- Validates duplicate prevention logic
- Tests multiple concurrent tasks handling
- All 5 tests passing
- Uses in-memory Keyv store for testing
@snomiao
feat: implement Working Tasks state management
e059a15 
- Added helper functions addWorkingTask() and removeWorkingTask()
- Replaced database queries with lightweight state key 'current-working-tasks'
- Updated --continue logic to read from working tasks state
- Integrated RestartManager for smart auto-restart on file changes
- Add task to working list when started (after quick response)
- Remove task from working list when completed or stopped by user
- Faster resume on restart, no database scanning needed
@snomiao
fix: update pr-bot CLI to use shorter slack upload command
95301f4 
- Added 'slack upload' as alias for 'slack upload-file'
- Fixed yargs command matching issue by reordering commands
- Updated examples to use shorter pr-bot slack upload syntax
- Moved upload command after post-with-files to fix partial matching
@snomiao
refactor: move bot-start.sh to bot/up.sh
951c2e4 
- Created bot/up.sh as new location for startup script
- Added deprecation notice to bot-start.sh
- bot-start.sh now shows warning and redirects to bot/up.sh
- Kept bot-start.sh for backward compatibility
- Both scripts have executable permissions
@snomiao
docs: add Smart Restart Manager documentation
b2f0f75 
- Added comprehensive user guide (RESTART.md)
- Added implementation summary (RESTART-SUMMARY.md)
- Added flow diagrams and workflows (RESTART-FLOW.md)
- Added quick reference guide (RESTART-QUICKREF.md)
- Documents usage, configuration, troubleshooting
- Includes comparison with --watch flag
@snomiao
docs: add Working Tasks state management documentation
4bc28d1 
- Comprehensive guide to new working tasks system
- Documents state structure and task lifecycle
- Explains resume on restart behavior
- Includes benefits comparison with old approach
- Troubleshooting and future improvements section
snomiao and others added 9 commits January 8, 2026 07:15
@snomiao
docs: update CLAUDE.md with new features
eee77f9 
- Added Working Tasks State Management section
- Updated Smart Restart Manager documentation
- Updated file paths to reference docs/ directory
- Updated bot/up.sh references (formerly bot-start.sh)
- Added usage examples and state structure
@snomiao
fix: handle undefined chunks in terminal render
5f2e7d5 
- Added null/undefined check before calling tr.write()
- Prevents TypeError when chunk is undefined
- Fixes runtime error: 'undefined is not an object (evaluating data.length)'
@snomiao
debug: add comprehensive logging for process stream issues
a29a1c3 
- Added stderr logging to capture process errors
- Added stdout/stderr availability checks
- Added buffer undefined/null checks with logging
- Added terminal render error handling with detailed logging
- Convert undefined buffers to empty strings to prevent crashes
- Log chunk type and length on terminal render errors

This will help diagnose why chunks are undefined in production.
@snomiao
Update bot configuration and startup script
f0d0b9d
@snomiao
Update bot index
825a66a
@snomiao
Update bot index
36cd878
@snomiao
Merge branch 'main' of github.com:Comfy-Org/Comfy-PR into sno-bot
c845d55
@snomiao
Update bot index
0879407
@snomiao @claude
Improve bot message filtering to prevent debugging messages from bein…
c48569d 
…g shown to users

- Created filterDebugMessages utility to remove debugging info from terminal output
- Filters: retry messages, file paths, timestamps, ANSI codes, process info, git internals
- Updated bot/index.ts to use filterInternalThoughts() before sending to Slack
- Added logging of filtered content to BOT-LOGS.md for debugging
- Enhanced prompt instructions with explicit examples of filtered patterns
- Added comprehensive test suite with 18 tests covering all filter patterns
- Updated CLAUDE.md documentation with usage examples and benefits

This ensures users only see clean, relevant task progress without internal debugging messages.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@vercel
Copy link
Contributor

vercel bot commented Jan 9, 2026 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
comfy-pr Error Error Jan 9, 2026 6:46am

Base automatically changed from sno-bot to main January 15, 2026 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@comfy-pr-bot @snomiao