Skip to content

Enhance Delve integration and improve agent functionality#1

Open
Ataxia123 wants to merge 178 commits intomainfrom
feat/bonfires_integration
Open

Enhance Delve integration and improve agent functionality#1
Ataxia123 wants to merge 178 commits intomainfrom
feat/bonfires_integration

Conversation

@Ataxia123
Copy link

Changes:

  • Added DELVE_BONFIRE_ID to .env.example for agent registration with Delve.
  • Introduced new SQL migration for delve_agent_config table to manage agent configurations and registration statuses.
  • Updated package.json to include a test script for better testing capabilities.
  • Enhanced TypeScript configuration with allowImportingTsExtensions for improved module handling.
  • Integrated Delve client in agent chat API to send messages to the Delve knowledge graph.
  • Implemented new components for managing knowledge context and episode timelines in the UI.
  • Added tests for Delve client and related utilities to ensure reliability.

This update lays the groundwork for improved AI agent knowledge management and interaction with the Delve knowledge graph.

### Changes:
- Added DELVE_BONFIRE_ID to .env.example for agent registration with Delve.
- Introduced new SQL migration for `delve_agent_config` table to manage agent configurations and registration statuses.
- Updated package.json to include a test script for better testing capabilities.
- Enhanced TypeScript configuration with `allowImportingTsExtensions` for improved module handling.
- Integrated Delve client in agent chat API to send messages to the Delve knowledge graph.
- Implemented new components for managing knowledge context and episode timelines in the UI.
- Added tests for Delve client and related utilities to ensure reliability.

This update lays the groundwork for improved AI agent knowledge management and interaction with the Delve knowledge graph.
- Only use force=true when Create2 fails (proves contract exists)
- On timeout, do actual on-chain verification before marking as deployed
- Prevents vaults from being marked deployed when they're not
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Delve knowledge-graph integration across backend, shared utilities, and UI to support agent registration/configuration, KG search, and episode timelines.

Changes:

  • Introduces a Delve client + types/utilities (KG context, episodes, categories) with Node tests.
  • Adds API routes for Delve settings/search/episodes and integrates KG context + stack updates into agent chat.
  • Adds UI components/tabs for knowledge collection toggles, KG browsing, and episode timeline views.

Reviewed changes

Copilot reviewed 31 out of 32 changed files in this pull request and generated 11 comments.

Show a summary per file
File Description
tsconfig.json Enables importing .ts extensions to support the new TS-module import style/tests.
src/lib/delve/types.ts Defines Delve API types and shared error/type utilities.
src/lib/delve/settings.ts Adds client-side helpers to fetch/update Delve settings and map registration status UI text.
src/lib/delve/settings.test.ts Adds unit tests for registration-status mapping.
src/lib/delve/registration.ts Adds a Supabase-backed registration + retry service for Delve agent records.
src/lib/delve/knowledgeGraphUtils.ts Adds client-side utilities for grouping/filtering KG entities and indexing relationships.
src/lib/delve/knowledgeGraphUtils.test.ts Adds unit tests for KG utility functions.
src/lib/delve/index.ts Adds a barrel export for Delve modules.
src/lib/delve/episodeUtils.ts Adds parsing/formatting utilities for episodes, timestamps, status, entities, categories, counts.
src/lib/delve/episodeUtils.test.ts Adds unit tests for episode utilities.
src/lib/delve/client.ts Implements the Delve HTTP client (register, stack, KG search, episodes, details).
src/lib/delve/client.test.ts Adds unit tests ensuring expected payloads for Delve client requests.
src/lib/delve/chatContext.ts Adds KG-context mapping and stack-message construction for chat integration.
src/lib/delve/chatContext.test.ts Adds unit tests for KG-context mapping.
src/hooks/useAgents.ts Extends chat message model to include optional KG context from the chat API.
src/components/KnowledgeGraphList.tsx Adds UI to search/browse KG entities/relationships and jump to timeline.
src/components/KnowledgeCollectionToggle.tsx Adds UI toggle to enable/disable knowledge collection and show registration status.
src/components/EpisodeTimelineModal.tsx Adds UI modal/panel to list episodes with infinite scroll and detail chips.
src/components/ChatKnowledgeContext.tsx Adds UI to show KG context attached to assistant messages.
src/components/CategoriesTab.tsx Adds UI to browse/sort/filter categories derived from KG search results.
src/components/AgentsSection.tsx Wires timeline modal + knowledge modal tab/query deep-links from chat/agent actions.
src/components/AgentKnowledgeModal.tsx Adds new knowledge tabs (categories/timeline/KG) and Delve settings display/toggle.
src/components/AgentChatModal.tsx Shows “learning” indicator and renders KG context for assistant messages; adds callbacks.
src/app/api/agents/[id]/delve/shared.ts Adds shared API helpers for error mapping, episode limit parsing, KG response shaping.
src/app/api/agents/[id]/delve/shared.test.ts Adds unit tests for the shared Delve API helpers.
src/app/api/agents/[id]/delve/settings/route.ts Adds GET/PUT endpoint for Delve settings backed by delve_agent_config.
src/app/api/agents/[id]/delve/search/route.ts Adds endpoint to search KG via Delve and normalize results for the UI.
src/app/api/agents/[id]/delve/episodes/route.ts Adds endpoint to fetch latest episodes for a registered Delve agent.
src/app/api/agents/[id]/chat/route.ts Integrates KG search context into the prompt and sends paired messages to Delve stack.
package.json Adds a test script using Node’s test runner.
migrations/delve_integration.sql Creates delve_agent_config table and indexes + RLS policy.
.env.example Documents DELVE_BONFIRE_ID for Delve integration configuration.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +134 to +142
.update({
knowledge_collection_enabled,
updated_at: new Date().toISOString(),
})
.eq("agent_id", id)
.select(
"knowledge_collection_enabled, registration_status, delve_agent_id, registration_error",
)
.maybeSingle();
Copy link

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This uses update() on delve_agent_config, which won’t create a row for agents that don’t yet have config (and the handler later returns 404). If the UI toggle is meant to work before registration runs, switch to an upsert/insert-on-missing strategy so enabling/disabling knowledge collection works even when no config row exists yet.

Suggested change
.update({
knowledge_collection_enabled,
updated_at: new Date().toISOString(),
})
.eq("agent_id", id)
.select(
"knowledge_collection_enabled, registration_status, delve_agent_id, registration_error",
)
.maybeSingle();
.upsert(
{
agent_id: id,
knowledge_collection_enabled,
updated_at: new Date().toISOString(),
},
{
onConflict: "agent_id",
},
)
.select(
"knowledge_collection_enabled, registration_status, delve_agent_id, registration_error",
)
.single();

Copilot uses AI. Check for mistakes.
Comment on lines +1324 to +1330
.then((result) => {
console.log("[Delve] Sent messages to stack", {
agentId: id,
delveAgentId,
messageCount: result.message_count,
});
})
Copy link

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New console.log usage here will trigger the repo’s no-console lint rule (eslint.config.mjs warns on console.log/info/debug and recommends using the logger utility). Consider switching this to createLogger (or an existing module logger) or using console.warn/error only if you intentionally want console output.

Copilot uses AI. Check for mistakes.
Comment on lines +316 to +317
// Singleton instance for convenience
export const delveClient = new DelveClient();
Copy link

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This eagerly constructs a DelveClient at module load. If DELVE_API_URL/DELVE_API_KEY aren’t set, importing the module will throw before route-level try/catch can handle it. Make the singleton lazy (create on first use), or remove the exported singleton entirely and require explicit instantiation where needed.

Copilot uses AI. Check for mistakes.
Comment on lines +1 to +3
export * from "./types.ts";
export { DelveClient, delveClient } from "./client.ts";
export { DelveRegistrationService, delveRegistrationService } from "./registration.ts";
Copy link

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-exporting singletons from this barrel file forces evaluation of client.ts/registration.ts (and their side effects) whenever @/lib/delve is imported. This can crash unrelated routes/builds if required env vars aren’t present. Consider exporting only the classes/types here, and expose singleton getters (or a separate server-only entrypoint) to avoid module-load side effects.

Copilot uses AI. Check for mistakes.
}
}

export const delveRegistrationService = new DelveRegistrationService();
Copy link

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This eagerly constructs delveRegistrationService at module load and will throw if Supabase/Delve env vars are missing. Prefer exporting a factory/lazy singleton (created on first call) so importing the module doesn’t hard-fail in environments where Delve registration isn’t configured.

Copilot uses AI. Check for mistakes.
DelveClientError,
RegisterAgentResponse,
RegistrationResult,
RegistrationStatus,
Copy link

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused import RegistrationStatus.

Suggested change
RegistrationStatus,

Copilot uses AI. Check for mistakes.
<CategoriesTab
agentId={agent.id}
userAddress={userAddress}
isActive={isOpen && activeTab === "categories"}
Copy link

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This use of variable 'isOpen' always evaluates to true.

Copilot uses AI. Check for mistakes.
<EpisodeTimelinePanel
agentId={agent.id}
userAddress={userAddress}
isActive={isOpen && activeTab === "timeline"}
Copy link

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This use of variable 'isOpen' always evaluates to true.

Copilot uses AI. Check for mistakes.
<KnowledgeGraphList
agentId={agent.id}
userAddress={userAddress}
isActive={isOpen && activeTab === "graph"}
Copy link

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This use of variable 'isOpen' always evaluates to true.

Copilot uses AI. Check for mistakes.
CREATE POLICY "Users can manage delve config"
ON delve_agent_config
FOR ALL
USING (true);
Copy link

Copilot AI Jan 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The RLS policy "Users can manage delve config" is defined as FOR ALL USING (true), which effectively grants any role with basic table privileges full read/write access to every row in delve_agent_config. In a typical Supabase setup this allows arbitrary clients (including those using the public anon key) to view and modify other agents’ Delve configuration (e.g., bonfire_id, delve_agent_id, knowledge_collection_enabled, registration_status), enabling cross-tenant data exposure and tampering. Restrict this policy to the appropriate roles and owners (for example, scoping access to the agent owner or to service_role only and adding a WITH CHECK clause) so that users can only read and update configuration tied to their own agents.

Suggested change
USING (true);
USING (auth.role() = 'service_role')
WITH CHECK (auth.role() = 'service_role');

Copilot uses AI. Check for mistakes.
kmjones1979 and others added 25 commits January 21, 2026 17:14
- Safe expects (bytes, bytes, uint256[2]) not (bytes, bytes, uint256, uint256)
- Encode r and s as uint256[2] array instead of separate values
- Fixes GS026 (invalid owner) error during vault transaction execution
Safe's WebAuthn verifier expects the closing } to be included in clientDataFields
since it does NOT add it during reconstruction. Also fixed the extraction logic
to properly find the challenge end quote.
Based on permissionless.js reference implementation:
1. clientDataFields must be STRING type, not bytes
2. clientDataFields must NOT include leading comma
3. clientDataFields must NOT include trailing closing brace

Format: 'origin":"https://...","crossOrigin":false' (no comma prefix, no closing brace)
When a vault has a Smart Wallet owner (which itself has a WebAuthn owner),
we need two levels of EIP-1271 contract signature wrapping:

1. INNER: For Smart Wallet's checkNSignatures
   - r = SafeWebAuthnSharedSigner (0x94a4F6...5c2)
   - s = offset
   - v = 0
   - dynamic = WebAuthn ABI-encoded signature

2. OUTER: For Vault's checkNSignatures
   - r = Smart Wallet address
   - s = offset
   - v = 0
   - dynamic = complete INNER signature

This allows the vault to verify the Smart Wallet owner, which then
verifies the WebAuthn signature through SafeWebAuthnSharedSigner.
- Fix EVM/Solana auth to check server session first (not re-verify expired nonces)
- Add visibility change listeners to auto-refresh sessions when app becomes visible
- Update PasskeyProvider and EmailAuthProvider with session-first approach
- Users now stay logged in for 7 days without requiring re-authentication
- Session automatically extends when user returns to app
- Added link to https://docs.spritz.chat
- Added link to https://docs.spritz.chat/blog
- Shortened Privacy Policy and Terms of Service labels for cleaner layout
- Removed WalletConnect link to reduce clutter
- Complete rewrite with detailed legal terms for decentralized dApp
- Added prominent Financial Risk and Loss Disclaimer section
- Users explicitly acknowledge no liability for crypto/digital asset losses
- Covers blockchain risks, irreversible transactions, wallet security
- Added table of contents with navigation
- Emphasized decentralization, no data custody, end-to-end encryption
- Added IPFS storage warnings and third-party technology disclaimers
- Limitation of liability capped at $100 USD
- Added Docs/Blog links to login screen footer
- Add localStorage caching for smart wallet addresses (fixes 'Unable to load wallet address' on mobile PWA when session expires)
- API now returns stored address for passkey users who lost their credentials
- Updated WalletModal UI to show appropriate warnings for lost passkey scenario
- Removed redundant session lock screen (all wallet operations require signatures)
- Verified 7-day session cookie duration
Set wallet_intentionally_disconnected flag at start of logout to ensure
all auth tabs are shown instead of forcing wallet sign-in
- Skip DM toast notifications when chat is open with that user
- Skip channel toast notifications when that channel is open
- Prevents redundant notifications while already viewing messages
- Add messagingKey.ts for deterministic key derivation from EOA signature or passkey PRF
- Add useMessagingKey hook for React integration
- Add MessagingKeyProvider context for app-wide state management
- Add EnableMessagingModal component for user onboarding
- Add MessagingStatus component for settings display

Key features:
- EOA wallet users: sign once, key derived from wallet signature
- Passkey users: authenticate with PRF extension for deterministic key
- Email/Digital ID users: must add passkey to enable messaging
- Keys persisted in localStorage - no re-signing on page refresh
- Public key only stored on server for ECDH key exchange
- Backward compatible with existing keypairs
- Use X25519 via TweetNaCl for deterministic key derivation
- EOA wallets: Same signature → Same key on ANY device
- Passkey PRF: Same passkey → Same key (with synced passkeys)
- Zero server storage: Only public keys stored on Supabase
- No backup needed: Keys are derived deterministically

Security improvements:
- Private keys never leave device
- No encrypted keys stored on server
- True zero-knowledge architecture
- Add MessagingKeyUpgradeBanner for legacy key users (one-time prompt)
- Add MessagingKeyStatusInline component for Settings page
- Simplified messaging encryption status in Settings (no backup UI)
- Auto-detect legacy vs deterministic keys
- Upgrade button for wallet users with legacy keys
- Enable button for users without messaging key
- Shows key source and cross-device status
- Banner auto-dismisses after upgrade or 'Later' click
- Update upgrade banner to use app's dark zinc theme (not blue/purple gradient)
- Use Spritz orange (#FF5500) for buttons and accent colors
- Only show upgrade banner for wallet users (passkey users excluded)
- Add authType prop to Settings for proper messaging key context
- Show appropriate status messages for passkey/email/digitalid users
- Passkey users see 'Your passkey provides secure messaging' instead of upgrade CTA
- passkey-fallback is now treated as 'good' for passkey auth users
- Shows 'Passkey' label instead of 'Passkey (Legacy)' for passkey users
- Green 'Active' indicator instead of amber 'Legacy'
- Message: 'Secured by your passkey' for passkey users
BUG: When EOA users added a passkey, their wallet_type was changed to 'passkey'
This caused them to see 'Passkey Required' when trying to send from Smart Wallet

FIX:
1. smart-wallet API now uses session.authMethod as source of truth
2. Auto-fixes database wallet_type if user logged in as 'wallet' but DB says 'passkey'
3. Passkey register/login no longer changes wallet_type for existing 'wallet' users

This fixes the issue for user grixl and any other EOA users affected.
…lRootRef type in EpisodeTimelineModal

### Changes:
- Simplified episode ID usage in ChatKnowledgeContext by introducing a local variable for clarity.
- Updated the type of scrollRootRef in EpisodeTimelineModal to allow for null values, enhancing type safety.
BUG: Passkeys with signer capability were showing 'Login Only' even though signing worked
- The on-chain ownership check was failing due to address format/chain differences

FIX:
1. If passkey has signer capability (public_key_x + safe_signer_address), trust it
2. For passkey users, treat these as 'Wallet Key' / 'active' status
3. Block deletion of any passkey with signer capability for non-wallet users
4. Don't rely solely on on-chain check which can fail for various reasons

This prevents users from accidentally deleting passkeys that control their funds.
… upgraded' message

1. Passkey users with ANY messaging key now show 'Active' (not 'Legacy')
   - They can't sign with a wallet to upgrade, so any key is valid for them
   - Source label shows 'Passkey' instead of 'Legacy' for passkey users

2. Changed chat status from 'Encrypted (peer hasn't upgraded yet)' to just 'Encrypted'
   - The old message was misleading - could be either user's key not synced
   - Simpler UX, messages are still encrypted either way
Direct check for authType === 'passkey' in the render, rather than relying
on isDeterministic state which may not update when authType changes.

For passkey users:
- Always shows green 'Active' status
- Shows '✓ Secured by your passkey' message
- Never shows upgrade prompt (they can't sign with wallet)
When users clear site data or have legacy keys:
- EOA users: Shows banner prompting to sign and restore deterministic key
- Passkey users: Shows banner directing them to Settings

Scenarios handled:
1. No key at all (site data cleared)
2. Legacy/random key instead of deterministic
3. Key exists locally but not synced to Supabase

For EOA users, signing restores the SAME deterministic key,
allowing them to decrypt all previous messages.

This does NOT affect Smart Wallet signing - completely separate systems.
- Fetch passkey credential ID from Supabase for passkey users
- Pass it to SettingsModal so MessagingKeyStatus can enable with passkey
- Now passkey users can enable messaging from Settings with 'Enable with Passkey' button
- Remove allowCredentials restriction so browser shows passkey picker
- User can now choose from ALL their passkeys for this site
- Matches the UX of login and transaction signing
- Also fixed rpId normalization to use parent domain (spritz.chat)
- Fixed chat sorting for channels/groups with fallback timestamps
kmjones1979 and others added 30 commits January 27, 2026 10:17
Duplicate Detection:
- Fingerprint-based matching (normalized name + date + city)
- Show duplicate badges in preview mode
- Auto-deselect duplicates in preview
- Within-batch duplicate prevention

Efficiency:
- Skip past events toggle (enabled by default)
- Content hash for unchanged page detection (skip re-scrape)
- Batch geocoding with caching (1 request per unique location)
- Rate limiting for Nominatim API (1/sec)

UI Improvements:
- Preview shows new/duplicate counts
- Result shows duplicate count separately
- Skip Past Events toggle in scrape modal
- Duplicate events highlighted in yellow

Migration Update:
- Added content_hash column to event_sources
AdminLayout:
- Use 100dvh for proper full-viewport height (better PWA/mobile support)
- Convert to flex layout with header/main/nav sections
- Move navigation to center of header on desktop (md+)
- Reduce header padding for more content space
- Make main content area scrollable via flex-1 overflow-hidden

Events Page:
- Compact inline stats instead of card grid
- Compact filter bar with smaller inputs
- Smaller bulk action bar (inline when active)
- Scrollable events container with proper flex layout
- More columns in grid view (up to 6 on 2xl screens)
- Compact event cards with less padding
- Compact table view with smaller cells
- Responsive column hiding (Type hidden on mobile, Location on smaller screens)
- Sticky compact pagination at bottom of scroll area

Mobile/PWA:
- Proper safe-area-inset handling for notched devices
- Bottom nav visible without cutting content
- Touch-friendly but space-efficient controls
- Remove max-w-[1800px] from AdminLayout header and main
- Use full width with responsive padding (px-3 sm:px-4 lg:px-6 xl:px-8)
- Increase modal widths (max-w-4xl, max-w-3xl, max-w-5xl)
- Add more grid columns on ultra-wide screens (2xl:grid-cols-7)
- Increase gap spacing on larger screens (lg:gap-4)
- Store sourceUrl variable to handle preview saves correctly
- Add fallback for sourceId generation when URL is missing
- Wrap URL parsing in try-catch to prevent crashes
- Add error handling for AI API calls with detailed logging
- Don't fail entire request if source save fails
- Better error messages for debugging
Extraction Improvements:
- Increase content limit from 50k to 100k characters
- Increase AI output tokens from 8k to 16k for more events
- Better AI prompt emphasizing 'extract ALL events'
- More detailed logging at each step

Filtering Changes:
- Change skip_past_events default to false (less aggressive)
- Change skip_if_unchanged default to false (always extract)
- Add detailed logging for skipped events (past, invalid, duplicates)
- Log fingerprint matching for debugging

Debugging:
- Log extracted count before/after filtering
- Log sample events for verification
- Log duplicate detection details
- Log final results summary
- Log at every step of the scrape process
- Log request start with timestamp
- Log configuration checks (DB, AI, Firecrawl)
- Log admin verification
- Log request body keys and options
- Log fetchContent calls and results
- Enhanced error logging with full stack traces
- Log mode (preview vs full scrape vs save)
- All logs prefixed with [Event Scrape] for easy filtering
- Increase modal width from max-w-3xl to max-w-6xl for better visibility
- Auto-close modal 1.5 seconds after successful scrape
- Auto-close modal after saving preview events
- Clear preview state when modal closes
- Add 'Crons' button in main header next to Scrape and Add Event buttons
- Button opens Scheduled Sources panel to manage all cron jobs
- Panel shows: active status, scrape interval, next scrape time, last scrape, errors
- Actions available: toggle active/pause, manual scrape, delete source
- Modal now uses 95vw width (full browser width with small padding)
- Modal height set to 95vh (fits viewport height)
- Form content is scrollable, buttons are sticky at bottom
- Buttons always visible without scrolling
- Responsive design: smaller padding/text on mobile
- Compact button labels on mobile (Scrape vs Scrape & Save)
- Form has id for submit button outside form element
- Handle markdown code blocks in AI response
- Strip code block markers
- Fix common JSON issues (trailing commas, quote escaping)
- Better error messages with more context
- Validate parsed result is an array
- Multiple parsing strategies with fallbacks
- Enhanced logging for debugging parse failures
- Initialize extractedEvents as empty array
- Add validation that extractedEvents is an array before processing
- Add null/empty checks for JSON parsing
- Better error handling for database queries
- More defensive coding to catch edge cases
Backend:
- Better error serialization (handle non-serializable errors)
- Include error stack trace in response (first 10 lines)
- Add timestamp to error response
- More detailed error info for debugging

Frontend:
- Check HTTP status before parsing JSON
- Display detailed error messages from API
- Show error details in alerts
- Better error logging in console
- Try multiple ways to access response.text (property, function, nested)
- Log response object structure for debugging
- Better error messages for unexpected response formats
- More detailed error logging for AI API failures
- Use response.text directly as property (matches other code)
- Better error logging when response is empty
- Log response structure for debugging
- More detailed error information
- Return proper error response instead of throwing
- Log result structure for debugging
- More detailed error information for fetchContent failures
- Check if result is null/undefined before accessing properties
- Better error message for insufficient content
- Log content length for debugging
…atabase schema fixes

- Add batch database inserts (100 events at a time) for 50-100x faster performance
- Optimize duplicate detection query (only check same source or recent events)
- Add retry mechanism for AI extraction when few events found
- Improve geocoding with parallel batch processing
- Enhance AI prompt with more explicit instructions to extract ALL events
- Increase token limit to 32k and content analysis to 200k chars
- Add comprehensive logging for debugging
- Fix infinite scroll auto-detection for cryptonomads.org
- Add migration for missing content_hash column in shout_event_sources
- Add database schema verification scripts
- Added comprehensive duplicate detection using source_id, URLs, and fingerprints
- Improved scrape route to check duplicates in priority order (source_id → URLs → fingerprints)
- Added cleanup script for removing duplicate events from database
- Added 'Select All Drafts' button to admin events page for bulk publishing
- Enhanced URL normalization for better duplicate detection
- Improved event name extraction and RSVP URL capture in AI prompts
- Optimized default scrape settings for infinite scroll pages like cryptonomads.org
- Fixed JSON parsing to handle truncated AI responses
- Added batch duplicate detection within scrape operations
… update Alien ID SDK

- Add deduplication logic to /api/events to prevent duplicate events from displaying
  - Uses multi-strategy fingerprinting (source_id, URLs, name+date+location)
  - Keeps oldest event when duplicates found
  - Applied before pagination for accurate counts

- Add event interests feature (Interested/Going)
  - New migration: shout_event_interests table
  - API endpoint: /api/events/[id]/interest
  - Display counts and attendee lists on events page
  - Restricted attendee list to authenticated users

- Add calendar integration
  - New calendar utility library with ICS, Google, Outlook support
  - Add to Calendar button on events page

- Update Alien ID SDK to v1.0.39
  - Update @alien_org/sso-sdk-react package
  - Add alien.org and *.alien.org to CSP policy (script-src, connect-src, frame-src)

- Enhance events page display
  - Show date ranges, full location, organizer website
  - Display blockchain focus tags and description previews
…ndling

- Events page: tabbed nav (All, Main, Side, Conferences, Hackathons, etc.)
- Events page: uniform card heights, main/side event badges
- Events page: admin edit from /events (modal + API), RSVP primary CTA
- Events page: RSVP from rsvp_url or registration-style event_url (lu.ma, etc.)
- Admin scrape: suggested sources (ETHDenver Side Events, Cryptonomads main)
- Admin scrape: SideEvents URL pattern → higher scroll count
- Scrape prompt: stronger rsvp_url extraction, side-events categorization
- Admin scrape: friendly error on Failed to fetch / network change
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants