feat: comprehensive RL module overhaul with human-readable names, cold start fixes, and total turns correction

[jjroelofs]

Summary

Fixes #7, Fixes #9, Fixes #10, Fixes #11

This PR implements a comprehensive overhaul of the RL module, introducing human-readable experiment names, complete cold start handling, total turns calculation fixes, enhanced symlink support, and production-ready code quality improvements.

Problems Solved

• Debug code in production (Production-ready updates: symlink support and debug cleanup #7): Console logs and debug statements cluttered production deployments
• Thompson sampling determinism (Thompson sampling returns identical scores during cold start #9): Identical beta values caused predictable sorting instead of exploration
• Cryptic experiment names (Reports show cryptic SHA1 hashes instead of human-readable experiment names #10): Reports showed SHA1 hashes instead of readable names like "content_recent:default"
• Total turns miscalculation (Total turns showing 0 in overview reports despite individual experiments having data #11): Overview reports showed 0 total turns despite individual experiments having data
• Symlink compatibility (Production-ready updates: symlink support and debug cleanup #7): Module failed in development setups with symlinked installations
• Incomplete cold start: New content didn't get proper exploration scores for fair exposure

Changes Made

Human-Readable Experiment Names (#10)

• Database schema enhancement: Added experiment_name field to rl_experiment_registry table
• Registry interface expansion: ExperimentRegistryInterface::register() now accepts optional experiment names
• Reports integration: ReportsController displays stored names instead of UUIDs in admin interface
• Backward compatibility: Existing experiments continue working, new ones get readable names
• Database update hook: rl_update_8002() adds field to existing installations

Total Turns Calculation Fix (#11)

• Fixed recordTurns() method: Now correctly increments total_turns by arm count instead of calling recordTurn() multiple times
• Proper multi-armed bandit accounting: Total turns = sum of all individual arm exposures
• Database consistency: Updated existing incorrect totals to match arm data
• Accurate reporting: Overview reports now show correct total turn counts

Complete Cold Start Solution

• Enhanced ExperimentManager: getThompsonScores() now accepts requested_arms parameter
• Proper initialization: New arms get 0 turns/0 rewards for maximum exploration scoring
• Centralized logic: All cold start handling consolidated in RL module instead of consuming modules
• Comprehensive coverage: Handles both partial cold start (some arms missing) and complete cold start (no experiment data)

Thompson Sampling Improvements (#9)

• Tie-breaker mechanism: Added micro-randomization to prevent identical scores during cold start
• Statistical integrity: Maintains beta distribution properties while ensuring unique scores
• Exploration guarantee: Prevents deterministic ordering when all arms have identical statistics
• Cold start optimization: Critical for proper randomization during initial learning phase

Enhanced rl.php Endpoint (#7)

• Symlink compatibility: Robust path detection using $_SERVER['SCRIPT_FILENAME'] fallback
• Security hardening: Regex validation for experiment UUIDs and arm IDs
• Error handling: Proper HTTP status codes (400, 500) with descriptive messages
• Drupal 10/11 compatibility: Updated deprecated FILTER_SANITIZE_STRING usage
• Performance optimization: Streamlined bootstrap and error handling

Production Code Quality (#7)

• Debug cleanup: Removed all console.log and debug statements from production code
• Coding standards: Full Drupal coding standards compliance with automated fixing
• Documentation: Essential comments retained, verbose explanations removed
• Error handling: Comprehensive exception handling with meaningful messages

Technical Implementation

Human-Readable Names Storage (#10)

-- New database field
ALTER TABLE rl_experiment_registry 
ADD COLUMN experiment_name VARCHAR(255) NULL 
COMMENT 'Human-readable experiment name';

Enhanced Cold Start Logic

// Initialize missing arms for cold start
if (\!empty($requested_arms)) {
  foreach ($requested_arms as $arm_id) {
    if (\!isset($arms_data[$arm_id])) {
      $arms_data[$arm_id] = (object) [
        'arm_id' => $arm_id,
        'turns' => 0,
        'rewards' => 0,
      ];
    }
  }
}

Corrected Total Turns Calculation (#11)

// Record total turns = number of arms shown (sum of individual turns)
$this->database->merge('rl_experiment_totals')
  ->key(['experiment_uuid' => $experiment_uuid])
  ->expression('total_turns', 'total_turns + :inc', [':inc' => $arm_count])
  ->execute();

Thompson Sampling with Tie-Breaker (#9)

$base_score = $this->randBeta($alpha, $beta);
$tie_breaker = mt_rand(1, 999) / 1000000;
$scores[$id] = $base_score + $tie_breaker;

Database Schema Changes

New Field Added (#10)

• rl_experiment_registry.experiment_name (VARCHAR 255, nullable)
• Stores human-readable experiment names like "blog_posts:default"
• Backward compatible - existing experiments continue working

Update Hook

• rl_update_8002() safely adds field to existing installations
• No data migration required
• Automatic execution during drush updb

Data Correction (#11)

-- Existing totals corrected automatically  
UPDATE rl_experiment_totals et 
SET total_turns = (
  SELECT SUM(turns) FROM rl_arm_data ad 
  WHERE ad.experiment_uuid = et.experiment_uuid
);

Benefits

✅ User-friendly reporting - Experiment names show as "content_recent:default" instead of SHA1 hashes (#10)
✅ Accurate statistics - Total turns display correctly as sum of all arm exposures (#11)
✅ Proper cold start - New content gets appropriate exploration scores for fair exposure
✅ Universal compatibility - Works with standard and symlinked Drupal installations (#7)
✅ Production ready - Clean code without debug statements or verbose logging (#7)
✅ Performance optimized - Efficient database operations and streamlined endpoint
✅ Developer friendly - Clear error messages and comprehensive documentation
✅ Statistically sound - Proper multi-armed bandit implementation with exploration guarantees (#9)

Reporting Improvements

Before (#10, #11):

Experiment ID: 6da7b208a42c9db4cb166b294f19a41f54f03b44
Total Turns: 0 (despite individual arms having data)

After:

Experiment ID: content_recent:block_1
Total Turns: 45 (correct sum of all arm exposures)

Testing

✅ Human-readable experiment names displaying correctly in admin reports (#10)
✅ Total turns calculation verified with multiple arm configurations (#11)
✅ Cold start behavior confirmed for new experiments and new arms
✅ Symlink compatibility tested in development environments (#7)
✅ Thompson sampling randomization verified during cold start conditions (#9)
✅ Database update hook tested on existing installations
✅ Drupal coding standards compliance verified
✅ Backward compatibility confirmed with existing experiment data

Compatibility

• Drupal 10/11 compatible with deprecated function updates
• Standard installations - Works with traditional module placement
• Symlinked installations - Enhanced path detection for development setups (Production-ready updates: symlink support and debug cleanup #7)
• Existing data preservation - No breaking changes to experiment data
• Progressive enhancement - New features available immediately, old experiments continue working

Related Issues & PRs

• Production updates (Production-ready updates: symlink support and debug cleanup #7): Debug cleanup and symlink support implemented
• Thompson sampling fix (Thompson sampling returns identical scores during cold start #9): Tie-breaker mechanism added for proper exploration
• Human-readable names (Reports show cryptic SHA1 hashes instead of human-readable experiment names #10): Complete solution with database schema and UI updates
• Total turns calculation (Total turns showing 0 in overview reports despite individual experiments having data #11): Fixed accounting and corrected existing data
• Enables AI Sorting enhancements: feat: comprehensive AI Sorting overhaul with entity-agnostic support and human-readable experiment names ai_sorting#11 (entity-agnostic support, cold start fixes)
• Powers AI Sorting reporting: Integration with human-readable experiment names
• Supports AI Sorting entity expansion: Proper cold start for any content type

This comprehensive overhaul transforms the RL module into a production-ready, statistically sound machine learning platform with proper cold start handling, accurate reporting, and user-friendly administration interfaces that work reliably across all Drupal installation scenarios.

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com