Add event log for historical posterior visualization

[jjroelofs]

Summary

Add an event logging system to enable visualization of how posterior beliefs evolve over time during experiments. This allows marketers to see charts showing the full history of an experiment's learning process.

Background

Currently, the module only stores cumulative totals in rl_arm_data. To visualize how posteriors updated over time, we need historical snapshots at key trial points.

Requirements

New Module Configuration

Add the following settings to rl.settings:

Setting	Type	Default	Description
enable_event_log	boolean	false	Enable/disable event logging module-wide
event_log_max_rows	integer	100000	Maximum total rows in the event log table

Note: Configuration is module-level only, not per-experiment.

Sampling Strategy: Fixed Budget

Use a fixed budget approach with max 250 snapshots per arm:

• First 100 trials: Every trial (full resolution)
• Middle section: Evenly spaced snapshots from remaining budget
• Last 100 trials: Every trial (full resolution, rolling window)

This ensures:

• Dense data at experiment start (see early belief formation)
• Dense data at experiment end (see current/recent state)
• Sparse but representative data in middle (posteriors converge smoothly)

Adaptive Budget for Large Experiments

Experiments may have vastly different arm counts:

• A/B tests: 2-5 arms
• Multivariate: 5-20 arms
• Content recommendations: 100-500+ arms

To prevent large experiments from consuming the entire row budget, scale snapshots per arm based on arm count:

function getSnapshotsPerArm(int $armCount): int {
  $maxTotalPerExperiment = 10000; // Cap per experiment
  $idealPerArm = 250;
  
  return min($idealPerArm, max(20, floor($maxTotalPerExperiment / $armCount)));
}

Arm count	Snapshots per arm	Total rows
2	250	500
5	250	1,250
40	250	10,000
100	100	10,000
500	20	10,000

The first/last trial counts scale proportionally:

• 250 snapshots → first 100 + last 100 + 50 middle
• 100 snapshots → first 40 + last 40 + 20 middle
• 20 snapshots → first 8 + last 8 + 4 middle

Database Schema

Add new table rl_arm_snapshots:

$schema['rl_arm_snapshots'] = [
  'description' => 'Stores historical snapshots of arm state for visualization.',
  'fields' => [
    'id' => [
      'type' => 'serial',
      'unsigned' => TRUE,
      'not null' => TRUE,
    ],
    'experiment_id' => [
      'type' => 'varchar',
      'length' => 255,
      'not null' => TRUE,
    ],
    'arm_id' => [
      'type' => 'varchar',
      'length' => 255,
      'not null' => TRUE,
    ],
    'turns' => [
      'type' => 'int',
      'unsigned' => TRUE,
      'not null' => TRUE,
      'description' => 'Cumulative turns for this arm at snapshot time.',
    ],
    'rewards' => [
      'type' => 'int',
      'unsigned' => TRUE,
      'not null' => TRUE,
      'description' => 'Cumulative rewards for this arm at snapshot time.',
    ],
    'total_experiment_turns' => [
      'type' => 'int',
      'unsigned' => TRUE,
      'not null' => TRUE,
      'description' => 'Total experiment turns at snapshot time (x-axis for charts).',
    ],
    'created' => [
      'type' => 'int',
      'unsigned' => TRUE,
      'not null' => TRUE,
      'description' => 'Unix timestamp when snapshot was created.',
    ],
    'is_milestone' => [
      'type' => 'int',
      'size' => 'tiny',
      'unsigned' => TRUE,
      'not null' => TRUE,
      'default' => 0,
      'description' => 'Whether this is a permanent milestone (1) or recent window (0).',
    ],
  ],
  'primary key' => ['id'],
  'indexes' => [
    'experiment_lookup' => ['experiment_id', 'total_experiment_turns'],
  ],
];

Write Logic

On each turn/reward:

1. Check if enable_event_log is true
2. Get current arm count for experiment, calculate snapshots budget
3. Determine if snapshot should be taken based on scaled thresholds
4. Insert snapshot with is_milestone = 1 for budget points, is_milestone = 0 for recent window

Cleanup Logic (Cron)

Periodic cleanup to maintain constraints:

1. Per-arm budget: For each experiment/arm, keep:

   • All rows where is_milestone = 1
   • Recent window rows where is_milestone = 0 (scaled by arm count)
   • Delete older non-milestone rows

2. Global max rows: If total rows exceed event_log_max_rows:

   • Delete oldest non-milestone rows across all experiments
   • Ensure at least some recent data per experiment is preserved

Retrieval Method

Add method to fetch snapshot history for charting:

public function getSnapshotHistory(string $experimentId): array {
  // Returns all snapshots for experiment, ordered by total_experiment_turns
  // Each row contains: arm_id, turns, rewards, total_experiment_turns, created
}

Implementation Notes

• Snapshot logging happens in the same request as turn/reward recording (minimal overhead - one INSERT)
• No indexes beyond what's needed for retrieval by experiment
• created timestamp enables optional date-based x-axis on charts
• Cleanup runs via cron, not on every request
• Arm count lookup can be cached to avoid extra query on each write

Out of Scope

• Chart visualization UI (separate issue)
• Per-experiment configuration
• Time-based retention policies