Skip to content

Conversation

@daveey
Copy link
Contributor

@daveey daveey commented Dec 4, 2025

TL;DR

Added a new script to generate a WandB table for experiment tracking.

What changed?

Created a new Python script make_wandb_table.py that:

  • Connects to WandB API to fetch runs from the "metta-research/metta" project
  • Parses run configurations to extract key parameters:
    • Total GPUs used (from run name patterns like "1x4")
    • Muesli model enablement status
    • Unroll steps (from config or run name)
    • Core ResNet layers (from config or run name)
  • Processes runs in parallel using ThreadPoolExecutor for efficiency
  • Outputs a formatted table with the extracted information

How to test?

  1. Ensure you have the required dependencies: pandas and wandb
  2. Run the script: python make_wandb_table.py
  3. Verify that it connects to WandB, fetches runs with names starting with "daveey.", and displays a table with the extracted parameters

Why make this change?

This script provides a convenient way to analyze and compare experiment configurations across multiple WandB runs. It extracts key parameters that affect model performance and training efficiency, making it easier to track and compare different experimental setups without having to manually inspect each run in the WandB UI.

Copy link
Contributor Author

daveey commented Dec 4, 2025

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@daveey daveey changed the title cp Add script to create WandB table with run metadata Dec 6, 2025
@daveey daveey force-pushed the daveey-crl branch 2 times, most recently from 774be36 to fdea434 Compare December 8, 2025 08:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants