Weave leaderboard configure DOCS-1994 #2051
Conversation
|
Images automagically compressed by Calibre's image-actions ✨ Compression reduced images by 79.5%, saving 251.6 KB.
|
📚 Mintlify Preview Links✨ Added (1 total)📄 Pages (1)
📝 Changed (1 total)⚙️ Other (1)
🤖 Generated automatically when Mintlify deployment succeeds |
🔗 Link Checker Results✅ All links are valid! No broken links were detected in the changed files. Tip Redirects detected: If you see redirects for internal docs.wandb.ai links, check if they have trailing slashes. Mintlify automatically removes trailing slashes, causing redirects like:
Fix: Remove trailing slashes from links to avoid unnecessary redirects. |
|
Images automagically compressed by Calibre's image-actions ✨ Compression reduced images by 77.8%, saving 383.5 KB.
|
zbirenbaum
left a comment
zbirenbaum
left a comment
There was a problem hiding this comment.
Awesome work, thank you! One line I had a suggestion on but totally your call, not blocking
|
|
||
| The Leaderboard automatically updates to include them, without requiring manual reconfiguration. | ||
|
|
||
| This lets you use views as living leaderboards that evolve alongside your experiments. |
There was a problem hiding this comment.
What do you think about: This lets you use views as persistent leaderboards which evolve alongside your experiments
I get what you are trying to say here but something feels a little off with the phrasing 'living'
dbrian57
left a comment
dbrian57
left a comment
There was a problem hiding this comment.
Hey this looks good. Just requesting changes because I'm curious about the last section of the doc.
|
|
||
|
|
||
|
|
||
| When working with Weave Evaluations, you can easily visualize and customize your experiment results as Leaderboards. Dynamic Leaderboard views automatically stay up to date as new evaluation runs are added. |
There was a problem hiding this comment.
| When working with Weave Evaluations, you can easily visualize and customize your experiment results as Leaderboards. Dynamic Leaderboard views automatically stay up to date as new evaluation runs are added. | |
| When working with Weave Evaluations, you can visualize and customize your experiment results as Leaderboards. Dynamic Leaderboard views automatically stay up to date as new evaluation runs are added. |
I see it thrown around a lot, but I think we should try to avoid the word "easily" if possible: https://developers.google.com/style/word-list#easy
|
|
||
| ## Visualize Evaluation results in a Leaderboard | ||
|
|
||
| When your project contains Weave Evaluation data, you can use the evaluation table to quickly create a Weave Leaderboard view based on a filtered subset of results. |
There was a problem hiding this comment.
We don't have a set standard or anything, but I think leading into the instructions with a transition clause is good, like this:
"To create a Weave Leaderboard:
- Navigate..."
Up to you
| - Remain clickable so you can still open the underlying reference in the side panel | ||
| - Automatically propagate anywhere the Leaderboard view is used | ||
|
|
||
| This makes it easier to compare experiments using meaningful, human-readable names without changing the underlying objects. |
There was a problem hiding this comment.
experiments or evaluations?
|
|
||
| ### Switch between saved views | ||
|
|
||
| Click the **menu icon (☰)** next to the Evaluations page title to open saved views. You can: |
| @@ -0,0 +1,99 @@ | |||
| --- | |||
| title: "Dynamic Leaderboards in Evaluations" | |||
There was a problem hiding this comment.
We're trying to get away from the sort of noun phrase stuff so I'd opt for something like
| title: "Dynamic Leaderboards in Evaluations" | |
| title: "Create dynamic Leaderboards in Evaluations" |
|
|
||
| --- | ||
|
|
||
| ## Dynamic updates as evaluations change |
There was a problem hiding this comment.
Do we need this section? I feel like this could be a part of the intro.
Description
Scott has indicated that the flow of creating a leaderboard FROM the evaluation data (using ‘visualize’ button) is the preferred path to educate customers instead of using the Weave sidebar to enter directly through ‘Leaderboard’ and then have to try to pick out the data you want.
(Adding as new page under Evaluations)
New Leaderboard flow
From eng loom the following is new and of interest to customers:
In Evaluations: New Configure button/panel
Shows models: active/deactive;
Rename models, datasets, and scorers. Name updates automatically in Leaderboard
Metrics: control coloring by color inverting what is green - higher better or lower better.
Save to view: view can be pulled up as a view of the evaluation (to show that customized leaderboard).
Testing
mint dev)mint broken-links)Related issues