From c44b9f07660eadaef27e8eadf0f7bc7c9ade9583 Mon Sep 17 00:00:00 2001 From: Matt Linville Date: Tue, 18 Nov 2025 15:33:06 -0800 Subject: [PATCH 1/3] Weave generated reference improvements (#48) * Fix Weave reference generation bugs Observed in https://github.com/wandb/docs/pull/1888 * Temporarily include fixes summary report --- WEAVE-REFERENCE-FIXES-SUMMARY.md | 159 ++++++++++++++++++ .../reference-generation/weave/fix_casing.py | 109 +++--------- .../weave/generate_python_sdk_docs.py | 4 + .../weave/generate_service_api_spec.py | 2 - .../weave/generate_typescript_sdk_docs.py | 27 +-- .../weave/sync_openapi_spec.py | 70 ++++++++ .../weave/update_service_api_landing.py | 15 +- .../weave/update_weave_toc.py | 42 ++--- 8 files changed, 296 insertions(+), 132 deletions(-) create mode 100644 WEAVE-REFERENCE-FIXES-SUMMARY.md diff --git a/WEAVE-REFERENCE-FIXES-SUMMARY.md b/WEAVE-REFERENCE-FIXES-SUMMARY.md new file mode 100644 index 0000000000..61128c0b41 --- /dev/null +++ b/WEAVE-REFERENCE-FIXES-SUMMARY.md @@ -0,0 +1,159 @@ +# Weave Reference Generation Script Fixes + +This document summarizes the fixes applied to the Weave reference documentation generation scripts based on PR #1888 feedback. + +## Issues Fixed + +### 1. Models Reference Files Being Renamed (CRITICAL BUG) +**Problem**: `fix_casing.py` was incorrectly targeting `models/ref/python/public-api` files instead of Weave reference docs. + +**Fix**: Updated `fix_casing.py` to only target `weave/reference/python-sdk` files. +- Changed path from `models/ref/python/public-api` to `weave/reference/python-sdk` +- Removed the logic that was renaming Models API files (ArtifactCollection, etc.) +- Added clear comments indicating this should NEVER touch Models reference docs + +**Files Modified**: +- `scripts/reference-generation/weave/fix_casing.py` + +### 2. TypeScript SDK Using PascalCase Filenames +**Problem**: TypeScript SDK files were being generated with PascalCase filenames (e.g., `Dataset.mdx`, `WeaveClient.mdx`), which causes Git case-sensitivity issues. + +**Fix**: Updated generation scripts to use lowercase filenames throughout. +- Modified `generate_typescript_sdk_docs.py` to convert filenames to lowercase when creating `.mdx` files +- Updated function and type-alias extraction to use lowercase filenames +- Updated internal links to use lowercase paths + +**Files Modified**: +- `scripts/reference-generation/weave/generate_typescript_sdk_docs.py` (lines 259, 319-320, 369-370, 379) +- `scripts/reference-generation/weave/fix_casing.py` (simplified to just convert to lowercase) + +### 3. H1 in service-api/index.mdx +**Problem**: The generated `service-api/index.mdx` had both a frontmatter title and an H1, which is redundant in Mintlify. + +**Fix**: Removed the H1 heading since Mintlify uses the frontmatter title. + +**Files Modified**: +- `scripts/reference-generation/weave/generate_service_api_spec.py` (line 31) + +### 4. Duplicate H3 Headings in service-api.mdx +**Problem**: The `service-api.mdx` file had duplicate category sections (e.g., "### Calls" appeared on both line 23 and line 158), listing the same endpoints twice. + +**Fix**: Added deduplication logic to prevent duplicate categories and duplicate endpoints. +- Track which categories have been written to prevent duplicate H3 headings +- Deduplicate endpoints within each category by (method, path) tuple +- This prevents the same endpoint from being listed multiple times if it appears in the OpenAPI spec with duplicate tags + +**Files Modified**: +- `scripts/reference-generation/weave/update_service_api_landing.py` (lines 99-118) + +### 5. Markdown Table Formatting Errors (------ lines) +**Problem**: Python SDK docs contained standalone lines with just dashes (`------`) which break markdown parsing. + +**Example**: In `trace_server_interface.mdx`, lines like 22, 30, 39, etc. had `------` that created invalid table structures. + +**Fix**: Added regex pattern to remove these malformed table separators. +- Pattern: `\n\s*------+\s*\n` → `\n\n` +- This removes lines that are just dashes with optional whitespace + +**Files Modified**: +- `scripts/reference-generation/weave/generate_python_sdk_docs.py` (lines 258-260) + +## Testing Recommendation + +Before merging, test the fixes by running the reference generation locally: + +```bash +# From the docs repository root +cd scripts/reference-generation/weave +python generate_weave_reference.py +``` + +Then verify: +1. No files in `models/ref/python/public-api` were modified +2. All TypeScript SDK files in `weave/reference/typescript-sdk/` have lowercase filenames +3. `weave/reference/service-api/index.mdx` has no H1 heading +4. `weave/reference/service-api.mdx` has no duplicate H3 category headings +5. No `------` lines in `weave/reference/python-sdk/trace_server/trace_server_interface.mdx` +6. In `docs.json`, modules under `weave/reference/python-sdk/trace/` are grouped as "Core" (not "Other") +7. In `docs.json`, the Service API `openapi` configuration uses the local spec (not a GitHub URL) if sync_openapi_spec.py was run with `--use-local` + +### 6. Incorrect Section Grouping ("Core" → "Other") +**Problem**: Python SDK modules in the `trace/` directory were being incorrectly grouped as "Other" instead of "Core" in docs.json navigation. + +**Root Cause**: The path checking logic in `update_weave_toc.py` was checking `if parts[0] == "weave"`, but paths are relative to `python-sdk/`, so `parts[0]` is actually the module subdirectory (`trace`, `trace_server`, etc.), not `weave`. + +**Fix**: Corrected the path checking logic to check the actual first path component. +- Changed from checking `parts[0] == "weave"` then `parts[1] == "trace"` +- To directly checking `parts[0] == "trace"`, `parts[0] == "trace_server"`, etc. + +**Files Modified**: +- `scripts/reference-generation/weave/update_weave_toc.py` (lines 33-45) + +### 7. OpenAPI Configuration Being Overwritten +**Problem**: `update_weave_toc.py` was unconditionally overwriting the OpenAPI spec configuration in docs.json to use a remote URL, ignoring the local spec that `sync_openapi_spec.py` downloads and configures. + +**Impact**: Even though `sync_openapi_spec.py` downloads the OpenAPI spec locally and can configure docs.json to use it, `update_weave_toc.py` would immediately overwrite it with a remote GitHub URL, defeating the purpose of the local spec. + +**Fix**: Removed the Service API OpenAPI configuration code from `update_weave_toc.py`. This script should only manage Python/TypeScript SDK navigation, not the OpenAPI spec source. +- Deleted lines 209-224 that were setting `page["openapi"]` to remote URLs +- Added comment noting that OpenAPI configuration is managed by `sync_openapi_spec.py` + +**Files Modified**: +- `scripts/reference-generation/weave/update_weave_toc.py` (lines 206-207) + +### 8. Missing Root Module Documentation (CRITICAL - WEAVE PACKAGE REGRESSION) +**Problem**: The generated `python-sdk.mdx` file is only 8 lines (just frontmatter), completely missing all the important API documentation for functions like `init()`, `publish()`, `ref()`, `get()`, etc. + +**Expected**: The current version (Weave 0.52.10) has 2074 lines documenting all the core Weave functions and classes. + +**Root Cause**: **This is a WEAVE PACKAGE REGRESSION, not a script bug.** + +Something changed in Weave between versions **0.52.10** (current docs) and **0.52.16** (PR version) that broke documentation generation for the root `weave` module. The generation scripts haven't changed, and lazydocs hasn't changed - so this is an upstream issue in the Weave package itself. + +Possible causes: +1. Changes to `weave/__init__.py` that affect how the module exports its public API +2. Module structure refactoring that lazydocs can't handle +3. New import patterns or lazy loading that breaks introspection + +**Status**: **CRITICAL UPSTREAM BUG** - This makes the Python SDK documentation completely unusable for version 0.52.16. + +**Action Required**: Report this to the Weave team immediately: +1. File an issue: https://github.com/wandb/weave/issues +2. Include: "Documentation generation broken in 0.52.16 - root module exports not discoverable by lazydocs" +3. Mention: "Works fine in 0.52.10, broken in 0.52.16" +4. Tag: @dbrian57 or relevant Weave maintainers + +**Recommendation**: +- **DO NOT MERGE PR #1888** - it will break Python SDK documentation +- Either: Fix the Weave package and regenerate docs +- Or: Stay on 0.52.10 documentation until the Weave package is fixed + +**Files to Investigate** (in Weave repo): +- `weave/__init__.py` between versions 0.52.10 and 0.52.16 +- Any structural changes to the weave package in that version range + +### 9. OpenAPI Spec Validation (New Feature) +**Enhancement**: Added validation to detect issues in the OpenAPI spec itself, which can help identify upstream problems. + +**Features**: +- Detects duplicate endpoint definitions (same method+path defined multiple times) +- Identifies endpoints appearing in multiple categories/tags +- Warns when critical issues like duplicate endpoints are found +- Suggests reporting issues to the Weave team when spec problems are detected + +**Files Modified**: +- `scripts/reference-generation/weave/sync_openapi_spec.py` (added `validate_spec()` function and integration in `main()`) + +This will help identify if duplicate H3s or other issues originate from the OpenAPI spec rather than our generation scripts. + +## Files Modified Summary + +1. `scripts/reference-generation/weave/fix_casing.py` +2. `scripts/reference-generation/weave/generate_typescript_sdk_docs.py` +3. `scripts/reference-generation/weave/generate_service_api_spec.py` +4. `scripts/reference-generation/weave/update_service_api_landing.py` +5. `scripts/reference-generation/weave/generate_python_sdk_docs.py` +6. `scripts/reference-generation/weave/update_weave_toc.py` +7. `scripts/reference-generation/weave/sync_openapi_spec.py` (new validation feature) + +All fixes are backward compatible and will take effect on the next reference documentation generation run. diff --git a/scripts/reference-generation/weave/fix_casing.py b/scripts/reference-generation/weave/fix_casing.py index b1225629c5..7971d5475d 100755 --- a/scripts/reference-generation/weave/fix_casing.py +++ b/scripts/reference-generation/weave/fix_casing.py @@ -12,108 +12,47 @@ from pathlib import Path def fix_typescript_casing(base_path): - """Fix TypeScript SDK file casing.""" - print("Fixing TypeScript SDK file casing...") + """Fix TypeScript SDK file casing - ensure all files use lowercase.""" + print("Fixing TypeScript SDK file casing to lowercase...") - ts_base = Path(base_path) / "weave/reference/typescript-sdk/weave" + ts_base = Path(base_path) / "weave/reference/typescript-sdk" if not ts_base.exists(): print(f" TypeScript SDK path not found: {ts_base}") return - # Define correct names for each directory - casing_rules = { - "classes": { - "dataset": "Dataset", - "evaluation": "Evaluation", - "weaveclient": "WeaveClient", - "weaveobject": "WeaveObject", - }, - "interfaces": { - "callschema": "CallSchema", - "callsfilter": "CallsFilter", - "weaveaudio": "WeaveAudio", - "weaveimage": "WeaveImage", - }, - "functions": { - # Functions should be lowercase/camelCase - "init": "init", - "login": "login", - "op": "op", - "requirecurrentcallstackentry": "requireCurrentCallStackEntry", - "requirecurrentchildsummary": "requireCurrentChildSummary", - "weaveaudio": "weaveAudio", - "weaveimage": "weaveImage", - "wrapopenai": "wrapOpenAI", - }, - "type-aliases": { - "op": "Op", # Type alias Op is uppercase - "opdecorator": "OpDecorator", - "messagesprompt": "MessagesPrompt", - "stringprompt": "StringPrompt", - } - } + # All TypeScript SDK files should use lowercase filenames for consistency + # This applies to classes, functions, interfaces, and type-aliases + subdirs_to_check = ["classes", "functions", "interfaces", "type-aliases"] - for dir_name, rules in casing_rules.items(): - dir_path = ts_base / dir_name + for subdir in subdirs_to_check: + dir_path = ts_base / subdir if not dir_path.exists(): continue for file in dir_path.glob("*.mdx"): - basename = file.stem.lower() - if basename in rules: - correct_name = rules[basename] - if file.stem != correct_name: - new_path = file.parent / f"{correct_name}.mdx" - print(f" Renaming: {file.name} → {correct_name}.mdx") - shutil.move(str(file), str(new_path)) + # Convert filename to lowercase + lowercase_name = file.stem.lower() + if file.stem != lowercase_name: + new_path = file.parent / f"{lowercase_name}.mdx" + print(f" Renaming: {file.name} → {lowercase_name}.mdx") + shutil.move(str(file), str(new_path)) def fix_python_casing(base_path): - """Fix Python SDK file casing.""" - print("Fixing Python SDK file casing...") + """Fix Python SDK file casing for WEAVE reference docs only.""" + print("Fixing Weave Python SDK file casing...") - py_base = Path(base_path) / "models/ref/python/public-api" + # IMPORTANT: This should ONLY touch Weave reference docs, never Models reference docs + py_base = Path(base_path) / "weave/reference/python-sdk" if not py_base.exists(): - print(f" Python SDK path not found: {py_base}") + print(f" Weave Python SDK path not found: {py_base}") return - # Python class files that should be uppercase - uppercase_files = { - "artifactcollection": "ArtifactCollection", - "artifactcollections": "ArtifactCollections", - "artifactfiles": "ArtifactFiles", - "artifacttype": "ArtifactType", - "artifacttypes": "ArtifactTypes", - "betareport": "BetaReport", - "file": "File", - "member": "Member", - "project": "Project", - "registry": "Registry", - "run": "Run", - "runartifacts": "RunArtifacts", - "sweep": "Sweep", - "team": "Team", - "user": "User", - } - - # Files that should remain lowercase - lowercase_files = ["api", "artifacts", "automations", "files", "projects", - "reports", "runs", "sweeps", "_index"] + # For Weave Python SDK, we generally want lowercase filenames + # Only specific files might need special casing - currently none known + # Most Weave modules use lowercase with underscores (e.g., weave_client.mdx) - for file in py_base.glob("*.mdx"): - basename = file.stem.lower() - - if basename in uppercase_files: - correct_name = uppercase_files[basename] - if file.stem != correct_name: - new_path = file.parent / f"{correct_name}.mdx" - print(f" Renaming: {file.name} → {correct_name}.mdx") - shutil.move(str(file), str(new_path)) - elif basename in lowercase_files: - # Ensure these stay lowercase - if file.stem != basename: - new_path = file.parent / f"{basename}.mdx" - print(f" Renaming: {file.name} → {basename}.mdx") - shutil.move(str(file), str(new_path)) + print(f" Weave Python SDK files are generated with correct casing") + print(f" No casing changes needed for Weave reference documentation") def main(): """Main function to fix all casing issues.""" diff --git a/scripts/reference-generation/weave/generate_python_sdk_docs.py b/scripts/reference-generation/weave/generate_python_sdk_docs.py index 46f6d0e9bd..63f22ee16b 100755 --- a/scripts/reference-generation/weave/generate_python_sdk_docs.py +++ b/scripts/reference-generation/weave/generate_python_sdk_docs.py @@ -255,6 +255,10 @@ def generate_module_docs(module, module_name: str, src_root_path: str, version: # Remove ` at the start of lines that don't have a closing content = re.sub(r'^- `([^`\n]*?)$', r'- \1', content, flags=re.MULTILINE) + # Remove malformed table separators that lazydocs sometimes generates + # These appear as standalone lines with just dashes (------) which break markdown parsing + content = re.sub(r'\n\s*------+\s*\n', '\n\n', content) + # Fix parameter lists that have been broken by lazydocs # Strategy: Parse all parameters into a structured format, then reconstruct them properly def fix_parameter_lists(text): diff --git a/scripts/reference-generation/weave/generate_service_api_spec.py b/scripts/reference-generation/weave/generate_service_api_spec.py index ad7bb0f456..6f078d713c 100755 --- a/scripts/reference-generation/weave/generate_service_api_spec.py +++ b/scripts/reference-generation/weave/generate_service_api_spec.py @@ -28,8 +28,6 @@ def main(): description: "REST API endpoints for the Weave service" --- -# Weave Service API - The Weave Service API provides REST endpoints for interacting with the Weave tracing service. ## Available Endpoints diff --git a/scripts/reference-generation/weave/generate_typescript_sdk_docs.py b/scripts/reference-generation/weave/generate_typescript_sdk_docs.py index d980f999ff..7d189e06b8 100755 --- a/scripts/reference-generation/weave/generate_typescript_sdk_docs.py +++ b/scripts/reference-generation/weave/generate_typescript_sdk_docs.py @@ -255,8 +255,9 @@ def convert_to_mintlify_format(docs_dir): # Just ensure .md extension is removed (already done above) pass - # Write as .mdx file - mdx_file = md_file.with_suffix('.mdx') + # Write as .mdx file with lowercase filename (avoid Git case sensitivity issues) + lowercase_stem = md_file.stem.lower() + mdx_file = md_file.parent / f"{lowercase_stem}.mdx" mdx_file.write_text(content) # Remove original .md file @@ -314,17 +315,18 @@ def extract_members_to_separate_files(docs_path): {func_content.replace(f'### {func_name}', f'# {func_name}')}""" - # Write the function file - func_file = functions_dir / f"{func_name}.mdx" + # Write the function file with lowercase filename (avoid Git case sensitivity issues) + func_filename = func_name.lower() + func_file = functions_dir / f"{func_filename}.mdx" func_file.write_text(func_file_content) - functions_found.append(func_name) - print(f" ✓ Extracted {func_name}.mdx") + functions_found.append(func_filename) + print(f" ✓ Extracted {func_filename}.mdx") if functions_found: # Remove the detailed function documentation from index content = function_pattern.sub('', content) - # Update the Functions section with links + # Update the Functions section with links (functions_found already has lowercase names) functions_section = "\n### Functions\n\n" for func in functions_found: functions_section += f"- [{func}](functions/{func})\n" @@ -363,17 +365,18 @@ def extract_members_to_separate_files(docs_path): {alias_content.replace(f'### {alias_name}', f'# {alias_name}')}""" - # Write the type alias file - alias_file = type_aliases_dir / f"{alias_name}.mdx" + # Write the type alias file with lowercase filename (avoid Git case sensitivity issues) + alias_filename = alias_name.lower() + alias_file = type_aliases_dir / f"{alias_filename}.mdx" alias_file.write_text(alias_file_content) - print(f" ✓ Extracted {alias_name}.mdx") + print(f" ✓ Extracted {alias_filename}.mdx") # Remove all extracted type aliases from index content = type_alias_pattern.sub('', content) - # Update Type Aliases section with links to all extracted type aliases + # Update Type Aliases section with links to all extracted type aliases (use lowercase filenames) if type_aliases: - type_aliases_links = [f"- [{name}](type-aliases/{name})" for _, name in type_aliases if _.startswith(f"### {name}\n\nƬ ")] + type_aliases_links = [f"- [{name}](type-aliases/{name.lower()})" for _, name in type_aliases if _.startswith(f"### {name}\n\nƬ ")] if type_aliases_links: type_aliases_section = "\n### Type Aliases\n\n" + "\n".join(sorted(type_aliases_links)) + "\n" diff --git a/scripts/reference-generation/weave/sync_openapi_spec.py b/scripts/reference-generation/weave/sync_openapi_spec.py index fff938e415..d42690e639 100755 --- a/scripts/reference-generation/weave/sync_openapi_spec.py +++ b/scripts/reference-generation/weave/sync_openapi_spec.py @@ -44,6 +44,62 @@ def spec_hash(spec: dict) -> str: return hashlib.sha256(spec_str.encode()).hexdigest() +def validate_spec(spec: dict) -> list: + """ + Validate the OpenAPI spec for potential issues. + Returns list of warning messages about spec issues. + """ + warnings = [] + + paths = spec.get("paths", {}) + + # Track endpoint definitions to detect duplicates + endpoint_map = {} # (method, path) -> [operation_ids] + tag_endpoint_map = {} # tag -> [(method, path)] + + for path, path_item in paths.items(): + for method in ["get", "post", "put", "delete", "patch"]: + if method not in path_item: + continue + + operation = path_item[method] + operation_id = operation.get("operationId", "") + tags = operation.get("tags", []) + + # Check for duplicate endpoint definitions + endpoint_key = (method.upper(), path) + if endpoint_key not in endpoint_map: + endpoint_map[endpoint_key] = [] + endpoint_map[endpoint_key].append(operation_id) + + # Track endpoints by tag to detect if endpoints appear in multiple tags + for tag in tags: + if tag not in tag_endpoint_map: + tag_endpoint_map[tag] = [] + tag_endpoint_map[tag].append(endpoint_key) + + # Check for actual duplicates (same endpoint with different operation IDs) + for endpoint_key, operation_ids in endpoint_map.items(): + if len(operation_ids) > 1: + method, path = endpoint_key + warnings.append(f" ⚠ Duplicate endpoint: {method} {path} defined {len(operation_ids)} times with operation IDs: {operation_ids}") + + # Check for endpoints appearing in multiple categories (tags) + endpoint_tag_count = {} + for tag, endpoints in tag_endpoint_map.items(): + for endpoint in endpoints: + if endpoint not in endpoint_tag_count: + endpoint_tag_count[endpoint] = [] + endpoint_tag_count[endpoint].append(tag) + + for endpoint, tags in endpoint_tag_count.items(): + if len(tags) > 1: + method, path = endpoint + warnings.append(f" ℹ Endpoint {method} {path} appears in multiple categories: {tags}") + + return warnings + + def compare_specs(local_spec: dict, remote_spec: dict) -> Tuple[bool, list]: """ Compare local and remote specs. @@ -131,6 +187,20 @@ def main(): print(" ✗ No local spec and couldn't fetch remote spec") return 1 + # Validate the remote spec for issues + print("\n Validating OpenAPI spec...") + spec_warnings = validate_spec(remote_spec) + if spec_warnings: + print(" ⚠ OpenAPI spec validation warnings:") + for warning in spec_warnings: + print(warning) + if any("Duplicate endpoint" in w for w in spec_warnings): + print("\n ⚠ CRITICAL: Duplicate endpoint definitions found!") + print(" This may indicate an issue in the upstream OpenAPI spec.") + print(" Consider reporting this to the Weave team: https://github.com/wandb/weave/issues") + else: + print(" ✓ OpenAPI spec validation passed") + # Load local spec local_spec = load_local_spec(local_spec_path) diff --git a/scripts/reference-generation/weave/update_service_api_landing.py b/scripts/reference-generation/weave/update_service_api_landing.py index 77abeb8052..bcfd038378 100755 --- a/scripts/reference-generation/weave/update_service_api_landing.py +++ b/scripts/reference-generation/weave/update_service_api_landing.py @@ -96,13 +96,24 @@ def generate_endpoints_section(endpoints: Dict[str, List[Tuple[str, str, str, st if tag not in category_order: category_order.append(tag) + # Track which categories we've already written to prevent duplicate H3s + written_categories = set() + for category in category_order: - if category not in endpoints: + if category not in endpoints or category in written_categories: continue - + + written_categories.add(category) lines.append(f"\n### {category}\n\n") + # Deduplicate endpoints within the category by (method, path) to avoid listing the same endpoint twice + seen_endpoints = set() for method, path, operation_id, summary in endpoints[category]: + endpoint_key = (method, path) + if endpoint_key in seen_endpoints: + continue + seen_endpoints.add(endpoint_key) + url = generate_endpoint_url(operation_id, category) lines.append(f"- **[{method} {path}]({url})** - {summary}\n") diff --git a/scripts/reference-generation/weave/update_weave_toc.py b/scripts/reference-generation/weave/update_weave_toc.py index aef71eda26..b08883f521 100755 --- a/scripts/reference-generation/weave/update_weave_toc.py +++ b/scripts/reference-generation/weave/update_weave_toc.py @@ -30,17 +30,15 @@ def get_generated_python_modules(): parts = list(rel_path.parts) parts[-1] = parts[-1].replace('.mdx', '') - # Determine the group based on path - if len(parts) >= 2: - if parts[0] == "weave": - if parts[1] == "trace": - group = "Core" - elif parts[1] == "trace_server": - group = "Trace Server" - elif parts[1] == "trace_server_bindings": - group = "Trace Server Bindings" - else: - group = "Other" + # Determine the group based on path structure + # Paths are relative to python-sdk/, so parts[0] is the module directory + if len(parts) >= 1: + if parts[0] == "trace": + group = "Core" + elif parts[0] == "trace_server": + group = "Trace Server" + elif parts[0] == "trace_server_bindings": + group = "Trace Server Bindings" else: group = "Other" else: @@ -205,26 +203,8 @@ def update_docs_json(python_modules, typescript_items, service_endpoints): print(f"✓ Updated TypeScript SDK with {sum(len(items) for items in typescript_items.values())} items") updated = True - # Update Service API - for page in reference_pages: - if isinstance(page, dict) and page.get("group") == "Service API": - # Point to the versioned OpenAPI spec in the Weave repo - # This spec already has the necessary filters applied - import os - version = os.environ.get('WEAVE_VERSION', 'latest') - - # The version should already be resolved by the Python SDK generation - # which converts "latest" to an actual version number - if version and version != "latest": - # Use the specific version tag - openapi_url = f"https://raw.githubusercontent.com/wandb/weave/v{version}/sdks/node/weave.openapi.json" - else: - # Fallback to master if somehow we don't have a version - openapi_url = "https://raw.githubusercontent.com/wandb/weave/master/sdks/node/weave.openapi.json" - - page["openapi"] = openapi_url - print(f"✓ Updated Service API to use versioned OpenAPI spec: {openapi_url}") - updated = True + # Note: Service API OpenAPI configuration is managed by sync_openapi_spec.py + # We don't modify it here to preserve the local vs remote spec choice break break From 29d49b9d2094f8ad2130e88f37845e21a95ad7d5 Mon Sep 17 00:00:00 2001 From: Matt Linville Date: Tue, 18 Nov 2025 15:48:59 -0800 Subject: [PATCH 2/3] fix: Add root pages and remove service-api/index.mdx generation Fixes issues from PR review: - Add weave/reference/python-sdk as first page in Python SDK section - Add weave/reference/typescript-sdk as first page in TypeScript SDK section - Remove service-api/index.mdx generation (landing page is service-api.mdx) - Keep OpenAPI config managed by sync_openapi_spec.py only This ensures proper navigation structure in docs.json. --- .../weave/generate_service_api_spec.py | 54 +++---------------- .../weave/update_weave_toc.py | 15 ++---- 2 files changed, 12 insertions(+), 57 deletions(-) diff --git a/scripts/reference-generation/weave/generate_service_api_spec.py b/scripts/reference-generation/weave/generate_service_api_spec.py index 6f078d713c..9a8df03e8f 100755 --- a/scripts/reference-generation/weave/generate_service_api_spec.py +++ b/scripts/reference-generation/weave/generate_service_api_spec.py @@ -12,57 +12,19 @@ def main(): """Main function.""" print("Service API configuration:") - print(" Using remote OpenAPI spec: https://trace.wandb.ai/openapi.json") - print(" Mintlify will generate documentation for all 41 endpoints") + print(" Using OpenAPI spec from sync_openapi_spec.py") + print(" Mintlify will generate documentation for endpoints") print("") - # Create the service-api directory structure + # Create the service-api directory structure for openapi.json + # Note: The landing page is service-api.mdx (not service-api/index.mdx) + # and is managed by update_service_api_landing.py service_api_dir = Path("weave/reference/service-api") service_api_dir.mkdir(parents=True, exist_ok=True) - # Create an index file if it doesn't exist - index_file = service_api_dir / "index.mdx" - if not index_file.exists(): - index_content = """--- -title: "Service API" -description: "REST API endpoints for the Weave service" ---- - -The Weave Service API provides REST endpoints for interacting with the Weave tracing service. - -## Available Endpoints - -This documentation is automatically generated from the OpenAPI specification at https://trace.wandb.ai/openapi.json. - -The API includes endpoints for: -- **Calls**: Start, end, update, query, and manage traces -- **Tables**: Create, update, and query data tables -- **Files**: Upload and manage file attachments -- **Objects**: Store and retrieve versioned objects -- **Feedback**: Collect and query user feedback -- **Costs**: Track and query usage costs -- **Inference**: OpenAI-compatible inference endpoints - -## Authentication - -Most endpoints require authentication. Include your W&B API key in the request headers: - -``` -Authorization: Bearer YOUR_API_KEY -``` - -## Base URL - -All API requests should be made to: - -``` -https://trace.wandb.ai -``` -""" - index_file.write_text(index_content) - print(f"✓ Created Service API index at {index_file}") - - print("✓ Service API setup complete!") + print("✓ Service API directory structure ready") + print(" Note: Landing page at weave/reference/service-api.mdx") + print(" Note: OpenAPI spec at weave/reference/service-api/openapi.json") if __name__ == "__main__": diff --git a/scripts/reference-generation/weave/update_weave_toc.py b/scripts/reference-generation/weave/update_weave_toc.py index b08883f521..9c342273ed 100755 --- a/scripts/reference-generation/weave/update_weave_toc.py +++ b/scripts/reference-generation/weave/update_weave_toc.py @@ -129,12 +129,8 @@ def update_docs_json(python_modules, typescript_items, service_endpoints): # Update Python SDK for page in reference_pages: if isinstance(page, dict) and page.get("group") == "Python SDK": - # Keep the index page if it exists - new_pages = [] - for existing_page in page.get("pages", []): - if isinstance(existing_page, str) and existing_page.endswith("/index"): - new_pages.append(existing_page) - break + # Always start with the root page + new_pages = ["weave/reference/python-sdk"] # Add the grouped modules if "Core" in python_modules and python_modules["Core"]: @@ -168,11 +164,8 @@ def update_docs_json(python_modules, typescript_items, service_endpoints): # Update TypeScript SDK for page in reference_pages: if isinstance(page, dict) and page.get("group") == "TypeScript SDK": - # Keep the index and README if they exist - new_pages = [] - for existing_page in page.get("pages", []): - if isinstance(existing_page, str) and (existing_page.endswith("/index") or existing_page.endswith("/README")): - new_pages.append(existing_page) + # Always start with the root page + new_pages = ["weave/reference/typescript-sdk"] # Add the categorized items with proper casing if "classes" in typescript_items and typescript_items["classes"]: From 51104346d6f149c304784548b1f85f33469152f4 Mon Sep 17 00:00:00 2001 From: mdlinville <7674613+mdlinville@users.noreply.github.com> Date: Wed, 19 Nov 2025 00:02:36 +0000 Subject: [PATCH 3/3] chore: Update reference documentation (Weave 0.52.16) --- docs.json | 43 +- weave/reference/python-sdk.mdx | 2065 -------- weave/reference/python-sdk/trace/feedback.mdx | 86 +- weave/reference/python-sdk/trace/op.mdx | 197 +- weave/reference/python-sdk/trace/util.mdx | 197 +- .../python-sdk/trace/weave_client.mdx | 262 +- .../trace_server/trace_server_interface.mdx | 3081 ++++++++--- .../remote_http_trace_server.mdx | 502 +- weave/reference/service-api.mdx | 135 + weave/reference/service-api/openapi.json | 4510 +++++++++++++++-- .../classes/EvaluationLogger.mdx | 110 - .../classes/{Dataset.mdx => dataset.mdx} | 26 +- .../{Evaluation.mdx => evaluation.mdx} | 16 +- .../classes/evaluationlogger.mdx | 163 + ...{MessagesPrompt.mdx => messagesprompt.mdx} | 16 +- .../typescript-sdk/classes/objectref.mdx | 134 + .../{ScoreLogger.mdx => scorelogger.mdx} | 30 +- .../{StringPrompt.mdx => stringprompt.mdx} | 16 +- .../{WeaveClient.mdx => weaveclient.mdx} | 44 +- .../{WeaveObject.mdx => weaveobject.mdx} | 12 +- ...y.mdx => requirecurrentcallstackentry.mdx} | 0 ...ary.mdx => requirecurrentchildsummary.mdx} | 0 .../{weaveAudio.mdx => weaveaudio.mdx} | 0 .../{weaveImage.mdx => weaveimage.mdx} | 0 .../{wrapOpenAI.mdx => wrapopenai.mdx} | 0 .../{CallSchema.mdx => callschema.mdx} | 32 +- .../{CallsFilter.mdx => callsfilter.mdx} | 18 +- .../{WeaveAudio.mdx => weaveaudio.mdx} | 6 +- .../{WeaveImage.mdx => weaveimage.mdx} | 6 +- weave/reference/typescript-sdk/readme.mdx | 379 ++ .../type-aliases/{Op.mdx => op.mdx} | 0 .../{OpDecorator.mdx => opdecorator.mdx} | 0 32 files changed, 8614 insertions(+), 3472 deletions(-) delete mode 100644 weave/reference/typescript-sdk/classes/EvaluationLogger.mdx rename weave/reference/typescript-sdk/classes/{Dataset.mdx => dataset.mdx} (66%) rename weave/reference/typescript-sdk/classes/{Evaluation.mdx => evaluation.mdx} (81%) create mode 100644 weave/reference/typescript-sdk/classes/evaluationlogger.mdx rename weave/reference/typescript-sdk/classes/{MessagesPrompt.mdx => messagesprompt.mdx} (63%) create mode 100644 weave/reference/typescript-sdk/classes/objectref.mdx rename weave/reference/typescript-sdk/classes/{ScoreLogger.mdx => scorelogger.mdx} (61%) rename weave/reference/typescript-sdk/classes/{StringPrompt.mdx => stringprompt.mdx} (62%) rename weave/reference/typescript-sdk/classes/{WeaveClient.mdx => weaveclient.mdx} (71%) rename weave/reference/typescript-sdk/classes/{WeaveObject.mdx => weaveobject.mdx} (61%) rename weave/reference/typescript-sdk/functions/{requireCurrentCallStackEntry.mdx => requirecurrentcallstackentry.mdx} (100%) rename weave/reference/typescript-sdk/functions/{requireCurrentChildSummary.mdx => requirecurrentchildsummary.mdx} (100%) rename weave/reference/typescript-sdk/functions/{weaveAudio.mdx => weaveaudio.mdx} (100%) rename weave/reference/typescript-sdk/functions/{weaveImage.mdx => weaveimage.mdx} (100%) rename weave/reference/typescript-sdk/functions/{wrapOpenAI.mdx => wrapopenai.mdx} (100%) rename weave/reference/typescript-sdk/interfaces/{CallSchema.mdx => callschema.mdx} (78%) rename weave/reference/typescript-sdk/interfaces/{CallsFilter.mdx => callsfilter.mdx} (80%) rename weave/reference/typescript-sdk/interfaces/{WeaveAudio.mdx => weaveaudio.mdx} (63%) rename weave/reference/typescript-sdk/interfaces/{WeaveImage.mdx => weaveimage.mdx} (63%) create mode 100644 weave/reference/typescript-sdk/readme.mdx rename weave/reference/typescript-sdk/type-aliases/{Op.mdx => op.mdx} (100%) rename weave/reference/typescript-sdk/type-aliases/{OpDecorator.mdx => opdecorator.mdx} (100%) diff --git a/docs.json b/docs.json index 906e189a1d..0ae296c681 100644 --- a/docs.json +++ b/docs.json @@ -982,14 +982,15 @@ { "group": "Classes", "pages": [ - "weave/reference/typescript-sdk/classes/Dataset", - "weave/reference/typescript-sdk/classes/Evaluation", - "weave/reference/typescript-sdk/classes/EvaluationLogger", - "weave/reference/typescript-sdk/classes/MessagesPrompt", - "weave/reference/typescript-sdk/classes/ScoreLogger", - "weave/reference/typescript-sdk/classes/StringPrompt", - "weave/reference/typescript-sdk/classes/WeaveClient", - "weave/reference/typescript-sdk/classes/WeaveObject" + "weave/reference/typescript-sdk/classes/dataset", + "weave/reference/typescript-sdk/classes/evaluation", + "weave/reference/typescript-sdk/classes/evaluationlogger", + "weave/reference/typescript-sdk/classes/messagesprompt", + "weave/reference/typescript-sdk/classes/objectref", + "weave/reference/typescript-sdk/classes/scorelogger", + "weave/reference/typescript-sdk/classes/stringprompt", + "weave/reference/typescript-sdk/classes/weaveclient", + "weave/reference/typescript-sdk/classes/weaveobject" ] }, { @@ -998,27 +999,27 @@ "weave/reference/typescript-sdk/functions/init", "weave/reference/typescript-sdk/functions/login", "weave/reference/typescript-sdk/functions/op", - "weave/reference/typescript-sdk/functions/requireCurrentCallStackEntry", - "weave/reference/typescript-sdk/functions/requireCurrentChildSummary", - "weave/reference/typescript-sdk/functions/weaveAudio", - "weave/reference/typescript-sdk/functions/weaveImage", - "weave/reference/typescript-sdk/functions/wrapOpenAI" + "weave/reference/typescript-sdk/functions/requirecurrentcallstackentry", + "weave/reference/typescript-sdk/functions/requirecurrentchildsummary", + "weave/reference/typescript-sdk/functions/weaveaudio", + "weave/reference/typescript-sdk/functions/weaveimage", + "weave/reference/typescript-sdk/functions/wrapopenai" ] }, { "group": "Interfaces", "pages": [ - "weave/reference/typescript-sdk/interfaces/CallSchema", - "weave/reference/typescript-sdk/interfaces/CallsFilter", - "weave/reference/typescript-sdk/interfaces/WeaveAudio", - "weave/reference/typescript-sdk/interfaces/WeaveImage" + "weave/reference/typescript-sdk/interfaces/callschema", + "weave/reference/typescript-sdk/interfaces/callsfilter", + "weave/reference/typescript-sdk/interfaces/weaveaudio", + "weave/reference/typescript-sdk/interfaces/weaveimage" ] }, { "group": "Type Aliases", "pages": [ - "weave/reference/typescript-sdk/type-aliases/Op", - "weave/reference/typescript-sdk/type-aliases/OpDecorator" + "weave/reference/typescript-sdk/type-aliases/op", + "weave/reference/typescript-sdk/type-aliases/opdecorator" ] } ] @@ -2317,7 +2318,7 @@ { "source": "/guides/integrations/pytorch", "destination": "/models/integrations" - }, + }, { "source": "/guides/launch/:slug*", "destination": "/platform/launch/:slug*" @@ -2504,4 +2505,4 @@ } ], "baseUrl": "https://docs.wandb.ai" -} +} \ No newline at end of file diff --git a/weave/reference/python-sdk.mdx b/weave/reference/python-sdk.mdx index 2949788cdf..1e0515db8e 100644 --- a/weave/reference/python-sdk.mdx +++ b/weave/reference/python-sdk.mdx @@ -6,2068 +6,3 @@ description: "Python SDK reference for weave" # API Overview ---- - - - -### function `init` - -```python -init( - project_name: 'str', - settings: 'UserSettings | dict[str, Any] | None' = None, - autopatch_settings: 'AutopatchSettings | None' = None, - global_postprocess_inputs: 'PostprocessInputsFunc | None' = None, - global_postprocess_output: 'PostprocessOutputFunc | None' = None, - global_attributes: 'dict[str, Any] | None' = None -) → WeaveClient -``` - -Initialize weave tracking, logging to a wandb project. - -Logging is initialized globally, so you do not need to keep a reference to the return value of init. - -Following init, calls of weave.op() decorated functions will be logged to the specified project. - -**Args:** - - -NOTE: Global postprocessing settings are applied to all ops after each op's own postprocessing. The order is always: 1. Op-specific postprocessing 2. Global postprocessing - - - `project_name`: The name of the Weights & Biases team and project to log to. If you don't specify a team, your default entity is used. To find or update your default entity, refer to [User Settings](https://docs.wandb.ai/guides/models/app/settings-page/user-settings/#default-team) in the W&B Models documentation. - - `settings`: Configuration for the Weave client generally. - - `autopatch_settings`: (Deprecated) Configuration for autopatch integrations. Use explicit patching instead. - - `global_postprocess_inputs`: A function that will be applied to all inputs of all ops. - - `global_postprocess_output`: A function that will be applied to all outputs of all ops. - - `global_attributes`: A dictionary of attributes that will be applied to all traces. -**Returns:** - A Weave client. - ---- - - - -### function `publish` - -```python -publish(obj: 'Any', name: 'str | None' = None) → ObjectRef -``` - -Save and version a Python object. - -Weave creates a new version of the object if the object's name already exists and its content hash does not match the latest version of that object. - -**Args:** - - - - `obj`: The object to save and version. - - `name`: The name to save the object under. -**Returns:** - A Weave Ref to the saved object. - ---- - - - -### function `ref` - -```python -ref(location: 'str') → ObjectRef -``` - -Creates a Ref to an existing Weave object. This does not directly retrieve the object but allows you to pass it to other Weave API functions. - -**Args:** - - - - `location`: A Weave Ref URI, or if `weave.init()` has been called, `name:version` or `name`. If no version is provided, `latest` is used. -**Returns:** - A Weave Ref to the object. - ---- - - - -### function `get` - -```python -get(uri: 'str | ObjectRef') → Any -``` - -A convenience function for getting an object from a URI. - -Many objects logged by Weave are automatically registered with the Weave server. This function allows you to retrieve those objects by their URI. - -**Args:** - - - - `uri`: A fully-qualified weave ref URI. -**Returns:** - The object. - -**Example:** -```python -weave.init("weave_get_example") -dataset = weave.Dataset(rows=[{"a": 1, "b": 2}]) -ref = weave.publish(dataset) - -dataset2 = weave.get(ref) # same as dataset! -``` - ---- - - - -### function `require_current_call` - -```python -require_current_call() → Call -``` - -Get the Call object for the currently executing Op, within that Op. - -This allows you to access attributes of the Call such as its id or feedback while it is running. - -```python -@weave.op -def hello(name: str) -> None: - print(f"Hello {name}!") - current_call = weave.require_current_call() - print(current_call.id) -``` - -It is also possible to access a Call after the Op has returned. - -If you have the Call's id, perhaps from the UI, you can use the `get_call` method on the `WeaveClient` returned from `weave.init` to retrieve the Call object. - -```python -client = weave.init("") -mycall = client.get_call("") -``` - -Alternately, after defining your Op you can use its `call` method. For example: - -```python -@weave.op -def add(a: int, b: int) -> int: - return a + b - -result, call = add.call(1, 2) -print(call.id) -``` - -**Returns:** - The Call object for the currently executing Op - -**Raises:** - - - `NoCurrentCallError`: If tracking has not been initialized or this method is invoked outside an Op. - ---- - - - -### function `get_current_call` - -```python -get_current_call() → Call | None -``` - -Get the Call object for the currently executing Op, within that Op. - -**Returns:** - The Call object for the currently executing Op, or None if tracking has not been initialized or this method is invoked outside an Op. - -**Note:** - -> The returned Call's ``attributes`` dictionary becomes immutable once the call starts. Use :func:`weave.attributes` to set call metadata before invoking an Op. The ``summary`` field may be updated while the Op executes and will be merged with computed summary information when the call finishes. - ---- - - - -### function `finish` - -```python -finish() → None -``` - -Stops logging to weave. - -Following finish, calls of weave.op() decorated functions will no longer be logged. You will need to run weave.init() again to resume logging. - ---- - - - -### function `op` - -```python -op( - func: 'Callable[P, R] | None' = None, - name: 'str | None' = None, - call_display_name: 'str | CallDisplayNameFunc | None' = None, - postprocess_inputs: 'PostprocessInputsFunc | None' = None, - postprocess_output: 'PostprocessOutputFunc | None' = None, - tracing_sample_rate: 'float' = 1.0, - enable_code_capture: 'bool' = True, - accumulator: 'Callable[[Any | None, Any], Any] | None' = None -) → Callable[[Callable[P, R]], Op[P, R]] | Op[P, R] -``` - -A decorator to weave op-ify a function or method. Works for both sync and async. Automatically detects iterator functions and applies appropriate behavior. - ---- - - - -### function `attributes` - -```python -attributes(attributes: 'dict[str, Any]') → Iterator -``` - -Context manager for setting attributes on a call. - -**Example:** -```python -with weave.attributes({'env': 'production'}): - print(my_function.call("World")) -``` - ---- - - - -### function `thread` - -```python -thread( - thread_id: 'str | None | object' = -) → Iterator[ThreadContext] -``` - -Context manager for setting thread_id on calls within the context. - -**Examples:** -```python -# Auto-generate thread_id -with weave.thread() as t: - print(f"Thread ID: {t.thread_id}") - result = my_function("input") # This call will have the auto-generated thread_id - print(f"Current turn: {t.turn_id}") - -# Explicit thread_id -with weave.thread("custom_thread") as t: - result = my_function("input") # This call will have thread_id="custom_thread" - -# Disable threading -with weave.thread(None) as t: - result = my_function("input") # This call will have thread_id=None -``` - -**Args:** - - - - `thread_id`: The thread identifier to associate with calls in this context. If not provided, a UUID v7 will be auto-generated. If None, thread tracking will be disabled. -**Yields:** - - - `ThreadContext`: An object providing access to thread_id and current turn_id. - ---- - - - -## class `Object` -Base class for Weave objects that can be tracked and versioned. - -This class extends Pydantic's BaseModel to provide Weave-specific functionality for object tracking, referencing, and serialization. Objects can have names, descriptions, and references that allow them to be stored and retrieved from the Weave system. - -**Attributes:** - - - `name` (Optional[str]): A human-readable name for the object. - - `description` (Optional[str]): A description of what the object represents. - - `ref` (Optional[ObjectRef]): A reference to the object in the Weave system. - -**Examples:** -```python -# Create a simple object -obj = Object(name="my_object", description="A test object") - -# Create an object from a URI -obj = Object.from_uri("weave:///entity/project/object:digest") -``` - -**Pydantic Fields:** - -- `name`: `typing.Optional[str]` -- `description`: `typing.Optional[str]` -- `ref`: `typing.Optional[trace.refs.ObjectRef]` - ---------- - - - -### classmethod `from_uri` - -```python -from_uri(uri: str, objectify: bool = True) → Self -``` - -Create an object instance from a Weave URI. - -**Args:** - - - `uri` (str): The Weave URI pointing to the object. - - `objectify` (bool): Whether to objectify the result. Defaults to True. - -**Returns:** - - - `Self`: An instance of the class created from the URI. - -**Raises:** - - - `NotImplementedError`: If the class doesn't implement the required methods for deserialization. - -**Examples:** -```python -obj = MyObject.from_uri("weave:///entity/project/object:digest") -``` - ---- - - - -### classmethod `handle_relocatable_object` - -```python -handle_relocatable_object( - v: Any, - handler: ValidatorFunctionWrapHandler, - info: ValidationInfo -) → Any -``` - -Handle validation of relocatable objects including ObjectRef and WeaveObject. - -This validator handles special cases where the input is an ObjectRef or WeaveObject that needs to be properly converted to a standard Object instance. It ensures that references are preserved and that ignored types are handled correctly during the validation process. - -**Args:** - - - `v` (Any): The value to validate. - - `handler` (ValidatorFunctionWrapHandler): The standard pydantic validation handler. - - `info` (ValidationInfo): Validation context information. - -**Returns:** - - - `Any`: The validated object instance. - -**Examples:** - This method is called automatically during object creation and validation. It handles cases like: ```python - # When an ObjectRef is passed - obj = MyObject(some_object_ref) - - # When a WeaveObject is passed - obj = MyObject(some_weave_object) - ``` - ---- - - - -## class `Dataset` -Dataset object with easy saving and automatic versioning. - -**Examples:** -```python -# Create a dataset -dataset = Dataset(name='grammar', rows=[ - {'id': '0', 'sentence': "He no likes ice cream.", 'correction': "He doesn't like ice cream."}, - {'id': '1', 'sentence': "She goed to the store.", 'correction': "She went to the store."}, - {'id': '2', 'sentence': "They plays video games all day.", 'correction': "They play video games all day."} -]) - -# Publish the dataset -weave.publish(dataset) - -# Retrieve the dataset -dataset_ref = weave.ref('grammar').get() - -# Access a specific example -example_label = dataset_ref.rows[2]['sentence'] -``` - -**Pydantic Fields:** - -- `name`: `typing.Optional[str]` -- `description`: `typing.Optional[str]` -- `ref`: `typing.Optional[trace.refs.ObjectRef]` -- `rows`: `typing.Union[trace.table.Table, trace.vals.WeaveTable]` - ---------- - - - -### method `add_rows` - -```python -add_rows(rows: Iterable[dict]) → Dataset -``` - -Create a new dataset version by appending rows to the existing dataset. - -This is useful for adding examples to large datasets without having to load the entire dataset into memory. - -**Args:** - - - - `rows`: The rows to add to the dataset. -**Returns:** - The updated dataset. - ---- - - - -### classmethod `convert_to_table` - -```python -convert_to_table(rows: Any) → Union[Table, WeaveTable] -``` - ---- - - - -### classmethod `from_calls` - -```python -from_calls(calls: Iterable[Call]) → Self -``` - ---- - - - -### classmethod `from_hf` - -```python -from_hf( - hf_dataset: Union[ForwardRef('HFDataset'), ForwardRef('HFDatasetDict')] -) → Self -``` - ---- - - - -### classmethod `from_obj` - -```python -from_obj(obj: WeaveObject) → Self -``` - ---- - - - -### classmethod `from_pandas` - -```python -from_pandas(df: 'DataFrame') → Self -``` - ---- - - - -### method `select` - -```python -select(indices: Iterable[int]) → Self -``` - -Select rows from the dataset based on the provided indices. - -**Args:** - - - - `indices`: An iterable of integer indices specifying which rows to select. -**Returns:** - A new Dataset object containing only the selected rows. - ---- - - - -### method `to_hf` - -```python -to_hf() → HFDataset -``` - ---- - - - -### method `to_pandas` - -```python -to_pandas() → DataFrame -``` - ---- - - - -## class `Model` -Intended to capture a combination of code and data the operates on an input. For example it might call an LLM with a prompt to make a prediction or generate text. - -When you change the attributes or the code that defines your model, these changes will be logged and the version will be updated. This ensures that you can compare the predictions across different versions of your model. Use this to iterate on prompts or to try the latest LLM and compare predictions across different settings - -**Examples:** -```python -class YourModel(Model): - attribute1: str - attribute2: int - - @weave.op() - def predict(self, input_data: str) -> dict: - # Model logic goes here - prediction = self.attribute1 + ' ' + input_data - return {'pred': prediction} -``` - -**Pydantic Fields:** - -- `name`: `typing.Optional[str]` -- `description`: `typing.Optional[str]` -- `ref`: `typing.Optional[trace.refs.ObjectRef]` - ---------- - - - -### method `get_infer_method` - -```python -get_infer_method() → Callable -``` - ---- - - - -## class `Prompt` - -**Pydantic Fields:** - -- `name`: `typing.Optional[str]` -- `description`: `typing.Optional[str]` -- `ref`: `typing.Optional[trace.refs.ObjectRef]` - ---------- - - - -### method `format` - -```python -format(**kwargs: Any) → Any -``` - ---- - - - -## class `StringPrompt` - - - -### method `__init__` - -```python -__init__(content: str) -``` - -**Pydantic Fields:** - -- `name`: `typing.Optional[str]` -- `description`: `typing.Optional[str]` -- `ref`: `typing.Optional[trace.refs.ObjectRef]` -- `content`: `` - ---------- - - - -### method `format` - -```python -format(**kwargs: Any) → str -``` - ---- - - - -### classmethod `from_obj` - -```python -from_obj(obj: WeaveObject) → Self -``` - ---- - - - -## class `MessagesPrompt` - - - -### method `__init__` - -```python -__init__(messages: list[dict]) -``` - -**Pydantic Fields:** - -- `name`: `typing.Optional[str]` -- `description`: `typing.Optional[str]` -- `ref`: `typing.Optional[trace.refs.ObjectRef]` -- `messages`: `list[dict]` - ---------- - - - -### method `format` - -```python -format(**kwargs: Any) → list -``` - ---- - - - -### method `format_message` - -```python -format_message(message: dict, **kwargs: Any) → dict -``` - ---- - - - -### classmethod `from_obj` - -```python -from_obj(obj: WeaveObject) → Self -``` - ---- - - - -## class `Evaluation` -Sets up an evaluation which includes a set of scorers and a dataset. - -Calling evaluation.evaluate(model) will pass in rows from a dataset into a model matching the names of the columns of the dataset to the argument names in model.predict. - -Then it will call all of the scorers and save the results in weave. - -If you want to preprocess the rows from the dataset you can pass in a function to preprocess_model_input. - -**Examples:** -```python -# Collect your examples -examples = [ - {"question": "What is the capital of France?", "expected": "Paris"}, - {"question": "Who wrote 'To Kill a Mockingbird'?", "expected": "Harper Lee"}, - {"question": "What is the square root of 64?", "expected": "8"}, -] - -# Define any custom scoring function -@weave.op() -def match_score1(expected: str, model_output: dict) -> dict: - # Here is where you'd define the logic to score the model output - return {'match': expected == model_output['generated_text']} - -@weave.op() -def function_to_evaluate(question: str): - # here's where you would add your LLM call and return the output - return {'generated_text': 'Paris'} - -# Score your examples using scoring functions -evaluation = Evaluation( - dataset=examples, scorers=[match_score1] -) - -# Start tracking the evaluation -weave.init('intro-example') -# Run the evaluation -asyncio.run(evaluation.evaluate(function_to_evaluate)) -``` - -**Pydantic Fields:** - -- `name`: `typing.Optional[str]` -- `description`: `typing.Optional[str]` -- `ref`: `typing.Optional[trace.refs.ObjectRef]` -- `dataset`: `` -- `scorers`: `typing.Optional[list[typing.Annotated[typing.Union[trace.op_protocol.Op, flow.scorer.Scorer], BeforeValidator(func=, json_schema_input_type=PydanticUndefined)]]]` -- `preprocess_model_input`: `typing.Optional[typing.Callable[[dict], dict]]` -- `trials`: `` -- `metadata`: `typing.Optional[dict[str, typing.Any]]` -- `evaluation_name`: `typing.Union[str, typing.Callable[[trace.call.Call], str], NoneType]` - ---------- - - - -### method `evaluate` - -```python -evaluate(model: Union[Op, Model]) → dict -``` - ---- - - - -### classmethod `from_obj` - -```python -from_obj(obj: WeaveObject) → Self -``` - ---- - - - -### method `get_eval_results` - -```python -get_eval_results(model: Union[Op, Model]) → EvaluationResults -``` - ---- - - - -### method `get_evaluate_calls` - -```python -get_evaluate_calls() → PaginatedIterator[CallSchema, WeaveObject] -``` - -Retrieve all evaluation calls that used this Evaluation object. - -Note that this returns a CallsIter instead of a single call because it's possible to have multiple evaluation calls for a single evaluation (e.g. if you run the same evaluation multiple times). - -**Returns:** - - - `CallsIter`: An iterator over Call objects representing evaluation runs. - -**Raises:** - - - `ValueError`: If the evaluation has no ref (hasn't been saved/run yet). - -**Examples:** -```python -evaluation = Evaluation(dataset=examples, scorers=[scorer]) -await evaluation.evaluate(model) # Run evaluation first -calls = evaluation.get_evaluate_calls() -for call in calls: - print(f"Evaluation run: {call.id} at {call.started_at}") -``` - ---- - - - -### method `get_score_calls` - -```python -get_score_calls() → dict[str, list[Call]] -``` - -Retrieve scorer calls for each evaluation run, grouped by trace ID. - -**Returns:** - - - `dict[str, list[Call]]`: A dictionary mapping trace IDs to lists of scorer Call objects. Each trace ID represents one evaluation run, and the list contains all scorer calls executed during that run. - -**Examples:** -```python -evaluation = Evaluation(dataset=examples, scorers=[accuracy_scorer, f1_scorer]) -await evaluation.evaluate(model) -score_calls = evaluation.get_score_calls() -for trace_id, calls in score_calls.items(): - print(f"Trace {trace_id}: {len(calls)} scorer calls") - for call in calls: - scorer_name = call.summary.get("weave", {}).get("trace_name") - print(f" Scorer: {scorer_name}, Output: {call.output}") -``` - ---- - - - -### method `get_scores` - -```python -get_scores() → dict[str, dict[str, list[Any]]] -``` - -Extract and organize scorer outputs from evaluation runs. - -**Returns:** - - - `dict[str, dict[str, list[Any]]]`: A nested dictionary structure where: - - First level keys are trace IDs (evaluation runs) - - Second level keys are scorer names - - Values are lists of scorer outputs for that run and scorer - -**Examples:** -```python -evaluation = Evaluation(dataset=examples, scorers=[accuracy_scorer, f1_scorer]) -await evaluation.evaluate(model) -scores = evaluation.get_scores() -# Access scores by trace and scorer -for trace_id, trace_scores in scores.items(): - print(f"Evaluation run {trace_id}:") - for scorer_name, outputs in trace_scores.items(): - print(f" {scorer_name}: {outputs}") -``` - -Expected output: - -``` -{ - "trace_123": { - "accuracy_scorer": [{"accuracy": 0.85}], - "f1_scorer": [{"f1": 0.78}] - } -} -``` - ---- - - - -### method `model_post_init` - -```python -model_post_init(_Evaluation__context: Any) → None -``` - ---- - - - -### method `predict_and_score` - -```python -predict_and_score(model: Union[Op, Model], example: dict) → dict -``` - ---- - - - -### method `summarize` - -```python -summarize(eval_table: EvaluationResults) → dict -``` - ---- - - - -## class `EvaluationLogger` -This class provides an imperative interface for logging evaluations. - -An evaluation is started automatically when the first prediction is logged using the `log_prediction` method, and finished when the `log_summary` method is called. - -Each time you log a prediction, you will get back a `ScoreLogger` object. You can use this object to log scores and metadata for that specific prediction. For more information, see the `ScoreLogger` class. - -Basic usage - log predictions with inputs and outputs directly: - -```python -ev = EvaluationLogger() - -# Log predictions with known inputs/outputs -pred = ev.log_prediction(inputs={'q': 'Hello'}, outputs={'a': 'Hi there!'}) -pred.log_score("correctness", 0.9) - -# Finish the evaluation -ev.log_summary({"avg_score": 0.9}) -``` - -Advanced usage - use context manager for dynamic outputs and nested operations: - -```python -ev = EvaluationLogger() - -# Use context manager when you need to capture nested operations -with ev.log_prediction(inputs={'q': 'Hello'}) as pred: - # Any operations here (like LLM calls) automatically become - # children of the predict call - response = your_llm_call(...) - pred.output = response.content - pred.log_score("correctness", 0.9) - -# Finish the evaluation -ev.log_summary({"avg_score": 0.9}) -``` - -**Pydantic Fields:** - -- `name`: `str | None` -- `model`: `flow.model.Model | dict | str` -- `dataset`: `dataset.dataset.Dataset | list[dict] | str` -- `eval_attributes`: `dict[str, typing.Any]` -- `scorers`: `list[str] | None` - ---- - -#### property attributes - ---------- - -#### property ui_url - ---- - - - -### method `fail` - -```python -fail(exception: 'BaseException') → None -``` - -Convenience method to fail the evaluation with an exception. - ---- - - - -### method `finish` - -```python -finish(exception: 'BaseException | None' = None) → None -``` - -Clean up the evaluation resources explicitly without logging a summary. - -Ensures all prediction calls and the main evaluation call are finalized. This is automatically called if the logger is used as a context manager. - ---- - - - -### method `log_prediction` - -```python -log_prediction( - inputs: 'dict[str, Any]', - output: 'Any | _NotSetType' = -) → ScoreLogger | _LogPredictionContext -``` - -Log a prediction to the Evaluation. - -If output is provided, returns a ScoreLogger for direct use. If output is not provided, returns a PredictionContext for use as a context manager. - -**Args:** - - - - `inputs`: The input data for the prediction - - `output`: The output value. If not provided, use as context manager to set later. -**Returns:** - ScoreLogger if output provided, PredictionContext if not. - -Example (direct): - - `pred = ev.log_prediction({'q'`: '...'}, output="answer") pred.log_score("correctness", 0.9) pred.finish() - -Example (context manager): - - `with ev.log_prediction({'q'`: '...'}) as pred: response = model(...) pred.output = response pred.log_score("correctness", 0.9) - ---- - - - -### method `log_summary` - -```python -log_summary(summary: 'dict | None' = None, auto_summarize: 'bool' = True) → None -``` - -Log a summary dict to the Evaluation. - -This will calculate the summary, call the summarize op, and then finalize the evaluation, meaning no more predictions or scores can be logged. - ---- - - - -### method `model_post_init` - -```python -model_post_init(_EvaluationLogger__context: 'Any') → None -``` - -Initialize the pseudo evaluation with the dataset from the model. - ---- - - - -### method `set_view` - -```python -set_view( - name: 'str', - content: 'Content | str', - extension: 'str | None' = None, - mimetype: 'str | None' = None, - metadata: 'dict[str, Any] | None' = None, - encoding: 'str' = 'utf-8' -) → None -``` - -Attach a view to the evaluation's main call summary under `weave.views`. - -Saves the provided content as an object in the project and writes its reference URI under `summary.weave.views.` for the evaluation's `evaluate` call. String inputs are wrapped as text content using `Content.from_text` with the provided extension or mimetype. - -**Args:** - - - - `name`: The view name to display, used as the key under `summary.weave.views`. - - `content`: A `weave.Content` instance or string to serialize. - - `extension`: Optional file extension for string content inputs. - - `mimetype`: Optional MIME type for string content inputs. - - `metadata`: Optional metadata attached to newly created `Content`. - - `encoding`: Text encoding for string content inputs. -**Returns:** - None - -**Examples:** - ``` import weave``` - >>> ev = weave.EvaluationLogger() - >>> ev.set_view("report", "# Report", extension="md") - ---- - - - -## class `Scorer` - -**Pydantic Fields:** - -- `name`: `typing.Optional[str]` -- `description`: `typing.Optional[str]` -- `ref`: `typing.Optional[trace.refs.ObjectRef]` -- `column_map`: `typing.Optional[dict[str, str]]` - ---------- - - - -### classmethod `from_obj` - -```python -from_obj(obj: WeaveObject) → Self -``` - ---- - - - -### method `model_post_init` - -```python -model_post_init(_Scorer__context: Any) → None -``` - ---- - - - -### method `score` - -```python -score(output: Any, **kwargs: Any) → Any -``` - ---- - - - -### method `summarize` - -```python -summarize(score_rows: list) → Optional[dict] -``` - ---- - - - -## class `AnnotationSpec` - -**Pydantic Fields:** - -- `name`: `typing.Optional[str]` -- `description`: `typing.Optional[str]` -- `field_schema`: `dict[str, typing.Any]` -- `unique_among_creators`: `` -- `op_scope`: `typing.Optional[list[str]]` - ---------- - - - -### classmethod `preprocess_field_schema` - -```python -preprocess_field_schema(data: dict[str, Any]) → dict[str, Any] -``` - ---- - - - -### classmethod `validate_field_schema` - -```python -validate_field_schema(schema: dict[str, Any]) → dict[str, Any] -``` - ---- - - - -### method `value_is_valid` - -```python -value_is_valid(payload: Any) → bool -``` - -Validates a payload against this annotation spec's schema. - -**Args:** - - - - `payload`: The data to validate against the schema -**Returns:** - - - `bool`: True if validation succeeds, False otherwise - ---- - - - -## class `File` -A class representing a file with path, mimetype, and size information. - - - -### method `__init__` - -```python -__init__(path: 'str | Path', mimetype: 'str | None' = None) -``` - -Initialize a File object. - -**Args:** - - ---- - -#### property filename - -Get the filename of the file. - - - `path`: Path to the file (string or pathlib.Path) - - `mimetype`: Optional MIME type of the file - will be inferred from extension if not provided -**Returns:** - - - `str`: The name of the file without the directory path. - ---- - - - -### method `open` - -```python -open() → bool -``` - -Open the file using the operating system's default application. - -This method uses the platform-specific mechanism to open the file with the default application associated with the file's type. - -**Returns:** - - - `bool`: True if the file was successfully opened, False otherwise. - ---- - - - -### method `save` - -```python -save(dest: 'str | Path') → None -``` - -Copy the file to the specified destination path. - -**Args:** - - ---- - - - -## class `Content` -A class to represent content from various sources, resolving them to a unified byte-oriented representation with associated metadata. - -This class must be instantiated using one of its classmethods: -- from_path() -- from_bytes() -- from_text() -- from_url() -- from_base64() -- from_data_url() - - - -### method `__init__` - -```python -__init__(*args: 'Any', **kwargs: 'Any') → None -``` - -Direct initialization is disabled. Please use a classmethod like `Content.from_path()` to create an instance. - - - `dest`: Destination path where the file will be copied to (string or pathlib.Path) The destination path can be a file or a directory. -**Pydantic Fields:** - -- `data`: `` -- `size`: `` -- `mimetype`: `` -- `digest`: `` -- `filename`: `` -- `content_type`: `typing.Literal['bytes', 'text', 'base64', 'file', 'url', 'data_url', 'data_url:base64', 'data_url:encoding', 'data_url:encoding:base64']` -- `input_type`: `` -- `encoding`: `` -- `metadata`: `dict[str, typing.Any] | None` -- `extension`: `str | None` - ---- - -#### property art - ---------- - -#### property ref - ---- - - - -### method `as_string` - -```python -as_string() → str -``` - -Display the data as a string. Bytes are decoded using the `encoding` attribute If base64, the data will be re-encoded to base64 bytes then decoded to an ASCII string - -**Returns:** - str. - ---- - - - -### classmethod `from_base64` - -```python -from_base64( - b64_data: 'str | bytes', - extension: 'str | None' = None, - mimetype: 'str | None' = None, - metadata: 'dict[str, Any] | None' = None -) → Self -``` - -Initializes Content from a base64 encoded string or bytes. - ---- - - - -### classmethod `from_bytes` - -```python -from_bytes( - data: 'bytes', - extension: 'str | None' = None, - mimetype: 'str | None' = None, - metadata: 'dict[str, Any] | None' = None, - encoding: 'str' = 'utf-8' -) → Self -``` - -Initializes Content from raw bytes. - ---- - - - -### classmethod `from_data_url` - -```python -from_data_url(url: 'str', metadata: 'dict[str, Any] | None' = None) → Self -``` - -Initializes Content from a data URL. - ---- - - - -### classmethod `from_path` - -```python -from_path( - path: 'str | Path', - encoding: 'str' = 'utf-8', - mimetype: 'str | None' = None, - metadata: 'dict[str, Any] | None' = None -) → Self -``` - -Initializes Content from a local file path. - ---- - - - -### classmethod `from_text` - -```python -from_text( - text: 'str', - extension: 'str | None' = None, - mimetype: 'str | None' = None, - metadata: 'dict[str, Any] | None' = None, - encoding: 'str' = 'utf-8' -) → Self -``` - -Initializes Content from a string of text. - ---- - - - -### classmethod `from_url` - -```python -from_url( - url: 'str', - headers: 'dict[str, Any] | None' = None, - timeout: 'int | None' = 30, - metadata: 'dict[str, Any] | None' = None -) → Self -``` - -Initializes Content by fetching bytes from an HTTP(S) URL. - -Downloads the content, infers mimetype/extension from headers, URL path, and data, and constructs a Content object from the resulting bytes. - ---- - - - -### classmethod `model_validate` - -```python -model_validate( - obj: 'Any', - strict: 'bool | None' = None, - from_attributes: 'bool | None' = None, - context: 'dict[str, Any] | None' = None -) → Self -``` - -Override model_validate to handle Content reconstruction from dict. - ---- - - - -### classmethod `model_validate_json` - -```python -model_validate_json( - json_data: 'str | bytes | bytearray', - strict: 'bool | None' = None, - context: 'dict[str, Any] | None' = None -) → Self -``` - -Override model_validate_json to handle Content reconstruction from JSON. - ---- - - - -### method `open` - -```python -open() → bool -``` - -Open the file using the operating system's default application. - -This method uses the platform-specific mechanism to open the file with the default application associated with the file's type. - -**Returns:** - - - `bool`: True if the file was successfully opened, False otherwise. - ---- - - - -### method `save` - -```python -save(dest: 'str | Path') → None -``` - -Copy the file to the specified destination path. Updates the filename and the path of the content to reflect the last saved copy. - -**Args:** - - ---- - - - -### method `serialize_data` - -```python -serialize_data(data: 'bytes') → str -``` - -When dumping model in json mode - ---- - - - -### method `to_data_url` - -```python -to_data_url(use_base64: 'bool' = True) → str -``` - -Constructs a data URL from the content. - - - `dest`: Destination path where the file will be copied to (string or pathlib.Path) The destination path can be a file or a directory. If dest has no file extension (e.g. .txt), destination will be considered a directory. -**Args:** - - - - `use_base64`: If True, the data will be base64 encoded. Otherwise, it will be percent-encoded. Defaults to True. -**Returns:** - A data URL string. - ---- - - - -## class `Markdown` -A Markdown renderable. - -**Args:** - - - `markup` (str): A string containing markdown. - - `code_theme` (str, optional): Pygments theme for code blocks. Defaults to "monokai". See https://pygments.org/styles/ for code themes. - - `justify` (JustifyMethod, optional): Justify value for paragraphs. Defaults to None. - - `style` (Union[str, Style], optional): Optional style to apply to markdown. - - `hyperlinks` (bool, optional): Enable hyperlinks. Defaults to ``True``. - - - -### method `__init__` - -```python -__init__( - markup: 'str', - code_theme: 'str' = 'monokai', - justify: 'JustifyMethod | None' = None, - style: 'str | Style' = 'none', - hyperlinks: 'bool' = True, - inline_code_lexer: 'str | None' = None, - inline_code_theme: 'str | None' = None -) → None -``` - ---- - - - -## class `Monitor` -Sets up a monitor to score incoming calls automatically. - - - `inline_code_lexer`: (str, optional): Lexer to use if inline code highlighting is enabled. Defaults to None. - - `inline_code_theme`: (Optional[str], optional): Pygments theme for inline code highlighting, or None for no highlighting. Defaults to None. -**Examples:** -```python -import weave -from weave.scorers import ValidJSONScorer - -json_scorer = ValidJSONScorer() - -my_monitor = weave.Monitor( - name="my-monitor", - description="This is a test monitor", - sampling_rate=0.5, - op_names=["my_op"], - query={ - "$expr": { - "$gt": [ - { - "$getField": "started_at" - }, - { - "$literal": 1742540400 - } - ] - } - } - }, - scorers=[json_scorer], -) - -my_monitor.activate() -``` - -**Pydantic Fields:** - -- `name`: `typing.Optional[str]` -- `description`: `typing.Optional[str]` -- `ref`: `typing.Optional[trace.refs.ObjectRef]` -- `sampling_rate`: `` -- `scorers`: `list[flow.scorer.Scorer]` -- `op_names`: `list[str]` -- `query`: `typing.Optional[trace_server.interface.query.Query]` -- `active`: `` - ---------- - - - -### method `activate` - -```python -activate() → ObjectRef -``` - -Activates the monitor. - -**Returns:** - The ref to the monitor. - ---- - - - -### method `deactivate` - -```python -deactivate() → ObjectRef -``` - -Deactivates the monitor. - -**Returns:** - The ref to the monitor. - ---- - - - -### classmethod `from_obj` - -```python -from_obj(obj: WeaveObject) → Self -``` - ---- - - - -## class `SavedView` -A fluent-style class for working with SavedView objects. - - - -### method `__init__` - -```python -__init__(view_type: 'str' = 'traces', label: 'str' = 'SavedView') → None -``` - ---- - -#### property entity - ---- - -#### property label - ---- - -#### property project - ---- - -#### property view_type - ---- - - - -### method `add_column` - -```python -add_column(path: 'str | ObjectPath', label: 'str | None' = None) → SavedView -``` - ---- - - - -### method `add_columns` - -```python -add_columns(*columns: 'str') → SavedView -``` - -Convenience method for adding multiple columns to the grid. - ---- - - - -### method `add_filter` - -```python -add_filter( - field: 'str', - operator: 'str', - value: 'Any | None' = None -) → SavedView -``` - ---- - - - -### method `add_sort` - -```python -add_sort(field: 'str', direction: 'SortDirection') → SavedView -``` - ---- - - - -### method `column_index` - -```python -column_index(path: 'int | str | ObjectPath') → int -``` - ---- - - - -### method `filter_op` - -```python -filter_op(op_name: 'str | None') → SavedView -``` - ---- - - - -### method `get_calls` - -```python -get_calls( - limit: 'int | None' = None, - offset: 'int | None' = None, - include_costs: 'bool' = False, - include_feedback: 'bool' = False, - all_columns: 'bool' = False -) → CallsIter -``` - -Get calls matching this saved view's filters and settings. - ---- - - - -### method `get_known_columns` - -```python -get_known_columns(num_calls_to_query: 'int | None' = None) → list[str] -``` - -Get the set of columns that are known to exist. - ---- - - - -### method `get_table_columns` - -```python -get_table_columns() → list[TableColumn] -``` - ---- - - - -### method `hide_column` - -```python -hide_column(col_name: 'str') → SavedView -``` - ---- - - - -### method `insert_column` - -```python -insert_column( - idx: 'int', - path: 'str | ObjectPath', - label: 'str | None' = None -) → SavedView -``` - ---- - - - -### classmethod `load` - -```python -load(ref: 'str') → SavedView -``` - ---- - - - -### method `page_size` - -```python -page_size(page_size: 'int') → SavedView -``` - ---- - - - -### method `pin_column_left` - -```python -pin_column_left(col_name: 'str') → SavedView -``` - ---- - - - -### method `pin_column_right` - -```python -pin_column_right(col_name: 'str') → SavedView -``` - ---- - - - -### method `remove_column` - -```python -remove_column(path: 'int | str | ObjectPath') → SavedView -``` - ---- - - - -### method `remove_columns` - -```python -remove_columns(*columns: 'str') → SavedView -``` - -Remove columns from the saved view. - ---- - - - -### method `remove_filter` - -```python -remove_filter(index_or_field: 'int | str') → SavedView -``` - ---- - - - -### method `remove_filters` - -```python -remove_filters() → SavedView -``` - -Remove all filters from the saved view. - ---- - - - -### method `rename` - -```python -rename(label: 'str') → SavedView -``` - ---- - - - -### method `rename_column` - -```python -rename_column(path: 'int | str | ObjectPath', label: 'str') → SavedView -``` - ---- - - - -### method `save` - -```python -save() → SavedView -``` - -Publish the saved view to the server. - ---- - - - -### method `set_columns` - -```python -set_columns(*columns: 'str') → SavedView -``` - -Set the columns to be displayed in the grid. - ---- - - - -### method `show_column` - -```python -show_column(col_name: 'str') → SavedView -``` - ---- - - - -### method `sort_by` - -```python -sort_by(field: 'str', direction: 'SortDirection') → SavedView -``` - ---- - - - -### method `to_grid` - -```python -to_grid(limit: 'int | None' = None) → Grid -``` - ---- - - - -### method `to_rich_table_str` - -```python -to_rich_table_str() → str -``` - ---- - - - -### method `ui_url` - -```python -ui_url() → str | None -``` - -URL to show this saved view in the UI. - -Note this is the "result" page with traces etc, not the URL for the view object. - ---- - - - -### method `unpin_column` - -```python -unpin_column(col_name: 'str') → SavedView -``` - ---- - - - -## class `Audio` -A class representing audio data in a supported format (wav or mp3). - -This class handles audio data storage and provides methods for loading from different sources and exporting to files. - -**Attributes:** - - - `format`: The audio format (currently supports 'wav' or 'mp3') - - `data`: The raw audio data as bytes - -**Args:** - - - - `data`: The audio data (bytes or base64 encoded string) - - `format`: The audio format ('wav' or 'mp3') - - `validate_base64`: Whether to attempt base64 decoding of the input data -**Raises:** - - - `ValueError`: If audio data is empty or format is not supported - - - -### method `__init__` - -```python -__init__( - data: 'bytes', - format: 'SUPPORTED_FORMATS_TYPE', - validate_base64: 'bool' = True -) → None -``` - ---- - - - -### method `export` - -```python -export(path: 'str | bytes | Path | PathLike') → None -``` - -Export audio data to a file. - -**Args:** - - ---- - - - -### classmethod `from_data` - -```python -from_data(data: 'str | bytes', format: 'str') → Audio -``` - -Create an Audio object from raw data and specified format. - - - `path`: Path where the audio file should be written -**Args:** - - - - `data`: Audio data as bytes or base64 encoded string - - `format`: Audio format ('wav' or 'mp3') -**Returns:** - - - `Audio`: A new Audio instance - -**Raises:** - - - `ValueError`: If format is not supported - ---- - - - -### classmethod `from_path` - -```python -from_path(path: 'str | bytes | Path | PathLike') → Audio -``` - -Create an Audio object from a file path. - -**Args:** - - - - `path`: Path to an audio file (must have .wav or .mp3 extension) -**Returns:** - - - `Audio`: A new Audio instance loaded from the file - -**Raises:** - - - `ValueError`: If file doesn't exist or has unsupported extension diff --git a/weave/reference/python-sdk/trace/feedback.mdx b/weave/reference/python-sdk/trace/feedback.mdx index 2c63b36097..c286d9c40c 100644 --- a/weave/reference/python-sdk/trace/feedback.mdx +++ b/weave/reference/python-sdk/trace/feedback.mdx @@ -8,79 +8,77 @@ description: "Python SDK reference for weave.trace.feedback" --- - + -## class `Feedbacks` -A collection of Feedback objects with utilities. +## class `FeedbackQuery` +Lazy-loading object for fetching feedback from the server. - + ### method `__init__` ```python __init__( - show_refs: 'bool', - feedbacks: 'Iterable[Feedback] | None' = None -) → None + entity: 'str', + project: 'str', + query: 'Query', + offset: 'int | None' = None, + limit: 'int | None' = None, + show_refs: 'bool' = False +) ``` --- - + -### method `refs` +### method `execute` ```python -refs() → Refs +execute() → Feedbacks ``` -Return the unique refs associated with these feedbacks. - --- - + -## class `FeedbackQuery` -Lazy-loading object for fetching feedback from the server. - - - -### method `__init__` +### method `refresh` ```python -__init__( - entity: 'str', - project: 'str', - query: 'Query', - offset: 'int | None' = None, - limit: 'int | None' = None, - show_refs: 'bool' = False -) +refresh() → Feedbacks ``` --- - + -### method `execute` +### method `refs` ```python -execute() → Feedbacks +refs() → Refs ``` --- - + -### method `refresh` +## class `Feedbacks` +A collection of Feedback objects with utilities. + + + +### method `__init__` ```python -refresh() → Feedbacks +__init__( + show_refs: 'bool', + feedbacks: 'Iterable[Feedback] | None' = None +) → None ``` --- - + ### method `refs` @@ -88,14 +86,16 @@ refresh() → Feedbacks refs() → Refs ``` +Return the unique refs associated with these feedbacks. + --- - + ## class `RefFeedbackQuery` Object for interacting with feedback associated with a particular ref. - + ### method `__init__` @@ -105,7 +105,7 @@ __init__(ref: 'str') → None --- - + ### method `add` @@ -125,7 +125,7 @@ feedback_type: A string identifying the type of feedback. The "wandb." prefix is --- - + ### method `add_note` @@ -135,7 +135,7 @@ add_note(note: 'str', creator: 'str | None' = None) → str --- - + ### method `add_reaction` @@ -145,7 +145,7 @@ add_reaction(emoji: 'str', creator: 'str | None' = None) → str --- - + ### method `execute` @@ -155,7 +155,7 @@ execute() → Feedbacks --- - + ### method `purge` @@ -165,7 +165,7 @@ purge(feedback_id: 'str') → None --- - + ### method `refresh` @@ -175,7 +175,7 @@ refresh() → Feedbacks --- - + ### method `refs` diff --git a/weave/reference/python-sdk/trace/op.mdx b/weave/reference/python-sdk/trace/op.mdx index 8c454c5cd1..0d08bedd9a 100644 --- a/weave/reference/python-sdk/trace/op.mdx +++ b/weave/reference/python-sdk/trace/op.mdx @@ -8,7 +8,68 @@ description: "Python SDK reference for weave.trace.op" --- - + + +## class `DisplayNameFuncError` + +--- + + + +## class `OpCallError` + +--- + + + +## class `OpKwargs` +TypedDict for op() keyword arguments. + +--- + + + +## class `Sentinel` +Sentinel(package: 'str', path: 'str', name: 'str') + + + +### method `__init__` + +```python +__init__(package: 'str', path: 'str', name: 'str') → None +``` + +--- + + + +## class `WeaveKwargs` + +--- + + + +### function `as_op` + +```python +as_op(fn: 'Callable[P, R]') → Op[P, R] +``` + +Given a @weave.op() decorated function, return its Op. + +@weave.op() decorated functions are instances of Op already, so this function should be a no-op at runtime. But you can use it to satisfy type checkers if you need to access OpDef attributes in a typesafe way. + +**Args:** + + + - `fn`: A weave.op() decorated function. +**Returns:** + The Op of the function. + +--- + + ### function `call` @@ -39,7 +100,7 @@ result, call = add.call(1, 2) --- - + ### function `calls` @@ -60,3 +121,135 @@ calls = add.calls() for call in calls: print(call) ``` + +--- + + + +### function `get_captured_code` + +```python +get_captured_code(op: 'Op') → str +``` + +Get the captured code of the op. + +This only works when you get an op back from a ref. The pattern is: + +ref = weave.publish(func) op = ref.get() captured_code = op.get_captured_code() + +--- + + + +### function `is_op` + +```python +is_op(obj: 'Any') → TypeIs[Op] +``` + +Check if an object is an Op. + +--- + + + +### function `is_placeholder_call` + +```python +is_placeholder_call(call: 'Call') → TypeIs[NoOpCall] +``` + +--- + + + +### function `is_tracing_setting_disabled` + +```python +is_tracing_setting_disabled() → bool +``` + +--- + + + +### function `maybe_bind_method` + +```python +maybe_bind_method(func: 'Callable', self: 'Any' = None) → Callable | MethodType +``` + +Bind a function to any object (even if it's not a class). + +If self is None, return the function as is. + +--- + + + +### function `maybe_unbind_method` + +```python +maybe_unbind_method(oplike: 'Op | MethodType | partial') → Op +``` + +Unbind an Op-like method or partial to a plain Op function. + +For: +- methods, remove set `self` param +- partials, remove any preset params + +--- + + + +### function `op` + +```python +op( + func: 'Callable[P, R] | None' = None, + name: 'str | None' = None, + call_display_name: 'str | CallDisplayNameFunc | None' = None, + postprocess_inputs: 'PostprocessInputsFunc | None' = None, + postprocess_output: 'PostprocessOutputFunc | None' = None, + tracing_sample_rate: 'float' = 1.0, + enable_code_capture: 'bool' = True, + accumulator: 'Callable[[Any | None, Any], Any] | None' = None +) → Callable[[Callable[P, R]], Op[P, R]] | Op[P, R] +``` + +A decorator to weave op-ify a function or method. Works for both sync and async. Automatically detects iterator functions and applies appropriate behavior. + +--- + + + +### function `placeholder_call` + +```python +placeholder_call() → Call +``` + +--- + + + +### function `setup_dunder_weave_dict` + +```python +setup_dunder_weave_dict(d: 'WeaveKwargs | None' = None) → WeaveKwargs +``` + +Sets up a __weave dict used to pass WeaveKwargs to ops. + +--- + + + +### function `should_skip_tracing_for_op` + +```python +should_skip_tracing_for_op(op: 'Op') → bool +``` + diff --git a/weave/reference/python-sdk/trace/util.mdx b/weave/reference/python-sdk/trace/util.mdx index e849391cda..1d8da0ae11 100644 --- a/weave/reference/python-sdk/trace/util.mdx +++ b/weave/reference/python-sdk/trace/util.mdx @@ -8,7 +8,81 @@ description: "Python SDK reference for weave.trace.util" --- - + + +## class `ContextAwareThread` +A Thread that runs functions with the context of the caller. + +This is a drop-in replacement for threading.Thread that ensures calls behave as expected inside the thread. Weave requires certain contextvars to be set (see call_context.py), but new threads do not automatically copy context from the parent, which can cause the call context to be lost -- not good! This class automates contextvar copying so using this thread "just works" as the user probably expects. + +You can achieve the same effect without this class by instead writing: + +```python +def run_with_context(func, *args, **kwargs): + context = copy_context() + def wrapper(): + context.run(func, *args, **kwargs) + return wrapper + +thread = threading.Thread(target=run_with_context(your_func, *args, **kwargs)) +thread.start() +``` + + + +### method `__init__` + +```python +__init__(*args: 'Any', **kwargs: 'Any') → None +``` + +--- + +#### property daemon + +A boolean value indicating whether this thread is a daemon thread. + +This must be set before start() is called, otherwise RuntimeError is raised. Its initial value is inherited from the creating thread; the main thread is not a daemon thread and therefore all threads created in the main thread default to daemon = False. + +The entire Python program exits when only daemon threads are left. + +--- + +#### property ident + +Thread identifier of this thread or None if it has not been started. + +This is a nonzero integer. See the get_ident() function. Thread identifiers may be recycled when a thread exits and another thread is created. The identifier is available even after the thread has exited. + +--- + +#### property name + +A string used for identification purposes only. + +It has no semantics. Multiple threads may be given the same name. The initial name is set by the constructor. + +--- + +#### property native_id + +Native integral thread ID of this thread, or None if it has not been started. + +This is a non-negative integer. See the get_native_id() function. This represents the Thread ID as reported by the kernel. + +--- + + + +### method `run` + +```python +run() → None +``` + +--- + + ## class `ContextAwareThreadPoolExecutor` A ThreadPoolExecutor that runs functions with the context of the caller. @@ -27,7 +101,7 @@ with concurrent.futures.ThreadPoolExecutor() as executor: executor.map(_wrapped_fn, vals) ``` - + ### method `__init__` @@ -37,7 +111,7 @@ __init__(*args: 'Any', **kwargs: 'Any') → None --- - + ### method `map` @@ -52,7 +126,7 @@ map( --- - + ### method `submit` @@ -62,7 +136,7 @@ submit(fn: 'Callable', *args: 'Any', **kwargs: 'Any') → Any --- - + ## class `ContextAwareThread` A Thread that runs functions with the context of the caller. @@ -82,7 +156,7 @@ thread = threading.Thread(target=run_with_context(your_func, *args, **kwargs)) thread.start() ``` - + ### method `__init__` @@ -126,7 +200,7 @@ This is a non-negative integer. See the get_native_id() function. This represent --- - + ### method `run` @@ -134,3 +208,112 @@ This is a non-negative integer. See the get_native_id() function. This represent run() → None ``` +--- + + + +## class `ContextAwareThreadPoolExecutor` +A ThreadPoolExecutor that runs functions with the context of the caller. + +This is a drop-in replacement for concurrent.futures.ThreadPoolExecutor that ensures weave calls behave as expected inside the executor. Weave requires certain contextvars to be set (see call_context.py), but new threads do not automatically copy context from the parent, which can cause the call context to be lost -- not good! This class automates contextvar copying so using this executor "just works" as the user probably expects. + +You can achieve the same effect without this class by instead writing: + +```python +with concurrent.futures.ThreadPoolExecutor() as executor: + contexts = [copy_context() for _ in range(len(vals))] + + def _wrapped_fn(*args): + return contexts.pop().run(fn, *args) + + executor.map(_wrapped_fn, vals) +``` + + + +### method `__init__` + +```python +__init__(*args: 'Any', **kwargs: 'Any') → None +``` + +--- + + + +### method `map` + +```python +map( + fn: 'Callable', + *iterables: 'Iterable[Any]', + timeout: 'float | None' = None, + chunksize: 'int' = 1 +) → Iterator +``` + +--- + + + +### method `submit` + +```python +submit(fn: 'Callable', *args: 'Any', **kwargs: 'Any') → Any +``` + +--- + + + +### function `deprecated` + +```python +deprecated(new_name: 'str') → Callable[[Callable[, Any]], Callable[, Any]] +``` + +Decorator to mark a function as deprecated and redirect users to `new_name`. + +--- + + + +### function `is_colab` + +```python +is_colab() +``` + +--- + + + +### function `is_notebook` + +```python +is_notebook() → bool +``` + +--- + + + +### function `log_once` + +```python +log_once(log_method: 'Callable[[str], None]', message: 'str') → None +``` + +Logs a message once, suppressing subsequent messages of the same type. This is useful for notifying the user about errors without spamming the logs. + +This is mostly useful for cases where the same error message might occur many times. For example, if an op fails to save, it is likely going to happen every time that op is called. Or, if we have an error in our patched iterator, then it likely happens every time we iterate over the result. This allows use to inform the user about the error without clogging up their logs. + +**Args:** + + + - `log_method`: The method to use to log the message. This should accept a string argument. + - `message`: The message to log. +**Example:** +```python +log_once(logger.error, "Failed to save op") +``` diff --git a/weave/reference/python-sdk/trace/weave_client.mdx b/weave/reference/python-sdk/trace/weave_client.mdx index 81b55842fd..d2fc0fe4d3 100644 --- a/weave/reference/python-sdk/trace/weave_client.mdx +++ b/weave/reference/python-sdk/trace/weave_client.mdx @@ -8,11 +8,25 @@ description: "Python SDK reference for weave.trace.weave_client" --- - + + +## class `FlushStatus` +Status information about the current flush operation. + +--- + + + +## class `PendingJobCounts` +Counts of pending jobs for each type. + +--- + + ## class `WeaveClient` - + ### method `__init__` @@ -39,7 +53,7 @@ This property can be used to check the progress of background tasks without bloc --- - + ### method `add_cost` @@ -78,30 +92,30 @@ client.add_cost(llm_id="my_expensive_custom_model", prompt_token_cost=500, compl --- - + -### method `call` +### method `clear_wandb_run_context` ```python -call(call_id: 'str', include_costs: 'bool' = False) → WeaveObject +clear_wandb_run_context() → None ``` ---- +Clear wandb run context override. - - -### method `calls` +After calling this, calls will fall back to using the global wandb.run (if available) for run_id and step information. +**Examples:** ```python -calls( - filter: 'CallsFilter | None' = None, - include_costs: 'bool' = False -) → CallsIter +client = weave.init("my-project") +client.set_wandb_run_context(run_id="my-run-id", step=5) +# ... make some calls ... +client.clear_wandb_run_context() +# Now calls will use global wandb.run again ``` --- - + ### method `create_call` @@ -133,7 +147,7 @@ Create, log, and push a call onto the runtime stack. --- - + ### method `delete_all_object_versions` @@ -152,7 +166,7 @@ Delete all versions of an object. --- - + ### method `delete_all_op_versions` @@ -171,7 +185,7 @@ Delete all versions of an op. --- - + ### method `delete_call` @@ -181,7 +195,7 @@ delete_call(call: 'Call') → None --- - + ### method `delete_calls` @@ -198,7 +212,7 @@ Deleting a call will also delete all of its children. --- - + ### method `delete_object_version` @@ -208,7 +222,7 @@ delete_object_version(object: 'ObjectRef') → None --- - + ### method `delete_object_versions` @@ -229,7 +243,7 @@ Delete specific versions of an object. --- - + ### method `delete_op_version` @@ -239,7 +253,7 @@ delete_op_version(op: 'OpRef') → None --- - + ### method `fail_call` @@ -251,22 +265,7 @@ Fail a call with an exception. This is a convenience method for finish_call. --- - - -### method `feedback` - -```python -feedback( - query: 'Query | str | None' = None, - reaction: 'str | None' = None, - offset: 'int' = 0, - limit: 'int' = 100 -) → FeedbackQuery -``` - ---- - - + ### method `finish` @@ -286,7 +285,7 @@ This method blocks until all currently enqueued jobs are processed, displaying a --- - + ### method `finish_call` @@ -305,7 +304,7 @@ Any values present in ``call.summary`` are deep-merged with computed summary sta --- - + ### method `flush` @@ -317,7 +316,7 @@ Flushes background asynchronous tasks, safe to call multiple times. --- - + ### method `get` @@ -327,7 +326,7 @@ get(ref: 'ObjectRef', objectify: 'bool' = True) → Any --- - + ### method `get_call` @@ -356,7 +355,7 @@ Get a single call by its ID. --- - + ### method `get_calls` @@ -413,7 +412,7 @@ for call in calls: --- - + ### method `get_evaluation` @@ -449,7 +448,7 @@ print(evaluation.name) --- - + ### method `get_evaluations` @@ -474,7 +473,7 @@ for eval in evaluations: --- - + ### method `get_feedback` @@ -528,7 +527,7 @@ client.get_feedback(query=query) --- - + ### method `purge_costs` @@ -549,7 +548,7 @@ client.purge_costs(ids) --- - + ### method `query_costs` @@ -588,7 +587,7 @@ client.query_costs(llm_ids=["gpt-4o-mini-2024-07-18"], limit=10) --- - + ### method `save` @@ -609,188 +608,91 @@ Do not call directly, use weave.publish() instead. --- - + -## class `Call` -A Call represents a single operation executed as part of a trace. - -``attributes`` are frozen once the call is created. Use :func:`weave.attributes` or ``create_call(..., attributes=...)`` to populate metadata beforehand. The ``summary`` dictionary may be modified while the call is running; its contents are deep-merged with computed summary values when :meth:`WeaveClient.finish_call` is invoked. - - - -### method `__init__` +### method `set_wandb_run_context` ```python -__init__( - _op_name: 'str | Future[str]', - trace_id: 'str', - project_id: 'str', - parent_id: 'str | None', - inputs: 'dict[str, Any]', - id: 'str | None' = None, - output: 'Any' = None, - exception: 'str | None' = None, - summary: 'dict[str, Any] | None' = <factory>, - _display_name: 'str | Callable[[Call], str] | None' = None, - attributes: 'dict[str, Any] | None' = None, - started_at: 'datetime | None' = None, - ended_at: 'datetime | None' = None, - deleted_at: 'datetime | None' = None, - thread_id: 'str | None' = None, - turn_id: 'str | None' = None, - wb_run_id: 'str | None' = None, - wb_run_step: 'int | None' = None, - wb_run_step_end: 'int | None' = None, - _children: 'list[Call]' = <factory>, - _feedback: 'RefFeedbackQuery | None' = None -) → None +set_wandb_run_context(run_id: 'str', step: 'int | None' = None) → None ``` ---- - -#### property display_name - ---- - -#### property feedback - ---- - -#### property func_name - -The decorated function's name that produced this call. +Override wandb run_id and step for calls created by this client. -This is different from `op_name` which is usually the ref of the op. - ---- - -#### property op_name - ---- - -#### property ref - ---- - -#### property ui_url - ---- - - - -### method `apply_scorer` - -```python -apply_scorer( - scorer: 'Op | Scorer', - additional_scorer_kwargs: 'dict[str, Any] | None' = None -) → ApplyScorerResult -``` - -`apply_scorer` is a method that applies a Scorer to a Call. This is useful for guarding application logic with a scorer and/or monitoring the quality of critical ops. Scorers are automatically logged to Weave as Feedback and can be used in queries & analysis. +This allows you to associate Weave calls with a specific WandB run that is not bound to the global wandb.run symbol. **Args:** - - `scorer`: The Scorer to apply. - - `additional_scorer_kwargs`: Additional kwargs to pass to the scorer. This is useful for passing in additional context that is not part of the call inputs.useful for passing in additional context that is not part of the call inputs. -**Returns:** - The result of the scorer application in the form of an `ApplyScorerResult`. - + - `run_id`: The run ID (not including entity/project prefix). The client will automatically add the entity/project prefix. + - `step`: The step number to use for calls. If None, step will not be set. +**Examples:** ```python -class ApplyScorerSuccess: - - - ` result`: Any - - - ` score_call`: Call -``` - -Example usage: +client = weave.init("my-project") +client.set_wandb_run_context(run_id="my-run-id", step=5) +# Now all calls will be associated with entity/project/my-run-id at step 5 -```python -my_scorer = ... # construct a scorer -prediction, prediction_call = my_op.call(input_data) -result, score_call = prediction.apply_scorer(my_scorer) +# Or without a step +client.set_wandb_run_context(run_id="my-run-id") +# Calls will be associated with entity/project/my-run-id with no step ``` --- - + -### method `children` +### function `get_obj_name` ```python -children(page_size: 'int' = 1000) → CallsIter +get_obj_name(val: 'Any') → str ``` -Get the children of the call. - -**Args:** - - - - `page_size`: Tune performance by changing the number of calls fetched at a time. -**Returns:** - An iterator of calls. - --- - + -### method `delete` +### function `get_parallelism_settings` ```python -delete() → bool +get_parallelism_settings() → tuple[int | None, int | None] ``` -Delete the call. - --- - + -### method `remove_display_name` +### function `map_to_refs` ```python -remove_display_name() → None +map_to_refs(obj: 'Any') → Any ``` --- - - -### method `set_display_name` - -```python -set_display_name(name: 'str | None') → None -``` - -Set the display name for the call. + -**Args:** - +### function `print_call_link` - - `name`: The display name to set for the call. -**Example:** ```python -result, call = my_function.call("World") -call.set_display_name("My Custom Display Name") +print_call_link(call: 'Call') → None ``` --- - + -### method `to_dict` +### function `redact_sensitive_keys` ```python -to_dict() → CallDict +redact_sensitive_keys(obj: 'Any') → Any ``` --- -### function `PaginatedIterator` + + +### function `sanitize_object_name` ```python -PaginatedIterator(*args, **kwargs) +sanitize_object_name(name: 'str') → str ``` diff --git a/weave/reference/python-sdk/trace_server/trace_server_interface.mdx b/weave/reference/python-sdk/trace_server/trace_server_interface.mdx index 5e86827716..ed1fbddba3 100644 --- a/weave/reference/python-sdk/trace_server/trace_server_interface.mdx +++ b/weave/reference/python-sdk/trace_server/trace_server_interface.mdx @@ -8,7 +8,7 @@ description: "Python SDK reference for weave.trace_server.trace_server_interface --- - + ## class `ActionsExecuteBatchReq` @@ -19,28 +19,22 @@ description: "Python SDK reference for weave.trace_server.trace_server_interface - `call_ids`: `list[str]` - `wb_user_id`: `typing.Optional[str]` ------- - --- - + ## class `ActionsExecuteBatchRes` ------- - --- - + ## class `BaseModelStrict` Base model with strict validation that forbids extra fields. ------- - --- - + ## class `CallBatchEndMode` @@ -49,11 +43,9 @@ Base model with strict validation that forbids extra fields. - `mode`: `` - `req`: `` ------- - --- - + ## class `CallBatchStartMode` @@ -62,11 +54,9 @@ Base model with strict validation that forbids extra fields. - `mode`: `` - `req`: `` ------- - --- - + ## class `CallCreateBatchReq` @@ -74,11 +64,9 @@ Base model with strict validation that forbids extra fields. - `batch`: `list[typing.Union[CallBatchStartMode, CallBatchEndMode]]` ------- - --- - + ## class `CallCreateBatchRes` @@ -86,11 +74,9 @@ Base model with strict validation that forbids extra fields. - `res`: `list[typing.Union[CallStartRes, CallEndRes]]` ------- - --- - + ## class `CallEndReq` @@ -98,19 +84,15 @@ Base model with strict validation that forbids extra fields. - `end`: `` ------- - --- - + ## class `CallEndRes` ------- - --- - + ## class `CallReadReq` @@ -122,11 +104,9 @@ Base model with strict validation that forbids extra fields. - `include_storage_size`: `typing.Optional[bool]` - `include_total_storage_size`: `typing.Optional[bool]` ------- - --- - + ## class `CallReadRes` @@ -134,11 +114,9 @@ Base model with strict validation that forbids extra fields. - `call`: `typing.Optional[CallSchema]` ------- - --- - + ## class `CallSchema` @@ -167,9 +145,7 @@ Base model with strict validation that forbids extra fields. - `storage_size_bytes`: `typing.Optional[int]` - `total_storage_size_bytes`: `typing.Optional[int]` ---------- - - + ### method `serialize_typed_dicts` @@ -179,7 +155,7 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] --- - + ## class `CallStartReq` @@ -187,11 +163,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `start`: `` ------- - --- - + ## class `CallStartRes` @@ -200,11 +174,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `id`: `` - `trace_id`: `` ------- - --- - + ## class `CallUpdateReq` @@ -215,19 +187,15 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `display_name`: `typing.Optional[str]` - `wb_user_id`: `typing.Optional[str]` ------- - --- - + ## class `CallUpdateRes` ------- - --- - + ## class `CallsDeleteReq` @@ -237,19 +205,19 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `call_ids`: `list[str]` - `wb_user_id`: `typing.Optional[str]` ------- - --- - + ## class `CallsDeleteRes` ------- +**Pydantic Fields:** + +- `num_deleted`: `` --- - + ## class `CallsFilter` @@ -267,11 +235,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `wb_user_ids`: `typing.Optional[list[str]]` - `wb_run_ids`: `typing.Optional[list[str]]` ------- - --- - + ## class `CallsQueryReq` @@ -291,11 +257,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `expand_columns`: `typing.Optional[list[str]]` - `return_expanded_column_values`: `typing.Optional[bool]` ------- - --- - + ## class `CallsQueryRes` @@ -303,11 +267,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `calls`: `list[CallSchema]` ------- - --- - + ## class `CallsQueryStatsReq` @@ -320,11 +282,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `include_total_storage_size`: `typing.Optional[bool]` - `expand_columns`: `typing.Optional[list[str]]` ------- - --- - + ## class `CallsQueryStatsRes` @@ -333,11 +293,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `count`: `` - `total_storage_size_bytes`: `typing.Optional[int]` ------- - --- - + ## class `CompletionsCreateReq` @@ -348,11 +306,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `wb_user_id`: `typing.Optional[str]` - `track_llm_call`: `typing.Optional[bool]` ------- - --- - + ## class `CompletionsCreateRequestInputs` @@ -385,11 +341,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `function_call`: `typing.Optional[str]` - `api_version`: `typing.Optional[str]` ------- - --- - + ## class `CompletionsCreateRes` @@ -398,11 +352,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `response`: `dict[str, typing.Any]` - `weave_call_id`: `typing.Optional[str]` ------- - --- - + ## class `CostCreateInput` @@ -415,11 +367,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `effective_date`: `typing.Optional[datetime.datetime]` - `provider_id`: `typing.Optional[str]` ------- - --- - + ## class `CostCreateReq` @@ -429,11 +379,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `costs`: `dict[str, CostCreateInput]` - `wb_user_id`: `typing.Optional[str]` ------- - --- - + ## class `CostCreateRes` @@ -441,11 +389,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `ids`: `list[tuple[str, str]]` ------- - --- - + ## class `CostPurgeReq` @@ -454,19 +400,15 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `project_id`: `` - `query`: `` ------- - --- - + ## class `CostPurgeRes` ------- - --- - + ## class `CostQueryOutput` @@ -481,11 +423,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `effective_date`: `typing.Optional[datetime.datetime]` - `provider_id`: `typing.Optional[str]` ------- - --- - + ## class `CostQueryReq` @@ -498,11 +438,9 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `limit`: `typing.Optional[int]` - `offset`: `typing.Optional[int]` ------- - --- - + ## class `CostQueryRes` @@ -510,821 +448,2448 @@ serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] - `results`: `list[CostQueryOutput]` ------- - --- - + -## class `EndedCallSchemaForInsert` +## class `DatasetCreateV2Body` **Pydantic Fields:** -- `project_id`: `` -- `id`: `` -- `ended_at`: `` -- `exception`: `typing.Optional[str]` -- `output`: `typing.Optional[typing.Any]` -- `summary`: `` -- `wb_run_step_end`: `typing.Optional[int]` +- `name`: `typing.Optional[str]` +- `description`: `typing.Optional[str]` +- `rows`: `list[dict[str, typing.Any]]` ---------- +--- - + -### method `serialize_typed_dicts` +## class `DatasetCreateV2Req` -```python -serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] -``` +**Pydantic Fields:** + +- `name`: `typing.Optional[str]` +- `description`: `typing.Optional[str]` +- `rows`: `list[dict[str, typing.Any]]` +- `project_id`: `` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `EnsureProjectExistsRes` +## class `DatasetCreateV2Res` **Pydantic Fields:** -- `project_name`: `` - ------- +- `digest`: `` +- `object_id`: `` +- `version_index`: `` --- - + -## class `EvaluateModelReq` +## class `DatasetDeleteV2Req` **Pydantic Fields:** - `project_id`: `` -- `evaluation_ref`: `` -- `model_ref`: `` +- `object_id`: `` +- `digests`: `typing.Optional[list[str]]` - `wb_user_id`: `typing.Optional[str]` ------- - --- - + -## class `EvaluateModelRes` +## class `DatasetDeleteV2Res` **Pydantic Fields:** -- `call_id`: `` - ------- +- `num_deleted`: `` --- - + -## class `EvaluationStatusComplete` +## class `DatasetListV2Req` **Pydantic Fields:** -- `code`: `typing.Literal['complete']` -- `output`: `dict[str, typing.Any]` - ------- +- `project_id`: `` +- `limit`: `typing.Optional[int]` +- `offset`: `typing.Optional[int]` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `EvaluationStatusFailed` +## class `DatasetReadV2Req` **Pydantic Fields:** -- `code`: `typing.Literal['failed']` -- `error`: `typing.Optional[str]` - ------- +- `project_id`: `` +- `object_id`: `` +- `digest`: `` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `EvaluationStatusNotFound` +## class `DatasetReadV2Res` **Pydantic Fields:** -- `code`: `typing.Literal['not_found']` - ------- +- `object_id`: `` +- `digest`: `` +- `version_index`: `` +- `created_at`: `` +- `name`: `` +- `description`: `typing.Optional[str]` +- `rows`: `` --- - + -## class `EvaluationStatusReq` +## class `EndedCallSchemaForInsert` **Pydantic Fields:** - `project_id`: `` -- `call_id`: `` +- `id`: `` +- `ended_at`: `` +- `exception`: `typing.Optional[str]` +- `output`: `typing.Optional[typing.Any]` +- `summary`: `` +- `wb_run_step_end`: `typing.Optional[int]` ------- + + +### method `serialize_typed_dicts` + +```python +serialize_typed_dicts(v: dict[str, Any]) → dict[str, Any] +``` --- - + -## class `EvaluationStatusRes` +## class `EnsureProjectExistsRes` **Pydantic Fields:** -- `status`: `typing.Union[EvaluationStatusNotFound, EvaluationStatusRunning, EvaluationStatusFailed, EvaluationStatusComplete]` - ------- +- `project_name`: `` --- - + -## class `EvaluationStatusRunning` +## class `EvaluateModelReq` **Pydantic Fields:** -- `code`: `typing.Literal['running']` -- `completed_rows`: `` -- `total_rows`: `` - ------- +- `project_id`: `` +- `evaluation_ref`: `` +- `model_ref`: `` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `ExportTracePartialSuccess` +## class `EvaluateModelRes` **Pydantic Fields:** -- `rejected_spans`: `` -- `error_message`: `` - ------- +- `call_id`: `` --- - + -## class `ExtraKeysTypedDict` +## class `EvaluationCreateV2Body` + +**Pydantic Fields:** + +- `name`: `` +- `description`: `typing.Optional[str]` +- `dataset`: `` +- `scorers`: `typing.Optional[list[str]]` +- `trials`: `` +- `evaluation_name`: `typing.Optional[str]` +- `eval_attributes`: `typing.Optional[dict[str, typing.Any]]` --- - + -## class `Feedback` +## class `EvaluationCreateV2Req` **Pydantic Fields:** -- `id`: `` +- `name`: `` +- `description`: `typing.Optional[str]` +- `dataset`: `` +- `scorers`: `typing.Optional[list[str]]` +- `trials`: `` +- `evaluation_name`: `typing.Optional[str]` +- `eval_attributes`: `typing.Optional[dict[str, typing.Any]]` - `project_id`: `` -- `weave_ref`: `` -- `creator`: `typing.Optional[str]` -- `feedback_type`: `` -- `payload`: `dict[str, typing.Any]` -- `annotation_ref`: `typing.Optional[str]` -- `runnable_ref`: `typing.Optional[str]` -- `call_ref`: `typing.Optional[str]` -- `trigger_ref`: `typing.Optional[str]` - `wb_user_id`: `typing.Optional[str]` -- `created_at`: `` - ------- --- - + -## class `FeedbackCreateBatchReq` +## class `EvaluationCreateV2Res` **Pydantic Fields:** -- `batch`: `list[FeedbackCreateReq]` - ------- +- `digest`: `` +- `object_id`: `` +- `version_index`: `` +- `evaluation_ref`: `` --- - + -## class `FeedbackCreateBatchRes` +## class `EvaluationDeleteV2Req` **Pydantic Fields:** -- `res`: `list[FeedbackCreateRes]` - ------- +- `project_id`: `` +- `object_id`: `` +- `digests`: `typing.Optional[list[str]]` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `FeedbackCreateReq` +## class `EvaluationDeleteV2Res` **Pydantic Fields:** -- `id`: `typing.Optional[str]` -- `project_id`: `` -- `weave_ref`: `` -- `creator`: `typing.Optional[str]` -- `feedback_type`: `` -- `payload`: `dict[str, typing.Any]` -- `annotation_ref`: `typing.Optional[str]` -- `runnable_ref`: `typing.Optional[str]` -- `call_ref`: `typing.Optional[str]` -- `trigger_ref`: `typing.Optional[str]` -- `wb_user_id`: `typing.Optional[str]` - ------- +- `num_deleted`: `` --- - + -## class `FeedbackCreateRes` +## class `EvaluationListV2Req` **Pydantic Fields:** -- `id`: `` -- `created_at`: `` -- `wb_user_id`: `` -- `payload`: `dict[str, typing.Any]` - ------- +- `project_id`: `` +- `limit`: `typing.Optional[int]` +- `offset`: `typing.Optional[int]` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `FeedbackDict` +## class `EvaluationReadV2Req` + +**Pydantic Fields:** + +- `project_id`: `` +- `object_id`: `` +- `digest`: `` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `FeedbackPurgeReq` +## class `EvaluationReadV2Res` **Pydantic Fields:** -- `project_id`: `` -- `query`: `` - ------- +- `object_id`: `` +- `digest`: `` +- `version_index`: `` +- `created_at`: `` +- `name`: `` +- `description`: `typing.Optional[str]` +- `dataset`: `` +- `scorers`: `list[str]` +- `trials`: `` +- `evaluation_name`: `typing.Optional[str]` +- `evaluate_op`: `typing.Optional[str]` +- `predict_and_score_op`: `typing.Optional[str]` +- `summarize_op`: `typing.Optional[str]` --- - + -## class `FeedbackPurgeRes` +## class `EvaluationRunCreateV2Body` + +**Pydantic Fields:** ------- +- `evaluation`: `` +- `model`: `` --- - + -## class `FeedbackQueryReq` +## class `EvaluationRunCreateV2Req` **Pydantic Fields:** +- `evaluation`: `` +- `model`: `` - `project_id`: `` -- `fields`: `typing.Optional[list[str]]` -- `query`: `typing.Optional[weave.trace_server.interface.query.Query]` -- `sort_by`: `typing.Optional[list[SortBy]]` -- `limit`: `typing.Optional[int]` -- `offset`: `typing.Optional[int]` - ------- +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `FeedbackQueryRes` +## class `EvaluationRunCreateV2Res` **Pydantic Fields:** -- `result`: `list[dict[str, typing.Any]]` - ------- +- `evaluation_run_id`: `` --- - + -## class `FeedbackReplaceReq` +## class `EvaluationRunDeleteV2Req` **Pydantic Fields:** -- `id`: `typing.Optional[str]` - `project_id`: `` -- `weave_ref`: `` -- `creator`: `typing.Optional[str]` -- `feedback_type`: `` -- `payload`: `dict[str, typing.Any]` -- `annotation_ref`: `typing.Optional[str]` -- `runnable_ref`: `typing.Optional[str]` -- `call_ref`: `typing.Optional[str]` -- `trigger_ref`: `typing.Optional[str]` +- `evaluation_run_ids`: `list[str]` - `wb_user_id`: `typing.Optional[str]` -- `feedback_id`: `` - ------- --- - + -## class `FeedbackReplaceRes` +## class `EvaluationRunDeleteV2Res` **Pydantic Fields:** -- `id`: `` -- `created_at`: `` -- `wb_user_id`: `` -- `payload`: `dict[str, typing.Any]` - ------- +- `num_deleted`: `` --- - + -## class `FileContentReadReq` +## class `EvaluationRunFilterV2` **Pydantic Fields:** -- `project_id`: `` -- `digest`: `` - ------- +- `evaluations`: `typing.Optional[list[str]]` +- `models`: `typing.Optional[list[str]]` +- `evaluation_run_ids`: `typing.Optional[list[str]]` --- - + -## class `FileContentReadRes` +## class `EvaluationRunFinishV2Body` +Request body for finishing an evaluation run via REST API. -**Pydantic Fields:** +This model excludes project_id and evaluation_run_id since they come from the URL path in RESTful endpoints. -- `content`: `` +**Pydantic Fields:** ------- +- `summary`: `typing.Optional[dict[str, typing.Any]]` --- - + -## class `FileCreateReq` +## class `EvaluationRunFinishV2Req` **Pydantic Fields:** +- `summary`: `typing.Optional[dict[str, typing.Any]]` - `project_id`: `` -- `name`: `` -- `content`: `` - ------- +- `evaluation_run_id`: `` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `FileCreateRes` +## class `EvaluationRunFinishV2Res` **Pydantic Fields:** -- `digest`: `` - ------- +- `success`: `` --- - + -## class `FilesStatsReq` +## class `EvaluationRunListV2Req` **Pydantic Fields:** - `project_id`: `` +- `filter`: `typing.Optional[EvaluationRunFilterV2]` +- `limit`: `typing.Optional[int]` +- `offset`: `typing.Optional[int]` + +--- + + + +## class `EvaluationRunReadV2Req` + +**Pydantic Fields:** + +- `project_id`: `` +- `evaluation_run_id`: `` + +--- + + + +## class `EvaluationRunReadV2Res` + +**Pydantic Fields:** + +- `evaluation_run_id`: `` +- `evaluation`: `` +- `model`: `` +- `status`: `typing.Optional[str]` +- `started_at`: `typing.Optional[datetime.datetime]` +- `finished_at`: `typing.Optional[datetime.datetime]` +- `summary`: `typing.Optional[dict[str, typing.Any]]` + +--- + + + +## class `EvaluationStatusComplete` + +**Pydantic Fields:** + +- `code`: `typing.Literal['complete']` +- `output`: `dict[str, typing.Any]` + +--- + + + +## class `EvaluationStatusFailed` + +**Pydantic Fields:** + +- `code`: `typing.Literal['failed']` +- `error`: `typing.Optional[str]` + +--- + + + +## class `EvaluationStatusNotFound` + +**Pydantic Fields:** + +- `code`: `typing.Literal['not_found']` + +--- + + + +## class `EvaluationStatusReq` + +**Pydantic Fields:** + +- `project_id`: `` +- `call_id`: `` + +--- + + + +## class `EvaluationStatusRes` + +**Pydantic Fields:** + +- `status`: `typing.Union[EvaluationStatusNotFound, EvaluationStatusRunning, EvaluationStatusFailed, EvaluationStatusComplete]` + +--- + + + +## class `EvaluationStatusRunning` + +**Pydantic Fields:** + +- `code`: `typing.Literal['running']` +- `completed_rows`: `` +- `total_rows`: `` + +--- + + + +## class `ExportTracePartialSuccess` + +**Pydantic Fields:** + +- `rejected_spans`: `` +- `error_message`: `` + +--- + + + +## class `ExtraKeysTypedDict` + +--- + + + +## class `Feedback` + +**Pydantic Fields:** + +- `id`: `` +- `project_id`: `` +- `weave_ref`: `` +- `creator`: `typing.Optional[str]` +- `feedback_type`: `` +- `payload`: `dict[str, typing.Any]` +- `annotation_ref`: `typing.Optional[str]` +- `runnable_ref`: `typing.Optional[str]` +- `call_ref`: `typing.Optional[str]` +- `trigger_ref`: `typing.Optional[str]` +- `wb_user_id`: `typing.Optional[str]` +- `created_at`: `` + +--- + + + +## class `FeedbackCreateBatchReq` + +**Pydantic Fields:** + +- `batch`: `list[FeedbackCreateReq]` + +--- + + + +## class `FeedbackCreateBatchRes` + +**Pydantic Fields:** + +- `res`: `list[FeedbackCreateRes]` + +--- + + + +## class `FeedbackCreateReq` + +**Pydantic Fields:** + +- `id`: `typing.Optional[str]` +- `project_id`: `` +- `weave_ref`: `` +- `creator`: `typing.Optional[str]` +- `feedback_type`: `` +- `payload`: `dict[str, typing.Any]` +- `annotation_ref`: `typing.Optional[str]` +- `runnable_ref`: `typing.Optional[str]` +- `call_ref`: `typing.Optional[str]` +- `trigger_ref`: `typing.Optional[str]` +- `wb_user_id`: `typing.Optional[str]` + +--- + + + +## class `FeedbackCreateRes` + +**Pydantic Fields:** + +- `id`: `` +- `created_at`: `` +- `wb_user_id`: `` +- `payload`: `dict[str, typing.Any]` + +--- + + + +## class `FeedbackDict` + +--- + + + +## class `FeedbackPurgeReq` + +**Pydantic Fields:** + +- `project_id`: `` +- `query`: `` + +--- + + + +## class `FeedbackPurgeRes` + +--- + + + +## class `FeedbackQueryReq` + +**Pydantic Fields:** + +- `project_id`: `` +- `fields`: `typing.Optional[list[str]]` +- `query`: `typing.Optional[weave.trace_server.interface.query.Query]` +- `sort_by`: `typing.Optional[list[SortBy]]` +- `limit`: `typing.Optional[int]` +- `offset`: `typing.Optional[int]` + +--- + + + +## class `FeedbackQueryRes` + +**Pydantic Fields:** + +- `result`: `list[dict[str, typing.Any]]` + +--- + + + +## class `FeedbackReplaceReq` + +**Pydantic Fields:** + +- `id`: `typing.Optional[str]` +- `project_id`: `` +- `weave_ref`: `` +- `creator`: `typing.Optional[str]` +- `feedback_type`: `` +- `payload`: `dict[str, typing.Any]` +- `annotation_ref`: `typing.Optional[str]` +- `runnable_ref`: `typing.Optional[str]` +- `call_ref`: `typing.Optional[str]` +- `trigger_ref`: `typing.Optional[str]` +- `wb_user_id`: `typing.Optional[str]` +- `feedback_id`: `` + +--- + + + +## class `FeedbackReplaceRes` + +**Pydantic Fields:** + +- `id`: `` +- `created_at`: `` +- `wb_user_id`: `` +- `payload`: `dict[str, typing.Any]` + +--- + + + +## class `FileContentReadReq` + +**Pydantic Fields:** + +- `project_id`: `` +- `digest`: `` + +--- + + + +## class `FileContentReadRes` + +**Pydantic Fields:** + +- `content`: `` + +--- + + + +## class `FileCreateReq` + +**Pydantic Fields:** + +- `project_id`: `` +- `name`: `` +- `content`: `` + +--- + + + +## class `FileCreateRes` + +**Pydantic Fields:** + +- `digest`: `` + +--- + + + +## class `FilesStatsReq` + +**Pydantic Fields:** + +- `project_id`: `` + +--- + + + +## class `FilesStatsRes` + +**Pydantic Fields:** + +- `total_size_bytes`: `` + +--- + + + +## class `FullTraceServerInterface` +Complete trace server interface supporting both V1 and V2 APIs. + +This protocol represents a trace server implementation that supports the full set of APIs - both legacy V1 endpoints and modern V2 endpoints. Use this type for implementations that need to support both API versions. + +--- + + + +### method `actions_execute_batch` + +```python +actions_execute_batch(req: ActionsExecuteBatchReq) → ActionsExecuteBatchRes +``` + +--- + + + +### method `call_end` + +```python +call_end(req: CallEndReq) → CallEndRes +``` + +--- + + + +### method `call_read` + +```python +call_read(req: CallReadReq) → CallReadRes +``` + +--- + + + +### method `call_start` + +```python +call_start(req: CallStartReq) → CallStartRes +``` + +--- + + + +### method `call_start_batch` + +```python +call_start_batch(req: CallCreateBatchReq) → CallCreateBatchRes +``` + +--- + + + +### method `call_update` + +```python +call_update(req: CallUpdateReq) → CallUpdateRes +``` + +--- + + + +### method `calls_delete` + +```python +calls_delete(req: CallsDeleteReq) → CallsDeleteRes +``` + +--- + + + +### method `calls_query` + +```python +calls_query(req: CallsQueryReq) → CallsQueryRes +``` + +--- + + + +### method `calls_query_stats` + +```python +calls_query_stats(req: CallsQueryStatsReq) → CallsQueryStatsRes +``` + +--- + + + +### method `calls_query_stream` + +```python +calls_query_stream(req: CallsQueryReq) → Iterator[CallSchema] +``` + +--- + + + +### method `completions_create` + +```python +completions_create(req: CompletionsCreateReq) → CompletionsCreateRes +``` + +--- + + + +### method `completions_create_stream` + +```python +completions_create_stream(req: CompletionsCreateReq) → Iterator[dict[str, Any]] +``` + +--- + + + +### method `cost_create` + +```python +cost_create(req: CostCreateReq) → CostCreateRes +``` + +--- + + + +### method `cost_purge` + +```python +cost_purge(req: CostPurgeReq) → CostPurgeRes +``` + +--- + + + +### method `cost_query` + +```python +cost_query(req: CostQueryReq) → CostQueryRes +``` + +--- + + + +### method `dataset_create_v2` + +```python +dataset_create_v2(req: DatasetCreateV2Req) → DatasetCreateV2Res +``` + +--- + + + +### method `dataset_delete_v2` + +```python +dataset_delete_v2(req: DatasetDeleteV2Req) → DatasetDeleteV2Res +``` + +--- + + + +### method `dataset_list_v2` + +```python +dataset_list_v2(req: DatasetListV2Req) → Iterator[DatasetReadV2Res] +``` + +--- + + + +### method `dataset_read_v2` + +```python +dataset_read_v2(req: DatasetReadV2Req) → DatasetReadV2Res +``` + +--- + + + +### method `ensure_project_exists` + +```python +ensure_project_exists(entity: str, project: str) → EnsureProjectExistsRes +``` + +--- + + + +### method `evaluate_model` + +```python +evaluate_model(req: EvaluateModelReq) → EvaluateModelRes +``` + +--- + + + +### method `evaluation_create_v2` + +```python +evaluation_create_v2(req: EvaluationCreateV2Req) → EvaluationCreateV2Res +``` + +--- + + + +### method `evaluation_delete_v2` + +```python +evaluation_delete_v2(req: EvaluationDeleteV2Req) → EvaluationDeleteV2Res +``` + +--- + + + +### method `evaluation_list_v2` + +```python +evaluation_list_v2(req: EvaluationListV2Req) → Iterator[EvaluationReadV2Res] +``` + +--- + + + +### method `evaluation_read_v2` + +```python +evaluation_read_v2(req: EvaluationReadV2Req) → EvaluationReadV2Res +``` + +--- + + + +### method `evaluation_run_create_v2` + +```python +evaluation_run_create_v2( + req: EvaluationRunCreateV2Req +) → EvaluationRunCreateV2Res +``` + +--- + + + +### method `evaluation_run_delete_v2` + +```python +evaluation_run_delete_v2( + req: EvaluationRunDeleteV2Req +) → EvaluationRunDeleteV2Res +``` + +--- + + + +### method `evaluation_run_finish_v2` + +```python +evaluation_run_finish_v2( + req: EvaluationRunFinishV2Req +) → EvaluationRunFinishV2Res +``` + +--- + + + +### method `evaluation_run_list_v2` + +```python +evaluation_run_list_v2( + req: EvaluationRunListV2Req +) → Iterator[EvaluationRunReadV2Res] +``` + +--- + + + +### method `evaluation_run_read_v2` + +```python +evaluation_run_read_v2(req: EvaluationRunReadV2Req) → EvaluationRunReadV2Res +``` + +--- + + + +### method `evaluation_status` + +```python +evaluation_status(req: EvaluationStatusReq) → EvaluationStatusRes +``` + +--- + + + +### method `feedback_create` + +```python +feedback_create(req: FeedbackCreateReq) → FeedbackCreateRes +``` + +--- + + + +### method `feedback_create_batch` + +```python +feedback_create_batch(req: FeedbackCreateBatchReq) → FeedbackCreateBatchRes +``` + +--- + + + +### method `feedback_purge` + +```python +feedback_purge(req: FeedbackPurgeReq) → FeedbackPurgeRes +``` + +--- + + + +### method `feedback_query` + +```python +feedback_query(req: FeedbackQueryReq) → FeedbackQueryRes +``` + +--- + + + +### method `feedback_replace` + +```python +feedback_replace(req: FeedbackReplaceReq) → FeedbackReplaceRes +``` + +--- + + + +### method `file_content_read` + +```python +file_content_read(req: FileContentReadReq) → FileContentReadRes +``` + +--- + + + +### method `file_create` + +```python +file_create(req: FileCreateReq) → FileCreateRes +``` + +--- + + + +### method `files_stats` + +```python +files_stats(req: FilesStatsReq) → FilesStatsRes +``` + +--- + + + +### method `image_create` + +```python +image_create(req: ImageGenerationCreateReq) → ImageGenerationCreateRes +``` + +--- + + + +### method `model_create_v2` + +```python +model_create_v2(req: ModelCreateV2Req) → ModelCreateV2Res +``` + +--- + + + +### method `model_delete_v2` + +```python +model_delete_v2(req: ModelDeleteV2Req) → ModelDeleteV2Res +``` + +--- + + + +### method `model_list_v2` + +```python +model_list_v2(req: ModelListV2Req) → Iterator[ModelReadV2Res] +``` + +--- + + + +### method `model_read_v2` + +```python +model_read_v2(req: ModelReadV2Req) → ModelReadV2Res +``` + +--- + + + +### method `obj_create` + +```python +obj_create(req: ObjCreateReq) → ObjCreateRes +``` + +--- + + + +### method `obj_delete` + +```python +obj_delete(req: ObjDeleteReq) → ObjDeleteRes +``` + +--- + + + +### method `obj_read` + +```python +obj_read(req: ObjReadReq) → ObjReadRes +``` + +--- + + + +### method `objs_query` + +```python +objs_query(req: ObjQueryReq) → ObjQueryRes +``` + +--- + + + +### method `op_create` + +```python +op_create(req: OpCreateReq) → OpCreateRes +``` + +--- + + + +### method `op_create_v2` + +```python +op_create_v2(req: OpCreateV2Req) → OpCreateV2Res +``` + +--- + + + +### method `op_delete_v2` + +```python +op_delete_v2(req: OpDeleteV2Req) → OpDeleteV2Res +``` + +--- + + + +### method `op_list_v2` + +```python +op_list_v2(req: OpListV2Req) → Iterator[OpReadV2Res] +``` + +--- + + + +### method `op_read` + +```python +op_read(req: OpReadReq) → OpReadRes +``` + +--- + + + +### method `op_read_v2` + +```python +op_read_v2(req: OpReadV2Req) → OpReadV2Res +``` + +--- + + + +### method `ops_query` + +```python +ops_query(req: OpQueryReq) → OpQueryRes +``` + +--- + + + +### method `otel_export` + +```python +otel_export(req: OtelExportReq) → OtelExportRes +``` + +--- + + + +### method `prediction_create_v2` + +```python +prediction_create_v2(req: PredictionCreateV2Req) → PredictionCreateV2Res +``` + +--- + + + +### method `prediction_delete_v2` + +```python +prediction_delete_v2(req: PredictionDeleteV2Req) → PredictionDeleteV2Res +``` + +--- + + + +### method `prediction_finish_v2` + +```python +prediction_finish_v2(req: PredictionFinishV2Req) → PredictionFinishV2Res +``` + +--- + + + +### method `prediction_list_v2` + +```python +prediction_list_v2(req: PredictionListV2Req) → Iterator[PredictionReadV2Res] +``` + +--- + + + +### method `prediction_read_v2` + +```python +prediction_read_v2(req: PredictionReadV2Req) → PredictionReadV2Res +``` + +--- + + + +### method `project_stats` + +```python +project_stats(req: ProjectStatsReq) → ProjectStatsRes +``` + +--- + + + +### method `refs_read_batch` + +```python +refs_read_batch(req: RefsReadBatchReq) → RefsReadBatchRes +``` + +--- + + + +### method `score_create_v2` + +```python +score_create_v2(req: ScoreCreateV2Req) → ScoreCreateV2Res +``` + +--- + + + +### method `score_delete_v2` + +```python +score_delete_v2(req: ScoreDeleteV2Req) → ScoreDeleteV2Res +``` + +--- + + + +### method `score_list_v2` + +```python +score_list_v2(req: ScoreListV2Req) → Iterator[ScoreReadV2Res] +``` + +--- + + + +### method `score_read_v2` + +```python +score_read_v2(req: ScoreReadV2Req) → ScoreReadV2Res +``` + +--- + + + +### method `scorer_create_v2` + +```python +scorer_create_v2(req: ScorerCreateV2Req) → ScorerCreateV2Res +``` + +--- + + + +### method `scorer_delete_v2` + +```python +scorer_delete_v2(req: ScorerDeleteV2Req) → ScorerDeleteV2Res +``` + +--- + + + +### method `scorer_list_v2` + +```python +scorer_list_v2(req: ScorerListV2Req) → Iterator[ScorerReadV2Res] +``` + +--- + + + +### method `scorer_read_v2` + +```python +scorer_read_v2(req: ScorerReadV2Req) → ScorerReadV2Res +``` + +--- + + + +### method `table_create` + +```python +table_create(req: TableCreateReq) → TableCreateRes +``` + +--- + + + +### method `table_create_from_digests` + +```python +table_create_from_digests( + req: TableCreateFromDigestsReq +) → TableCreateFromDigestsRes +``` + +--- + + + +### method `table_query` + +```python +table_query(req: TableQueryReq) → TableQueryRes +``` + +--- + + + +### method `table_query_stats` + +```python +table_query_stats(req: TableQueryStatsReq) → TableQueryStatsRes +``` + +--- + + + +### method `table_query_stats_batch` + +```python +table_query_stats_batch(req: TableQueryStatsBatchReq) → TableQueryStatsBatchRes +``` + +--- + + + +### method `table_query_stream` + +```python +table_query_stream(req: TableQueryReq) → Iterator[TableRowSchema] +``` + +--- + + + +### method `table_update` + +```python +table_update(req: TableUpdateReq) → TableUpdateRes +``` + +--- + + + +### method `threads_query_stream` + +```python +threads_query_stream(req: ThreadsQueryReq) → Iterator[ThreadSchema] +``` + +--- + + + +## class `ImageGenerationCreateReq` + +**Pydantic Fields:** + +- `project_id`: `` +- `inputs`: `` +- `wb_user_id`: `typing.Optional[str]` +- `track_llm_call`: `typing.Optional[bool]` + +--- + + + +## class `ImageGenerationCreateRes` + +**Pydantic Fields:** + +- `response`: `dict[str, typing.Any]` +- `weave_call_id`: `typing.Optional[str]` + +--- + + + +## class `ImageGenerationRequestInputs` + +**Pydantic Fields:** + +- `model`: `` +- `prompt`: `` +- `n`: `typing.Optional[int]` + +--- + + + +## class `LLMCostSchema` + +--- + + + +## class `LLMUsageSchema` + +--- + + + +## class `ModelCreateV2Body` + +**Pydantic Fields:** + +- `name`: `` +- `description`: `typing.Optional[str]` +- `source_code`: `` +- `attributes`: `typing.Optional[dict[str, typing.Any]]` + +--- + + + +## class `ModelCreateV2Req` + +**Pydantic Fields:** + +- `name`: `` +- `description`: `typing.Optional[str]` +- `source_code`: `` +- `attributes`: `typing.Optional[dict[str, typing.Any]]` +- `project_id`: `` +- `wb_user_id`: `typing.Optional[str]` + +--- + + + +## class `ModelCreateV2Res` + +**Pydantic Fields:** + +- `digest`: `` +- `object_id`: `` +- `version_index`: `` +- `model_ref`: `` + +--- + + + +## class `ModelDeleteV2Req` + +**Pydantic Fields:** + +- `project_id`: `` +- `object_id`: `` +- `digests`: `typing.Optional[list[str]]` +- `wb_user_id`: `typing.Optional[str]` + +--- + + + +## class `ModelDeleteV2Res` + +**Pydantic Fields:** + +- `num_deleted`: `` + +--- + + + +## class `ModelListV2Req` + +**Pydantic Fields:** + +- `project_id`: `` +- `limit`: `typing.Optional[int]` +- `offset`: `typing.Optional[int]` + +--- + + + +## class `ModelReadV2Req` + +**Pydantic Fields:** + +- `project_id`: `` +- `object_id`: `` +- `digest`: `` + +--- + + + +## class `ModelReadV2Res` + +**Pydantic Fields:** + +- `object_id`: `` +- `digest`: `` +- `version_index`: `` +- `created_at`: `` +- `name`: `` +- `description`: `typing.Optional[str]` +- `source_code`: `` +- `attributes`: `typing.Optional[dict[str, typing.Any]]` + +--- + + + +## class `ObjCreateReq` + +**Pydantic Fields:** + +- `obj`: `` + +--- + + + +## class `ObjCreateRes` + +**Pydantic Fields:** + +- `digest`: `` +- `object_id`: `typing.Optional[str]` + +--- + + + +## class `ObjDeleteReq` + +**Pydantic Fields:** + +- `project_id`: `` +- `object_id`: `` +- `digests`: `typing.Optional[list[str]]` + +--- + + + +## class `ObjDeleteRes` + +**Pydantic Fields:** + +- `num_deleted`: `` + +--- + + + +## class `ObjQueryReq` + +**Pydantic Fields:** + +- `project_id`: `` +- `filter`: `typing.Optional[ObjectVersionFilter]` +- `limit`: `typing.Optional[int]` +- `offset`: `typing.Optional[int]` +- `sort_by`: `typing.Optional[list[SortBy]]` +- `metadata_only`: `typing.Optional[bool]` +- `include_storage_size`: `typing.Optional[bool]` + +--- + + + +## class `ObjQueryRes` + +**Pydantic Fields:** + +- `objs`: `list[ObjSchema]` + +--- + + + +## class `ObjReadReq` + +**Pydantic Fields:** + +- `project_id`: `` +- `object_id`: `` +- `digest`: `` +- `metadata_only`: `typing.Optional[bool]` + +--- + + + +## class `ObjReadRes` + +**Pydantic Fields:** + +- `obj`: `` + +--- + + + +## class `ObjSchema` + +**Pydantic Fields:** + +- `project_id`: `` +- `object_id`: `` +- `created_at`: `` +- `deleted_at`: `typing.Optional[datetime.datetime]` +- `digest`: `` +- `version_index`: `` +- `is_latest`: `` +- `kind`: `` +- `base_object_class`: `typing.Optional[str]` +- `leaf_object_class`: `typing.Optional[str]` +- `val`: `typing.Any` +- `wb_user_id`: `typing.Optional[str]` +- `size_bytes`: `typing.Optional[int]` + +--- + + + +## class `ObjSchemaForInsert` + +**Pydantic Fields:** + +- `project_id`: `` +- `object_id`: `` +- `val`: `typing.Any` +- `builtin_object_class`: `typing.Optional[str]` +- `set_base_object_class`: `typing.Optional[str]` +- `wb_user_id`: `typing.Optional[str]` + + + +### method `model_post_init` + +```python +model_post_init(_ObjSchemaForInsert__context: Any) → None +``` + +--- + + + +## class `ObjectVersionFilter` + +**Pydantic Fields:** + +- `base_object_classes`: `typing.Optional[list[str]]` +- `exclude_base_object_classes`: `typing.Optional[list[str]]` +- `leaf_object_classes`: `typing.Optional[list[str]]` +- `object_ids`: `typing.Optional[list[str]]` +- `is_op`: `typing.Optional[bool]` +- `latest_only`: `typing.Optional[bool]` + +--- + + + +## class `OpCreateReq` + +**Pydantic Fields:** + +- `op_obj`: `` + +--- + + + +## class `OpCreateRes` + +**Pydantic Fields:** + +- `digest`: `` + +--- + + + +## class `OpCreateV2Body` +Request body for creating an Op object via REST API. + +This model excludes project_id since it comes from the URL path in RESTful endpoints. + +**Pydantic Fields:** + +- `name`: `typing.Optional[str]` +- `source_code`: `typing.Optional[str]` + +--- + + + +## class `OpCreateV2Req` +Request model for creating an Op object. + +Extends OpCreateV2Body by adding project_id for internal API usage. + +**Pydantic Fields:** + +- `name`: `typing.Optional[str]` +- `source_code`: `typing.Optional[str]` +- `project_id`: `` +- `wb_user_id`: `typing.Optional[str]` + +--- + + + +## class `OpCreateV2Res` +Response model for creating an Op object. + +**Pydantic Fields:** + +- `digest`: `` +- `object_id`: `` +- `version_index`: `` + +--- + + + +## class `OpDeleteV2Req` + +**Pydantic Fields:** + +- `project_id`: `` +- `object_id`: `` +- `digests`: `typing.Optional[list[str]]` +- `wb_user_id`: `typing.Optional[str]` + +--- + + + +## class `OpDeleteV2Res` + +**Pydantic Fields:** + +- `num_deleted`: `` + +--- + + + +## class `OpListV2Req` + +**Pydantic Fields:** + +- `project_id`: `` +- `limit`: `typing.Optional[int]` +- `offset`: `typing.Optional[int]` +- `wb_user_id`: `typing.Optional[str]` + +--- + + + +## class `OpQueryReq` + +**Pydantic Fields:** + +- `project_id`: `` +- `filter`: `typing.Optional[OpVersionFilter]` + +--- + + + +## class `OpQueryRes` + +**Pydantic Fields:** + +- `op_objs`: `list[ObjSchema]` + +--- + + + +## class `OpReadReq` + +**Pydantic Fields:** + +- `project_id`: `` +- `name`: `` +- `digest`: `` + +--- + + + +## class `OpReadRes` + +**Pydantic Fields:** + +- `op_obj`: `` + +--- + + + +## class `OpReadV2Req` + +**Pydantic Fields:** ------- +- `project_id`: `` +- `object_id`: `` +- `digest`: `` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `FilesStatsRes` +## class `OpReadV2Res` +Response model for reading an Op object. + +The code field contains the actual source code of the op. **Pydantic Fields:** -- `total_size_bytes`: `` +- `object_id`: `` +- `digest`: `` +- `version_index`: `` +- `created_at`: `` +- `code`: `` ------- +--- + + + +## class `OpVersionFilter` + +**Pydantic Fields:** + +- `op_names`: `typing.Optional[list[str]]` +- `latest_only`: `typing.Optional[bool]` --- - + -## class `ImageGenerationCreateReq` +## class `OtelExportReq` **Pydantic Fields:** - `project_id`: `` -- `inputs`: `` +- `traces`: `typing.Any` +- `wb_run_id`: `typing.Optional[str]` - `wb_user_id`: `typing.Optional[str]` -- `track_llm_call`: `typing.Optional[bool]` - ------- --- - + -## class `ImageGenerationCreateRes` +## class `OtelExportRes` **Pydantic Fields:** -- `response`: `dict[str, typing.Any]` -- `weave_call_id`: `typing.Optional[str]` - ------- +- `partial_success`: `typing.Optional[ExportTracePartialSuccess]` --- - + -## class `ImageGenerationRequestInputs` +## class `PredictionCreateV2Body` +Request body for creating a Prediction via REST API. + +This model excludes project_id since it comes from the URL path in RESTful endpoints. **Pydantic Fields:** - `model`: `` -- `prompt`: `` -- `n`: `typing.Optional[int]` - ------- +- `inputs`: `dict[str, typing.Any]` +- `output`: `typing.Any` +- `evaluation_run_id`: `typing.Optional[str]` --- - + -## class `LLMCostSchema` +## class `PredictionCreateV2Req` +Request model for creating a Prediction. ---- +Extends PredictionCreateV2Body by adding project_id for internal API usage. - +**Pydantic Fields:** -## class `LLMUsageSchema` +- `model`: `` +- `inputs`: `dict[str, typing.Any]` +- `output`: `typing.Any` +- `evaluation_run_id`: `typing.Optional[str]` +- `project_id`: `` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `ObjCreateReq` +## class `PredictionCreateV2Res` **Pydantic Fields:** -- `obj`: `` - ------- +- `prediction_id`: `` --- - + -## class `ObjCreateRes` +## class `PredictionDeleteV2Req` **Pydantic Fields:** -- `digest`: `` +- `project_id`: `` +- `prediction_ids`: `list[str]` +- `wb_user_id`: `typing.Optional[str]` + +--- + + + +## class `PredictionDeleteV2Res` + +**Pydantic Fields:** ------- +- `num_deleted`: `` --- - + -## class `ObjDeleteReq` +## class `PredictionFinishV2Req` **Pydantic Fields:** - `project_id`: `` -- `object_id`: `` -- `digests`: `typing.Optional[list[str]]` - ------- +- `prediction_id`: `` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `ObjDeleteRes` +## class `PredictionFinishV2Res` **Pydantic Fields:** -- `num_deleted`: `` - ------- +- `success`: `` --- - + -## class `ObjQueryReq` +## class `PredictionListV2Req` **Pydantic Fields:** - `project_id`: `` -- `filter`: `typing.Optional[ObjectVersionFilter]` +- `evaluation_run_id`: `typing.Optional[str]` - `limit`: `typing.Optional[int]` - `offset`: `typing.Optional[int]` -- `sort_by`: `typing.Optional[list[SortBy]]` -- `metadata_only`: `typing.Optional[bool]` -- `include_storage_size`: `typing.Optional[bool]` - ------- +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `ObjQueryRes` +## class `PredictionListV2Res` **Pydantic Fields:** -- `objs`: `list[ObjSchema]` - ------- +- `predictions`: `list[PredictionReadV2Res]` --- - + -## class `ObjReadReq` +## class `PredictionReadV2Req` **Pydantic Fields:** - `project_id`: `` -- `object_id`: `` -- `digest`: `` -- `metadata_only`: `typing.Optional[bool]` - ------- +- `prediction_id`: `` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `ObjReadRes` +## class `PredictionReadV2Res` **Pydantic Fields:** -- `obj`: `` - ------- +- `prediction_id`: `` +- `model`: `` +- `inputs`: `dict[str, typing.Any]` +- `output`: `typing.Any` +- `evaluation_run_id`: `typing.Optional[str]` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `ObjSchema` +## class `ProjectStatsReq` **Pydantic Fields:** - `project_id`: `` -- `object_id`: `` -- `created_at`: `` -- `deleted_at`: `typing.Optional[datetime.datetime]` -- `digest`: `` -- `version_index`: `` -- `is_latest`: `` -- `kind`: `` -- `base_object_class`: `typing.Optional[str]` -- `leaf_object_class`: `typing.Optional[str]` -- `val`: `typing.Any` -- `wb_user_id`: `typing.Optional[str]` -- `size_bytes`: `typing.Optional[int]` - ------- +- `include_trace_storage_size`: `typing.Optional[bool]` +- `include_object_storage_size`: `typing.Optional[bool]` +- `include_table_storage_size`: `typing.Optional[bool]` +- `include_file_storage_size`: `typing.Optional[bool]` --- - + -## class `ObjSchemaForInsert` +## class `ProjectStatsRes` **Pydantic Fields:** -- `project_id`: `` -- `object_id`: `` -- `val`: `typing.Any` -- `builtin_object_class`: `typing.Optional[str]` -- `set_base_object_class`: `typing.Optional[str]` -- `wb_user_id`: `typing.Optional[str]` +- `trace_storage_size_bytes`: `` +- `objects_storage_size_bytes`: `` +- `tables_storage_size_bytes`: `` +- `files_storage_size_bytes`: `` + +--- ---------- + - +## class `RefsReadBatchReq` -### method `model_post_init` +**Pydantic Fields:** -```python -model_post_init(_ObjSchemaForInsert__context: Any) → None -``` +- `refs`: `list[str]` --- - + -## class `ObjectVersionFilter` +## class `RefsReadBatchRes` **Pydantic Fields:** -- `base_object_classes`: `typing.Optional[list[str]]` -- `exclude_base_object_classes`: `typing.Optional[list[str]]` -- `leaf_object_classes`: `typing.Optional[list[str]]` -- `object_ids`: `typing.Optional[list[str]]` -- `is_op`: `typing.Optional[bool]` -- `latest_only`: `typing.Optional[bool]` - ------- +- `vals`: `list[typing.Any]` --- - + -## class `OpCreateReq` +## class `ScoreCreateV2Body` +Request body for creating a Score via REST API. -**Pydantic Fields:** +This model excludes project_id since it comes from the URL path in RESTful endpoints. -- `op_obj`: `` +**Pydantic Fields:** ------- +- `prediction_id`: `` +- `scorer`: `` +- `value`: `` +- `evaluation_run_id`: `typing.Optional[str]` --- - + -## class `OpCreateRes` +## class `ScoreCreateV2Req` +Request model for creating a Score. + +Extends ScoreCreateV2Body by adding project_id for internal API usage. **Pydantic Fields:** -- `digest`: `` +- `prediction_id`: `` +- `scorer`: `` +- `value`: `` +- `evaluation_run_id`: `typing.Optional[str]` +- `project_id`: `` +- `wb_user_id`: `typing.Optional[str]` + +--- + + + +## class `ScoreCreateV2Res` + +**Pydantic Fields:** ------- +- `score_id`: `` --- - + -## class `OpQueryReq` +## class `ScoreDeleteV2Req` **Pydantic Fields:** - `project_id`: `` -- `filter`: `typing.Optional[OpVersionFilter]` - ------- +- `score_ids`: `list[str]` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `OpQueryRes` +## class `ScoreDeleteV2Res` **Pydantic Fields:** -- `op_objs`: `list[ObjSchema]` - ------- +- `num_deleted`: `` --- - + -## class `OpReadReq` +## class `ScoreListV2Req` **Pydantic Fields:** - `project_id`: `` -- `name`: `` -- `digest`: `` - ------- +- `evaluation_run_id`: `typing.Optional[str]` +- `limit`: `typing.Optional[int]` +- `offset`: `typing.Optional[int]` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `OpReadRes` +## class `ScoreReadV2Req` **Pydantic Fields:** -- `op_obj`: `` - ------- +- `project_id`: `` +- `score_id`: `` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `OpVersionFilter` +## class `ScoreReadV2Res` **Pydantic Fields:** -- `op_names`: `typing.Optional[list[str]]` -- `latest_only`: `typing.Optional[bool]` +- `score_id`: `` +- `scorer`: `` +- `value`: `` +- `evaluation_run_id`: `typing.Optional[str]` +- `wb_user_id`: `typing.Optional[str]` + +--- + + ------- +## class `ScorerCreateV2Body` + +**Pydantic Fields:** + +- `name`: `` +- `description`: `typing.Optional[str]` +- `op_source_code`: `` --- - + -## class `OtelExportReq` +## class `ScorerCreateV2Req` **Pydantic Fields:** +- `name`: `` +- `description`: `typing.Optional[str]` +- `op_source_code`: `` - `project_id`: `` -- `traces`: `typing.Any` - `wb_user_id`: `typing.Optional[str]` ------- - --- - + -## class `OtelExportRes` +## class `ScorerCreateV2Res` **Pydantic Fields:** -- `partial_success`: `typing.Optional[ExportTracePartialSuccess]` - ------- +- `digest`: `` +- `object_id`: `` +- `version_index`: `` +- `scorer`: `` --- - + -## class `ProjectStatsReq` +## class `ScorerDeleteV2Req` **Pydantic Fields:** - `project_id`: `` -- `include_trace_storage_size`: `typing.Optional[bool]` -- `include_object_storage_size`: `typing.Optional[bool]` -- `include_table_storage_size`: `typing.Optional[bool]` -- `include_file_storage_size`: `typing.Optional[bool]` - ------- +- `object_id`: `` +- `digests`: `typing.Optional[list[str]]` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `ProjectStatsRes` +## class `ScorerDeleteV2Res` **Pydantic Fields:** -- `trace_storage_size_bytes`: `` -- `objects_storage_size_bytes`: `` -- `tables_storage_size_bytes`: `` -- `files_storage_size_bytes`: `` - ------- +- `num_deleted`: `` --- - + -## class `RefsReadBatchReq` +## class `ScorerListV2Req` **Pydantic Fields:** -- `refs`: `list[str]` - ------- +- `project_id`: `` +- `limit`: `typing.Optional[int]` +- `offset`: `typing.Optional[int]` +- `wb_user_id`: `typing.Optional[str]` --- - + -## class `RefsReadBatchRes` +## class `ScorerReadV2Req` **Pydantic Fields:** -- `vals`: `list[typing.Any]` +- `project_id`: `` +- `object_id`: `` +- `digest`: `` +- `wb_user_id`: `typing.Optional[str]` + +--- + + + +## class `ScorerReadV2Res` + +**Pydantic Fields:** ------- +- `object_id`: `` +- `digest`: `` +- `version_index`: `` +- `created_at`: `` +- `name`: `` +- `description`: `typing.Optional[str]` +- `score_op`: `` --- - + ## class `SortBy` @@ -1333,11 +2898,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `field`: `` - `direction`: `typing.Literal['asc', 'desc']` ------- - --- - + ## class `StartedCallSchemaForInsert` @@ -1354,27 +2917,26 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `started_at`: `` - `attributes`: `dict[str, typing.Any]` - `inputs`: `dict[str, typing.Any]` +- `otel_dump`: `typing.Optional[dict[str, typing.Any]]` - `wb_user_id`: `typing.Optional[str]` - `wb_run_id`: `typing.Optional[str]` - `wb_run_step`: `typing.Optional[int]` ------- - --- - + ## class `SummaryInsertMap` --- - + ## class `SummaryMap` --- - + ## class `TableAppendSpec` @@ -1382,11 +2944,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `append`: `` ------- - --- - + ## class `TableAppendSpecPayload` @@ -1394,11 +2954,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `row`: `dict[str, typing.Any]` ------- - --- - + ## class `TableCreateFromDigestsReq` @@ -1407,11 +2965,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `project_id`: `` - `row_digests`: `list[str]` ------- - --- - + ## class `TableCreateFromDigestsRes` @@ -1419,11 +2975,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `digest`: `` ------- - --- - + ## class `TableCreateReq` @@ -1431,11 +2985,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `table`: `` ------- - --- - + ## class `TableCreateRes` @@ -1444,11 +2996,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `digest`: `` - `row_digests`: `list[str]` ------- - --- - + ## class `TableInsertSpec` @@ -1456,11 +3006,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `insert`: `` ------- - --- - + ## class `TableInsertSpecPayload` @@ -1469,11 +3017,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `index`: `` - `row`: `dict[str, typing.Any]` ------- - --- - + ## class `TablePopSpec` @@ -1481,11 +3027,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `pop`: `` ------- - --- - + ## class `TablePopSpecPayload` @@ -1493,11 +3037,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `index`: `` ------- - --- - + ## class `TableQueryReq` @@ -1510,11 +3052,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `offset`: `typing.Optional[int]` - `sort_by`: `typing.Optional[list[SortBy]]` ------- - --- - + ## class `TableQueryRes` @@ -1522,11 +3062,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `rows`: `list[TableRowSchema]` ------- - --- - + ## class `TableQueryStatsBatchReq` @@ -1536,11 +3074,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `digests`: `typing.Optional[list[str]]` - `include_storage_size`: `typing.Optional[bool]` ------- - --- - + ## class `TableQueryStatsBatchRes` @@ -1548,11 +3084,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `tables`: `list[TableStatsRow]` ------- - --- - + ## class `TableQueryStatsReq` @@ -1561,11 +3095,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `project_id`: `` - `digest`: `` ------- - --- - + ## class `TableQueryStatsRes` @@ -1573,11 +3105,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `count`: `` ------- - --- - + ## class `TableRowFilter` @@ -1585,11 +3115,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `row_digests`: `typing.Optional[list[str]]` ------- - --- - + ## class `TableRowSchema` @@ -1599,11 +3127,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `val`: `typing.Any` - `original_index`: `typing.Optional[int]` ------- - --- - + ## class `TableSchemaForInsert` @@ -1612,11 +3138,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `project_id`: `` - `rows`: `list[dict[str, typing.Any]]` ------- - --- - + ## class `TableStatsRow` @@ -1626,11 +3150,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `digest`: `` - `storage_size_bytes`: `typing.Optional[int]` ------- - --- - + ## class `TableUpdateReq` @@ -1640,11 +3162,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `base_digest`: `` - `updates`: `list[typing.Union[TableAppendSpec, TablePopSpec, TableInsertSpec]]` ------- - --- - + ## class `TableUpdateRes` @@ -1653,11 +3173,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `digest`: `` - `updated_row_digests`: `list[str]` ------- - --- - + ## class `ThreadSchema` @@ -1672,11 +3190,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `p50_turn_duration_ms`: `typing.Optional[float]` - `p99_turn_duration_ms`: `typing.Optional[float]` ------- - --- - + ## class `ThreadsQueryFilter` @@ -1686,11 +3202,9 @@ model_post_init(_ObjSchemaForInsert__context: Any) → None - `before_datetime`: `typing.Optional[datetime.datetime]` - `thread_ids`: `typing.Optional[list[str]]` ------- - --- - + ## class `ThreadsQueryReq` Query threads with aggregated statistics based on turn calls only. @@ -1705,17 +3219,15 @@ Turn calls are the immediate children of thread contexts (where call.id == turn_ - `offset`: `typing.Optional[int]` - `sort_by`: `typing.Optional[list[SortBy]]` ------- - --- - + ## class `TraceServerInterface` --- - + ### method `actions_execute_batch` @@ -1725,7 +3237,7 @@ actions_execute_batch(req: ActionsExecuteBatchReq) → ActionsExecuteBatchRes --- - + ### method `call_end` @@ -1735,7 +3247,7 @@ call_end(req: CallEndReq) → CallEndRes --- - + ### method `call_read` @@ -1745,7 +3257,7 @@ call_read(req: CallReadReq) → CallReadRes --- - + ### method `call_start` @@ -1755,7 +3267,7 @@ call_start(req: CallStartReq) → CallStartRes --- - + ### method `call_start_batch` @@ -1765,7 +3277,7 @@ call_start_batch(req: CallCreateBatchReq) → CallCreateBatchRes --- - + ### method `call_update` @@ -1775,7 +3287,7 @@ call_update(req: CallUpdateReq) → CallUpdateRes --- - + ### method `calls_delete` @@ -1785,7 +3297,7 @@ calls_delete(req: CallsDeleteReq) → CallsDeleteRes --- - + ### method `calls_query` @@ -1795,7 +3307,7 @@ calls_query(req: CallsQueryReq) → CallsQueryRes --- - + ### method `calls_query_stats` @@ -1805,7 +3317,7 @@ calls_query_stats(req: CallsQueryStatsReq) → CallsQueryStatsRes --- - + ### method `calls_query_stream` @@ -1815,7 +3327,7 @@ calls_query_stream(req: CallsQueryReq) → Iterator[CallSchema] --- - + ### method `completions_create` @@ -1825,7 +3337,7 @@ completions_create(req: CompletionsCreateReq) → CompletionsCreateRes --- - + ### method `completions_create_stream` @@ -1835,7 +3347,7 @@ completions_create_stream(req: CompletionsCreateReq) → Iterator[dict[str, Any] --- - + ### method `cost_create` @@ -1845,7 +3357,7 @@ cost_create(req: CostCreateReq) → CostCreateRes --- - + ### method `cost_purge` @@ -1855,7 +3367,7 @@ cost_purge(req: CostPurgeReq) → CostPurgeRes --- - + ### method `cost_query` @@ -1865,7 +3377,7 @@ cost_query(req: CostQueryReq) → CostQueryRes --- - + ### method `ensure_project_exists` @@ -1875,7 +3387,7 @@ ensure_project_exists(entity: str, project: str) → EnsureProjectExistsRes --- - + ### method `evaluate_model` @@ -1885,7 +3397,7 @@ evaluate_model(req: EvaluateModelReq) → EvaluateModelRes --- - + ### method `evaluation_status` @@ -1895,7 +3407,7 @@ evaluation_status(req: EvaluationStatusReq) → EvaluationStatusRes --- - + ### method `feedback_create` @@ -1905,7 +3417,7 @@ feedback_create(req: FeedbackCreateReq) → FeedbackCreateRes --- - + ### method `feedback_create_batch` @@ -1915,7 +3427,7 @@ feedback_create_batch(req: FeedbackCreateBatchReq) → FeedbackCreateBatchRes --- - + ### method `feedback_purge` @@ -1925,7 +3437,7 @@ feedback_purge(req: FeedbackPurgeReq) → FeedbackPurgeRes --- - + ### method `feedback_query` @@ -1935,7 +3447,7 @@ feedback_query(req: FeedbackQueryReq) → FeedbackQueryRes --- - + ### method `feedback_replace` @@ -1945,7 +3457,7 @@ feedback_replace(req: FeedbackReplaceReq) → FeedbackReplaceRes --- - + ### method `file_content_read` @@ -1955,7 +3467,7 @@ file_content_read(req: FileContentReadReq) → FileContentReadRes --- - + ### method `file_create` @@ -1965,7 +3477,7 @@ file_create(req: FileCreateReq) → FileCreateRes --- - + ### method `files_stats` @@ -1975,7 +3487,7 @@ files_stats(req: FilesStatsReq) → FilesStatsRes --- - + ### method `image_create` @@ -1985,7 +3497,7 @@ image_create(req: ImageGenerationCreateReq) → ImageGenerationCreateRes --- - + ### method `obj_create` @@ -1995,7 +3507,7 @@ obj_create(req: ObjCreateReq) → ObjCreateRes --- - + ### method `obj_delete` @@ -2005,7 +3517,7 @@ obj_delete(req: ObjDeleteReq) → ObjDeleteRes --- - + ### method `obj_read` @@ -2015,7 +3527,7 @@ obj_read(req: ObjReadReq) → ObjReadRes --- - + ### method `objs_query` @@ -2025,7 +3537,7 @@ objs_query(req: ObjQueryReq) → ObjQueryRes --- - + ### method `op_create` @@ -2035,7 +3547,7 @@ op_create(req: OpCreateReq) → OpCreateRes --- - + ### method `op_read` @@ -2045,7 +3557,7 @@ op_read(req: OpReadReq) → OpReadRes --- - + ### method `ops_query` @@ -2055,7 +3567,7 @@ ops_query(req: OpQueryReq) → OpQueryRes --- - + ### method `otel_export` @@ -2065,7 +3577,7 @@ otel_export(req: OtelExportReq) → OtelExportRes --- - + ### method `project_stats` @@ -2075,7 +3587,7 @@ project_stats(req: ProjectStatsReq) → ProjectStatsRes --- - + ### method `refs_read_batch` @@ -2085,7 +3597,7 @@ refs_read_batch(req: RefsReadBatchReq) → RefsReadBatchRes --- - + ### method `table_create` @@ -2095,7 +3607,7 @@ table_create(req: TableCreateReq) → TableCreateRes --- - + ### method `table_create_from_digests` @@ -2107,7 +3619,7 @@ table_create_from_digests( --- - + ### method `table_query` @@ -2117,7 +3629,7 @@ table_query(req: TableQueryReq) → TableQueryRes --- - + ### method `table_query_stats` @@ -2127,7 +3639,7 @@ table_query_stats(req: TableQueryStatsReq) → TableQueryStatsRes --- - + ### method `table_query_stats_batch` @@ -2137,7 +3649,7 @@ table_query_stats_batch(req: TableQueryStatsBatchReq) → TableQueryStatsBatchRe --- - + ### method `table_query_stream` @@ -2147,7 +3659,7 @@ table_query_stream(req: TableQueryReq) → Iterator[TableRowSchema] --- - + ### method `table_update` @@ -2157,7 +3669,7 @@ table_update(req: TableUpdateReq) → TableUpdateRes --- - + ### method `threads_query_stream` @@ -2167,13 +3679,370 @@ threads_query_stream(req: ThreadsQueryReq) → Iterator[ThreadSchema] --- - + + +## class `TraceServerInterfaceV2` +V2 API endpoints for Trace Server. + +This protocol contains the next generation of trace server APIs that provide cleaner, more RESTful interfaces. Implementations should support both this protocol and TraceServerInterface to maintain backward compatibility. + +--- + + + +### method `dataset_create_v2` + +```python +dataset_create_v2(req: DatasetCreateV2Req) → DatasetCreateV2Res +``` + +--- + + + +### method `dataset_delete_v2` + +```python +dataset_delete_v2(req: DatasetDeleteV2Req) → DatasetDeleteV2Res +``` + +--- + + + +### method `dataset_list_v2` + +```python +dataset_list_v2(req: DatasetListV2Req) → Iterator[DatasetReadV2Res] +``` + +--- + + + +### method `dataset_read_v2` + +```python +dataset_read_v2(req: DatasetReadV2Req) → DatasetReadV2Res +``` + +--- + + + +### method `evaluation_create_v2` + +```python +evaluation_create_v2(req: EvaluationCreateV2Req) → EvaluationCreateV2Res +``` + +--- + + + +### method `evaluation_delete_v2` + +```python +evaluation_delete_v2(req: EvaluationDeleteV2Req) → EvaluationDeleteV2Res +``` + +--- + + + +### method `evaluation_list_v2` + +```python +evaluation_list_v2(req: EvaluationListV2Req) → Iterator[EvaluationReadV2Res] +``` + +--- + + + +### method `evaluation_read_v2` + +```python +evaluation_read_v2(req: EvaluationReadV2Req) → EvaluationReadV2Res +``` + +--- + + + +### method `evaluation_run_create_v2` + +```python +evaluation_run_create_v2( + req: EvaluationRunCreateV2Req +) → EvaluationRunCreateV2Res +``` + +--- + + + +### method `evaluation_run_delete_v2` + +```python +evaluation_run_delete_v2( + req: EvaluationRunDeleteV2Req +) → EvaluationRunDeleteV2Res +``` + +--- + + + +### method `evaluation_run_finish_v2` + +```python +evaluation_run_finish_v2( + req: EvaluationRunFinishV2Req +) → EvaluationRunFinishV2Res +``` + +--- + + + +### method `evaluation_run_list_v2` + +```python +evaluation_run_list_v2( + req: EvaluationRunListV2Req +) → Iterator[EvaluationRunReadV2Res] +``` + +--- + + + +### method `evaluation_run_read_v2` + +```python +evaluation_run_read_v2(req: EvaluationRunReadV2Req) → EvaluationRunReadV2Res +``` + +--- + + + +### method `model_create_v2` + +```python +model_create_v2(req: ModelCreateV2Req) → ModelCreateV2Res +``` + +--- + + + +### method `model_delete_v2` + +```python +model_delete_v2(req: ModelDeleteV2Req) → ModelDeleteV2Res +``` + +--- + + + +### method `model_list_v2` + +```python +model_list_v2(req: ModelListV2Req) → Iterator[ModelReadV2Res] +``` + +--- + + + +### method `model_read_v2` + +```python +model_read_v2(req: ModelReadV2Req) → ModelReadV2Res +``` + +--- + + + +### method `op_create_v2` + +```python +op_create_v2(req: OpCreateV2Req) → OpCreateV2Res +``` + +--- + + + +### method `op_delete_v2` + +```python +op_delete_v2(req: OpDeleteV2Req) → OpDeleteV2Res +``` + +--- + + + +### method `op_list_v2` + +```python +op_list_v2(req: OpListV2Req) → Iterator[OpReadV2Res] +``` + +--- + + + +### method `op_read_v2` + +```python +op_read_v2(req: OpReadV2Req) → OpReadV2Res +``` + +--- + + + +### method `prediction_create_v2` + +```python +prediction_create_v2(req: PredictionCreateV2Req) → PredictionCreateV2Res +``` + +--- + + + +### method `prediction_delete_v2` + +```python +prediction_delete_v2(req: PredictionDeleteV2Req) → PredictionDeleteV2Res +``` + +--- + + + +### method `prediction_finish_v2` + +```python +prediction_finish_v2(req: PredictionFinishV2Req) → PredictionFinishV2Res +``` + +--- + + + +### method `prediction_list_v2` + +```python +prediction_list_v2(req: PredictionListV2Req) → Iterator[PredictionReadV2Res] +``` + +--- + + + +### method `prediction_read_v2` + +```python +prediction_read_v2(req: PredictionReadV2Req) → PredictionReadV2Res +``` + +--- + + + +### method `score_create_v2` + +```python +score_create_v2(req: ScoreCreateV2Req) → ScoreCreateV2Res +``` + +--- + + + +### method `score_delete_v2` + +```python +score_delete_v2(req: ScoreDeleteV2Req) → ScoreDeleteV2Res +``` + +--- + + + +### method `score_list_v2` + +```python +score_list_v2(req: ScoreListV2Req) → Iterator[ScoreReadV2Res] +``` + +--- + + + +### method `score_read_v2` + +```python +score_read_v2(req: ScoreReadV2Req) → ScoreReadV2Res +``` + +--- + + + +### method `scorer_create_v2` + +```python +scorer_create_v2(req: ScorerCreateV2Req) → ScorerCreateV2Res +``` + +--- + + + +### method `scorer_delete_v2` + +```python +scorer_delete_v2(req: ScorerDeleteV2Req) → ScorerDeleteV2Res +``` + +--- + + + +### method `scorer_list_v2` + +```python +scorer_list_v2(req: ScorerListV2Req) → Iterator[ScorerReadV2Res] +``` + +--- + + + +### method `scorer_read_v2` + +```python +scorer_read_v2(req: ScorerReadV2Req) → ScorerReadV2Res +``` + +--- + + ## class `TraceStatus` --- - + ## class `WeaveSummarySchema` diff --git a/weave/reference/python-sdk/trace_server_bindings/remote_http_trace_server.mdx b/weave/reference/python-sdk/trace_server_bindings/remote_http_trace_server.mdx index b548360151..10a6e31c2a 100644 --- a/weave/reference/python-sdk/trace_server_bindings/remote_http_trace_server.mdx +++ b/weave/reference/python-sdk/trace_server_bindings/remote_http_trace_server.mdx @@ -8,11 +8,11 @@ description: "Python SDK reference for weave.trace_server_bindings.remote_http_t --- - + ## class `RemoteHTTPTraceServer` - + ### method `__init__` @@ -28,7 +28,7 @@ __init__( --- - + ### method `actions_execute_batch` @@ -40,7 +40,7 @@ actions_execute_batch( --- - + ### method `call_end` @@ -50,7 +50,7 @@ call_end(req: Union[CallEndReq, dict[str, Any]]) → CallEndRes --- - + ### method `call_read` @@ -60,7 +60,7 @@ call_read(req: Union[CallReadReq, dict[str, Any]]) → CallReadRes --- - + ### method `call_start` @@ -70,7 +70,7 @@ call_start(req: Union[CallStartReq, dict[str, Any]]) → CallStartRes --- - + ### method `call_start_batch` @@ -80,7 +80,7 @@ call_start_batch(req: CallCreateBatchReq) → CallCreateBatchRes --- - + ### method `call_update` @@ -90,7 +90,7 @@ call_update(req: Union[CallUpdateReq, dict[str, Any]]) → CallUpdateRes --- - + ### method `calls_delete` @@ -100,7 +100,7 @@ calls_delete(req: Union[CallsDeleteReq, dict[str, Any]]) → CallsDeleteRes --- - + ### method `calls_query` @@ -110,7 +110,7 @@ calls_query(req: Union[CallsQueryReq, dict[str, Any]]) → CallsQueryRes --- - + ### method `calls_query_stats` @@ -122,7 +122,7 @@ calls_query_stats( --- - + ### method `calls_query_stream` @@ -134,7 +134,7 @@ calls_query_stream( --- - + ### method `completions_create` @@ -144,7 +144,7 @@ completions_create(req: CompletionsCreateReq) → CompletionsCreateRes --- - + ### method `completions_create_stream` @@ -154,7 +154,7 @@ completions_create_stream(req: CompletionsCreateReq) → Iterator[dict[str, Any] --- - + ### method `cost_create` @@ -164,7 +164,7 @@ cost_create(req: Union[CostCreateReq, dict[str, Any]]) → CostCreateRes --- - + ### method `cost_purge` @@ -174,7 +174,7 @@ cost_purge(req: Union[CostPurgeReq, dict[str, Any]]) → CostPurgeRes --- - + ### method `cost_query` @@ -184,7 +184,63 @@ cost_query(req: Union[CostQueryReq, dict[str, Any]]) → CostQueryRes --- - + + +### method `dataset_create_v2` + +```python +dataset_create_v2( + req: Union[DatasetCreateV2Req, dict[str, Any]] +) → DatasetCreateV2Res +``` + +--- + + + +### method `dataset_delete_v2` + +```python +dataset_delete_v2( + req: Union[DatasetDeleteV2Req, dict[str, Any]] +) → DatasetDeleteV2Res +``` + +--- + + + +### method `dataset_list_v2` + +```python +dataset_list_v2( + req: Union[DatasetListV2Req, dict[str, Any]] +) → Iterator[DatasetReadV2Res] +``` + +--- + + + +### method `dataset_read_v2` + +```python +dataset_read_v2(req: Union[DatasetReadV2Req, dict[str, Any]]) → DatasetReadV2Res +``` + +--- + + + +### method `delete` + +```python +delete(url: str, *args: Any, **kwargs: Any) → Response +``` + +--- + + ### method `ensure_project_exists` @@ -194,7 +250,7 @@ ensure_project_exists(entity: str, project: str) → EnsureProjectExistsRes --- - + ### method `evaluate_model` @@ -204,7 +260,115 @@ evaluate_model(req: EvaluateModelReq) → EvaluateModelRes --- - + + +### method `evaluation_create_v2` + +```python +evaluation_create_v2( + req: Union[EvaluationCreateV2Req, dict[str, Any]] +) → EvaluationCreateV2Res +``` + +--- + + + +### method `evaluation_delete_v2` + +```python +evaluation_delete_v2( + req: Union[EvaluationDeleteV2Req, dict[str, Any]] +) → EvaluationDeleteV2Res +``` + +--- + + + +### method `evaluation_list_v2` + +```python +evaluation_list_v2( + req: Union[EvaluationListV2Req, dict[str, Any]] +) → Iterator[EvaluationReadV2Res] +``` + +--- + + + +### method `evaluation_read_v2` + +```python +evaluation_read_v2( + req: Union[EvaluationReadV2Req, dict[str, Any]] +) → EvaluationReadV2Res +``` + +--- + + + +### method `evaluation_run_create_v2` + +```python +evaluation_run_create_v2( + req: Union[EvaluationRunCreateV2Req, dict[str, Any]] +) → EvaluationRunCreateV2Res +``` + +--- + + + +### method `evaluation_run_delete_v2` + +```python +evaluation_run_delete_v2( + req: Union[EvaluationRunDeleteV2Req, dict[str, Any]] +) → EvaluationRunDeleteV2Res +``` + +--- + + + +### method `evaluation_run_finish_v2` + +```python +evaluation_run_finish_v2( + req: Union[EvaluationRunFinishV2Req, dict[str, Any]] +) → EvaluationRunFinishV2Res +``` + +--- + + + +### method `evaluation_run_list_v2` + +```python +evaluation_run_list_v2( + req: Union[EvaluationRunListV2Req, dict[str, Any]] +) → Iterator[EvaluationRunReadV2Res] +``` + +--- + + + +### method `evaluation_run_read_v2` + +```python +evaluation_run_read_v2( + req: Union[EvaluationRunReadV2Req, dict[str, Any]] +) → EvaluationRunReadV2Res +``` + +--- + + ### method `evaluation_status` @@ -214,7 +378,7 @@ evaluation_status(req: EvaluationStatusReq) → EvaluationStatusRes --- - + ### method `feedback_create` @@ -226,7 +390,7 @@ feedback_create( --- - + ### method `feedback_create_batch` @@ -236,7 +400,7 @@ feedback_create_batch(req: FeedbackCreateBatchReq) → FeedbackCreateBatchRes --- - + ### method `feedback_purge` @@ -246,7 +410,7 @@ feedback_purge(req: Union[FeedbackPurgeReq, dict[str, Any]]) → FeedbackPurgeRe --- - + ### method `feedback_query` @@ -256,7 +420,7 @@ feedback_query(req: Union[FeedbackQueryReq, dict[str, Any]]) → FeedbackQueryRe --- - + ### method `feedback_replace` @@ -268,7 +432,7 @@ feedback_replace( --- - + ### method `file_content_read` @@ -278,7 +442,7 @@ file_content_read(req: FileContentReadReq) → FileContentReadRes --- - + ### method `file_create` @@ -288,7 +452,7 @@ file_create(req: FileCreateReq) → FileCreateRes --- - + ### method `files_stats` @@ -298,7 +462,7 @@ files_stats(req: FilesStatsReq) → FilesStatsRes --- - + ### classmethod `from_env` @@ -308,7 +472,7 @@ from_env(should_batch: bool = False) → RemoteHTTPTraceServer --- - + ### method `get` @@ -318,7 +482,7 @@ get(url: str, *args: Any, **kwargs: Any) → Response --- - + ### method `get_call_processor` @@ -330,7 +494,7 @@ Custom method not defined on the formal TraceServerInterface to expose the under --- - + ### method `get_feedback_processor` @@ -342,7 +506,7 @@ Custom method not defined on the formal TraceServerInterface to expose the under --- - + ### method `image_create` @@ -352,7 +516,49 @@ image_create(req: ImageGenerationCreateReq) → ImageGenerationCreateRes --- - + + +### method `model_create_v2` + +```python +model_create_v2(req: Union[ModelCreateV2Req, dict[str, Any]]) → ModelCreateV2Res +``` + +--- + + + +### method `model_delete_v2` + +```python +model_delete_v2(req: Union[ModelDeleteV2Req, dict[str, Any]]) → ModelDeleteV2Res +``` + +--- + + + +### method `model_list_v2` + +```python +model_list_v2( + req: Union[ModelListV2Req, dict[str, Any]] +) → Iterator[ModelReadV2Res] +``` + +--- + + + +### method `model_read_v2` + +```python +model_read_v2(req: Union[ModelReadV2Req, dict[str, Any]]) → ModelReadV2Res +``` + +--- + + ### method `obj_create` @@ -362,7 +568,7 @@ obj_create(req: Union[ObjCreateReq, dict[str, Any]]) → ObjCreateRes --- - + ### method `obj_delete` @@ -372,7 +578,7 @@ obj_delete(req: ObjDeleteReq) → ObjDeleteRes --- - + ### method `obj_read` @@ -382,7 +588,7 @@ obj_read(req: Union[ObjReadReq, dict[str, Any]]) → ObjReadRes --- - + ### method `objs_query` @@ -392,7 +598,7 @@ objs_query(req: Union[ObjQueryReq, dict[str, Any]]) → ObjQueryRes --- - + ### method `op_create` @@ -402,7 +608,37 @@ op_create(req: Union[OpCreateReq, dict[str, Any]]) → OpCreateRes --- - + + +### method `op_create_v2` + +```python +op_create_v2(req: Union[OpCreateV2Req, dict[str, Any]]) → OpCreateV2Res +``` + +--- + + + +### method `op_delete_v2` + +```python +op_delete_v2(req: Union[OpDeleteV2Req, dict[str, Any]]) → OpDeleteV2Res +``` + +--- + + + +### method `op_list_v2` + +```python +op_list_v2(req: Union[OpListV2Req, dict[str, Any]]) → Iterator[OpReadV2Res] +``` + +--- + + ### method `op_read` @@ -412,7 +648,17 @@ op_read(req: Union[OpReadReq, dict[str, Any]]) → OpReadRes --- - + + +### method `op_read_v2` + +```python +op_read_v2(req: Union[OpReadV2Req, dict[str, Any]]) → OpReadV2Res +``` + +--- + + ### method `ops_query` @@ -422,7 +668,7 @@ ops_query(req: Union[OpQueryReq, dict[str, Any]]) → OpQueryRes --- - + ### method `otel_export` @@ -432,7 +678,7 @@ otel_export(req: OtelExportReq) → OtelExportRes --- - + ### method `post` @@ -442,7 +688,67 @@ post(url: str, *args: Any, **kwargs: Any) → Response --- - + + +### method `prediction_create_v2` + +```python +prediction_create_v2( + req: Union[PredictionCreateV2Req, dict[str, Any]] +) → PredictionCreateV2Res +``` + +--- + + + +### method `prediction_delete_v2` + +```python +prediction_delete_v2( + req: Union[PredictionDeleteV2Req, dict[str, Any]] +) → PredictionDeleteV2Res +``` + +--- + + + +### method `prediction_finish_v2` + +```python +prediction_finish_v2( + req: Union[PredictionFinishV2Req, dict[str, Any]] +) → PredictionFinishV2Res +``` + +--- + + + +### method `prediction_list_v2` + +```python +prediction_list_v2( + req: Union[PredictionListV2Req, dict[str, Any]] +) → Iterator[PredictionReadV2Res] +``` + +--- + + + +### method `prediction_read_v2` + +```python +prediction_read_v2( + req: Union[PredictionReadV2Req, dict[str, Any]] +) → PredictionReadV2Res +``` + +--- + + ### method `project_stats` @@ -452,7 +758,7 @@ project_stats(req: ProjectStatsReq) → ProjectStatsRes --- - + ### method `refs_read_batch` @@ -462,7 +768,95 @@ refs_read_batch(req: Union[RefsReadBatchReq, dict[str, Any]]) → RefsReadBatchR --- - + + +### method `score_create_v2` + +```python +score_create_v2(req: Union[ScoreCreateV2Req, dict[str, Any]]) → ScoreCreateV2Res +``` + +--- + + + +### method `score_delete_v2` + +```python +score_delete_v2(req: Union[ScoreDeleteV2Req, dict[str, Any]]) → ScoreDeleteV2Res +``` + +--- + + + +### method `score_list_v2` + +```python +score_list_v2( + req: Union[ScoreListV2Req, dict[str, Any]] +) → Iterator[ScoreReadV2Res] +``` + +--- + + + +### method `score_read_v2` + +```python +score_read_v2(req: Union[ScoreReadV2Req, dict[str, Any]]) → ScoreReadV2Res +``` + +--- + + + +### method `scorer_create_v2` + +```python +scorer_create_v2( + req: Union[ScorerCreateV2Req, dict[str, Any]] +) → ScorerCreateV2Res +``` + +--- + + + +### method `scorer_delete_v2` + +```python +scorer_delete_v2( + req: Union[ScorerDeleteV2Req, dict[str, Any]] +) → ScorerDeleteV2Res +``` + +--- + + + +### method `scorer_list_v2` + +```python +scorer_list_v2( + req: Union[ScorerListV2Req, dict[str, Any]] +) → Iterator[ScorerReadV2Res] +``` + +--- + + + +### method `scorer_read_v2` + +```python +scorer_read_v2(req: Union[ScorerReadV2Req, dict[str, Any]]) → ScorerReadV2Res +``` + +--- + + ### method `server_info` @@ -472,7 +866,7 @@ server_info() → ServerInfoRes --- - + ### method `set_auth` @@ -482,7 +876,7 @@ set_auth(auth: tuple[str, str]) → None --- - + ### method `table_create` @@ -492,7 +886,7 @@ table_create(req: Union[TableCreateReq, dict[str, Any]]) → TableCreateRes --- - + ### method `table_create_from_digests` @@ -506,7 +900,7 @@ Create a table by specifying row digests instead of actual rows. --- - + ### method `table_query` @@ -516,7 +910,7 @@ table_query(req: Union[TableQueryReq, dict[str, Any]]) → TableQueryRes --- - + ### method `table_query_stats` @@ -528,7 +922,7 @@ table_query_stats( --- - + ### method `table_query_stats_batch` @@ -540,7 +934,7 @@ table_query_stats_batch( --- - + ### method `table_query_stream` @@ -550,7 +944,7 @@ table_query_stream(req: TableQueryReq) → Iterator[TableRowSchema] --- - + ### method `table_update` @@ -562,7 +956,7 @@ Similar to `calls/batch_upsert`, we can dynamically adjust the payload size due --- - + ### method `threads_query_stream` diff --git a/weave/reference/service-api.mdx b/weave/reference/service-api.mdx index e5da909ac3..4088ab750a 100644 --- a/weave/reference/service-api.mdx +++ b/weave/reference/service-api.mdx @@ -22,6 +22,141 @@ curl -H "Authorization: Bearer YOUR_API_KEY" https://trace.wandb.ai/... ### Calls +- **[POST /call/end](https://docs.wandb.ai/weave/reference/service-api/calls/call-end-call-end)** - Call End +- **[POST /call/read](https://docs.wandb.ai/weave/reference/service-api/calls/call-read-call-read)** - Call Read +- **[POST /call/start](https://docs.wandb.ai/weave/reference/service-api/calls/call-start-call-start)** - Call Start +- **[POST /call/update](https://docs.wandb.ai/weave/reference/service-api/calls/call-update-call-update)** - Call Update +- **[POST /call/upsert_batch](https://docs.wandb.ai/weave/reference/service-api/calls/call-start-batch-call-upsert-batch)** - Call Start Batch +- **[POST /calls/delete](https://docs.wandb.ai/weave/reference/service-api/calls/calls-delete-calls-delete)** - Calls Delete +- **[POST /calls/query_stats](https://docs.wandb.ai/weave/reference/service-api/calls/calls-query-stats-calls-query-stats)** - Calls Query Stats +- **[POST /calls/stream_query](https://docs.wandb.ai/weave/reference/service-api/calls/calls-query-stream-calls-stream-query)** - Calls Query Stream + +### Costs + +- **[POST /cost/create](https://docs.wandb.ai/weave/reference/service-api/costs/cost-create-cost-create)** - Cost Create +- **[POST /cost/purge](https://docs.wandb.ai/weave/reference/service-api/costs/cost-purge-cost-purge)** - Cost Purge +- **[POST /cost/query](https://docs.wandb.ai/weave/reference/service-api/costs/cost-query-cost-query)** - Cost Query + +### Feedback + +- **[POST /feedback/batch/create](https://docs.wandb.ai/weave/reference/service-api/feedback/feedback-create-batch-feedback-batch-create)** - Feedback Create Batch +- **[POST /feedback/create](https://docs.wandb.ai/weave/reference/service-api/feedback/feedback-create-feedback-create)** - Feedback Create +- **[POST /feedback/purge](https://docs.wandb.ai/weave/reference/service-api/feedback/feedback-purge-feedback-purge)** - Feedback Purge +- **[POST /feedback/query](https://docs.wandb.ai/weave/reference/service-api/feedback/feedback-query-feedback-query)** - Feedback Query +- **[POST /feedback/replace](https://docs.wandb.ai/weave/reference/service-api/feedback/feedback-replace-feedback-replace)** - Feedback Replace + +### Files + +- **[POST /file/content](https://docs.wandb.ai/weave/reference/service-api/files/file-content-file-content)** - File Content +- **[POST /file/create](https://docs.wandb.ai/weave/reference/service-api/files/file-create-file-create)** - File Create +- **[POST /files/query_stats](https://docs.wandb.ai/weave/reference/service-api/files/files-stats-files-query-stats)** - Files Stats + +### Objects + +- **[POST /obj/create](https://docs.wandb.ai/weave/reference/service-api/objects/obj-create-obj-create)** - Obj Create +- **[POST /obj/delete](https://docs.wandb.ai/weave/reference/service-api/objects/obj-delete-obj-delete)** - Obj Delete +- **[POST /obj/read](https://docs.wandb.ai/weave/reference/service-api/objects/obj-read-obj-read)** - Obj Read +- **[POST /objs/query](https://docs.wandb.ai/weave/reference/service-api/objects/objs-query-objs-query)** - Objs Query + +### OpenTelemetry + +- **[POST /otel/v1/traces](https://docs.wandb.ai/weave/reference/service-api/opentelemetry/export-trace-otel-v1-traces)** - Export Trace + +### Refs + +- **[POST /refs/read_batch](https://docs.wandb.ai/weave/reference/service-api/refs/refs-read-batch-refs-read-batch)** - Refs Read Batch + +### Service + +- **[GET /geolocate](https://docs.wandb.ai/weave/reference/service-api/service/get-caller-location-geolocate)** - Get Caller Location +- **[GET /health](https://docs.wandb.ai/weave/reference/service-api/service/read-root-health)** - Read Root +- **[GET /server_info](https://docs.wandb.ai/weave/reference/service-api/service/server-info-server-info)** - Server Info +- **[GET /version](https://docs.wandb.ai/weave/reference/service-api/service/read-version-version)** - Read Version + +### Tables + +- **[POST /table/create](https://docs.wandb.ai/weave/reference/service-api/tables/table-create-table-create)** - Table Create +- **[POST /table/create_from_digests](https://docs.wandb.ai/weave/reference/service-api/tables/table-create-from-digests-table-create-from-digests)** - Table Create From Digests +- **[POST /table/query](https://docs.wandb.ai/weave/reference/service-api/tables/table-query-table-query)** - Table Query +- **[POST /table/query_stats](https://docs.wandb.ai/weave/reference/service-api/tables/table-query-stats-table-query-stats)** - Table Query Stats +- **[POST /table/query_stats_batch](https://docs.wandb.ai/weave/reference/service-api/tables/table-query-stats-batch-table-query-stats-batch)** - Table Query Stats Batch +- **[POST /table/update](https://docs.wandb.ai/weave/reference/service-api/tables/table-update-table-update)** - Table Update + +### Threads + +- **[POST /threads/stream_query](https://docs.wandb.ai/weave/reference/service-api/threads/threads-query-stream-threads-stream-query)** - Threads Query Stream + +### Inference + +- **[GET /inference/router/openrouter/models](https://docs.wandb.ai/weave/reference/service-api/inference/inference-router-openrouter-models-inference-router-openrouter-models)** - Inference Router Openrouter Models +- **[GET /inference/v1{path}](https://docs.wandb.ai/weave/reference/service-api/inference/inference-get-inference-v1-path-)** - Inference Get +- **[POST /inference/v1{path}](https://docs.wandb.ai/weave/reference/service-api/inference/inference-post-inference-v1-path-)** - Inference Post + +### Evaluations + +- **[POST /evaluations/evaluate_model](https://docs.wandb.ai/weave/reference/service-api/evaluations/evaluate-model-evaluations-evaluate-model)** - Evaluate Model +- **[POST /evaluations/status](https://docs.wandb.ai/weave/reference/service-api/evaluations/evaluation-status-evaluations-status)** - Evaluation Status +- **[GET /v2/{entity}/{project}/evaluations](https://docs.wandb.ai/weave/reference/service-api/evaluations/evaluation-list-v2--entity---project--evaluations)** - Evaluation List +- **[POST /v2/{entity}/{project}/evaluations](https://docs.wandb.ai/weave/reference/service-api/evaluations/evaluation-create-v2--entity---project--evaluations)** - Evaluation Create +- **[DELETE /v2/{entity}/{project}/evaluations/{object_id}](https://docs.wandb.ai/weave/reference/service-api/evaluations/evaluation-delete-v2--entity---project--evaluations--object-id-)** - Evaluation Delete +- **[GET /v2/{entity}/{project}/evaluations/{object_id}/versions/{digest}](https://docs.wandb.ai/weave/reference/service-api/evaluations/evaluation-read-v2--entity---project--evaluations--object-id--versions--digest-)** - Evaluation Read + +### Images + +- **[POST /image/create](https://docs.wandb.ai/weave/reference/service-api/images/image-create-image-create)** - Image Create + +### Ops + +- **[GET /v2/{entity}/{project}/ops](https://docs.wandb.ai/weave/reference/service-api/ops/op-list-v2--entity---project--ops)** - Op List +- **[POST /v2/{entity}/{project}/ops](https://docs.wandb.ai/weave/reference/service-api/ops/op-create-v2--entity---project--ops)** - Op Create +- **[DELETE /v2/{entity}/{project}/ops/{object_id}](https://docs.wandb.ai/weave/reference/service-api/ops/op-delete-v2--entity---project--ops--object-id-)** - Op Delete +- **[GET /v2/{entity}/{project}/ops/{object_id}/versions/{digest}](https://docs.wandb.ai/weave/reference/service-api/ops/op-read-v2--entity---project--ops--object-id--versions--digest-)** - Op Read + +### Datasets + +- **[GET /v2/{entity}/{project}/datasets](https://docs.wandb.ai/weave/reference/service-api/datasets/dataset-list-v2--entity---project--datasets)** - Dataset List +- **[POST /v2/{entity}/{project}/datasets](https://docs.wandb.ai/weave/reference/service-api/datasets/dataset-create-v2--entity---project--datasets)** - Dataset Create +- **[DELETE /v2/{entity}/{project}/datasets/{object_id}](https://docs.wandb.ai/weave/reference/service-api/datasets/dataset-delete-v2--entity---project--datasets--object-id-)** - Dataset Delete +- **[GET /v2/{entity}/{project}/datasets/{object_id}/versions/{digest}](https://docs.wandb.ai/weave/reference/service-api/datasets/dataset-read-v2--entity---project--datasets--object-id--versions--digest-)** - Dataset Read + +### Scorers + +- **[GET /v2/{entity}/{project}/scorers](https://docs.wandb.ai/weave/reference/service-api/scorers/scorer-list-v2--entity---project--scorers)** - Scorer List +- **[POST /v2/{entity}/{project}/scorers](https://docs.wandb.ai/weave/reference/service-api/scorers/scorer-create-v2--entity---project--scorers)** - Scorer Create +- **[DELETE /v2/{entity}/{project}/scorers/{object_id}](https://docs.wandb.ai/weave/reference/service-api/scorers/scorer-delete-v2--entity---project--scorers--object-id-)** - Scorer Delete +- **[GET /v2/{entity}/{project}/scorers/{object_id}/versions/{digest}](https://docs.wandb.ai/weave/reference/service-api/scorers/scorer-read-v2--entity---project--scorers--object-id--versions--digest-)** - Scorer Read + +### Models + +- **[GET /v2/{entity}/{project}/models](https://docs.wandb.ai/weave/reference/service-api/models/model-list-v2--entity---project--models)** - Model List +- **[POST /v2/{entity}/{project}/models](https://docs.wandb.ai/weave/reference/service-api/models/model-create-v2--entity---project--models)** - Model Create +- **[DELETE /v2/{entity}/{project}/models/{object_id}](https://docs.wandb.ai/weave/reference/service-api/models/model-delete-v2--entity---project--models--object-id-)** - Model Delete +- **[GET /v2/{entity}/{project}/models/{object_id}/versions/{digest}](https://docs.wandb.ai/weave/reference/service-api/models/model-read-v2--entity---project--models--object-id--versions--digest-)** - Model Read + +### Evaluation Runs + +- **[DELETE /v2/{entity}/{project}/evaluation_runs](https://docs.wandb.ai/weave/reference/service-api/evaluation runs/evaluation-run-delete-v2--entity---project--evaluation-runs)** - Evaluation Run Delete +- **[GET /v2/{entity}/{project}/evaluation_runs](https://docs.wandb.ai/weave/reference/service-api/evaluation runs/evaluation-run-list-v2--entity---project--evaluation-runs)** - Evaluation Run List +- **[POST /v2/{entity}/{project}/evaluation_runs](https://docs.wandb.ai/weave/reference/service-api/evaluation runs/evaluation-run-create-v2--entity---project--evaluation-runs)** - Evaluation Run Create +- **[GET /v2/{entity}/{project}/evaluation_runs/{evaluation_run_id}](https://docs.wandb.ai/weave/reference/service-api/evaluation runs/evaluation-run-read-v2--entity---project--evaluation-runs--evaluation-run-id-)** - Evaluation Run Read +- **[POST /v2/{entity}/{project}/evaluation_runs/{evaluation_run_id}/finish](https://docs.wandb.ai/weave/reference/service-api/evaluation runs/evaluation-run-finish-v2--entity---project--evaluation-runs--evaluation-run-id--finish)** - Evaluation Run Finish + +### Predictions + +- **[DELETE /v2/{entity}/{project}/predictions](https://docs.wandb.ai/weave/reference/service-api/predictions/prediction-delete-v2--entity---project--predictions)** - Prediction Delete +- **[GET /v2/{entity}/{project}/predictions](https://docs.wandb.ai/weave/reference/service-api/predictions/prediction-list-v2--entity---project--predictions)** - Prediction List +- **[POST /v2/{entity}/{project}/predictions](https://docs.wandb.ai/weave/reference/service-api/predictions/prediction-create-v2--entity---project--predictions)** - Prediction Create +- **[GET /v2/{entity}/{project}/predictions/{prediction_id}](https://docs.wandb.ai/weave/reference/service-api/predictions/prediction-read-v2--entity---project--predictions--prediction-id-)** - Prediction Read +- **[POST /v2/{entity}/{project}/predictions/{prediction_id}/finish](https://docs.wandb.ai/weave/reference/service-api/predictions/prediction-finish-v2--entity---project--predictions--prediction-id--finish)** - Prediction Finish + +### Scores + +- **[DELETE /v2/{entity}/{project}/scores](https://docs.wandb.ai/weave/reference/service-api/scores/score-delete-v2--entity---project--scores)** - Score Delete +- **[GET /v2/{entity}/{project}/scores](https://docs.wandb.ai/weave/reference/service-api/scores/score-list-v2--entity---project--scores)** - Score List +- **[POST /v2/{entity}/{project}/scores](https://docs.wandb.ai/weave/reference/service-api/scores/score-create-v2--entity---project--scores)** - Score Create +- **[GET /v2/{entity}/{project}/scores/{score_id}](https://docs.wandb.ai/weave/reference/service-api/scores/score-read-v2--entity---project--scores--score-id-)** - Score Read +### Calls + - **[POST /call/start](https://docs.wandb.ai/weave/reference/service-api/calls/call-start)** - Call Start - **[POST /call/end](https://docs.wandb.ai/weave/reference/service-api/calls/call-end)** - Call End - **[POST /call/upsert_batch](https://docs.wandb.ai/weave/reference/service-api/calls/call-start-batch)** - Call Start Batch diff --git a/weave/reference/service-api/openapi.json b/weave/reference/service-api/openapi.json index 6929dd4cae..341c56a1a4 100644 --- a/weave/reference/service-api/openapi.json +++ b/weave/reference/service-api/openapi.json @@ -952,7 +952,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/TableCreateRes" + "$ref": "#/components/schemas/TableCreateFromDigestsRes" } } } @@ -1848,180 +1848,2839 @@ } ] } - } - }, - "components": { - "schemas": { - "AndOperation": { - "properties": { - "$and": { - "items": { - "anyOf": [ - { - "$ref": "#/components/schemas/LiteralOperation" - }, - { - "$ref": "#/components/schemas/GetFieldOperator" - }, - { - "$ref": "#/components/schemas/ConvertOperation" - }, - { - "$ref": "#/components/schemas/AndOperation" - }, - { - "$ref": "#/components/schemas/OrOperation" - }, - { - "$ref": "#/components/schemas/NotOperation" - }, - { - "$ref": "#/components/schemas/EqOperation" - }, - { - "$ref": "#/components/schemas/GtOperation" - }, - { - "$ref": "#/components/schemas/GteOperation" - }, - { - "$ref": "#/components/schemas/InOperation" - }, - { - "$ref": "#/components/schemas/ContainsOperation" - } - ] - }, - "type": "array", - "title": "$And" + }, + "/v2/{entity}/{project}/ops": { + "post": { + "tags": [ + "Ops" + ], + "summary": "Op Create", + "description": "Create an op object.", + "operationId": "op_create_v2__entity___project__ops_post", + "security": [ + { + "HTTPBasic": [] } - }, - "type": "object", - "required": [ - "$and" ], - "title": "AndOperation", - "description": "Logical AND. All conditions must evaluate to true.\n\nExample:\n ```\n {\n \"$and\": [\n {\"$eq\": [{\"$getField\": \"op_name\"}, {\"$literal\": \"predict\"}]},\n {\"$gt\": [{\"$getField\": \"summary.usage.tokens\"}, {\"$literal\": 1000}]}\n ]\n }\n ```" - }, - "Body_file_create_file_create_post": { - "properties": { - "project_id": { - "type": "string", - "title": "Project Id" + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } }, - "file": { - "type": "string", - "format": "binary", - "title": "File" + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } } - }, - "type": "object", - "required": [ - "project_id", - "file" ], - "title": "Body_file_create_file_create_post" - }, - "CallBatchEndMode": { - "properties": { - "mode": { - "type": "string", - "title": "Mode", - "default": "end" - }, - "req": { - "$ref": "#/components/schemas/CallEndReq" + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/OpCreateBody" + } + } } }, - "type": "object", - "required": [ - "req" - ], - "title": "CallBatchEndMode" - }, - "CallBatchStartMode": { - "properties": { - "mode": { - "type": "string", - "title": "Mode", - "default": "start" + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/OpCreateRes" + } + } + } }, - "req": { - "$ref": "#/components/schemas/CallStartReq" + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } } - }, - "type": "object", - "required": [ - "req" - ], - "title": "CallBatchStartMode" + } }, - "CallCreateBatchReq": { - "properties": { - "batch": { - "items": { + "get": { + "tags": [ + "Ops" + ], + "summary": "Op List", + "description": "List op objects.", + "operationId": "op_list_v2__entity___project__ops_get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "limit", + "in": "query", + "required": false, + "schema": { "anyOf": [ { - "$ref": "#/components/schemas/CallBatchStartMode" + "type": "integer" }, { - "$ref": "#/components/schemas/CallBatchEndMode" + "type": "null" } - ] + ], + "description": "Maximum number of ops to return", + "title": "Limit" }, - "type": "array", - "title": "Batch" - } - }, - "type": "object", - "required": [ - "batch" - ], - "title": "CallCreateBatchReq" - }, - "CallCreateBatchRes": { - "properties": { - "res": { - "items": { + "description": "Maximum number of ops to return" + }, + { + "name": "offset", + "in": "query", + "required": false, + "schema": { "anyOf": [ { - "$ref": "#/components/schemas/CallStartRes" + "type": "integer" }, { - "$ref": "#/components/schemas/CallEndRes" + "type": "null" } - ] + ], + "description": "Number of ops to skip", + "title": "Offset" }, - "type": "array", - "title": "Res" + "description": "Number of ops to skip" } - }, - "type": "object", - "required": [ - "res" ], - "title": "CallCreateBatchRes" - }, - "CallEndReq": { - "properties": { - "end": { - "$ref": "#/components/schemas/EndedCallSchemaForInsert" + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": {} + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } } - }, - "additionalProperties": false, - "type": "object", - "required": [ - "end" + } + } + }, + "/v2/{entity}/{project}/ops/{object_id}/versions/{digest}": { + "get": { + "tags": [ + "Ops" ], - "title": "CallEndReq" - }, - "CallEndRes": { - "properties": {}, - "type": "object", - "title": "CallEndRes" - }, - "CallReadReq": { - "properties": { + "summary": "Op Read", + "description": "Get an op object.", + "operationId": "op_read_v2__entity___project__ops__object_id__versions__digest__get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "object_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Object Id" + } + }, + { + "name": "digest", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Digest" + } + }, + { + "name": "eager", + "in": "query", + "required": false, + "schema": { + "type": "boolean", + "description": "Whether to eagerly load the op code", + "default": false, + "title": "Eager" + }, + "description": "Whether to eagerly load the op code" + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/OpReadRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/ops/{object_id}": { + "delete": { + "tags": [ + "Ops" + ], + "summary": "Op Delete", + "description": "Delete an op object. If digests are provided, only those versions are deleted. Otherwise, all versions are deleted.", + "operationId": "op_delete_v2__entity___project__ops__object_id__delete", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "object_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Object Id" + } + }, + { + "name": "digests", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "array", + "items": { + "type": "string" + } + }, + { + "type": "null" + } + ], + "description": "List of digests to delete. If not provided, all digests for the op will be deleted.", + "title": "Digests" + }, + "description": "List of digests to delete. If not provided, all digests for the op will be deleted." + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/OpDeleteRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/datasets": { + "post": { + "tags": [ + "Datasets" + ], + "summary": "Dataset Create", + "description": "Create a dataset object.", + "operationId": "dataset_create_v2__entity___project__datasets_post", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + } + ], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/DatasetCreateBody" + } + } + } + }, + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/DatasetCreateRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + }, + "get": { + "tags": [ + "Datasets" + ], + "summary": "Dataset List", + "description": "List dataset objects.", + "operationId": "dataset_list_v2__entity___project__datasets_get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "limit", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "Maximum number of datasets to return", + "title": "Limit" + }, + "description": "Maximum number of datasets to return" + }, + { + "name": "offset", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "Number of datasets to skip", + "title": "Offset" + }, + "description": "Number of datasets to skip" + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": {} + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/datasets/{object_id}/versions/{digest}": { + "get": { + "tags": [ + "Datasets" + ], + "summary": "Dataset Read", + "description": "Get a dataset object.", + "operationId": "dataset_read_v2__entity___project__datasets__object_id__versions__digest__get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "object_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Object Id" + } + }, + { + "name": "digest", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Digest" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/DatasetReadRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/datasets/{object_id}": { + "delete": { + "tags": [ + "Datasets" + ], + "summary": "Dataset Delete", + "description": "Delete a dataset object. If digests are provided, only those versions are deleted. Otherwise, all versions are deleted.", + "operationId": "dataset_delete_v2__entity___project__datasets__object_id__delete", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "object_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Object Id" + } + }, + { + "name": "digests", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "array", + "items": { + "type": "string" + } + }, + { + "type": "null" + } + ], + "description": "List of digests to delete. If not provided, all digests for the dataset will be deleted.", + "title": "Digests" + }, + "description": "List of digests to delete. If not provided, all digests for the dataset will be deleted." + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/DatasetDeleteRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/scorers": { + "post": { + "tags": [ + "Scorers" + ], + "summary": "Scorer Create", + "description": "Create a scorer object.", + "operationId": "scorer_create_v2__entity___project__scorers_post", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + } + ], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ScorerCreateBody" + } + } + } + }, + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ScorerCreateRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + }, + "get": { + "tags": [ + "Scorers" + ], + "summary": "Scorer List", + "description": "List scorer objects.", + "operationId": "scorer_list_v2__entity___project__scorers_get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": {} + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/scorers/{object_id}/versions/{digest}": { + "get": { + "tags": [ + "Scorers" + ], + "summary": "Scorer Read", + "description": "Get a scorer object.", + "operationId": "scorer_read_v2__entity___project__scorers__object_id__versions__digest__get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "object_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Object Id" + } + }, + { + "name": "digest", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Digest" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ScorerReadRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/scorers/{object_id}": { + "delete": { + "tags": [ + "Scorers" + ], + "summary": "Scorer Delete", + "description": "Delete a scorer object.", + "operationId": "scorer_delete_v2__entity___project__scorers__object_id__delete", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "object_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Object Id" + } + }, + { + "name": "digests", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "array", + "items": { + "type": "string" + } + }, + { + "type": "null" + } + ], + "description": "List of digests to delete. If not provided, all digests for the scorer will be deleted.", + "title": "Digests" + }, + "description": "List of digests to delete. If not provided, all digests for the scorer will be deleted." + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ScorerDeleteRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/evaluations": { + "post": { + "tags": [ + "Evaluations" + ], + "summary": "Evaluation Create", + "description": "Create an evaluation object.", + "operationId": "evaluation_create_v2__entity___project__evaluations_post", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + } + ], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/EvaluationCreateBody" + } + } + } + }, + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/EvaluationCreateRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + }, + "get": { + "tags": [ + "Evaluations" + ], + "summary": "Evaluation List", + "description": "List evaluation objects.", + "operationId": "evaluation_list_v2__entity___project__evaluations_get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": {} + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/evaluations/{object_id}/versions/{digest}": { + "get": { + "tags": [ + "Evaluations" + ], + "summary": "Evaluation Read", + "description": "Get an evaluation object.", + "operationId": "evaluation_read_v2__entity___project__evaluations__object_id__versions__digest__get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "object_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Object Id" + } + }, + { + "name": "digest", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Digest" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/EvaluationReadRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/evaluations/{object_id}": { + "delete": { + "tags": [ + "Evaluations" + ], + "summary": "Evaluation Delete", + "description": "Delete an evaluation object.", + "operationId": "evaluation_delete_v2__entity___project__evaluations__object_id__delete", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "object_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Object Id" + } + }, + { + "name": "digests", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "array", + "items": { + "type": "string" + } + }, + { + "type": "null" + } + ], + "description": "List of digests to delete. If not provided, all digests for the evaluation will be deleted.", + "title": "Digests" + }, + "description": "List of digests to delete. If not provided, all digests for the evaluation will be deleted." + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/EvaluationDeleteRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/models": { + "post": { + "tags": [ + "Models" + ], + "summary": "Model Create", + "description": "Create a model object.", + "operationId": "model_create_v2__entity___project__models_post", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + } + ], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ModelCreateBody" + } + } + } + }, + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ModelCreateRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + }, + "get": { + "tags": [ + "Models" + ], + "summary": "Model List", + "description": "List model objects.", + "operationId": "model_list_v2__entity___project__models_get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "limit", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "Maximum number of models to return", + "title": "Limit" + }, + "description": "Maximum number of models to return" + }, + { + "name": "offset", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "Number of models to skip", + "title": "Offset" + }, + "description": "Number of models to skip" + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": {} + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/models/{object_id}/versions/{digest}": { + "get": { + "tags": [ + "Models" + ], + "summary": "Model Read", + "description": "Get a model object.", + "operationId": "model_read_v2__entity___project__models__object_id__versions__digest__get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "object_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Object Id" + } + }, + { + "name": "digest", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Digest" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ModelReadRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/models/{object_id}": { + "delete": { + "tags": [ + "Models" + ], + "summary": "Model Delete", + "description": "Delete a model object. If digests are provided, only those versions are deleted. Otherwise, all versions are deleted.", + "operationId": "model_delete_v2__entity___project__models__object_id__delete", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "object_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Object Id" + } + }, + { + "name": "digests", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "array", + "items": { + "type": "string" + } + }, + { + "type": "null" + } + ], + "description": "List of digests to delete. If not provided, all digests for the model will be deleted.", + "title": "Digests" + }, + "description": "List of digests to delete. If not provided, all digests for the model will be deleted." + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ModelDeleteRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/evaluation_runs": { + "post": { + "tags": [ + "Evaluation Runs" + ], + "summary": "Evaluation Run Create", + "description": "Create an evaluation run.", + "operationId": "evaluation_run_create_v2__entity___project__evaluation_runs_post", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + } + ], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/EvaluationRunCreateBody" + } + } + } + }, + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/EvaluationRunCreateRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + }, + "get": { + "tags": [ + "Evaluation Runs" + ], + "summary": "Evaluation Run List", + "description": "List evaluation runs.", + "operationId": "evaluation_run_list_v2__entity___project__evaluation_runs_get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "evaluations", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "array", + "items": { + "type": "string" + } + }, + { + "type": "null" + } + ], + "description": "Filter by evaluation references", + "title": "Evaluations" + }, + "description": "Filter by evaluation references" + }, + { + "name": "models", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "array", + "items": { + "type": "string" + } + }, + { + "type": "null" + } + ], + "description": "Filter by model references", + "title": "Models" + }, + "description": "Filter by model references" + }, + { + "name": "evaluation_run_ids", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "array", + "items": { + "type": "string" + } + }, + { + "type": "null" + } + ], + "description": "Filter by evaluation run IDs", + "title": "Evaluation Run Ids" + }, + "description": "Filter by evaluation run IDs" + }, + { + "name": "limit", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "Maximum number of evaluation runs to return", + "title": "Limit" + }, + "description": "Maximum number of evaluation runs to return" + }, + { + "name": "offset", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "Number of evaluation runs to skip", + "title": "Offset" + }, + "description": "Number of evaluation runs to skip" + } + ], + "responses": { + "200": { + "description": "Stream of data in JSONL format", + "content": { + "application/jsonl": { + "schema": { + "type": "array", + "items": { + "$ref": "#/components/schemas/Schema" + } + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + }, + "delete": { + "tags": [ + "Evaluation Runs" + ], + "summary": "Evaluation Run Delete", + "description": "Delete evaluation runs.", + "operationId": "evaluation_run_delete_v2__entity___project__evaluation_runs_delete", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "evaluation_run_ids", + "in": "query", + "required": true, + "schema": { + "type": "array", + "items": { + "type": "string" + }, + "description": "List of evaluation run IDs to delete", + "title": "Evaluation Run Ids" + }, + "description": "List of evaluation run IDs to delete" + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/EvaluationRunDeleteRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/evaluation_runs/{evaluation_run_id}": { + "get": { + "tags": [ + "Evaluation Runs" + ], + "summary": "Evaluation Run Read", + "description": "Read an evaluation run.", + "operationId": "evaluation_run_read_v2__entity___project__evaluation_runs__evaluation_run_id__get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "evaluation_run_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Evaluation Run Id" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/EvaluationRunReadRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/evaluation_runs/{evaluation_run_id}/finish": { + "post": { + "tags": [ + "Evaluation Runs" + ], + "summary": "Evaluation Run Finish", + "description": "Finish an evaluation run.", + "operationId": "evaluation_run_finish_v2__entity___project__evaluation_runs__evaluation_run_id__finish_post", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "evaluation_run_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Evaluation Run Id" + } + } + ], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/EvaluationRunFinishBody" + } + } + } + }, + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/EvaluationRunFinishRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/predictions": { + "post": { + "tags": [ + "Predictions" + ], + "summary": "Prediction Create", + "description": "Create a prediction.", + "operationId": "prediction_create_v2__entity___project__predictions_post", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + } + ], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/PredictionCreateBody" + } + } + } + }, + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/PredictionCreateRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + }, + "get": { + "tags": [ + "Predictions" + ], + "summary": "Prediction List", + "description": "List predictions.", + "operationId": "prediction_list_v2__entity___project__predictions_get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "evaluation_run_id", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "Filter by evaluation run ID", + "title": "Evaluation Run Id" + }, + "description": "Filter by evaluation run ID" + }, + { + "name": "limit", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "Maximum number of predictions to return", + "title": "Limit" + }, + "description": "Maximum number of predictions to return" + }, + { + "name": "offset", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "Number of predictions to skip", + "title": "Offset" + }, + "description": "Number of predictions to skip" + } + ], + "responses": { + "200": { + "description": "Stream of data in JSONL format", + "content": { + "application/jsonl": { + "schema": { + "type": "array", + "items": { + "$ref": "#/components/schemas/Schema" + } + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + }, + "delete": { + "tags": [ + "Predictions" + ], + "summary": "Prediction Delete", + "description": "Delete predictions.", + "operationId": "prediction_delete_v2__entity___project__predictions_delete", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "prediction_ids", + "in": "query", + "required": true, + "schema": { + "type": "array", + "items": { + "type": "string" + }, + "description": "List of prediction IDs to delete", + "title": "Prediction Ids" + }, + "description": "List of prediction IDs to delete" + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/PredictionDeleteRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/predictions/{prediction_id}": { + "get": { + "tags": [ + "Predictions" + ], + "summary": "Prediction Read", + "description": "Read a prediction.", + "operationId": "prediction_read_v2__entity___project__predictions__prediction_id__get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "prediction_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Prediction Id" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/PredictionReadRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/predictions/{prediction_id}/finish": { + "post": { + "tags": [ + "Predictions" + ], + "summary": "Prediction Finish", + "description": "Finish a prediction.", + "operationId": "prediction_finish_v2__entity___project__predictions__prediction_id__finish_post", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "prediction_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Prediction Id" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/PredictionFinishRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/scores": { + "post": { + "tags": [ + "Scores" + ], + "summary": "Score Create", + "description": "Create a score.", + "operationId": "score_create_v2__entity___project__scores_post", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + } + ], + "requestBody": { + "required": true, + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ScoreCreateBody" + } + } + } + }, + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ScoreCreateRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + }, + "get": { + "tags": [ + "Scores" + ], + "summary": "Score List", + "description": "List scores.", + "operationId": "score_list_v2__entity___project__scores_get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "evaluation_run_id", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "description": "Filter by evaluation run ID", + "title": "Evaluation Run Id" + }, + "description": "Filter by evaluation run ID" + }, + { + "name": "limit", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "Maximum number of scores to return", + "title": "Limit" + }, + "description": "Maximum number of scores to return" + }, + { + "name": "offset", + "in": "query", + "required": false, + "schema": { + "anyOf": [ + { + "type": "integer" + }, + { + "type": "null" + } + ], + "description": "Number of scores to skip", + "title": "Offset" + }, + "description": "Number of scores to skip" + } + ], + "responses": { + "200": { + "description": "Stream of data in JSONL format", + "content": { + "application/jsonl": { + "schema": { + "type": "array", + "items": { + "$ref": "#/components/schemas/Schema" + } + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + }, + "delete": { + "tags": [ + "Scores" + ], + "summary": "Score Delete", + "description": "Delete scores.", + "operationId": "score_delete_v2__entity___project__scores_delete", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "score_ids", + "in": "query", + "required": true, + "schema": { + "type": "array", + "items": { + "type": "string" + }, + "description": "List of score IDs to delete", + "title": "Score Ids" + }, + "description": "List of score IDs to delete" + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ScoreDeleteRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + }, + "/v2/{entity}/{project}/scores/{score_id}": { + "get": { + "tags": [ + "Scores" + ], + "summary": "Score Read", + "description": "Read a score.", + "operationId": "score_read_v2__entity___project__scores__score_id__get", + "security": [ + { + "HTTPBasic": [] + } + ], + "parameters": [ + { + "name": "entity", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Entity" + } + }, + { + "name": "project", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Project" + } + }, + { + "name": "score_id", + "in": "path", + "required": true, + "schema": { + "type": "string", + "title": "Score Id" + } + } + ], + "responses": { + "200": { + "description": "Successful Response", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/ScoreReadRes" + } + } + } + }, + "422": { + "description": "Validation Error", + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/HTTPValidationError" + } + } + } + } + } + } + } + }, + "components": { + "schemas": { + "AndOperation": { + "properties": { + "$and": { + "items": { + "anyOf": [ + { + "$ref": "#/components/schemas/LiteralOperation" + }, + { + "$ref": "#/components/schemas/GetFieldOperator" + }, + { + "$ref": "#/components/schemas/ConvertOperation" + }, + { + "$ref": "#/components/schemas/AndOperation" + }, + { + "$ref": "#/components/schemas/OrOperation" + }, + { + "$ref": "#/components/schemas/NotOperation" + }, + { + "$ref": "#/components/schemas/EqOperation" + }, + { + "$ref": "#/components/schemas/GtOperation" + }, + { + "$ref": "#/components/schemas/GteOperation" + }, + { + "$ref": "#/components/schemas/InOperation" + }, + { + "$ref": "#/components/schemas/ContainsOperation" + } + ] + }, + "type": "array", + "title": "$And" + } + }, + "type": "object", + "required": [ + "$and" + ], + "title": "AndOperation", + "description": "Logical AND. All conditions must evaluate to true.\n\nExample:\n ```\n {\n \"$and\": [\n {\"$eq\": [{\"$getField\": \"op_name\"}, {\"$literal\": \"predict\"}]},\n {\"$gt\": [{\"$getField\": \"summary.usage.tokens\"}, {\"$literal\": 1000}]}\n ]\n }\n ```" + }, + "Body_file_create_file_create_post": { + "properties": { + "project_id": { + "type": "string", + "title": "Project Id" + }, + "file": { + "type": "string", + "format": "binary", + "title": "File" + } + }, + "type": "object", + "required": [ + "project_id", + "file" + ], + "title": "Body_file_create_file_create_post" + }, + "CallBatchEndMode": { + "properties": { + "mode": { + "type": "string", + "title": "Mode", + "default": "end" + }, + "req": { + "$ref": "#/components/schemas/CallEndReq" + } + }, + "type": "object", + "required": [ + "req" + ], + "title": "CallBatchEndMode" + }, + "CallBatchStartMode": { + "properties": { + "mode": { + "type": "string", + "title": "Mode", + "default": "start" + }, + "req": { + "$ref": "#/components/schemas/CallStartReq" + } + }, + "type": "object", + "required": [ + "req" + ], + "title": "CallBatchStartMode" + }, + "CallCreateBatchReq": { + "properties": { + "batch": { + "items": { + "anyOf": [ + { + "$ref": "#/components/schemas/CallBatchStartMode" + }, + { + "$ref": "#/components/schemas/CallBatchEndMode" + } + ] + }, + "type": "array", + "title": "Batch" + } + }, + "type": "object", + "required": [ + "batch" + ], + "title": "CallCreateBatchReq" + }, + "CallCreateBatchRes": { + "properties": { + "res": { + "items": { + "anyOf": [ + { + "$ref": "#/components/schemas/CallStartRes" + }, + { + "$ref": "#/components/schemas/CallEndRes" + } + ] + }, + "type": "array", + "title": "Res" + } + }, + "type": "object", + "required": [ + "res" + ], + "title": "CallCreateBatchRes" + }, + "CallEndReq": { + "properties": { + "end": { + "$ref": "#/components/schemas/EndedCallSchemaForInsert" + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "end" + ], + "title": "CallEndReq" + }, + "CallEndRes": { + "properties": {}, + "type": "object", + "title": "CallEndRes" + }, + "CallReadReq": { + "properties": { "project_id": { "type": "string", "title": "Project Id" @@ -2162,10 +4821,12 @@ "title": "Started At" }, "attributes": { + "additionalProperties": true, "type": "object", "title": "Attributes" }, "inputs": { + "additionalProperties": true, "type": "object", "title": "Inputs" }, @@ -2202,6 +4863,7 @@ "title": "Output" }, "summary": { + "additionalProperties": true, "type": "object" }, "wb_user_id": { @@ -2408,8 +5070,17 @@ "title": "CallsDeleteReq" }, "CallsDeleteRes": { - "properties": {}, + "properties": { + "num_deleted": { + "type": "integer", + "title": "Num Deleted", + "description": "The number of calls deleted" + } + }, "type": "object", + "required": [ + "num_deleted" + ], "title": "CallsDeleteRes" }, "CallsFilter": { @@ -2986,6 +5657,7 @@ "logit_bias": { "anyOf": [ { + "additionalProperties": true, "type": "object" }, { @@ -3008,6 +5680,7 @@ "response_format": { "anyOf": [ { + "additionalProperties": true, "type": "object" }, {}, @@ -3046,6 +5719,7 @@ "type": "string" }, { + "additionalProperties": true, "type": "object" }, { @@ -3090,6 +5764,7 @@ "extra_headers": { "anyOf": [ { + "additionalProperties": true, "type": "object" }, { @@ -3131,6 +5806,31 @@ } ], "title": "Api Version" + }, + "prompt": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Prompt", + "description": "Reference to a Weave Prompt object (e.g., 'weave:///entity/project/object/prompt_name:version'). If provided, the messages from this prompt will be prepended to the messages in this request. Template variables in the prompt messages can be substituted using the template_vars parameter." + }, + "template_vars": { + "anyOf": [ + { + "additionalProperties": true, + "type": "object" + }, + { + "type": "null" + } + ], + "title": "Template Vars", + "description": "Dictionary of template variables to substitute in prompt messages. Variables in messages like '{variable_name}' will be replaced with the corresponding values. Applied to both prompt messages (if prompt is provided) and regular messages." } }, "type": "object", @@ -3690,41 +6390,180 @@ ] } }, - "additionalProperties": false, + "additionalProperties": false, + "type": "object", + "required": [ + "project_id" + ], + "title": "CostQueryReq" + }, + "CostQueryRes": { + "properties": { + "results": { + "items": { + "$ref": "#/components/schemas/CostQueryOutput" + }, + "type": "array", + "title": "Results" + } + }, + "type": "object", + "required": [ + "results" + ], + "title": "CostQueryRes" + }, + "Datacenter": { + "properties": { + "country_code": { + "type": "string", + "title": "Country Code" + } + }, + "type": "object", + "required": [ + "country_code" + ], + "title": "Datacenter" + }, + "DatasetCreateBody": { + "properties": { + "name": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Name", + "description": "The name of this dataset. Datasets with the same name will be versioned together." + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "A description of this dataset" + }, + "rows": { + "items": { + "additionalProperties": true, + "type": "object" + }, + "type": "array", + "title": "Rows", + "description": "Dataset rows" + } + }, + "type": "object", + "required": [ + "rows" + ], + "title": "DatasetCreateBody" + }, + "DatasetCreateRes": { + "properties": { + "digest": { + "type": "string", + "title": "Digest", + "description": "The digest of the created dataset" + }, + "object_id": { + "type": "string", + "title": "Object Id", + "description": "The ID of the created dataset" + }, + "version_index": { + "type": "integer", + "title": "Version Index", + "description": "The version index of the created dataset" + } + }, "type": "object", "required": [ - "project_id" + "digest", + "object_id", + "version_index" ], - "title": "CostQueryReq" + "title": "DatasetCreateRes" }, - "CostQueryRes": { + "DatasetDeleteRes": { "properties": { - "results": { - "items": { - "$ref": "#/components/schemas/CostQueryOutput" - }, - "type": "array", - "title": "Results" + "num_deleted": { + "type": "integer", + "title": "Num Deleted", + "description": "Number of dataset versions deleted" } }, "type": "object", "required": [ - "results" + "num_deleted" ], - "title": "CostQueryRes" + "title": "DatasetDeleteRes" }, - "Datacenter": { + "DatasetReadRes": { "properties": { - "country_code": { + "object_id": { "type": "string", - "title": "Country Code" + "title": "Object Id", + "description": "The dataset ID" + }, + "digest": { + "type": "string", + "title": "Digest", + "description": "The digest of the dataset object" + }, + "version_index": { + "type": "integer", + "title": "Version Index", + "description": "The version index of the object" + }, + "created_at": { + "type": "string", + "format": "date-time", + "title": "Created At", + "description": "When the object was created" + }, + "name": { + "type": "string", + "title": "Name", + "description": "The name of the dataset" + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "Description of the dataset" + }, + "rows": { + "type": "string", + "title": "Rows", + "description": "Reference to the dataset rows data" } }, "type": "object", "required": [ - "country_code" + "object_id", + "digest", + "version_index", + "created_at", + "name", + "rows" ], - "title": "Datacenter" + "title": "DatasetReadRes" }, "EndedCallSchemaForInsert": { "properties": { @@ -3864,80 +6703,481 @@ ] } ], - "type": "array", - "maxItems": 2, - "minItems": 2, - "title": "$Eq" + "type": "array", + "maxItems": 2, + "minItems": 2, + "title": "$Eq" + } + }, + "type": "object", + "required": [ + "$eq" + ], + "title": "EqOperation", + "description": "Equality check between two operands.\n\nExample:\n ```\n {\n \"$eq\": [{\"$getField\": \"op_name\"}, {\"$literal\": \"predict\"}]\n }\n ```" + }, + "EvaluateModelReq": { + "properties": { + "project_id": { + "type": "string", + "title": "Project Id" + }, + "evaluation_ref": { + "type": "string", + "title": "Evaluation Ref" + }, + "model_ref": { + "type": "string", + "title": "Model Ref" + }, + "wb_user_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Wb User Id", + "description": "Do not set directly. Server will automatically populate this field." + } + }, + "additionalProperties": false, + "type": "object", + "required": [ + "project_id", + "evaluation_ref", + "model_ref" + ], + "title": "EvaluateModelReq" + }, + "EvaluateModelRes": { + "properties": { + "call_id": { + "type": "string", + "title": "Call Id" + } + }, + "type": "object", + "required": [ + "call_id" + ], + "title": "EvaluateModelRes" + }, + "EvaluationCreateBody": { + "properties": { + "name": { + "type": "string", + "title": "Name", + "description": "The name of this evaluation. Evaluations with the same name will be versioned together." + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "A description of this evaluation" + }, + "dataset": { + "type": "string", + "title": "Dataset", + "description": "Reference to the dataset (weave:// URI)" + }, + "scorers": { + "anyOf": [ + { + "items": { + "type": "string" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "Scorers", + "description": "List of scorer references (weave:// URIs)" + }, + "trials": { + "type": "integer", + "title": "Trials", + "description": "Number of trials to run", + "default": 1 + }, + "evaluation_name": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Evaluation Name", + "description": "Name for the evaluation run" + }, + "eval_attributes": { + "anyOf": [ + { + "additionalProperties": true, + "type": "object" + }, + { + "type": "null" + } + ], + "title": "Eval Attributes", + "description": "Optional attributes for the evaluation" + } + }, + "type": "object", + "required": [ + "name", + "dataset" + ], + "title": "EvaluationCreateBody" + }, + "EvaluationCreateRes": { + "properties": { + "digest": { + "type": "string", + "title": "Digest", + "description": "The digest of the created evaluation" + }, + "object_id": { + "type": "string", + "title": "Object Id", + "description": "The ID of the created evaluation" + }, + "version_index": { + "type": "integer", + "title": "Version Index", + "description": "The version index of the created evaluation" + }, + "evaluation_ref": { + "type": "string", + "title": "Evaluation Ref", + "description": "Full reference to the created evaluation" + } + }, + "type": "object", + "required": [ + "digest", + "object_id", + "version_index", + "evaluation_ref" + ], + "title": "EvaluationCreateRes" + }, + "EvaluationDeleteRes": { + "properties": { + "num_deleted": { + "type": "integer", + "title": "Num Deleted", + "description": "Number of evaluation versions deleted" + } + }, + "type": "object", + "required": [ + "num_deleted" + ], + "title": "EvaluationDeleteRes" + }, + "EvaluationReadRes": { + "properties": { + "object_id": { + "type": "string", + "title": "Object Id", + "description": "The evaluation ID" + }, + "digest": { + "type": "string", + "title": "Digest", + "description": "The digest of the evaluation" + }, + "version_index": { + "type": "integer", + "title": "Version Index", + "description": "The version index of the evaluation" + }, + "created_at": { + "type": "string", + "format": "date-time", + "title": "Created At", + "description": "When the evaluation was created" + }, + "name": { + "type": "string", + "title": "Name", + "description": "The name of the evaluation" + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "A description of the evaluation" + }, + "dataset": { + "type": "string", + "title": "Dataset", + "description": "Dataset reference (weave:// URI)" + }, + "scorers": { + "items": { + "type": "string" + }, + "type": "array", + "title": "Scorers", + "description": "List of scorer references (weave:// URIs)" + }, + "trials": { + "type": "integer", + "title": "Trials", + "description": "Number of trials" + }, + "evaluation_name": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Evaluation Name", + "description": "Name for the evaluation run" + }, + "evaluate_op": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Evaluate Op", + "description": "Evaluate op reference (weave:// URI)" + }, + "predict_and_score_op": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Predict And Score Op", + "description": "Predict and score op reference (weave:// URI)" + }, + "summarize_op": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Summarize Op", + "description": "Summarize op reference (weave:// URI)" } }, "type": "object", "required": [ - "$eq" + "object_id", + "digest", + "version_index", + "created_at", + "name", + "dataset", + "scorers", + "trials" ], - "title": "EqOperation", - "description": "Equality check between two operands.\n\nExample:\n ```\n {\n \"$eq\": [{\"$getField\": \"op_name\"}, {\"$literal\": \"predict\"}]\n }\n ```" + "title": "EvaluationReadRes" }, - "EvaluateModelReq": { + "EvaluationRunCreateBody": { "properties": { - "project_id": { + "evaluation": { "type": "string", - "title": "Project Id" + "title": "Evaluation", + "description": "Reference to the evaluation (weave:// URI)" }, - "evaluation_ref": { + "model": { "type": "string", - "title": "Evaluation Ref" - }, - "model_ref": { + "title": "Model", + "description": "Reference to the model (weave:// URI)" + } + }, + "type": "object", + "required": [ + "evaluation", + "model" + ], + "title": "EvaluationRunCreateBody" + }, + "EvaluationRunCreateRes": { + "properties": { + "evaluation_run_id": { "type": "string", - "title": "Model Ref" - }, - "wb_user_id": { + "title": "Evaluation Run Id", + "description": "The ID of the created evaluation run" + } + }, + "type": "object", + "required": [ + "evaluation_run_id" + ], + "title": "EvaluationRunCreateRes" + }, + "EvaluationRunDeleteRes": { + "properties": { + "num_deleted": { + "type": "integer", + "title": "Num Deleted", + "description": "Number of evaluation runs deleted" + } + }, + "type": "object", + "required": [ + "num_deleted" + ], + "title": "EvaluationRunDeleteRes" + }, + "EvaluationRunFinishBody": { + "properties": { + "summary": { "anyOf": [ { - "type": "string" + "additionalProperties": true, + "type": "object" }, { "type": "null" } ], - "title": "Wb User Id", - "description": "Do not set directly. Server will automatically populate this field." + "title": "Summary", + "description": "Optional summary dictionary for the evaluation run" + } + }, + "type": "object", + "title": "EvaluationRunFinishBody", + "description": "Request body for finishing an evaluation run via REST API.\n\nThis model excludes project_id and evaluation_run_id since they come from the URL path in RESTful endpoints." + }, + "EvaluationRunFinishRes": { + "properties": { + "success": { + "type": "boolean", + "title": "Success", + "description": "Whether the evaluation run was finished successfully" } }, - "additionalProperties": false, "type": "object", "required": [ - "project_id", - "evaluation_ref", - "model_ref" + "success" ], - "title": "EvaluateModelReq" + "title": "EvaluationRunFinishRes" }, - "EvaluateModelRes": { + "EvaluationRunReadRes": { "properties": { - "call_id": { + "evaluation_run_id": { "type": "string", - "title": "Call Id" + "title": "Evaluation Run Id", + "description": "The evaluation run ID" + }, + "evaluation": { + "type": "string", + "title": "Evaluation", + "description": "Reference to the evaluation (weave:// URI)" + }, + "model": { + "type": "string", + "title": "Model", + "description": "Reference to the model (weave:// URI)" + }, + "status": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Status", + "description": "Status of the evaluation run" + }, + "started_at": { + "anyOf": [ + { + "type": "string", + "format": "date-time" + }, + { + "type": "null" + } + ], + "title": "Started At", + "description": "When the evaluation run started" + }, + "finished_at": { + "anyOf": [ + { + "type": "string", + "format": "date-time" + }, + { + "type": "null" + } + ], + "title": "Finished At", + "description": "When the evaluation run finished" + }, + "summary": { + "anyOf": [ + { + "additionalProperties": true, + "type": "object" + }, + { + "type": "null" + } + ], + "title": "Summary", + "description": "Summary data for the evaluation run" } }, "type": "object", "required": [ - "call_id" + "evaluation_run_id", + "evaluation", + "model" ], - "title": "EvaluateModelRes" + "title": "EvaluationRunReadRes" }, "EvaluationStatusComplete": { "properties": { "code": { "type": "string", - "enum": [ - "complete" - ], "const": "complete", "title": "Code", "default": "complete" }, "output": { + "additionalProperties": true, "type": "object", "title": "Output" } @@ -3953,9 +7193,6 @@ "properties": { "code": { "type": "string", - "enum": [ - "failed" - ], "const": "failed", "title": "Code", "default": "failed" @@ -3980,9 +7217,6 @@ "properties": { "code": { "type": "string", - "enum": [ - "not_found" - ], "const": "not_found", "title": "Code", "default": "not_found" @@ -4041,9 +7275,6 @@ "properties": { "code": { "type": "string", - "enum": [ - "running" - ], "const": "running", "title": "Code", "default": "running" @@ -4151,6 +7382,7 @@ ] }, "payload": { + "additionalProperties": true, "type": "object", "title": "Payload", "examples": [ @@ -4254,6 +7486,7 @@ "title": "Wb User Id" }, "payload": { + "additionalProperties": true, "type": "object", "title": "Payload" } @@ -4387,6 +7620,7 @@ "properties": { "result": { "items": { + "additionalProperties": true, "type": "object" }, "type": "array", @@ -4452,6 +7686,7 @@ ] }, "payload": { + "additionalProperties": true, "type": "object", "title": "Payload", "examples": [ @@ -4560,6 +7795,7 @@ "title": "Wb User Id" }, "payload": { + "additionalProperties": true, "type": "object", "title": "Payload" } @@ -4975,6 +8211,7 @@ "ImageGenerationCreateRes": { "properties": { "response": { + "additionalProperties": true, "type": "object", "title": "Response" }, @@ -5189,6 +8426,7 @@ "title": "Total Tokens" } }, + "additionalProperties": true, "type": "object", "title": "LLMUsageSchema" }, @@ -5215,24 +8453,185 @@ "type": "object" }, { - "items": { - "$ref": "#/components/schemas/LiteralOperation" - }, - "type": "array" + "items": { + "$ref": "#/components/schemas/LiteralOperation" + }, + "type": "array" + }, + { + "type": "null" + } + ], + "title": "$Literal" + } + }, + "type": "object", + "required": [ + "$literal" + ], + "title": "LiteralOperation", + "description": "Represents a constant value in the query language.\n\nThis can be any standard JSON-serializable value.\n\nExample:\n ```\n {\"$literal\": \"predict\"}\n ```" + }, + "ModelCreateBody": { + "properties": { + "name": { + "type": "string", + "title": "Name", + "description": "The name of this model. Models with the same name will be versioned together." + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "A description of this model" + }, + "source_code": { + "type": "string", + "title": "Source Code", + "description": "Complete source code for the Model class including imports" + }, + "attributes": { + "anyOf": [ + { + "additionalProperties": true, + "type": "object" + }, + { + "type": "null" + } + ], + "title": "Attributes", + "description": "Additional attributes to be stored with the model" + } + }, + "type": "object", + "required": [ + "name", + "source_code" + ], + "title": "ModelCreateBody" + }, + "ModelCreateRes": { + "properties": { + "digest": { + "type": "string", + "title": "Digest", + "description": "The digest of the created model" + }, + "object_id": { + "type": "string", + "title": "Object Id", + "description": "The ID of the created model" + }, + "version_index": { + "type": "integer", + "title": "Version Index", + "description": "The version index of the created model" + }, + "model_ref": { + "type": "string", + "title": "Model Ref", + "description": "Full reference to the created model" + } + }, + "type": "object", + "required": [ + "digest", + "object_id", + "version_index", + "model_ref" + ], + "title": "ModelCreateRes" + }, + "ModelDeleteRes": { + "properties": { + "num_deleted": { + "type": "integer", + "title": "Num Deleted", + "description": "Number of model versions deleted" + } + }, + "type": "object", + "required": [ + "num_deleted" + ], + "title": "ModelDeleteRes" + }, + "ModelReadRes": { + "properties": { + "object_id": { + "type": "string", + "title": "Object Id", + "description": "The model ID" + }, + "digest": { + "type": "string", + "title": "Digest", + "description": "The digest of the model" + }, + "version_index": { + "type": "integer", + "title": "Version Index", + "description": "The version index of the object" + }, + "created_at": { + "type": "string", + "format": "date-time", + "title": "Created At", + "description": "When the model was created" + }, + "name": { + "type": "string", + "title": "Name", + "description": "The name of the model" + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "Description of the model" + }, + "source_code": { + "type": "string", + "title": "Source Code", + "description": "The source code of the model" + }, + "attributes": { + "anyOf": [ + { + "additionalProperties": true, + "type": "object" }, { "type": "null" } ], - "title": "$Literal" + "title": "Attributes", + "description": "Additional attributes stored with the model" } }, "type": "object", "required": [ - "$literal" + "object_id", + "digest", + "version_index", + "created_at", + "name", + "source_code" ], - "title": "LiteralOperation", - "description": "Represents a constant value in the query language.\n\nThis can be any standard JSON-serializable value.\n\nExample:\n ```\n {\"$literal\": \"predict\"}\n ```" + "title": "ModelReadRes" }, "NotOperation": { "properties": { @@ -5307,6 +8706,17 @@ "digest": { "type": "string", "title": "Digest" + }, + "object_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Object Id" } }, "type": "object", @@ -5808,85 +9218,334 @@ "type": "null" } ], - "title": "Is Op", - "description": "Filter objects based on whether they are weave.ops or not. `True` will only return ops, `False` will return non-ops, and `None` will return all objects", - "examples": [ - true, - false, - null - ] + "title": "Is Op", + "description": "Filter objects based on whether they are weave.ops or not. `True` will only return ops, `False` will return non-ops, and `None` will return all objects", + "examples": [ + true, + false, + null + ] + }, + "latest_only": { + "anyOf": [ + { + "type": "boolean" + }, + { + "type": "null" + } + ], + "title": "Latest Only", + "description": "If True, return only the latest version of each object. `False` and `None` will return all versions", + "examples": [ + true, + false + ] + } + }, + "additionalProperties": false, + "type": "object", + "title": "ObjectVersionFilter" + }, + "OpCreateBody": { + "properties": { + "name": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Name", + "description": "The name of this op. Ops with the same name will be versioned together." + }, + "source_code": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Source Code", + "description": "Complete source code for this op, including imports" + } + }, + "type": "object", + "title": "OpCreateBody", + "description": "Request body for creating an Op object via REST API.\n\nThis model excludes project_id since it comes from the URL path in RESTful endpoints." + }, + "OpCreateRes": { + "properties": { + "digest": { + "type": "string", + "title": "Digest", + "description": "The digest of the created op" + }, + "object_id": { + "type": "string", + "title": "Object Id", + "description": "The ID of the created op" + }, + "version_index": { + "type": "integer", + "title": "Version Index", + "description": "The version index of the created op" + } + }, + "type": "object", + "required": [ + "digest", + "object_id", + "version_index" + ], + "title": "OpCreateRes", + "description": "Response model for creating an Op object." + }, + "OpDeleteRes": { + "properties": { + "num_deleted": { + "type": "integer", + "title": "Num Deleted", + "description": "Number of op versions deleted from this op" + } + }, + "type": "object", + "required": [ + "num_deleted" + ], + "title": "OpDeleteRes" + }, + "OpReadRes": { + "properties": { + "object_id": { + "type": "string", + "title": "Object Id", + "description": "The op ID" + }, + "digest": { + "type": "string", + "title": "Digest", + "description": "The digest of the op" + }, + "version_index": { + "type": "integer", + "title": "Version Index", + "description": "The version index of this op" + }, + "created_at": { + "type": "string", + "format": "date-time", + "title": "Created At", + "description": "When this op was created" + }, + "code": { + "type": "string", + "title": "Code", + "description": "The actual op source code" + } + }, + "type": "object", + "required": [ + "object_id", + "digest", + "version_index", + "created_at", + "code" + ], + "title": "OpReadRes", + "description": "Response model for reading an Op object.\n\nThe code field contains the actual source code of the op." + }, + "OrOperation": { + "properties": { + "$or": { + "items": { + "anyOf": [ + { + "$ref": "#/components/schemas/LiteralOperation" + }, + { + "$ref": "#/components/schemas/GetFieldOperator" + }, + { + "$ref": "#/components/schemas/ConvertOperation" + }, + { + "$ref": "#/components/schemas/AndOperation" + }, + { + "$ref": "#/components/schemas/OrOperation" + }, + { + "$ref": "#/components/schemas/NotOperation" + }, + { + "$ref": "#/components/schemas/EqOperation" + }, + { + "$ref": "#/components/schemas/GtOperation" + }, + { + "$ref": "#/components/schemas/GteOperation" + }, + { + "$ref": "#/components/schemas/InOperation" + }, + { + "$ref": "#/components/schemas/ContainsOperation" + } + ] + }, + "type": "array", + "title": "$Or" + } + }, + "type": "object", + "required": [ + "$or" + ], + "title": "OrOperation", + "description": "Logical OR. At least one condition must be true.\n\nExample:\n ```\n {\n \"$or\": [\n {\"$eq\": [{\"$getField\": \"op_name\"}, {\"$literal\": \"a\"}]},\n {\"$eq\": [{\"$getField\": \"op_name\"}, {\"$literal\": \"b\"}]}\n ]\n }\n ```" + }, + "PredictionCreateBody": { + "properties": { + "model": { + "type": "string", + "title": "Model", + "description": "The model reference (weave:// URI)" + }, + "inputs": { + "additionalProperties": true, + "type": "object", + "title": "Inputs", + "description": "The inputs to the prediction" + }, + "output": { + "title": "Output", + "description": "The output of the prediction" + }, + "evaluation_run_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Evaluation Run Id", + "description": "Optional evaluation run ID to link this prediction as a child call" + } + }, + "type": "object", + "required": [ + "model", + "inputs", + "output" + ], + "title": "PredictionCreateBody", + "description": "Request body for creating a Prediction via REST API.\n\nThis model excludes project_id since it comes from the URL path in RESTful endpoints." + }, + "PredictionCreateRes": { + "properties": { + "prediction_id": { + "type": "string", + "title": "Prediction Id", + "description": "The prediction ID" + } + }, + "type": "object", + "required": [ + "prediction_id" + ], + "title": "PredictionCreateRes" + }, + "PredictionDeleteRes": { + "properties": { + "num_deleted": { + "type": "integer", + "title": "Num Deleted", + "description": "Number of predictions deleted" + } + }, + "type": "object", + "required": [ + "num_deleted" + ], + "title": "PredictionDeleteRes" + }, + "PredictionFinishRes": { + "properties": { + "success": { + "type": "boolean", + "title": "Success", + "description": "Whether the prediction was finished successfully" + } + }, + "type": "object", + "required": [ + "success" + ], + "title": "PredictionFinishRes" + }, + "PredictionReadRes": { + "properties": { + "prediction_id": { + "type": "string", + "title": "Prediction Id", + "description": "The prediction ID" + }, + "model": { + "type": "string", + "title": "Model", + "description": "The model reference (weave:// URI)" + }, + "inputs": { + "additionalProperties": true, + "type": "object", + "title": "Inputs", + "description": "The inputs to the prediction" + }, + "output": { + "title": "Output", + "description": "The output of the prediction" + }, + "evaluation_run_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Evaluation Run Id", + "description": "Evaluation run ID if this prediction is linked to one" }, - "latest_only": { + "wb_user_id": { "anyOf": [ { - "type": "boolean" + "type": "string" }, { "type": "null" } ], - "title": "Latest Only", - "description": "If True, return only the latest version of each object. `False` and `None` will return all versions", - "examples": [ - true, - false - ] - } - }, - "additionalProperties": false, - "type": "object", - "title": "ObjectVersionFilter" - }, - "OrOperation": { - "properties": { - "$or": { - "items": { - "anyOf": [ - { - "$ref": "#/components/schemas/LiteralOperation" - }, - { - "$ref": "#/components/schemas/GetFieldOperator" - }, - { - "$ref": "#/components/schemas/ConvertOperation" - }, - { - "$ref": "#/components/schemas/AndOperation" - }, - { - "$ref": "#/components/schemas/OrOperation" - }, - { - "$ref": "#/components/schemas/NotOperation" - }, - { - "$ref": "#/components/schemas/EqOperation" - }, - { - "$ref": "#/components/schemas/GtOperation" - }, - { - "$ref": "#/components/schemas/GteOperation" - }, - { - "$ref": "#/components/schemas/InOperation" - }, - { - "$ref": "#/components/schemas/ContainsOperation" - } - ] - }, - "type": "array", - "title": "$Or" + "title": "Wb User Id", + "description": "Do not set directly. Server will automatically populate this field." } }, "type": "object", "required": [ - "$or" + "prediction_id", + "model", + "inputs", + "output" ], - "title": "OrOperation", - "description": "Logical OR. At least one condition must be true.\n\nExample:\n ```\n {\n \"$or\": [\n {\"$eq\": [{\"$getField\": \"op_name\"}, {\"$literal\": \"a\"}]},\n {\"$eq\": [{\"$getField\": \"op_name\"}, {\"$literal\": \"b\"}]}\n ]\n }\n ```" + "title": "PredictionReadRes" }, "Pricing": { "properties": { @@ -6122,6 +9781,258 @@ ], "title": "RouterOpenRouterModelsRes" }, + "ScoreCreateBody": { + "properties": { + "prediction_id": { + "type": "string", + "title": "Prediction Id", + "description": "The prediction ID" + }, + "scorer": { + "type": "string", + "title": "Scorer", + "description": "The scorer reference (weave:// URI)" + }, + "value": { + "type": "number", + "title": "Value", + "description": "The value of the score" + }, + "evaluation_run_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Evaluation Run Id", + "description": "Optional evaluation run ID to link this score as a child call" + } + }, + "type": "object", + "required": [ + "prediction_id", + "scorer", + "value" + ], + "title": "ScoreCreateBody", + "description": "Request body for creating a Score via REST API.\n\nThis model excludes project_id since it comes from the URL path in RESTful endpoints." + }, + "ScoreCreateRes": { + "properties": { + "score_id": { + "type": "string", + "title": "Score Id", + "description": "The score ID" + } + }, + "type": "object", + "required": [ + "score_id" + ], + "title": "ScoreCreateRes" + }, + "ScoreDeleteRes": { + "properties": { + "num_deleted": { + "type": "integer", + "title": "Num Deleted", + "description": "Number of scores deleted" + } + }, + "type": "object", + "required": [ + "num_deleted" + ], + "title": "ScoreDeleteRes" + }, + "ScoreReadRes": { + "properties": { + "score_id": { + "type": "string", + "title": "Score Id", + "description": "The score ID" + }, + "scorer": { + "type": "string", + "title": "Scorer", + "description": "The scorer reference (weave:// URI)" + }, + "value": { + "type": "number", + "title": "Value", + "description": "The value of the score" + }, + "evaluation_run_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Evaluation Run Id", + "description": "Evaluation run ID if this score is linked to one" + }, + "wb_user_id": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Wb User Id", + "description": "Do not set directly. Server will automatically populate this field." + } + }, + "type": "object", + "required": [ + "score_id", + "scorer", + "value" + ], + "title": "ScoreReadRes" + }, + "ScorerCreateBody": { + "properties": { + "name": { + "type": "string", + "title": "Name", + "description": "The name of this scorer. Scorers with the same name will be versioned together." + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "A description of this scorer" + }, + "op_source_code": { + "type": "string", + "title": "Op Source Code", + "description": "Complete source code for the Scorer.score op including imports" + } + }, + "type": "object", + "required": [ + "name", + "op_source_code" + ], + "title": "ScorerCreateBody" + }, + "ScorerCreateRes": { + "properties": { + "digest": { + "type": "string", + "title": "Digest", + "description": "The digest of the created scorer" + }, + "object_id": { + "type": "string", + "title": "Object Id", + "description": "The ID of the created scorer" + }, + "version_index": { + "type": "integer", + "title": "Version Index", + "description": "The version index of the created scorer" + }, + "scorer": { + "type": "string", + "title": "Scorer", + "description": "Full reference to the created scorer" + } + }, + "type": "object", + "required": [ + "digest", + "object_id", + "version_index", + "scorer" + ], + "title": "ScorerCreateRes" + }, + "ScorerDeleteRes": { + "properties": { + "num_deleted": { + "type": "integer", + "title": "Num Deleted", + "description": "Number of scorer versions deleted" + } + }, + "type": "object", + "required": [ + "num_deleted" + ], + "title": "ScorerDeleteRes" + }, + "ScorerReadRes": { + "properties": { + "object_id": { + "type": "string", + "title": "Object Id", + "description": "The scorer ID" + }, + "digest": { + "type": "string", + "title": "Digest", + "description": "The digest of the scorer" + }, + "version_index": { + "type": "integer", + "title": "Version Index", + "description": "The version index of the object" + }, + "created_at": { + "type": "string", + "format": "date-time", + "title": "Created At", + "description": "When the scorer was created" + }, + "name": { + "type": "string", + "title": "Name", + "description": "The name of the scorer" + }, + "description": { + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Description", + "description": "Description of the scorer" + }, + "score_op": { + "type": "string", + "title": "Score Op", + "description": "The Scorer.score op reference" + } + }, + "type": "object", + "required": [ + "object_id", + "digest", + "version_index", + "created_at", + "name", + "score_op" + ], + "title": "ScorerReadRes" + }, "ServerInfoRes": { "properties": { "min_required_weave_python_version": { @@ -6240,13 +10151,27 @@ "title": "Started At" }, "attributes": { + "additionalProperties": true, "type": "object", "title": "Attributes" }, "inputs": { + "additionalProperties": true, "type": "object", "title": "Inputs" }, + "otel_dump": { + "anyOf": [ + { + "additionalProperties": true, + "type": "object" + }, + { + "type": "null" + } + ], + "title": "Otel Dump" + }, "wb_user_id": { "anyOf": [ { @@ -6305,6 +10230,9 @@ "additionalProperties": { "type": "integer" }, + "propertyNames": { + "$ref": "#/components/schemas/TraceStatus" + }, "type": "object", "title": "Status Counts" } @@ -6328,6 +10256,7 @@ "TableAppendSpecPayload": { "properties": { "row": { + "additionalProperties": true, "type": "object", "title": "Row" } @@ -6360,6 +10289,19 @@ ], "title": "TableCreateFromDigestsReq" }, + "TableCreateFromDigestsRes": { + "properties": { + "digest": { + "type": "string", + "title": "Digest" + } + }, + "type": "object", + "required": [ + "digest" + ], + "title": "TableCreateFromDigestsRes" + }, "TableCreateReq": { "properties": { "table": { @@ -6413,6 +10355,7 @@ "title": "Index" }, "row": { + "additionalProperties": true, "type": "object", "title": "Row" } @@ -6732,6 +10675,7 @@ }, "rows": { "items": { + "additionalProperties": true, "type": "object" }, "type": "array", diff --git a/weave/reference/typescript-sdk/classes/EvaluationLogger.mdx b/weave/reference/typescript-sdk/classes/EvaluationLogger.mdx deleted file mode 100644 index 5242b4adad..0000000000 --- a/weave/reference/typescript-sdk/classes/EvaluationLogger.mdx +++ /dev/null @@ -1,110 +0,0 @@ ---- -title: "Class: EvaluationLogger" -description: "TypeScript SDK reference" ---- - -[weave](../) / EvaluationLogger - - - -EvaluationLogger enables incremental logging of predictions and scores. - -Unlike the traditional Evaluation class which requires upfront dataset and batch processing, -EvaluationLogger allows you to log predictions as they happen, with flexible scoring. - -`Example` - -```ts -const ev = new EvaluationLogger({name: 'my-eval', dataset: 'my-dataset'}); - -for (const example of streamingData) { - const output = await myModel.predict(example); - const pred = await ev.logPrediction(example, output); - - if (shouldScore(output)) { - await pred.logScore("accuracy", calculateAccuracy(output)); - } - await pred.finish(); -} - -await ev.logSummary(); -``` - -## Table of contents - -### Constructors - -- [constructor](./EvaluationLogger#constructor) - -### Methods - -- [logPrediction](./EvaluationLogger#logprediction) -- [logSummary](./EvaluationLogger#logsummary) - -## Constructors - -### constructor - -• **new EvaluationLogger**(`options`): [`EvaluationLogger`](./EvaluationLogger) - -#### Parameters - -| Name | Type | -| :------ | :------ | -| `options` | `EvaluationLoggerOptions` | - -#### Returns - -[`EvaluationLogger`](./EvaluationLogger) - -#### Defined in - -[evaluationLogger.ts:502](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/evaluationLogger.ts#L502) - -## Methods - -### logPrediction - -▸ **logPrediction**(`inputs`, `output`): `Promise`\<[`ScoreLogger`](./ScoreLogger)\> - -Log a prediction with its input and output. -Creates a predict_and_score call (with child predict call). -Returns a ScoreLogger for adding scores. - -#### Parameters - -| Name | Type | -| :------ | :------ | -| `inputs` | `Record`\<`string`, `any`\> | -| `output` | `any` | - -#### Returns - -`Promise`\<[`ScoreLogger`](./ScoreLogger)\> - -#### Defined in - -[evaluationLogger.ts:579](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/evaluationLogger.ts#L579) - -___ - -### logSummary - -▸ **logSummary**(`summary?`): `Promise`\<`void`\> - -Log a summary and finalize the evaluation. -Creates a summarize call and finishes the evaluate call. - -#### Parameters - -| Name | Type | -| :------ | :------ | -| `summary?` | `Record`\<`string`, `any`\> | - -#### Returns - -`Promise`\<`void`\> - -#### Defined in - -[evaluationLogger.ts:669](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/evaluationLogger.ts#L669) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/classes/Dataset.mdx b/weave/reference/typescript-sdk/classes/dataset.mdx similarity index 66% rename from weave/reference/typescript-sdk/classes/Dataset.mdx rename to weave/reference/typescript-sdk/classes/dataset.mdx index 322249cffd..7b3ac5dc38 100644 --- a/weave/reference/typescript-sdk/classes/Dataset.mdx +++ b/weave/reference/typescript-sdk/classes/dataset.mdx @@ -93,13 +93,13 @@ const ref = await dataset.save() #### Defined in -[dataset.ts:51](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/dataset.ts#L51) +[dataset.ts:51](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/dataset.ts#L51) ## Properties ### \_\_savedRef -• `Optional` **\_\_savedRef**: `ObjectRef` \| `Promise`\<`ObjectRef`\> +• `Optional` **\_\_savedRef**: [`ObjectRef`](./ObjectRef) \| `Promise`\<[`ObjectRef`](./ObjectRef)\> #### Inherited from @@ -107,7 +107,7 @@ const ref = await dataset.save() #### Defined in -[weaveObject.ts:49](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L49) +[weaveObject.ts:49](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L49) ___ @@ -117,7 +117,7 @@ ___ #### Defined in -[dataset.ts:49](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/dataset.ts#L49) +[dataset.ts:49](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/dataset.ts#L49) ## Accessors @@ -135,7 +135,7 @@ WeaveObject.description #### Defined in -[weaveObject.ts:76](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L76) +[weaveObject.ts:76](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L76) ___ @@ -149,7 +149,7 @@ ___ #### Defined in -[dataset.ts:64](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/dataset.ts#L64) +[dataset.ts:64](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/dataset.ts#L64) ___ @@ -167,7 +167,7 @@ WeaveObject.name #### Defined in -[weaveObject.ts:72](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L72) +[weaveObject.ts:72](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L72) ## Methods @@ -181,7 +181,7 @@ WeaveObject.name #### Defined in -[dataset.ts:68](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/dataset.ts#L68) +[dataset.ts:68](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/dataset.ts#L68) ___ @@ -201,21 +201,21 @@ ___ #### Defined in -[dataset.ts:74](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/dataset.ts#L74) +[dataset.ts:74](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/dataset.ts#L74) ___ ### save -▸ **save**(): `Promise`\<`ObjectRef`\> +▸ **save**(): `Promise`\<[`ObjectRef`](./ObjectRef)\> #### Returns -`Promise`\<`ObjectRef`\> +`Promise`\<[`ObjectRef`](./ObjectRef)\> #### Defined in -[dataset.ts:60](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/dataset.ts#L60) +[dataset.ts:60](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/dataset.ts#L60) ___ @@ -233,4 +233,4 @@ ___ #### Defined in -[weaveObject.ts:53](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L53) \ No newline at end of file +[weaveObject.ts:53](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L53) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/classes/Evaluation.mdx b/weave/reference/typescript-sdk/classes/evaluation.mdx similarity index 81% rename from weave/reference/typescript-sdk/classes/Evaluation.mdx rename to weave/reference/typescript-sdk/classes/evaluation.mdx index a61b755c71..6e37848eaf 100644 --- a/weave/reference/typescript-sdk/classes/Evaluation.mdx +++ b/weave/reference/typescript-sdk/classes/evaluation.mdx @@ -112,13 +112,13 @@ const results = await evaluation.evaluate({ model }); #### Defined in -[evaluation.ts:148](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/evaluation.ts#L148) +[evaluation.ts:148](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/evaluation.ts#L148) ## Properties ### \_\_savedRef -• `Optional` **\_\_savedRef**: `ObjectRef` \| `Promise`\<`ObjectRef`\> +• `Optional` **\_\_savedRef**: [`ObjectRef`](./ObjectRef) \| `Promise`\<[`ObjectRef`](./ObjectRef)\> #### Inherited from @@ -126,7 +126,7 @@ const results = await evaluation.evaluate({ model }); #### Defined in -[weaveObject.ts:49](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L49) +[weaveObject.ts:49](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L49) ## Accessors @@ -144,7 +144,7 @@ WeaveObject.description #### Defined in -[weaveObject.ts:76](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L76) +[weaveObject.ts:76](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L76) ___ @@ -162,7 +162,7 @@ WeaveObject.name #### Defined in -[weaveObject.ts:72](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L72) +[weaveObject.ts:72](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L72) ## Methods @@ -185,7 +185,7 @@ WeaveObject.name #### Defined in -[evaluation.ts:163](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/evaluation.ts#L163) +[evaluation.ts:163](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/evaluation.ts#L163) ___ @@ -208,7 +208,7 @@ ___ #### Defined in -[evaluation.ts:231](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/evaluation.ts#L231) +[evaluation.ts:231](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/evaluation.ts#L231) ___ @@ -226,4 +226,4 @@ ___ #### Defined in -[weaveObject.ts:53](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L53) \ No newline at end of file +[weaveObject.ts:53](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L53) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/classes/evaluationlogger.mdx b/weave/reference/typescript-sdk/classes/evaluationlogger.mdx new file mode 100644 index 0000000000..69c64c6973 --- /dev/null +++ b/weave/reference/typescript-sdk/classes/evaluationlogger.mdx @@ -0,0 +1,163 @@ +--- +title: "Class: EvaluationLogger" +description: "TypeScript SDK reference" +--- + +[weave](../) / EvaluationLogger + + + +EvaluationLogger enables incremental logging of predictions and scores. + +Unlike the traditional Evaluation class which requires upfront dataset and batch processing, +EvaluationLogger allows you to log predictions as they happen, with flexible scoring. + +`Example` + +```ts +const ev = new EvaluationLogger({name: 'my-eval', dataset: 'my-dataset'}); + +for (const example of streamingData) { + const output = await myModel.predict(example); + const pred = ev.logPrediction(example, output); + + if (shouldScore(output)) { + pred.logScore("accuracy", calculateAccuracy(output)); + } + pred.finish(); +} + +await ev.logSummary(); +``` + +## Table of contents + +### Constructors + +- [constructor](./EvaluationLogger#constructor) + +### Methods + +- [logPrediction](./EvaluationLogger#logprediction) +- [logPredictionAsync](./EvaluationLogger#logpredictionasync) +- [logSummary](./EvaluationLogger#logsummary) + +## Constructors + +### constructor + +• **new EvaluationLogger**(`options`): [`EvaluationLogger`](./EvaluationLogger) + +#### Parameters + +| Name | Type | +| :------ | :------ | +| `options` | `EvaluationLoggerOptions` | + +#### Returns + +[`EvaluationLogger`](./EvaluationLogger) + +#### Defined in + +[evaluationLogger.ts:554](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/evaluationLogger.ts#L554) + +## Methods + +### logPrediction + +▸ **logPrediction**(`inputs`, `output`): [`ScoreLogger`](./ScoreLogger) + +Log a prediction with its input and output (synchronous version). +Creates a predict_and_score call (with child predict call). +Returns a ScoreLogger immediately for adding scores. + +This method returns the ScoreLogger synchronously. Operations on the +ScoreLogger (logScore, finish) will be queued and executed when initialization completes. + +#### Parameters + +| Name | Type | +| :------ | :------ | +| `inputs` | `Record`\<`string`, `any`\> | +| `output` | `any` | + +#### Returns + +[`ScoreLogger`](./ScoreLogger) + +`Example` + +```ts +// Fire-and-forget style +const scoreLogger = evalLogger.logPrediction({input: 'test'}, 'output'); +scoreLogger.logScore('accuracy', 0.95); +scoreLogger.finish(); +await evalLogger.logSummary(); // Waits for everything +``` + +#### Defined in + +[evaluationLogger.ts:641](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/evaluationLogger.ts#L641) + +___ + +### logPredictionAsync + +▸ **logPredictionAsync**(`inputs`, `output`): `Promise`\<[`ScoreLogger`](./ScoreLogger)\> + +Log a prediction with its input and output (async version). +Like logPrediction() but returns a Promise that resolves when +the prediction call is fully initialized. + +Use this if you need to await the initialization before proceeding. + +#### Parameters + +| Name | Type | +| :------ | :------ | +| `inputs` | `Record`\<`string`, `any`\> | +| `output` | `any` | + +#### Returns + +`Promise`\<[`ScoreLogger`](./ScoreLogger)\> + +`Example` + +```ts +// Awaitable style +const scoreLogger = await evalLogger.logPredictionAsync({input: 'test'}, 'output'); +await scoreLogger.logScore('accuracy', 0.95); +await scoreLogger.finish(); +``` + +#### Defined in + +[evaluationLogger.ts:666](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/evaluationLogger.ts#L666) + +___ + +### logSummary + +▸ **logSummary**(`summary?`): `Promise`\<`void`\> + +Log a summary and finalize the evaluation. +Creates a summarize call and finishes the evaluate call. + +This method can be called without await (fire-and-forget), but internally +it will wait for all pending operations to complete. + +#### Parameters + +| Name | Type | +| :------ | :------ | +| `summary?` | `Record`\<`string`, `any`\> | + +#### Returns + +`Promise`\<`void`\> + +#### Defined in + +[evaluationLogger.ts:767](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/evaluationLogger.ts#L767) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/classes/MessagesPrompt.mdx b/weave/reference/typescript-sdk/classes/messagesprompt.mdx similarity index 63% rename from weave/reference/typescript-sdk/classes/MessagesPrompt.mdx rename to weave/reference/typescript-sdk/classes/messagesprompt.mdx index d6f23f76f3..dda994c2e6 100644 --- a/weave/reference/typescript-sdk/classes/MessagesPrompt.mdx +++ b/weave/reference/typescript-sdk/classes/messagesprompt.mdx @@ -56,13 +56,13 @@ Prompt.constructor #### Defined in -[prompt.ts:33](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/prompt.ts#L33) +[prompt.ts:33](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/prompt.ts#L33) ## Properties ### \_\_savedRef -• `Optional` **\_\_savedRef**: `ObjectRef` \| `Promise`\<`ObjectRef`\> +• `Optional` **\_\_savedRef**: [`ObjectRef`](./ObjectRef) \| `Promise`\<[`ObjectRef`](./ObjectRef)\> #### Inherited from @@ -70,7 +70,7 @@ Prompt.\_\_savedRef #### Defined in -[weaveObject.ts:49](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L49) +[weaveObject.ts:49](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L49) ___ @@ -80,7 +80,7 @@ ___ #### Defined in -[prompt.ts:31](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/prompt.ts#L31) +[prompt.ts:31](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/prompt.ts#L31) ## Accessors @@ -98,7 +98,7 @@ Prompt.description #### Defined in -[weaveObject.ts:76](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L76) +[weaveObject.ts:76](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L76) ___ @@ -116,7 +116,7 @@ Prompt.name #### Defined in -[weaveObject.ts:72](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L72) +[weaveObject.ts:72](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L72) ## Methods @@ -136,7 +136,7 @@ Prompt.name #### Defined in -[prompt.ts:60](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/prompt.ts#L60) +[prompt.ts:60](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/prompt.ts#L60) ___ @@ -154,4 +154,4 @@ Prompt.saveAttrs #### Defined in -[weaveObject.ts:53](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L53) \ No newline at end of file +[weaveObject.ts:53](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L53) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/classes/objectref.mdx b/weave/reference/typescript-sdk/classes/objectref.mdx new file mode 100644 index 0000000000..9d422a017c --- /dev/null +++ b/weave/reference/typescript-sdk/classes/objectref.mdx @@ -0,0 +1,134 @@ +--- +title: "Class: ObjectRef" +description: "TypeScript SDK reference" +--- + +[weave](../) / ObjectRef + + + +Represents a reference to a saved Weave object. + +Generally, end users will not need to interact with this class directly. + +An ObjectRef contains the project ID, object ID, and digest that uniquely identify +a saved object in Weave's storage system. + +`Example` + +```ts +const ref = new ObjectRef('my-project', 'abc123', 'def456'); +const uri = ref.uri(); // weave:///my-project/object/abc123:def456 +``` + +## Table of contents + +### Constructors + +- [constructor](./ObjectRef#constructor) + +### Properties + +- [digest](./ObjectRef#digest) +- [objectId](./ObjectRef#objectid) +- [projectId](./ObjectRef#projectid) + +### Methods + +- [get](./ObjectRef#get) +- [ui\_url](./ObjectRef#ui_url) +- [uri](./ObjectRef#uri) + +## Constructors + +### constructor + +• **new ObjectRef**(`projectId`, `objectId`, `digest`): [`ObjectRef`](./ObjectRef) + +#### Parameters + +| Name | Type | +| :------ | :------ | +| `projectId` | `string` | +| `objectId` | `string` | +| `digest` | `string` | + +#### Returns + +[`ObjectRef`](./ObjectRef) + +#### Defined in + +[weaveObject.ts:25](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L25) + +## Properties + +### digest + +• **digest**: `string` + +#### Defined in + +[weaveObject.ts:28](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L28) + +___ + +### objectId + +• **objectId**: `string` + +#### Defined in + +[weaveObject.ts:27](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L27) + +___ + +### projectId + +• **projectId**: `string` + +#### Defined in + +[weaveObject.ts:26](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L26) + +## Methods + +### get + +▸ **get**(): `Promise`\<`any`\> + +#### Returns + +`Promise`\<`any`\> + +#### Defined in + +[weaveObject.ts:42](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L42) + +___ + +### ui\_url + +▸ **ui_url**(): `string` + +#### Returns + +`string` + +#### Defined in + +[weaveObject.ts:37](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L37) + +___ + +### uri + +▸ **uri**(): `string` + +#### Returns + +`string` + +#### Defined in + +[weaveObject.ts:33](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L33) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/classes/ScoreLogger.mdx b/weave/reference/typescript-sdk/classes/scorelogger.mdx similarity index 61% rename from weave/reference/typescript-sdk/classes/ScoreLogger.mdx rename to weave/reference/typescript-sdk/classes/scorelogger.mdx index 4e99b4d0e1..01f3216888 100644 --- a/weave/reference/typescript-sdk/classes/ScoreLogger.mdx +++ b/weave/reference/typescript-sdk/classes/scorelogger.mdx @@ -25,6 +25,10 @@ await pred.finish(); // Finalizes the prediction - [constructor](./ScoreLogger#constructor) +### Accessors + +- [isFinishCalled](./ScoreLogger#isfinishcalled) + ### Methods - [finish](./ScoreLogger#finish) @@ -34,13 +38,12 @@ await pred.finish(); // Finalizes the prediction ### constructor -• **new ScoreLogger**(`predMeta`, `evalLogger`): [`ScoreLogger`](./ScoreLogger) +• **new ScoreLogger**(`evalLogger`): [`ScoreLogger`](./ScoreLogger) #### Parameters | Name | Type | | :------ | :------ | -| `predMeta` | `PredictAndScoreCallMetadata` | | `evalLogger` | [`EvaluationLogger`](./EvaluationLogger) | #### Returns @@ -49,7 +52,24 @@ await pred.finish(); // Finalizes the prediction #### Defined in -[evaluationLogger.ts:317](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/evaluationLogger.ts#L317) +[evaluationLogger.ts:319](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/evaluationLogger.ts#L319) + +## Accessors + +### isFinishCalled + +• `get` **isFinishCalled**(): `boolean` + +Check if finish() has been called. +Used by EvaluationLogger to detect unfinished predictions. + +#### Returns + +`boolean` + +#### Defined in + +[evaluationLogger.ts:349](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/evaluationLogger.ts#L349) ## Methods @@ -67,7 +87,7 @@ Updates incremental aggregates and frees memory. #### Defined in -[evaluationLogger.ts:410](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/evaluationLogger.ts#L410) +[evaluationLogger.ts:451](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/evaluationLogger.ts#L451) ___ @@ -91,4 +111,4 @@ Creates a scorer call as a child of predict_and_score. #### Defined in -[evaluationLogger.ts:332](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/evaluationLogger.ts#L332) \ No newline at end of file +[evaluationLogger.ts:360](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/evaluationLogger.ts#L360) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/classes/StringPrompt.mdx b/weave/reference/typescript-sdk/classes/stringprompt.mdx similarity index 62% rename from weave/reference/typescript-sdk/classes/StringPrompt.mdx rename to weave/reference/typescript-sdk/classes/stringprompt.mdx index 748b6d4fe8..aded25547b 100644 --- a/weave/reference/typescript-sdk/classes/StringPrompt.mdx +++ b/weave/reference/typescript-sdk/classes/stringprompt.mdx @@ -56,13 +56,13 @@ Prompt.constructor #### Defined in -[prompt.ts:16](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/prompt.ts#L16) +[prompt.ts:16](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/prompt.ts#L16) ## Properties ### \_\_savedRef -• `Optional` **\_\_savedRef**: `ObjectRef` \| `Promise`\<`ObjectRef`\> +• `Optional` **\_\_savedRef**: [`ObjectRef`](./ObjectRef) \| `Promise`\<[`ObjectRef`](./ObjectRef)\> #### Inherited from @@ -70,7 +70,7 @@ Prompt.\_\_savedRef #### Defined in -[weaveObject.ts:49](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L49) +[weaveObject.ts:49](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L49) ___ @@ -80,7 +80,7 @@ ___ #### Defined in -[prompt.ts:14](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/prompt.ts#L14) +[prompt.ts:14](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/prompt.ts#L14) ## Accessors @@ -98,7 +98,7 @@ Prompt.description #### Defined in -[weaveObject.ts:76](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L76) +[weaveObject.ts:76](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L76) ___ @@ -116,7 +116,7 @@ Prompt.name #### Defined in -[weaveObject.ts:72](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L72) +[weaveObject.ts:72](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L72) ## Methods @@ -136,7 +136,7 @@ Prompt.name #### Defined in -[prompt.ts:21](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/prompt.ts#L21) +[prompt.ts:21](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/prompt.ts#L21) ___ @@ -154,4 +154,4 @@ Prompt.saveAttrs #### Defined in -[weaveObject.ts:53](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L53) \ No newline at end of file +[weaveObject.ts:53](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L53) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/classes/WeaveClient.mdx b/weave/reference/typescript-sdk/classes/weaveclient.mdx similarity index 71% rename from weave/reference/typescript-sdk/classes/WeaveClient.mdx rename to weave/reference/typescript-sdk/classes/weaveclient.mdx index c8f406ad1b..00ea9abf5e 100644 --- a/weave/reference/typescript-sdk/classes/WeaveClient.mdx +++ b/weave/reference/typescript-sdk/classes/weaveclient.mdx @@ -58,7 +58,7 @@ description: "TypeScript SDK reference" #### Defined in -[weaveClient.ts:92](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L92) +[weaveClient.ts:92](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L92) ## Properties @@ -68,7 +68,7 @@ description: "TypeScript SDK reference" #### Defined in -[weaveClient.ts:95](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L95) +[weaveClient.ts:95](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L95) ___ @@ -78,7 +78,7 @@ ___ #### Defined in -[weaveClient.ts:96](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L96) +[weaveClient.ts:96](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L96) ___ @@ -88,7 +88,7 @@ ___ #### Defined in -[weaveClient.ts:93](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L93) +[weaveClient.ts:93](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L93) ## Methods @@ -114,7 +114,7 @@ Used in imperative evaluation to attach scorer results to predict calls. #### Defined in -[weaveClient.ts:841](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L841) +[weaveClient.ts:847](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L847) ___ @@ -143,7 +143,7 @@ ___ #### Defined in -[weaveClient.ts:701](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L701) +[weaveClient.ts:707](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L707) ___ @@ -169,7 +169,7 @@ ___ #### Defined in -[weaveClient.ts:748](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L748) +[weaveClient.ts:754](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L754) ___ @@ -194,7 +194,7 @@ ___ #### Defined in -[weaveClient.ts:788](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L788) +[weaveClient.ts:794](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L794) ___ @@ -206,7 +206,7 @@ ___ | Name | Type | | :------ | :------ | -| `ref` | `ObjectRef` | +| `ref` | [`ObjectRef`](./ObjectRef) | #### Returns @@ -214,7 +214,7 @@ ___ #### Defined in -[weaveClient.ts:281](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L281) +[weaveClient.ts:281](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L281) ___ @@ -235,7 +235,7 @@ ___ #### Defined in -[weaveClient.ts:212](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L212) +[weaveClient.ts:212](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L212) ___ @@ -249,7 +249,7 @@ ___ #### Defined in -[weaveClient.ts:629](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L629) +[weaveClient.ts:635](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L635) ___ @@ -271,7 +271,7 @@ ___ #### Defined in -[weaveClient.ts:222](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L222) +[weaveClient.ts:222](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L222) ___ @@ -293,13 +293,13 @@ ___ #### Defined in -[weaveClient.ts:236](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L236) +[weaveClient.ts:236](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L236) ___ ### publish -▸ **publish**(`obj`, `objId?`): `Promise`\<`ObjectRef`\> +▸ **publish**(`obj`, `objId?`): `Promise`\<[`ObjectRef`](./ObjectRef)\> #### Parameters @@ -310,11 +310,11 @@ ___ #### Returns -`Promise`\<`ObjectRef`\> +`Promise`\<[`ObjectRef`](./ObjectRef)\> #### Defined in -[weaveClient.ts:200](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L200) +[weaveClient.ts:200](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L200) ___ @@ -334,7 +334,7 @@ ___ #### Defined in -[weaveClient.ts:633](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L633) +[weaveClient.ts:639](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L639) ___ @@ -361,7 +361,7 @@ ___ #### Defined in -[weaveClient.ts:637](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L637) +[weaveClient.ts:643](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L643) ___ @@ -382,7 +382,7 @@ ___ #### Defined in -[weaveClient.ts:667](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L667) +[weaveClient.ts:673](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L673) ___ @@ -403,7 +403,7 @@ ___ #### Defined in -[weaveClient.ts:824](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L824) +[weaveClient.ts:830](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L830) ___ @@ -417,4 +417,4 @@ ___ #### Defined in -[weaveClient.ts:113](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveClient.ts#L113) \ No newline at end of file +[weaveClient.ts:113](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveClient.ts#L113) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/classes/WeaveObject.mdx b/weave/reference/typescript-sdk/classes/weaveobject.mdx similarity index 61% rename from weave/reference/typescript-sdk/classes/WeaveObject.mdx rename to weave/reference/typescript-sdk/classes/weaveobject.mdx index 25c084e292..b9bcfc4641 100644 --- a/weave/reference/typescript-sdk/classes/WeaveObject.mdx +++ b/weave/reference/typescript-sdk/classes/weaveobject.mdx @@ -52,17 +52,17 @@ description: "TypeScript SDK reference" #### Defined in -[weaveObject.ts:51](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L51) +[weaveObject.ts:51](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L51) ## Properties ### \_\_savedRef -• `Optional` **\_\_savedRef**: `ObjectRef` \| `Promise`\<`ObjectRef`\> +• `Optional` **\_\_savedRef**: [`ObjectRef`](./ObjectRef) \| `Promise`\<[`ObjectRef`](./ObjectRef)\> #### Defined in -[weaveObject.ts:49](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L49) +[weaveObject.ts:49](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L49) ## Accessors @@ -76,7 +76,7 @@ description: "TypeScript SDK reference" #### Defined in -[weaveObject.ts:76](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L76) +[weaveObject.ts:76](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L76) ___ @@ -90,7 +90,7 @@ ___ #### Defined in -[weaveObject.ts:72](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L72) +[weaveObject.ts:72](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L72) ## Methods @@ -104,4 +104,4 @@ ___ #### Defined in -[weaveObject.ts:53](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/weaveObject.ts#L53) \ No newline at end of file +[weaveObject.ts:53](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/weaveObject.ts#L53) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/functions/requireCurrentCallStackEntry.mdx b/weave/reference/typescript-sdk/functions/requirecurrentcallstackentry.mdx similarity index 100% rename from weave/reference/typescript-sdk/functions/requireCurrentCallStackEntry.mdx rename to weave/reference/typescript-sdk/functions/requirecurrentcallstackentry.mdx diff --git a/weave/reference/typescript-sdk/functions/requireCurrentChildSummary.mdx b/weave/reference/typescript-sdk/functions/requirecurrentchildsummary.mdx similarity index 100% rename from weave/reference/typescript-sdk/functions/requireCurrentChildSummary.mdx rename to weave/reference/typescript-sdk/functions/requirecurrentchildsummary.mdx diff --git a/weave/reference/typescript-sdk/functions/weaveAudio.mdx b/weave/reference/typescript-sdk/functions/weaveaudio.mdx similarity index 100% rename from weave/reference/typescript-sdk/functions/weaveAudio.mdx rename to weave/reference/typescript-sdk/functions/weaveaudio.mdx diff --git a/weave/reference/typescript-sdk/functions/weaveImage.mdx b/weave/reference/typescript-sdk/functions/weaveimage.mdx similarity index 100% rename from weave/reference/typescript-sdk/functions/weaveImage.mdx rename to weave/reference/typescript-sdk/functions/weaveimage.mdx diff --git a/weave/reference/typescript-sdk/functions/wrapOpenAI.mdx b/weave/reference/typescript-sdk/functions/wrapopenai.mdx similarity index 100% rename from weave/reference/typescript-sdk/functions/wrapOpenAI.mdx rename to weave/reference/typescript-sdk/functions/wrapopenai.mdx diff --git a/weave/reference/typescript-sdk/interfaces/CallSchema.mdx b/weave/reference/typescript-sdk/interfaces/callschema.mdx similarity index 78% rename from weave/reference/typescript-sdk/interfaces/CallSchema.mdx rename to weave/reference/typescript-sdk/interfaces/callschema.mdx index 4c66792c21..e354ddfe06 100644 --- a/weave/reference/typescript-sdk/interfaces/CallSchema.mdx +++ b/weave/reference/typescript-sdk/interfaces/callschema.mdx @@ -40,7 +40,7 @@ Attributes #### Defined in -[generated/traceServerApi.ts:118](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L118) +[generated/traceServerApi.ts:118](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L118) ___ @@ -52,7 +52,7 @@ Deleted At #### Defined in -[generated/traceServerApi.ts:133](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L133) +[generated/traceServerApi.ts:133](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L133) ___ @@ -64,7 +64,7 @@ Display Name #### Defined in -[generated/traceServerApi.ts:107](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L107) +[generated/traceServerApi.ts:107](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L107) ___ @@ -76,7 +76,7 @@ Ended At #### Defined in -[generated/traceServerApi.ts:122](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L122) +[generated/traceServerApi.ts:122](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L122) ___ @@ -88,7 +88,7 @@ Exception #### Defined in -[generated/traceServerApi.ts:124](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L124) +[generated/traceServerApi.ts:124](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L124) ___ @@ -100,7 +100,7 @@ Id #### Defined in -[generated/traceServerApi.ts:101](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L101) +[generated/traceServerApi.ts:101](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L101) ___ @@ -112,7 +112,7 @@ Inputs #### Defined in -[generated/traceServerApi.ts:120](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L120) +[generated/traceServerApi.ts:120](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L120) ___ @@ -124,7 +124,7 @@ Op Name #### Defined in -[generated/traceServerApi.ts:105](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L105) +[generated/traceServerApi.ts:105](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L105) ___ @@ -136,7 +136,7 @@ Output #### Defined in -[generated/traceServerApi.ts:126](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L126) +[generated/traceServerApi.ts:126](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L126) ___ @@ -148,7 +148,7 @@ Parent Id #### Defined in -[generated/traceServerApi.ts:111](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L111) +[generated/traceServerApi.ts:111](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L111) ___ @@ -160,7 +160,7 @@ Project Id #### Defined in -[generated/traceServerApi.ts:103](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L103) +[generated/traceServerApi.ts:103](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L103) ___ @@ -176,7 +176,7 @@ date-time #### Defined in -[generated/traceServerApi.ts:116](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L116) +[generated/traceServerApi.ts:116](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L116) ___ @@ -186,7 +186,7 @@ ___ #### Defined in -[generated/traceServerApi.ts:127](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L127) +[generated/traceServerApi.ts:127](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L127) ___ @@ -198,7 +198,7 @@ Trace Id #### Defined in -[generated/traceServerApi.ts:109](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L109) +[generated/traceServerApi.ts:109](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L109) ___ @@ -210,7 +210,7 @@ Wb Run Id #### Defined in -[generated/traceServerApi.ts:131](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L131) +[generated/traceServerApi.ts:131](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L131) ___ @@ -222,4 +222,4 @@ Wb User Id #### Defined in -[generated/traceServerApi.ts:129](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L129) \ No newline at end of file +[generated/traceServerApi.ts:129](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L129) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/interfaces/CallsFilter.mdx b/weave/reference/typescript-sdk/interfaces/callsfilter.mdx similarity index 80% rename from weave/reference/typescript-sdk/interfaces/CallsFilter.mdx rename to weave/reference/typescript-sdk/interfaces/callsfilter.mdx index 66fb8eb535..f27863205f 100644 --- a/weave/reference/typescript-sdk/interfaces/CallsFilter.mdx +++ b/weave/reference/typescript-sdk/interfaces/callsfilter.mdx @@ -33,7 +33,7 @@ Call Ids #### Defined in -[generated/traceServerApi.ts:196](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L196) +[generated/traceServerApi.ts:196](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L196) ___ @@ -45,7 +45,7 @@ Input Refs #### Defined in -[generated/traceServerApi.ts:188](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L188) +[generated/traceServerApi.ts:188](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L188) ___ @@ -57,7 +57,7 @@ Op Names #### Defined in -[generated/traceServerApi.ts:186](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L186) +[generated/traceServerApi.ts:186](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L186) ___ @@ -69,7 +69,7 @@ Output Refs #### Defined in -[generated/traceServerApi.ts:190](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L190) +[generated/traceServerApi.ts:190](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L190) ___ @@ -81,7 +81,7 @@ Parent Ids #### Defined in -[generated/traceServerApi.ts:192](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L192) +[generated/traceServerApi.ts:192](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L192) ___ @@ -93,7 +93,7 @@ Trace Ids #### Defined in -[generated/traceServerApi.ts:194](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L194) +[generated/traceServerApi.ts:194](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L194) ___ @@ -105,7 +105,7 @@ Trace Roots Only #### Defined in -[generated/traceServerApi.ts:198](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L198) +[generated/traceServerApi.ts:198](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L198) ___ @@ -117,7 +117,7 @@ Wb Run Ids #### Defined in -[generated/traceServerApi.ts:202](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L202) +[generated/traceServerApi.ts:202](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L202) ___ @@ -129,4 +129,4 @@ Wb User Ids #### Defined in -[generated/traceServerApi.ts:200](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/generated/traceServerApi.ts#L200) \ No newline at end of file +[generated/traceServerApi.ts:200](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/generated/traceServerApi.ts#L200) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/interfaces/WeaveAudio.mdx b/weave/reference/typescript-sdk/interfaces/weaveaudio.mdx similarity index 63% rename from weave/reference/typescript-sdk/interfaces/WeaveAudio.mdx rename to weave/reference/typescript-sdk/interfaces/weaveaudio.mdx index f1d1aa497e..5a3bf0b3b5 100644 --- a/weave/reference/typescript-sdk/interfaces/WeaveAudio.mdx +++ b/weave/reference/typescript-sdk/interfaces/weaveaudio.mdx @@ -29,7 +29,7 @@ description: "TypeScript SDK reference" #### Defined in -[media.ts:48](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/media.ts#L48) +[media.ts:48](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/media.ts#L48) ___ @@ -43,7 +43,7 @@ WeaveAudioInput.audioType #### Defined in -[media.ts:44](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/media.ts#L44) +[media.ts:44](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/media.ts#L44) ___ @@ -57,4 +57,4 @@ WeaveAudioInput.data #### Defined in -[media.ts:43](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/media.ts#L43) \ No newline at end of file +[media.ts:43](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/media.ts#L43) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/interfaces/WeaveImage.mdx b/weave/reference/typescript-sdk/interfaces/weaveimage.mdx similarity index 63% rename from weave/reference/typescript-sdk/interfaces/WeaveImage.mdx rename to weave/reference/typescript-sdk/interfaces/weaveimage.mdx index db986c371d..5d3e890dc6 100644 --- a/weave/reference/typescript-sdk/interfaces/WeaveImage.mdx +++ b/weave/reference/typescript-sdk/interfaces/weaveimage.mdx @@ -29,7 +29,7 @@ description: "TypeScript SDK reference" #### Defined in -[media.ts:14](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/media.ts#L14) +[media.ts:14](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/media.ts#L14) ___ @@ -43,7 +43,7 @@ WeaveImageInput.data #### Defined in -[media.ts:9](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/media.ts#L9) +[media.ts:9](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/media.ts#L9) ___ @@ -57,4 +57,4 @@ WeaveImageInput.imageType #### Defined in -[media.ts:10](https://github.com/wandb/weave/blob/5d9590d2658eeb69c9c9f04342e107bf9a0a4c90/sdks/node/src/media.ts#L10) \ No newline at end of file +[media.ts:10](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/media.ts#L10) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/readme.mdx b/weave/reference/typescript-sdk/readme.mdx new file mode 100644 index 0000000000..8017bc0005 --- /dev/null +++ b/weave/reference/typescript-sdk/readme.mdx @@ -0,0 +1,379 @@ +--- +title: "weave" +description: "TypeScript SDK reference" +--- + +weave + + + +## Table of contents + +### Classes + +- [Dataset](./typescript-sdk/classes/Dataset) +- [Evaluation](./typescript-sdk/classes/Evaluation) +- [EvaluationLogger](./typescript-sdk/classes/EvaluationLogger) +- [MessagesPrompt](./typescript-sdk/classes/MessagesPrompt) +- [ObjectRef](./typescript-sdk/classes/ObjectRef) +- [ScoreLogger](./typescript-sdk/classes/ScoreLogger) +- [StringPrompt](./typescript-sdk/classes/StringPrompt) +- [WeaveClient](./typescript-sdk/classes/WeaveClient) +- [WeaveObject](./typescript-sdk/classes/WeaveObject) + +### Interfaces + +- [CallSchema](./typescript-sdk/interfaces/CallSchema) +- [CallsFilter](./typescript-sdk/interfaces/CallsFilter) +- [WeaveAudio](./typescript-sdk/interfaces/WeaveAudio) +- [WeaveImage](./typescript-sdk/interfaces/WeaveImage) + +### Type Aliases + +- [Op](typescript-sdk#op) +- [OpDecorator](typescript-sdk#opdecorator) + +### Functions + +- [init](typescript-sdk#init) +- [login](typescript-sdk#login) +- [op](typescript-sdk#op-1) +- [requireCurrentCallStackEntry](typescript-sdk#requirecurrentcallstackentry) +- [requireCurrentChildSummary](typescript-sdk#requirecurrentchildsummary) +- [weaveAudio](typescript-sdk#weaveaudio) +- [weaveImage](typescript-sdk#weaveimage) +- [wrapOpenAI](typescript-sdk#wrapopenai) + +## Type Aliases + +### Op + +Ƭ **Op**\<`T`\>: \{ `__boundThis?`: [`WeaveObject`](./typescript-sdk/classes/WeaveObject) ; `__isOp`: ``true`` ; `__name`: `string` ; `__parameterNames?`: `ParameterNamesOption` ; `__savedRef?`: `OpRef` \| `Promise`\<`OpRef`\> ; `__wrappedFunction`: `T` ; `invoke`: `CallMethod`\<`T`\> } & `T` & (...`args`: `Parameters`\<`T`\>) => `ReturnType`\<`T`\> extends `AsyncIterable`\ ? `AsyncIterable`\<`Awaited`\<`U`\>\> : `Promise`\<`Awaited`\<`ReturnType`\<`T`\>\>\> + +#### Type parameters + +| Name | Type | +| :------ | :------ | +| `T` | extends (...`args`: `any`[]) => `any` | + +#### Defined in + +[opType.ts:7](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/opType.ts#L7) + +___ + +### OpDecorator + +Ƭ **OpDecorator**\<`T`\>: (`value`: `T`, `context`: `ClassMethodDecoratorContext`) => `T` \| `void` & (`target`: `Object`, `propertyKey`: `string` \| `symbol`, `descriptor`: `TypedPropertyDescriptor`\<`T`\>) => `TypedPropertyDescriptor`\<`T`\> \| `void` + +Helper type for decorators +This represents a decorator function that can be used with both legacy and Stage 3 decorators. + +For Stage 3 decorators: + target: The function being decorated (T) + context: MethodDecoratorContext + +For legacy decorators: + target: The prototype (instance methods) or constructor (static methods) + propertyKey: The method name + descriptor: The property descriptor containing the method + +#### Type parameters + +| Name | Type | +| :------ | :------ | +| `T` | extends (...`args`: `any`[]) => `any` | + +#### Defined in + +[opType.ts:41](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/opType.ts#L41) + +## Functions + +### init + +▸ **init**(`project`, `settings?`): `Promise`\<[`WeaveClient`](./typescript-sdk/classes/WeaveClient)\> + +Initialize the Weave client, which is required for weave tracing to work. + +#### Parameters + +| Name | Type | Description | +| :------ | :------ | :------ | +| `project` | `string` | The W&B project name (can be project or entity/project). If you don't specify a W&B team (e.g., 'team/project'), your default entity is used. To find or update your default entity, refer to User Settings at https://docs.wandb.ai/guides/models/app/settings-page/user-settings/#default-team | +| `settings?` | `Settings` | (Optional) Weave tracing settings | + +#### Returns + +`Promise`\<[`WeaveClient`](./typescript-sdk/classes/WeaveClient)\> + +A promise that resolves to the initialized Weave client. + +`Throws` + +If the initialization fails + +#### Defined in + +[clientApi.ts:83](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/clientApi.ts#L83) + +___ + +### login + +▸ **login**(`apiKey`, `host?`): `Promise`\<`void`\> + +Log in to Weights & Biases (W&B) using the provided API key. +This function attempts to save the credentials to your netrc file for future use, +but will continue even if it cannot write to the file system. + +#### Parameters + +| Name | Type | Description | +| :------ | :------ | :------ | +| `apiKey` | `string` | Your W&B API key. | +| `host?` | `string` | (Optional) The host name (usually only needed if you're using a custom W&B server). | + +#### Returns + +`Promise`\<`void`\> + +`Throws` + +If the API key is not specified or if the connection to the weave trace server cannot be verified. + +#### Defined in + +[clientApi.ts:23](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/clientApi.ts#L23) + +___ + +### op + +▸ **op**\<`T`\>(`fn`, `options?`): [`Op`](typescript-sdk#op)\<`T`\> + +#### Type parameters + +| Name | Type | +| :------ | :------ | +| `T` | extends (...`args`: `any`[]) => `any` | + +#### Parameters + +| Name | Type | +| :------ | :------ | +| `fn` | `T` | +| `options?` | `OpOptions`\<`T`\> | + +#### Returns + +[`Op`](typescript-sdk#op)\<`T`\> + +#### Defined in + +[op.ts:369](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/op.ts#L369) + +▸ **op**\<`T`\>(`thisArg`, `fn`, `options?`): [`Op`](typescript-sdk#op)\<`T`\> + +#### Type parameters + +| Name | Type | +| :------ | :------ | +| `T` | extends (...`args`: `any`[]) => `any` | + +#### Parameters + +| Name | Type | +| :------ | :------ | +| `thisArg` | `any` | +| `fn` | `T` | +| `options?` | `OpOptions`\<`T`\> | + +#### Returns + +[`Op`](typescript-sdk#op)\<`T`\> + +#### Defined in + +[op.ts:374](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/op.ts#L374) + +▸ **op**(`target`, `propertyKey`, `descriptor`): `TypedPropertyDescriptor`\<`any`\> + +#### Parameters + +| Name | Type | +| :------ | :------ | +| `target` | `Object` | +| `propertyKey` | `string` \| `symbol` | +| `descriptor` | `TypedPropertyDescriptor`\<`any`\> | + +#### Returns + +`TypedPropertyDescriptor`\<`any`\> + +#### Defined in + +[op.ts:380](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/op.ts#L380) + +▸ **op**\<`T`\>(`value`, `context`): [`Op`](typescript-sdk#op)\<`T`\> + +#### Type parameters + +| Name | Type | +| :------ | :------ | +| `T` | extends (...`args`: `any`[]) => `any` | + +#### Parameters + +| Name | Type | +| :------ | :------ | +| `value` | `T` | +| `context` | `MethodDecoratorContext` | + +#### Returns + +[`Op`](typescript-sdk#op)\<`T`\> + +#### Defined in + +[op.ts:386](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/op.ts#L386) + +▸ **op**(`options`): `MethodDecorator` + +#### Parameters + +| Name | Type | +| :------ | :------ | +| `options` | `Partial`\<`OpOptions`\<`any`\>\> | + +#### Returns + +`MethodDecorator` + +#### Defined in + +[op.ts:391](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/op.ts#L391) + +___ + +### requireCurrentCallStackEntry + +▸ **requireCurrentCallStackEntry**(): `CallStackEntry` + +#### Returns + +`CallStackEntry` + +#### Defined in + +[clientApi.ts:145](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/clientApi.ts#L145) + +___ + +### requireCurrentChildSummary + +▸ **requireCurrentChildSummary**(): `Object` + +#### Returns + +`Object` + +#### Defined in + +[clientApi.ts:157](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/clientApi.ts#L157) + +___ + +### weaveAudio + +▸ **weaveAudio**(`options`): [`WeaveAudio`](./typescript-sdk/interfaces/WeaveAudio) + +Create a new WeaveAudio object + +#### Parameters + +| Name | Type | Description | +| :------ | :------ | :------ | +| `options` | `WeaveAudioInput` | The options for this media type - data: The raw audio data as a Buffer - audioType: (Optional) The type of audio file, currently only 'wav' is supported | + +#### Returns + +[`WeaveAudio`](./typescript-sdk/interfaces/WeaveAudio) + +`Example` + +```ts +const audioBuffer = fs.readFileSync('path/to/audio.wav'); +const weaveAudio = weaveAudio({ data: audioBuffer }); +``` + +#### Defined in + +[media.ts:62](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/media.ts#L62) + +___ + +### weaveImage + +▸ **weaveImage**(`options`): [`WeaveImage`](./typescript-sdk/interfaces/WeaveImage) + +Create a new WeaveImage object + +#### Parameters + +| Name | Type | Description | +| :------ | :------ | :------ | +| `options` | `WeaveImageInput` | The options for this media type - data: The raw image data as a Buffer - imageType: (Optional) The type of image file, currently only 'png' is supported | + +#### Returns + +[`WeaveImage`](./typescript-sdk/interfaces/WeaveImage) + +`Example` + +```ts +const imageBuffer = fs.readFileSync('path/to/image.png'); +const weaveImage = weaveImage({ data: imageBuffer }); +``` + +#### Defined in + +[media.ts:28](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/media.ts#L28) + +___ + +### wrapOpenAI + +▸ **wrapOpenAI**\<`T`\>(`openai`): `T` + +Wraps the OpenAI API to enable function tracing for OpenAI calls. + +#### Type parameters + +| Name | Type | +| :------ | :------ | +| `T` | extends `OpenAIAPI` | + +#### Parameters + +| Name | Type | +| :------ | :------ | +| `openai` | `T` | + +#### Returns + +`T` + +`Example` + +```ts +const openai = wrapOpenAI(new OpenAI()); +const result = await openai.chat.completions.create({ + model: 'gpt-3.5-turbo', + messages: [{ role: 'user', content: 'Hello, world!' }] +}); +``` + +#### Defined in + +[integrations/openai.ts:469](https://github.com/wandb/weave/blob/01ba25415eecdd98ace095491cc3bdfc7f4ba384/sdks/node/src/integrations/openai.ts#L469) \ No newline at end of file diff --git a/weave/reference/typescript-sdk/type-aliases/Op.mdx b/weave/reference/typescript-sdk/type-aliases/op.mdx similarity index 100% rename from weave/reference/typescript-sdk/type-aliases/Op.mdx rename to weave/reference/typescript-sdk/type-aliases/op.mdx diff --git a/weave/reference/typescript-sdk/type-aliases/OpDecorator.mdx b/weave/reference/typescript-sdk/type-aliases/opdecorator.mdx similarity index 100% rename from weave/reference/typescript-sdk/type-aliases/OpDecorator.mdx rename to weave/reference/typescript-sdk/type-aliases/opdecorator.mdx