From b9919514831244f17bc323e4d9707211f5fe4adf Mon Sep 17 00:00:00 2001 From: Archan Datta Date: Thu, 8 Jan 2026 15:40:27 +0000 Subject: [PATCH 1/7] feat: add integration doc for web-bot-auth --- docs.json | 3 +- integrations/web-bot-auth.mdx | 110 ++++++++++++++++++++++++++++++++++ 2 files changed, 112 insertions(+), 1 deletion(-) create mode 100644 integrations/web-bot-auth.mdx diff --git a/docs.json b/docs.json index dedd1b8..c66c424 100644 --- a/docs.json +++ b/docs.json @@ -136,7 +136,8 @@ "integrations/vercel/ai-sdk", "integrations/vercel/marketplace" ] - } + }, + "integrations/web-bot-auth" ] }, { diff --git a/integrations/web-bot-auth.mdx b/integrations/web-bot-auth.mdx new file mode 100644 index 0000000..be0192e --- /dev/null +++ b/integrations/web-bot-auth.mdx @@ -0,0 +1,110 @@ +--- +title: "Web Bot Auth" +--- + +Cloudflare's [web-bot-auth](https://github.com/cloudflare/web-bot-auth) extension adds HTTP Message Signatures to browser requests, providing cryptographic proof of browser authenticity. This is particularly useful for bypassing bot detection systems that support this authentication method. + +## Quick Start + +**1. Prepare the extension:** + +```bash +kernel extensions prepare-web-bot-auth --output ./my-web-bot-auth +``` + +Output: +``` +✓ Web-bot-auth extension prepared successfully! + +Extension ID: +Output directory: ./my-web-bot-auth +``` + +**3. Upload the extension to Kernel:** + +```bash +kernel extensions upload ./my-web-bot-auth --name +``` + + +Use the extension ID as the name to ensure consistency with the generated policy files. + + +**3. Use in a browsers:** + + +```typescript Typescript/Javascript +import { Kernel } from '@onkernel/sdk'; + +const kernel = new Kernel(); +const browser = await kernel.browsers.create({ + extensions: [{ name: "" }], +}); + +// The extension is now active and signing requests +console.log('Browser created:', browser.session_id); +``` + +```python Python +from kernel import Kernel + +kernel = Kernel() +browser = kernel.browsers.create( + extensions=[{"name": ""}] +) + +# The extension is now active and signing requests +print(f"Browser created: {browser.session_id}") +``` + +```bash CLI +kernel browsers create --extension +``` + + + +The `` is different than the name. Extension ID can be retrieved from `kernel extensions list` command. + + +## Configuration Options + +The `prepare-web-bot-auth` command options: + +| Flag | Default | Description | +|------|---------|-------------| +| `--output` | `./web-bot-auth` | Output directory for the prepared extension | +| `--url` | `http://127.0.0.1:10001` | Base URL for extension updates | + +## Understanding the Build Process + +### Extension signing + +When you run `prepare-web-bot-auth`, it generates a `private_key.pem` file. This key: +- Determines the extension's unique ID +- Must be kept secure +- Should be reused for updates to maintain the same extension ID + + +Keep `private_key.pem` secure. Anyone with this key can create extensions with the same ID. + + +### What's in the output directory + +The generated `my-web-bot-auth` directory contains: + +``` +my-web-bot-auth +├── manifest.json # Extension manifest +├── background.mjs # Extension code +├── update.xml # Update manifest for Chrome +├── http-message-signatures-extension.crx # Signed extension package +└── private_key.pem # Private key not uploaded to Kernel +``` + +## References + +- [Cloudflare web-bot-auth Repository](https://github.com/cloudflare/web-bot-auth) +- [HTTP Message Signatures RFC 9421](https://datatracker.ietf.org/doc/html/rfc9421) +- [Chrome Enterprise Policy Documentation](https://chromeenterprise.google/policies/) +- [Chrome webRequest API](https://developer.chrome.com/docs/extensions/reference/api/webRequest) +- [Kernel Extensions Documentation](/browsers/extensions) From 47c3f62afa15655b1d6eefd44880cabd528ad346 Mon Sep 17 00:00:00 2001 From: Archan Datta Date: Thu, 8 Jan 2026 16:55:17 +0000 Subject: [PATCH 2/7] feat: add custom ent extension docs --- browsers/extensions.mdx | 51 +++++++++++++++++++++++++++++++++++++++ integrations/overview.mdx | 3 ++- 2 files changed, 53 insertions(+), 1 deletion(-) diff --git a/browsers/extensions.mdx b/browsers/extensions.mdx index 9b3189b..55457da 100644 --- a/browsers/extensions.mdx +++ b/browsers/extensions.mdx @@ -97,3 +97,54 @@ kernel browsers extensions upload ./my-extension Note that this will restart the browser process and break any connections to the browser CDP URL. + +## Extensions requiring enterprise policies + +For a complete list of available extension settings and policies, refer to the [Chrome Enterprise Policy documentation](https://chromeenterprise.google/policies/extension-settings/). + +### Uploading extensions requiring enterprise policies + +Some Chrome extensions require elevated permissions that Chrome will only grant when the extension is installed via enterprise policies. These extensions cannot be loaded with the standard `--load-extension` flag and require special handling. + +### What are enterprise policy extensions? + +Extensions that require enterprise policies typically: + +- Use permissions like `webRequestBlocking` or `webRequest` with blocking capabilities +- Need to intercept and modify network requests before they're sent +- Require installation via Chrome's `ExtensionInstallForcelist` policy + +Common examples include extensions for network filtering, request signing, or advanced content modification. + +### Required files for upload + +When uploading an extension that requires enterprise policies to Kernel, your extension directory or zip file must include: + +1. **Extension source files** - Your `manifest.json` and all extension code (background scripts, content scripts, etc.) +2. **`update.xml`** - A Chrome update manifest that points to the `.crx` file location +3. **`.crx` file** - The signed and packed extension file + + +The `.crx` file and `update.xml` are required for Kernel to serve the extension via Chrome's `ExtensionInstallForcelist` policy. If you're deploying extensions from the Chrome Web Store via `ExtensionInstallForcelist`, these files are optional since Chrome uses the Web Store's default update URL. + + +### Automatic detection and validation + +Kernel automatically detects extensions that require enterprise policies by analyzing the `manifest.json` file during upload. No manual configuration is needed. + +**Detection process:** +1. You upload an extension via CLI or API +2. Kernel scans the `manifest.json` for permissions like `webRequestBlocking` +3. If enterprise policies are required, Kernel validates the required files are present + +### How it works + +Once you successfully upload an enterprise policy extension, Kernel handles the rest automatically: + +1. **Upload** - You upload your extension with all required files +2. **Detection** - Kernel detects the enterprise policy requirement from the manifest +3. **Policy configuration** - Extension is automatically added to `ExtensionInstallForcelist` +4. **File serving** - The kernel-images server serves update files at `http://127.0.0.1:10001/extensions/{extension-id}/update.xml` +5. **Installation** - Chrome installs the extension via enterprise policy when the browser starts + +No additional HTTP server or manual policy configuration is needed. The extension works seamlessly in any browser session that its uploaded to. diff --git a/integrations/overview.mdx b/integrations/overview.mdx index 398a349..6db2a01 100644 --- a/integrations/overview.mdx +++ b/integrations/overview.mdx @@ -28,7 +28,8 @@ Kernel provides detailed guides for popular agent frameworks: - **[Notte](/integrations/notte)** - AI agent framework for browser automation - **[Val Town](/integrations/valtown)** - Serverless function runtime - **[Vercel](https://github.com/onkernel/vercel-template)** - Deploy browser automations to Vercel +- **[Web Bot Authentication](/integrations/web-bot-auth)** - Create signed Chrome extensions for web bot authentication ## Custom Integrations -Kernel works with any tool that supports CDP. Check out our [browser creation guide](/browsers/create-a-browser) to learn how to connect any other agent framework. \ No newline at end of file +Kernel works with any tool that supports CDP. Check out our [browser creation guide](/browsers/create-a-browser) to learn how to connect any other agent framework. From 9360d67065e383d5b823613b5372e103b14fe29a Mon Sep 17 00:00:00 2001 From: Archan Datta Date: Fri, 9 Jan 2026 13:07:23 +0000 Subject: [PATCH 3/7] review: typos --- browsers/extensions.mdx | 2 +- integrations/web-bot-auth.mdx | 4 +- reference/cli/extensions.mdx | 218 ++++++++++++++++++ testing/profile-loading-performance.md | 298 +++++++++++++++++++++++++ 4 files changed, 519 insertions(+), 3 deletions(-) create mode 100644 reference/cli/extensions.mdx create mode 100644 testing/profile-loading-performance.md diff --git a/browsers/extensions.mdx b/browsers/extensions.mdx index 55457da..e85c9a8 100644 --- a/browsers/extensions.mdx +++ b/browsers/extensions.mdx @@ -147,4 +147,4 @@ Once you successfully upload an enterprise policy extension, Kernel handles the 4. **File serving** - The kernel-images server serves update files at `http://127.0.0.1:10001/extensions/{extension-id}/update.xml` 5. **Installation** - Chrome installs the extension via enterprise policy when the browser starts -No additional HTTP server or manual policy configuration is needed. The extension works seamlessly in any browser session that its uploaded to. +No additional HTTP server or manual policy configuration is needed. The extension works seamlessly in any browser session that it's uploaded to. diff --git a/integrations/web-bot-auth.mdx b/integrations/web-bot-auth.mdx index be0192e..4b96731 100644 --- a/integrations/web-bot-auth.mdx +++ b/integrations/web-bot-auth.mdx @@ -20,7 +20,7 @@ Extension ID: Output directory: ./my-web-bot-auth ``` -**3. Upload the extension to Kernel:** +**2. Upload the extension to Kernel:** ```bash kernel extensions upload ./my-web-bot-auth --name @@ -30,7 +30,7 @@ kernel extensions upload ./my-web-bot-auth --name Use the extension ID as the name to ensure consistency with the generated policy files. -**3. Use in a browsers:** +**3. Use in a browser:** ```typescript Typescript/Javascript diff --git a/reference/cli/extensions.mdx b/reference/cli/extensions.mdx new file mode 100644 index 0000000..b0e2cc3 --- /dev/null +++ b/reference/cli/extensions.mdx @@ -0,0 +1,218 @@ +--- +title: "Extensions" +--- + +Manage browser extensions for use with Kernel browsers. + +## Extension Management + +### `kernel extensions list` + +List all extensions in your organization. + +**Output includes:** +- Extension ID +- Name (if assigned) +- Created date +- Size in bytes +- Last used date + +**Example:** + +```bash +kernel extensions list +``` + +### `kernel extensions upload ` + +Upload an unpacked browser extension directory or zip file. + +| Flag | Description | +|------|-------------| +| `--name ` | Optional unique name for the extension. Must be unique within your organization. | + +**Example:** + +```bash +# Upload with auto-generated ID +kernel extensions upload ./my-extension + +# Upload with custom name +kernel extensions upload ./my-extension --name my-custom-extension +``` + + +Extensions requiring enterprise policies must include `update.xml` and `.crx` files. See [Extensions requiring enterprise policies](/browsers/extensions#uploading-extensions-requiring-enterprise-policies) for details. + + +### `kernel extensions download ` + +Download an extension archive by ID or name. + +| Flag | Description | +|------|-------------| +| `--to ` | Output path for the downloaded zip file. | + +**Example:** + +```bash +# Download to current directory +kernel extensions download my-extension + +# Download to specific path +kernel extensions download my-extension --to ./downloads/extension.zip +``` + +### `kernel extensions delete ` + +Delete an extension by ID or name. + +| Flag | Description | +|------|-------------| +| `--yes`, `-y` | Skip confirmation prompt. | + +**Example:** + +```bash +# Delete with confirmation +kernel extensions delete my-extension + +# Delete without confirmation +kernel extensions delete my-extension --yes +``` + +### `kernel extensions download-web-store ` + +Download and unpack an extension directly from the Chrome Web Store. + +| Flag | Description | +|------|-------------| +| `--to ` | Output directory for the unpacked extension. | +| `--os ` | Target OS: `mac`, `win`, or `linux` (default: `linux`). | + +**Example:** + +```bash +kernel extensions download-web-store \ + https://chromewebstore.google.com/detail/extension-id \ + --to ./my-extension +``` + +After downloading, you can upload the extension to Kernel: + +```bash +kernel extensions upload ./my-extension --name my-extension +``` + +## Preparing Enterprise Extensions + +### `kernel extensions prepare-web-bot-auth` + +Download, build, and prepare the Cloudflare web-bot-auth extension with Kernel-specific configurations. This command creates a complete extension package ready to upload to Kernel. + +The web-bot-auth extension adds HTTP Message Signatures to browser requests, providing cryptographic proof of browser authenticity. It requires enterprise policies due to its use of the `webRequestBlocking` permission. + +| Flag | Default | Description | +|------|---------|-------------| +| `--output ` | `./web-bot-auth` | Output directory for the prepared extension. | +| `--url ` | `http://127.0.0.1:10001` | Base URL for extension updates and policy configuration. | +| `--version ` | `main` | GitHub branch or tag to download. | + +**Prerequisites:** +- Node.js and npm must be installed on your system +- Internet connection to download from GitHub + +**What this command does:** + +1. **Downloads** the latest web-bot-auth source from GitHub +2. **Modifies** configuration files to work with Kernel's infrastructure +3. **Builds** the extension using npm +4. **Signs** the extension with a stable extension ID using a generated private key +5. **Creates** a zip file ready to upload to Kernel + +**Output files:** + +- Extension directory with all source files +- `web-bot-auth.zip` - Ready to upload to Kernel +- `private_key.pem` - Signing key (keep secure!) +- `update.xml` - Chrome update manifest +- `.crx` file - Signed extension package +- Policy files for reference + +**Example:** + +```bash +# Prepare with defaults +kernel extensions prepare-web-bot-auth + +# Prepare with custom output directory +kernel extensions prepare-web-bot-auth --output ./my-web-bot-auth + +# Prepare specific version +kernel extensions prepare-web-bot-auth --version v1.0.0 +``` + +**Output:** + +``` +✓ Web-bot-auth extension prepared successfully! + +Extension ID: gilochgnhkffnkiiaooiedhngaiflfho +Output directory: ./web-bot-auth +Zip file: ./web-bot-auth/web-bot-auth.zip + +Next steps: +1. Upload: kernel extensions upload ./web-bot-auth/web-bot-auth.zip --name gilochgnhkffnkiiaooiedhngaiflfho +2. Use in browsers: kernel browsers create --extension gilochgnhkffnkiiaooiedhngaiflfho +``` + +**Next steps after preparation:** + +1. **Upload the extension:** + +```bash +kernel extensions upload ./web-bot-auth/web-bot-auth.zip --name +``` + + +Use the extension ID shown in the output as the name. This ensures consistency with the generated policy files. + + +2. **Use in browsers:** + + +```typescript Typescript/Javascript +import { Kernel } from '@onkernel/sdk'; + +const kernel = new Kernel(); +const browser = await kernel.browsers.create({ + extensions: [{ name: "gilochgnhkffnkiiaooiedhngaiflfho" }] +}); +``` + +```python Python +from kernel import Kernel + +kernel = Kernel() +browser = kernel.browsers.create( + extensions=[{"name": "gilochgnhkffnkiiaooiedhngaiflfho"}] +) +``` + +```bash CLI +kernel browsers create --extension gilochgnhkffnkiiaooiedhngaiflfho +``` + + + +Keep your `private_key.pem` file secure! Anyone with access to this key can create extensions with the same extension ID. Store it safely if you need to rebuild or update the extension in the future. + + +**How it works:** + +The kernel-images server automatically serves the extension's update files at `http://127.0.0.1:10001/extensions/{extension-id}/update.xml`. Chrome uses this URL to install the extension via the `ExtensionInstallForcelist` policy. No additional HTTP server or manual configuration is needed. + +**For more information:** + +- [Web Bot Authentication integration guide](/integrations/web-bot-auth) - Detailed guide with examples +- [Extensions requiring enterprise policies](/browsers/extensions#uploading-extensions-requiring-enterprise-policies) - General information about enterprise extensions diff --git a/testing/profile-loading-performance.md b/testing/profile-loading-performance.md new file mode 100644 index 0000000..ea701c6 --- /dev/null +++ b/testing/profile-loading-performance.md @@ -0,0 +1,298 @@ +# Profile Loading Performance Testing Guide + +## Overview + +This guide documents the process and results of testing profile loading performance with Google login across multiple SaaS sites. The goal is to understand if large profiles (mega-profiles) cause slow browser startup times and whether origin filtering could improve performance. + +## Background + +Some customers use Kernel to manage multiple SaaS accounts by: +1. Creating a synthetic Google account +2. Logging into Google once in a browser +3. Using "Login with Google" across many SaaS products +4. Saving all this state in a single mega-profile +5. Reusing this profile across many sessions + +This approach can result in very large profiles with hundreds of origins, thousands of cookies, and extensive localStorage/IndexedDB data. The question is: **Does this cause slow browser startup times?** + +## Test Setup + +### Prerequisites + +1. **Kernel API Key**: Set `KERNEL_API_KEY` environment variable +2. **Google Account**: A test Google account for logging in +3. **Node.js/Bun**: To run the test script +4. **Access to Live View**: To manually perform login steps + +### Running the Test + +```bash +cd kernel/packages/synthetic-tests +export KERNEL_API_KEY="your-api-key" +bun run src/profile-load-test.ts +``` + +Or with npm/pnpm: + +```bash +npm run dev src/profile-load-test.ts +# or +pnpm tsx src/profile-load-test.ts +``` + +## Test Workflow + +The automated test script performs the following steps: + +```mermaid +flowchart TD + A[Create Test Profile] --> B[Create Browser with save_changes=true] + B --> C[Output Live View URL] + C --> D[Wait for Manual Login] + D --> E[User: Login to Google] + E --> F[User: Navigate to Notion] + F --> G[User: Use Login with Google] + G --> H[User: Press Enter] + H --> I[Delete Browser to Save Profile] + I --> J[Download Profile via API] + J --> K[Analyze Profile Contents] + K --> L[Measure Browser Startup Time] + L --> M[Test Origin Filtering Hypothesis] + M --> N[Display Results] +``` + +### Manual Steps During Test + +When the test pauses, you need to: + +1. **Open the Live View URL** displayed in the terminal +2. **Navigate to Google**: Go to `https://accounts.google.com` +3. **Login**: Enter your test Google account credentials +4. **Navigate to Notion**: Go to `https://www.notion.so` +5. **Use "Login with Google"**: Click the button to authenticate +6. **Wait for Notion to load**: Ensure you're fully logged in +7. **Press Enter** in the terminal to continue + +## Metrics Collected + +### Profile Analysis + +The test analyzes the downloaded profile and reports: + +- **Total Size**: Profile JSON size in KB and bytes +- **Cookie Count**: Total number of cookies across all domains +- **Origin Count**: Number of unique origins with stored data +- **Top Origins by Cookies**: The 10 origins with the most cookies +- **Top Origins by LocalStorage**: The 10 origins with the most localStorage entries +- **Domain Breakdown**: Cookies, localStorage, and IndexedDB counts per domain + +### Timing Measurements + +The test measures browser startup time with the profile loaded: + +- **Browser Creation Time**: Time from API call to browser ready +- **Navigation Time**: Time to navigate to Notion after browser is ready +- **Total Time**: Combined time for full browser startup +- **Statistics**: Average, median, min, max across multiple iterations (default: 3) + +### Origin Filtering Analysis + +The test analyzes potential benefits of origin filtering: + +- **Google-related origins**: Count and percentage of total +- **Notion-related origins**: Count and percentage of total +- **Minimal profile size**: Size if only Google + Notion origins were loaded +- **Size reduction**: Percentage reduction from filtering + +## Interpreting Results + +### Profile Size + +- **< 100 KB**: Small profile, unlikely to cause performance issues +- **100 KB - 1 MB**: Medium profile, may have minor impact +- **1 MB - 10 MB**: Large profile, likely to impact startup time +- **> 10 MB**: Mega-profile, significant performance impact expected + +### Browser Startup Time + +- **< 5 seconds**: Fast startup, no optimization needed +- **5-10 seconds**: Moderate startup, optimization may help +- **10-30 seconds**: Slow startup, optimization recommended +- **> 30 seconds**: Very slow, optimization critical + +### Origin Filtering Potential + +If the minimal profile (Google + target site only) is significantly smaller than the full profile: + +- **> 50% reduction**: Strong candidate for origin filtering +- **25-50% reduction**: Moderate benefit from filtering +- **< 25% reduction**: Limited benefit, may not be worth complexity + +## Example Results + +### Sample Output + +``` +📊 PROFILE ANALYSIS +====================================================================== +📦 Total Size: 2,456.78 KB (2515712 bytes) +🍪 Total Cookies: 847 +🌐 Total Origins: 156 + +🔝 Top 10 Origins by Cookie Count: + 1. https://.google.com: 234 cookies + 2. https://.notion.so: 45 cookies + 3. https://.gstatic.com: 38 cookies + ... + +⏱️ TIMING STATISTICS +====================================================================== +📊 Average: 12,345.67ms +📊 Median: 12,234.00ms +📊 Min: 11,890ms +📊 Max: 13,012ms +📊 Range: 1,122ms + +🔍 ORIGIN FILTERING ANALYSIS +====================================================================== +📊 Google-related origins: 23 (14.7%) +📊 Notion-related origins: 8 (5.1%) +📊 Google-related cookies: 234 (27.6%) +📊 Notion-related cookies: 45 (5.3%) + +💡 Minimal profile size (Google + Notion only): 345.67 KB +💡 Size reduction: 85.9% (from 2,456.78 KB to 345.67 KB) +``` + +### Interpretation + +In this example: +- The profile is quite large (2.5 MB) +- Browser startup takes ~12 seconds on average +- Origin filtering could reduce size by 86% +- **Recommendation**: Implement origin filtering for this use case + +## Origin Filtering Implementation Approach + +If testing confirms that origin filtering would help, here's the recommended approach: + +### 1. Add API Parameter + +Add an optional `origins` parameter to the browser creation API: + +```typescript +await kernel.browsers.create({ + profile: { + name: 'my-profile', + origins: ['google.com', 'notion.so'], // Only load these origins + }, +}); +``` + +### 2. Filter Profile Data + +In `kernel/packages/api/lib/profiles/service_cdp.go`, modify `LoadProfileIntoSession`: + +```go +// Filter cookies by allowed origins +filteredCookies := state.Cookies +if len(allowedOrigins) > 0 { + filteredCookies = filterCookiesByOrigins(state.Cookies, allowedOrigins) +} + +// Filter origins by allowed list +filteredOrigins := state.Origins +if len(allowedOrigins) > 0 { + filteredOrigins = filterOriginsByList(state.Origins, allowedOrigins) +} +``` + +### 3. Domain Matching Logic + +Support flexible domain matching: +- `google.com` matches `.google.com`, `accounts.google.com`, etc. +- `notion.so` matches `.notion.so`, `www.notion.so`, etc. + +### 4. Performance Testing + +After implementation: +1. Run the same test with origin filtering enabled +2. Compare startup times with and without filtering +3. Verify login functionality still works +4. Document performance improvements + +## Troubleshooting + +### Profile Not Saving + +**Issue**: Profile appears empty after saving + +**Solution**: +- Ensure `save_changes: true` is set when creating the browser +- Wait at least 5 seconds after deleting the browser before downloading +- Check that you actually logged in and created session state + +### Browser Startup Timeout + +**Issue**: Browser creation times out when loading profile + +**Solution**: +- Profile may be too large (> 10 MB) +- Try with a smaller profile first +- Check Kernel logs for specific errors +- Increase timeout if necessary + +### Download Fails + +**Issue**: Profile download returns 404 or empty data + +**Solution**: +- Profile may not have any saved data yet +- Verify profile exists: `kernel.profiles.list()` +- Check that browser was deleted (not just closed) +- Wait longer for profile persistence + +## Next Steps + +After running the test: + +1. **Document your results** in this file under a "Test Results" section +2. **Share findings** with the team +3. **Decide on implementation** of origin filtering if beneficial +4. **Create follow-up tests** for other SaaS sites if needed + +## Test Results + +### Test Run: [Date] + +**Tester**: [Your Name] + +**Profile Details**: +- Size: X KB +- Cookies: X +- Origins: X + +**Timing Results**: +- Average startup: X ms +- Median startup: X ms +- Range: X ms + +**Origin Filtering Analysis**: +- Potential size reduction: X% +- Google origins: X +- Notion origins: X + +**Recommendation**: +[Your recommendation based on results] + +**Notes**: +[Any additional observations] + +--- + +## References + +- [Kernel Profiles Documentation](/browsers/profiles) +- [Profile API Reference](/api-reference/profiles/list-profiles) +- [Browser Creation API](/api-reference/browsers/create-a-browser) + From bb33244faa3e6aafe681ddfb8dc43b38c1f54d7e Mon Sep 17 00:00:00 2001 From: Archan Datta Date: Fri, 9 Jan 2026 19:11:16 +0000 Subject: [PATCH 4/7] review: update docs --- docs.json | 3 +- integrations/web-bot-auth.mdx | 182 +++++++++++++++++++++++----------- reference/cli.mdx | 3 + reference/cli/extensions.mdx | 118 +++------------------- 4 files changed, 143 insertions(+), 163 deletions(-) diff --git a/docs.json b/docs.json index c66c424..901c0e9 100644 --- a/docs.json +++ b/docs.json @@ -183,7 +183,8 @@ "reference/cli/create", "reference/cli/auth", "reference/cli/browsers", - "reference/cli/apps" + "reference/cli/apps", + "reference/cli/extensions" ] }, { diff --git a/integrations/web-bot-auth.mdx b/integrations/web-bot-auth.mdx index 4b96731..8da1d7e 100644 --- a/integrations/web-bot-auth.mdx +++ b/integrations/web-bot-auth.mdx @@ -1,110 +1,172 @@ --- title: "Web Bot Auth" +description: "Cryptographically sign browser requests with Cloudflare's Web Bot Auth" --- -Cloudflare's [web-bot-auth](https://github.com/cloudflare/web-bot-auth) extension adds HTTP Message Signatures to browser requests, providing cryptographic proof of browser authenticity. This is particularly useful for bypassing bot detection systems that support this authentication method. +[Web Bot Auth](https://github.com/cloudflare/web-bot-auth) is Cloudflare's implementation of cryptographic authentication for automated web agents. It uses [RFC 9421 HTTP Message Signatures](https://datatracker.ietf.org/doc/html/rfc9421) to sign outgoing HTTP requests, allowing websites to verify the identity and integrity of bot traffic. -## Quick Start +By integrating Web Bot Auth with Kernel, your browser automations can cryptographically prove their identity to websites that support signature verification. -**1. Prepare the extension:** +## How it works -```bash -kernel extensions prepare-web-bot-auth --output ./my-web-bot-auth -``` +Web Bot Auth works via a Chrome extension that intercepts all outgoing HTTP requests and adds cryptographic signature headers: -Output: -``` -✓ Web-bot-auth extension prepared successfully! +- **`Signature`**: The RFC 9421 signature of the request +- **`Signature-Input`**: Metadata about how the signature was created -Extension ID: -Output directory: ./my-web-bot-auth -``` +Websites can verify these signatures against your public key to confirm the request came from your authenticated agent. + +## Quick Start with Test Key -**2. Upload the extension to Kernel:** +The fastest way to get started is using Cloudflare's RFC9421 test key, which works with their [test verification site](https://http-message-signatures-example.research.cloudflare.com/). + +### 1. Build the extension + +Use the Kernel CLI to build the Web Bot Auth extension: ```bash -kernel extensions upload ./my-web-bot-auth --name +kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --upload ``` +This command: +- Downloads Cloudflare's web-bot-auth browser extension source +- Builds it with the default RFC9421 test key +- Uploads it to Kernel as `web-bot-auth` + -Use the extension ID as the name to ensure consistency with the generated policy files. +The build command requires Node.js and npm to be installed on your system. -**3. Use in a browser:** +### 2. Create a browser with the extension -```typescript Typescript/Javascript -import { Kernel } from '@onkernel/sdk'; +```bash CLI +# Create a browser with the web-bot-auth extension +kernel browsers create --extension web-bot-auth + +# The command outputs the browser ID and live view URL +# Open the live view URL in your browser, then navigate to: +# https://http-message-signatures-example.research.cloudflare.com/ +``` + +```typescript TypeScript +import { Kernel } from "@onkernel/sdk"; +import { chromium } from "playwright"; const kernel = new Kernel(); + +// Create browser with web-bot-auth extension const browser = await kernel.browsers.create({ - extensions: [{ name: "" }], + extensions: [{ name: "web-bot-auth" }], }); -// The extension is now active and signing requests -console.log('Browser created:', browser.session_id); +// Connect via Playwright +const pw = await chromium.connectOverCDP(browser.browser_url); +const context = pw.contexts()[0]; +const page = context?.pages()[0] || await context.newPage(); + +// Navigate to a page - requests will be automatically signed +await page.goto("https://http-message-signatures-example.research.cloudflare.com/"); ``` ```python Python from kernel import Kernel +from playwright.sync_api import sync_playwright kernel = Kernel() -browser = kernel.browsers.create( - extensions=[{"name": ""}] -) -# The extension is now active and signing requests -print(f"Browser created: {browser.session_id}") -``` +# Create browser with web-bot-auth extension +browser = kernel.browsers.create(extensions=[{"name": "web-bot-auth"}]) -```bash CLI -kernel browsers create --extension +# Connect via Playwright +with sync_playwright() as p: + pw = p.chromium.connect_over_cdp(browser.browser_url) + context = pw.contexts[0] + page = context.pages[0] if context.pages else context.new_page() + + # Navigate to a page - requests will be automatically signed + page.goto("https://http-message-signatures-example.research.cloudflare.com/") ``` + - -The `` is different than the name. Extension ID can be retrieved from `kernel extensions list` command. - +### 3. Verify it's working + +Navigate to Cloudflare's test site to verify your signatures are being accepted: + +``` +https://http-message-signatures-example.research.cloudflare.com/ +``` -## Configuration Options +This site validates requests signed with the RFC9421 test key and shows whether the signature was verified successfully. -The `prepare-web-bot-auth` command options: +## Using Your Own Keys -| Flag | Default | Description | -|------|---------|-------------| -| `--output` | `./web-bot-auth` | Output directory for the prepared extension | -| `--url` | `http://127.0.0.1:10001` | Base URL for extension updates | +For production use, you'll want to use your own signing keys instead of the test key. -## Understanding the Build Process +### 1. Generate an Ed25519 key pair -### Extension signing +Create a JWK file with your Ed25519 private key. The key must include both the public (`x`) and private (`d`) components: + +```json my-key.jwk +{ + "kty": "OKP", + "crv": "Ed25519", + "x": "YOUR_PUBLIC_KEY_BASE64URL", + "d": "YOUR_PRIVATE_KEY_BASE64URL" +} +``` + + +See [Cloudflare's web-bot-auth documentation](https://github.com/cloudflare/web-bot-auth) for tools to generate Ed25519 key pairs. + -When you run `prepare-web-bot-auth`, it generates a `private_key.pem` file. This key: -- Determines the extension's unique ID -- Must be kept secure -- Should be reused for updates to maintain the same extension ID +### 2. Build with your key - -Keep `private_key.pem` secure. Anyone with this key can create extensions with the same ID. - +```bash +kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --key ./my-key.jwk --upload --name my-web-bot-auth +``` -### What's in the output directory +### 3. Host your public key -The generated `my-web-bot-auth` directory contains: +For websites to verify your signatures, you need to host your public key at a well-known URL. Create a key directory at: ``` -my-web-bot-auth -├── manifest.json # Extension manifest -├── background.mjs # Extension code -├── update.xml # Update manifest for Chrome -├── http-message-signatures-extension.crx # Signed extension package -└── private_key.pem # Private key not uploaded to Kernel +https://yourdomain.com/.well-known/http-message-signatures-directory +``` + +The directory should contain your public keys in JWKS format: + +```json +{ + "keys": [ + { + "kty": "OKP", + "crv": "Ed25519", + "x": "YOUR_PUBLIC_KEY_BASE64URL", + "kid": "YOUR_KEY_ID" + } + ], + "purpose": "your-bot-purpose" +} ``` +### 4. Register with Cloudflare (optional) + +If you want Cloudflare-protected sites to recognize your bot, you can register your key directory with Cloudflare: + +1. Log into the Cloudflare dashboard +2. Navigate to **Manage Account > Configurations** +3. Select the **Bot Submission Form** tab +4. Choose **Request Signature** as the verification method +5. Enter your key directory URL + +See [Cloudflare's Web Bot Auth documentation](https://developers.cloudflare.com/bots/reference/bot-verification/web-bot-auth/) for complete details. + ## References -- [Cloudflare web-bot-auth Repository](https://github.com/cloudflare/web-bot-auth) -- [HTTP Message Signatures RFC 9421](https://datatracker.ietf.org/doc/html/rfc9421) -- [Chrome Enterprise Policy Documentation](https://chromeenterprise.google/policies/) -- [Chrome webRequest API](https://developer.chrome.com/docs/extensions/reference/api/webRequest) -- [Kernel Extensions Documentation](/browsers/extensions) +- [Web Bot Auth GitHub Repository](https://github.com/cloudflare/web-bot-auth) +- [Cloudflare Web Bot Auth Documentation](https://developers.cloudflare.com/bots/reference/bot-verification/web-bot-auth/) +- [RFC 9421 - HTTP Message Signatures](https://datatracker.ietf.org/doc/html/rfc9421) +- [Cloudflare Test Verification Site](https://http-message-signatures-example.research.cloudflare.com/) +- [Web Bot Auth Architecture Draft](https://thibmeu.github.io/http-message-signatures-directory/draft-meunier-web-bot-auth-architecture.html) diff --git a/reference/cli.mdx b/reference/cli.mdx index 364d175..90ec699 100644 --- a/reference/cli.mdx +++ b/reference/cli.mdx @@ -37,6 +37,9 @@ kernel --version Deploy apps, invoke actions, and stream logs. + + Upload, download, and build browser extensions. + Install Kernel MCP server configuration for AI tools. diff --git a/reference/cli/extensions.mdx b/reference/cli/extensions.mdx index b0e2cc3..3ad34e6 100644 --- a/reference/cli/extensions.mdx +++ b/reference/cli/extensions.mdx @@ -104,115 +104,29 @@ After downloading, you can upload the extension to Kernel: kernel extensions upload ./my-extension --name my-extension ``` -## Preparing Enterprise Extensions +### `kernel extensions build-web-bot-auth` +Build Cloudflare's [Web Bot Auth](/integrations/web-bot-auth) browser extension for signing HTTP requests with RFC 9421 signatures. -### `kernel extensions prepare-web-bot-auth` - -Download, build, and prepare the Cloudflare web-bot-auth extension with Kernel-specific configurations. This command creates a complete extension package ready to upload to Kernel. - -The web-bot-auth extension adds HTTP Message Signatures to browser requests, providing cryptographic proof of browser authenticity. It requires enterprise policies due to its use of the `webRequestBlocking` permission. - -| Flag | Default | Description | -|------|---------|-------------| -| `--output ` | `./web-bot-auth` | Output directory for the prepared extension. | -| `--url ` | `http://127.0.0.1:10001` | Base URL for extension updates and policy configuration. | -| `--version ` | `main` | GitHub branch or tag to download. | - -**Prerequisites:** -- Node.js and npm must be installed on your system -- Internet connection to download from GitHub - -**What this command does:** - -1. **Downloads** the latest web-bot-auth source from GitHub -2. **Modifies** configuration files to work with Kernel's infrastructure -3. **Builds** the extension using npm -4. **Signs** the extension with a stable extension ID using a generated private key -5. **Creates** a zip file ready to upload to Kernel - -**Output files:** - -- Extension directory with all source files -- `web-bot-auth.zip` - Ready to upload to Kernel -- `private_key.pem` - Signing key (keep secure!) -- `update.xml` - Chrome update manifest -- `.crx` file - Signed extension package -- Policy files for reference +| Flag | Description | +|------|-------------| +| `--to ` | Output directory for the built extension (required). | +| `--key ` | Path to JWK file with Ed25519 signing key (defaults to RFC9421 test key). | +| `--upload` | Upload the extension to Kernel after building. | +| `--name ` | Extension name when uploading (default: `web-bot-auth`). | -**Example:** +**Examples:** ```bash -# Prepare with defaults -kernel extensions prepare-web-bot-auth - -# Prepare with custom output directory -kernel extensions prepare-web-bot-auth --output ./my-web-bot-auth - -# Prepare specific version -kernel extensions prepare-web-bot-auth --version v1.0.0 -``` - -**Output:** - -``` -✓ Web-bot-auth extension prepared successfully! - -Extension ID: gilochgnhkffnkiiaooiedhngaiflfho -Output directory: ./web-bot-auth -Zip file: ./web-bot-auth/web-bot-auth.zip +# Build with default test key +kernel extensions build-web-bot-auth --to ./web-bot-auth-ext -Next steps: -1. Upload: kernel extensions upload ./web-bot-auth/web-bot-auth.zip --name gilochgnhkffnkiiaooiedhngaiflfho -2. Use in browsers: kernel browsers create --extension gilochgnhkffnkiiaooiedhngaiflfho -``` - -**Next steps after preparation:** - -1. **Upload the extension:** +# Build with custom key and upload +kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --key ./my-key.jwk --upload -```bash -kernel extensions upload ./web-bot-auth/web-bot-auth.zip --name +# Build with custom name +kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --upload --name ``` -Use the extension ID shown in the output as the name. This ensures consistency with the generated policy files. +This command requires Node.js and npm to be installed on your system. - -2. **Use in browsers:** - - -```typescript Typescript/Javascript -import { Kernel } from '@onkernel/sdk'; - -const kernel = new Kernel(); -const browser = await kernel.browsers.create({ - extensions: [{ name: "gilochgnhkffnkiiaooiedhngaiflfho" }] -}); -``` - -```python Python -from kernel import Kernel - -kernel = Kernel() -browser = kernel.browsers.create( - extensions=[{"name": "gilochgnhkffnkiiaooiedhngaiflfho"}] -) -``` - -```bash CLI -kernel browsers create --extension gilochgnhkffnkiiaooiedhngaiflfho -``` - - - -Keep your `private_key.pem` file secure! Anyone with access to this key can create extensions with the same extension ID. Store it safely if you need to rebuild or update the extension in the future. - - -**How it works:** - -The kernel-images server automatically serves the extension's update files at `http://127.0.0.1:10001/extensions/{extension-id}/update.xml`. Chrome uses this URL to install the extension via the `ExtensionInstallForcelist` policy. No additional HTTP server or manual configuration is needed. - -**For more information:** - -- [Web Bot Authentication integration guide](/integrations/web-bot-auth) - Detailed guide with examples -- [Extensions requiring enterprise policies](/browsers/extensions#uploading-extensions-requiring-enterprise-policies) - General information about enterprise extensions From 643bc56bd8e83da8f72b40d75c6af2bc5883f2a5 Mon Sep 17 00:00:00 2001 From: Archan Datta Date: Fri, 9 Jan 2026 19:13:36 +0000 Subject: [PATCH 5/7] review --- testing/profile-loading-performance.md | 298 ------------------------- 1 file changed, 298 deletions(-) delete mode 100644 testing/profile-loading-performance.md diff --git a/testing/profile-loading-performance.md b/testing/profile-loading-performance.md deleted file mode 100644 index ea701c6..0000000 --- a/testing/profile-loading-performance.md +++ /dev/null @@ -1,298 +0,0 @@ -# Profile Loading Performance Testing Guide - -## Overview - -This guide documents the process and results of testing profile loading performance with Google login across multiple SaaS sites. The goal is to understand if large profiles (mega-profiles) cause slow browser startup times and whether origin filtering could improve performance. - -## Background - -Some customers use Kernel to manage multiple SaaS accounts by: -1. Creating a synthetic Google account -2. Logging into Google once in a browser -3. Using "Login with Google" across many SaaS products -4. Saving all this state in a single mega-profile -5. Reusing this profile across many sessions - -This approach can result in very large profiles with hundreds of origins, thousands of cookies, and extensive localStorage/IndexedDB data. The question is: **Does this cause slow browser startup times?** - -## Test Setup - -### Prerequisites - -1. **Kernel API Key**: Set `KERNEL_API_KEY` environment variable -2. **Google Account**: A test Google account for logging in -3. **Node.js/Bun**: To run the test script -4. **Access to Live View**: To manually perform login steps - -### Running the Test - -```bash -cd kernel/packages/synthetic-tests -export KERNEL_API_KEY="your-api-key" -bun run src/profile-load-test.ts -``` - -Or with npm/pnpm: - -```bash -npm run dev src/profile-load-test.ts -# or -pnpm tsx src/profile-load-test.ts -``` - -## Test Workflow - -The automated test script performs the following steps: - -```mermaid -flowchart TD - A[Create Test Profile] --> B[Create Browser with save_changes=true] - B --> C[Output Live View URL] - C --> D[Wait for Manual Login] - D --> E[User: Login to Google] - E --> F[User: Navigate to Notion] - F --> G[User: Use Login with Google] - G --> H[User: Press Enter] - H --> I[Delete Browser to Save Profile] - I --> J[Download Profile via API] - J --> K[Analyze Profile Contents] - K --> L[Measure Browser Startup Time] - L --> M[Test Origin Filtering Hypothesis] - M --> N[Display Results] -``` - -### Manual Steps During Test - -When the test pauses, you need to: - -1. **Open the Live View URL** displayed in the terminal -2. **Navigate to Google**: Go to `https://accounts.google.com` -3. **Login**: Enter your test Google account credentials -4. **Navigate to Notion**: Go to `https://www.notion.so` -5. **Use "Login with Google"**: Click the button to authenticate -6. **Wait for Notion to load**: Ensure you're fully logged in -7. **Press Enter** in the terminal to continue - -## Metrics Collected - -### Profile Analysis - -The test analyzes the downloaded profile and reports: - -- **Total Size**: Profile JSON size in KB and bytes -- **Cookie Count**: Total number of cookies across all domains -- **Origin Count**: Number of unique origins with stored data -- **Top Origins by Cookies**: The 10 origins with the most cookies -- **Top Origins by LocalStorage**: The 10 origins with the most localStorage entries -- **Domain Breakdown**: Cookies, localStorage, and IndexedDB counts per domain - -### Timing Measurements - -The test measures browser startup time with the profile loaded: - -- **Browser Creation Time**: Time from API call to browser ready -- **Navigation Time**: Time to navigate to Notion after browser is ready -- **Total Time**: Combined time for full browser startup -- **Statistics**: Average, median, min, max across multiple iterations (default: 3) - -### Origin Filtering Analysis - -The test analyzes potential benefits of origin filtering: - -- **Google-related origins**: Count and percentage of total -- **Notion-related origins**: Count and percentage of total -- **Minimal profile size**: Size if only Google + Notion origins were loaded -- **Size reduction**: Percentage reduction from filtering - -## Interpreting Results - -### Profile Size - -- **< 100 KB**: Small profile, unlikely to cause performance issues -- **100 KB - 1 MB**: Medium profile, may have minor impact -- **1 MB - 10 MB**: Large profile, likely to impact startup time -- **> 10 MB**: Mega-profile, significant performance impact expected - -### Browser Startup Time - -- **< 5 seconds**: Fast startup, no optimization needed -- **5-10 seconds**: Moderate startup, optimization may help -- **10-30 seconds**: Slow startup, optimization recommended -- **> 30 seconds**: Very slow, optimization critical - -### Origin Filtering Potential - -If the minimal profile (Google + target site only) is significantly smaller than the full profile: - -- **> 50% reduction**: Strong candidate for origin filtering -- **25-50% reduction**: Moderate benefit from filtering -- **< 25% reduction**: Limited benefit, may not be worth complexity - -## Example Results - -### Sample Output - -``` -📊 PROFILE ANALYSIS -====================================================================== -📦 Total Size: 2,456.78 KB (2515712 bytes) -🍪 Total Cookies: 847 -🌐 Total Origins: 156 - -🔝 Top 10 Origins by Cookie Count: - 1. https://.google.com: 234 cookies - 2. https://.notion.so: 45 cookies - 3. https://.gstatic.com: 38 cookies - ... - -⏱️ TIMING STATISTICS -====================================================================== -📊 Average: 12,345.67ms -📊 Median: 12,234.00ms -📊 Min: 11,890ms -📊 Max: 13,012ms -📊 Range: 1,122ms - -🔍 ORIGIN FILTERING ANALYSIS -====================================================================== -📊 Google-related origins: 23 (14.7%) -📊 Notion-related origins: 8 (5.1%) -📊 Google-related cookies: 234 (27.6%) -📊 Notion-related cookies: 45 (5.3%) - -💡 Minimal profile size (Google + Notion only): 345.67 KB -💡 Size reduction: 85.9% (from 2,456.78 KB to 345.67 KB) -``` - -### Interpretation - -In this example: -- The profile is quite large (2.5 MB) -- Browser startup takes ~12 seconds on average -- Origin filtering could reduce size by 86% -- **Recommendation**: Implement origin filtering for this use case - -## Origin Filtering Implementation Approach - -If testing confirms that origin filtering would help, here's the recommended approach: - -### 1. Add API Parameter - -Add an optional `origins` parameter to the browser creation API: - -```typescript -await kernel.browsers.create({ - profile: { - name: 'my-profile', - origins: ['google.com', 'notion.so'], // Only load these origins - }, -}); -``` - -### 2. Filter Profile Data - -In `kernel/packages/api/lib/profiles/service_cdp.go`, modify `LoadProfileIntoSession`: - -```go -// Filter cookies by allowed origins -filteredCookies := state.Cookies -if len(allowedOrigins) > 0 { - filteredCookies = filterCookiesByOrigins(state.Cookies, allowedOrigins) -} - -// Filter origins by allowed list -filteredOrigins := state.Origins -if len(allowedOrigins) > 0 { - filteredOrigins = filterOriginsByList(state.Origins, allowedOrigins) -} -``` - -### 3. Domain Matching Logic - -Support flexible domain matching: -- `google.com` matches `.google.com`, `accounts.google.com`, etc. -- `notion.so` matches `.notion.so`, `www.notion.so`, etc. - -### 4. Performance Testing - -After implementation: -1. Run the same test with origin filtering enabled -2. Compare startup times with and without filtering -3. Verify login functionality still works -4. Document performance improvements - -## Troubleshooting - -### Profile Not Saving - -**Issue**: Profile appears empty after saving - -**Solution**: -- Ensure `save_changes: true` is set when creating the browser -- Wait at least 5 seconds after deleting the browser before downloading -- Check that you actually logged in and created session state - -### Browser Startup Timeout - -**Issue**: Browser creation times out when loading profile - -**Solution**: -- Profile may be too large (> 10 MB) -- Try with a smaller profile first -- Check Kernel logs for specific errors -- Increase timeout if necessary - -### Download Fails - -**Issue**: Profile download returns 404 or empty data - -**Solution**: -- Profile may not have any saved data yet -- Verify profile exists: `kernel.profiles.list()` -- Check that browser was deleted (not just closed) -- Wait longer for profile persistence - -## Next Steps - -After running the test: - -1. **Document your results** in this file under a "Test Results" section -2. **Share findings** with the team -3. **Decide on implementation** of origin filtering if beneficial -4. **Create follow-up tests** for other SaaS sites if needed - -## Test Results - -### Test Run: [Date] - -**Tester**: [Your Name] - -**Profile Details**: -- Size: X KB -- Cookies: X -- Origins: X - -**Timing Results**: -- Average startup: X ms -- Median startup: X ms -- Range: X ms - -**Origin Filtering Analysis**: -- Potential size reduction: X% -- Google origins: X -- Notion origins: X - -**Recommendation**: -[Your recommendation based on results] - -**Notes**: -[Any additional observations] - ---- - -## References - -- [Kernel Profiles Documentation](/browsers/profiles) -- [Profile API Reference](/api-reference/profiles/list-profiles) -- [Browser Creation API](/api-reference/browsers/create-a-browser) - From da82051c2a094c2678def074999e8fd34139dd23 Mon Sep 17 00:00:00 2001 From: Archan Datta Date: Mon, 12 Jan 2026 14:18:09 +0000 Subject: [PATCH 6/7] review: update docs --- integrations/web-bot-auth.mdx | 12 ++++++------ reference/cli/extensions.mdx | 5 ++--- 2 files changed, 8 insertions(+), 9 deletions(-) diff --git a/integrations/web-bot-auth.mdx b/integrations/web-bot-auth.mdx index 8da1d7e..f111c75 100644 --- a/integrations/web-bot-auth.mdx +++ b/integrations/web-bot-auth.mdx @@ -25,13 +25,13 @@ The fastest way to get started is using Cloudflare's RFC9421 test key, which wor Use the Kernel CLI to build the Web Bot Auth extension: ```bash -kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --upload +kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --upload my-web-bot-auth ``` This command: - Downloads Cloudflare's web-bot-auth browser extension source - Builds it with the default RFC9421 test key -- Uploads it to Kernel as `web-bot-auth` +- Uploads it to Kernel as `my-web-bot-auth` The build command requires Node.js and npm to be installed on your system. @@ -42,7 +42,7 @@ The build command requires Node.js and npm to be installed on your system. ```bash CLI # Create a browser with the web-bot-auth extension -kernel browsers create --extension web-bot-auth +kernel browsers create --extension my-web-bot-auth # The command outputs the browser ID and live view URL # Open the live view URL in your browser, then navigate to: @@ -57,7 +57,7 @@ const kernel = new Kernel(); // Create browser with web-bot-auth extension const browser = await kernel.browsers.create({ - extensions: [{ name: "web-bot-auth" }], + extensions: [{ name: "my-web-bot-auth" }], }); // Connect via Playwright @@ -76,7 +76,7 @@ from playwright.sync_api import sync_playwright kernel = Kernel() # Create browser with web-bot-auth extension -browser = kernel.browsers.create(extensions=[{"name": "web-bot-auth"}]) +browser = kernel.browsers.create(extensions=[{"name": "my-web-bot-auth"}]) # Connect via Playwright with sync_playwright() as p: @@ -124,7 +124,7 @@ See [Cloudflare's web-bot-auth documentation](https://github.com/cloudflare/web- ### 2. Build with your key ```bash -kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --key ./my-key.jwk --upload --name my-web-bot-auth +kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --key ./my-key.jwk --upload my-web-bot-auth ``` ### 3. Host your public key diff --git a/reference/cli/extensions.mdx b/reference/cli/extensions.mdx index 3ad34e6..705b964 100644 --- a/reference/cli/extensions.mdx +++ b/reference/cli/extensions.mdx @@ -111,8 +111,7 @@ Build Cloudflare's [Web Bot Auth](/integrations/web-bot-auth) browser extension |------|-------------| | `--to ` | Output directory for the built extension (required). | | `--key ` | Path to JWK file with Ed25519 signing key (defaults to RFC9421 test key). | -| `--upload` | Upload the extension to Kernel after building. | -| `--name ` | Extension name when uploading (default: `web-bot-auth`). | +| `--upload ` | Upload the extension to Kernel with specified name (e.g., --upload my-web-bot-auth). | **Examples:** @@ -124,7 +123,7 @@ kernel extensions build-web-bot-auth --to ./web-bot-auth-ext kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --key ./my-key.jwk --upload # Build with custom name -kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --upload --name +kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --upload my-web-bot-auth ``` From 636e91f8859b91977bc74d96d61585f4206eba5c Mon Sep 17 00:00:00 2001 From: Archan Datta Date: Mon, 12 Jan 2026 16:25:29 +0000 Subject: [PATCH 7/7] review --- reference/cli/extensions.mdx | 6 +- testing/profile-loading-performance.md | 187 +++++++++++++++++++++++++ 2 files changed, 190 insertions(+), 3 deletions(-) create mode 100644 testing/profile-loading-performance.md diff --git a/reference/cli/extensions.mdx b/reference/cli/extensions.mdx index 705b964..c16a44f 100644 --- a/reference/cli/extensions.mdx +++ b/reference/cli/extensions.mdx @@ -119,10 +119,10 @@ Build Cloudflare's [Web Bot Auth](/integrations/web-bot-auth) browser extension # Build with default test key kernel extensions build-web-bot-auth --to ./web-bot-auth-ext -# Build with custom key and upload -kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --key ./my-key.jwk --upload +# Build with custom key +kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --key ./my-key.jwk -# Build with custom name +# Build with custom name and upload kernel extensions build-web-bot-auth --to ./web-bot-auth-ext --upload my-web-bot-auth ``` diff --git a/testing/profile-loading-performance.md b/testing/profile-loading-performance.md new file mode 100644 index 0000000..8d6fd9e --- /dev/null +++ b/testing/profile-loading-performance.md @@ -0,0 +1,187 @@ +# Profile Loading Performance Testing Guide + +## Overview + +This guide documents the process for testing profile loading performance with Google login across SaaS sites. The goal is to understand if large profiles cause slow browser startup times and whether origin filtering could improve performance. + +## Background + +Some customers use Kernel to manage multiple SaaS accounts by: +1. Creating a synthetic Google account +2. Logging into Google once in a browser +3. Using "Login with Google" across many SaaS products +4. Saving all state in a single mega-profile +5. Reusing this profile across sessions + +This can result in large profiles with many origins and cookies. This test measures the performance impact. + +## Test Setup + +### Prerequisites + +1. **Kernel API Key**: Set `KERNEL_API_KEY` environment variable +2. **Google Account**: A test Google account for logging in +3. **Node.js/Bun**: To run the test script + +### Running the Test + +```bash +cd kernel/packages/synthetic-tests +export KERNEL_API_KEY="your-api-key" +npx tsx src/profile-load-test.ts +``` + +## Test Workflow + +The test performs these steps: + +1. **Create Test Profile** - Creates a new profile with unique name +2. **Create Browser** - Launches browser with `save_changes: true` +3. **Manual Login** - User logs into Google and Notion via live view +4. **Save Profile** - Deletes browser to persist profile data +5. **Download Profile** - Downloads profile via API +6. **Analyze Profile** - Measures size, cookies, origins +7. **Measure Startup** - Creates browsers with profile, measures timing +8. **Origin Analysis** - Calculates filtering potential + +### Manual Steps During Test + +When the test pauses, you need to: + +1. Open the Live View URL displayed in the terminal +2. Navigate to `https://accounts.google.com` +3. Log in to your test Google account +4. Navigate to `https://www.notion.so` +5. Click "Continue with Google" or "Login with Google" +6. Complete the login flow +7. Wait for Notion to load completely +8. Press Enter in the terminal to continue + +## Metrics Collected + +### Profile Analysis + +- **Total Size**: Profile JSON size in KB and bytes +- **Cookie Count**: Total cookies across all domains +- **Origin Count**: Unique origins with stored data +- **Top Origins**: Origins with most cookies/localStorage +- **Domain Breakdown**: Cookies, localStorage, IndexedDB per domain + +### Timing Measurements + +- **Browser Creation Time**: Time from API call to browser ready +- **Navigation Time**: Time to navigate to Notion +- **Total Time**: Combined time for full browser startup +- **Statistics**: Average, median, min, max across 3 iterations + +### Origin Filtering Analysis + +- **Google-related origins/cookies**: Count and percentage +- **Notion-related origins/cookies**: Count and percentage +- **Minimal profile size**: Size if only Google + Notion origins were loaded +- **Size reduction**: Percentage reduction from filtering + +## Interpreting Results + +### Profile Size + +- **< 100 KB**: Small profile, unlikely performance issues +- **100 KB - 1 MB**: Medium profile, minor impact possible +- **1 MB - 10 MB**: Large profile, likely performance impact +- **> 10 MB**: Mega-profile, significant impact expected + +### Browser Startup Time + +- **< 5 seconds**: Fast startup, no optimization needed +- **5-10 seconds**: Moderate startup, optimization may help +- **10-30 seconds**: Slow startup, optimization recommended +- **> 30 seconds**: Very slow, optimization critical + +### Origin Filtering Potential + +- **> 50% reduction**: Strong candidate for origin filtering +- **25-50% reduction**: Moderate benefit from filtering +- **< 25% reduction**: Limited benefit + +## Origin Filtering Implementation + +If testing confirms filtering would help, the recommended approach: + +### 1. API Parameter + +Add optional `origins` parameter to browser creation: + +```typescript +await kernel.browsers.create({ + profile: { + name: 'my-profile', + origins: ['google.com', 'notion.so'], + }, +}); +``` + +### 2. Backend Implementation + +In `kernel/packages/api/lib/profiles/service_cdp.go`, modify `LoadProfileIntoSession`: + +```go +// Filter cookies by allowed origins +if len(allowedOrigins) > 0 { + filteredCookies := filterCookiesByOrigins(state.Cookies, allowedOrigins) + filteredOrigins := filterOriginsByList(state.Origins, allowedOrigins) +} +``` + +### 3. Domain Matching + +Support flexible domain matching: +- `google.com` matches `.google.com`, `accounts.google.com`, etc. +- `notion.so` matches `.notion.so`, `www.notion.so`, etc. + +## Troubleshooting + +### Profile Not Saving + +- Ensure `save_changes: true` is set +- Wait at least 5 seconds after deleting browser +- Verify you created session state during login + +### Browser Startup Timeout + +- Profile may be too large +- Check network connection +- Increase timeout if necessary + +### Download Fails + +- Profile may not have saved data +- Browser must be deleted (not just closed) +- Wait longer for profile persistence + +## Test Results + +### Test Run: [Date] + +**Profile Details**: +- Size: X KB +- Cookies: X +- Origins: X + +**Timing Results**: +- Average startup: X ms +- Median startup: X ms +- Range: X ms + +**Origin Filtering Analysis**: +- Potential size reduction: X% +- Google origins: X +- Notion origins: X + +**Recommendation**: +[Based on results] + +## References + +- [Kernel Profiles Documentation](/browsers/profiles) +- [Profile API Reference](/api-reference/profiles/list-profiles) +- [Browser Creation API](/api-reference/browsers/create-a-browser)