Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,13 @@ jobs:
- name: Install dependencies
run: npm ci

- name: Package seeds directory
run: |
cd seeds
zip -r ../seeds.zip *.png
cd ..
echo "Created seeds.zip with $(unzip -l seeds.zip | tail -1 | awk '{print $2}') bytes"

- name: Build Tauri app
uses: tauri-apps/tauri-action@v0
env:
Expand All @@ -57,3 +64,10 @@ jobs:
releaseBody: 'See the assets below to download and install Biome.'
releaseDraft: false
prerelease: false

- name: Upload seeds.zip to release
if: matrix.platform == 'ubuntu-22.04'
uses: softprops/action-gh-release@v1
with:
files: seeds.zip
token: ${{ secrets.GITHUB_TOKEN }}
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,12 @@ dist-ssr
.env/
*.bak

.venv/
uv.lock
__pycache__
# seed directory created by standalone instance of server.py
src-tauri/world_engine

# env
.env
.vercel
Expand Down
17 changes: 8 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,18 @@

# Biome

**Explore AI-generated worlds in real-time, running locally on your GPU.**
**Explore AI-generated worlds in real-time, running locally on your GPU.**

[![Website](https://img.shields.io/badge/over.world-000000?logo=data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyNCIgaGVpZ2h0PSIyNCIgdmlld0JveD0iMCAwIDI0IDI0IiBmaWxsPSJub25lIiBzdHJva2U9IndoaXRlIiBzdHJva2Utd2lkdGg9IjIiIHN0cm9rZS1saW5lY2FwPSJyb3VuZCIgc3Ryb2tlLWxpbmVqb2luPSJyb3VuZCI+PGNpcmNsZSBjeD0iMTIiIGN5PSIxMiIgcj0iMTAiLz48bGluZSB4MT0iMiIgeTE9IjEyIiB4Mj0iMjIiIHkyPSIxMiIvPjxwYXRoIGQ9Ik0xMiAyYTE1LjMgMTUuMyAwIDAgMSA0IDEwIDE1LjMgMTUuMyAwIDAgMS00IDEwIDE1LjMgMTUuMyAwIDAgMS00LTEwIDE1LjMgMTUuMyAwIDAgMSA0LTEweiIvPjwvc3ZnPg==)](https://over.world)
[![Discord](https://img.shields.io/badge/Discord-5865F2?logo=discord&logoColor=white)](https://discord.gg/overworld)
[![X](https://img.shields.io/badge/X-000000?logo=x&logoColor=white)](https://x.com/overworld_ai)
[![Windows](https://img.shields.io/badge/Windows-0078D6?logo=windows&logoColor=white)](https://github.com/Overworldai/Biome/releases)
[![Linux](https://img.shields.io/badge/Linux-FCC624?logo=linux&logoColor=black)](https://github.com/Overworldai/Biome/releases)
[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)
[![Website](https://img.shields.io/badge/over.world-000000?logo=data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIyNCIgaGVpZ2h0PSIyNCIgdmlld0JveD0iMCAwIDI0IDI0IiBmaWxsPSJub25lIiBzdHJva2U9IndoaXRlIiBzdHJva2Utd2lkdGg9IjIiIHN0cm9rZS1saW5lY2FwPSJyb3VuZCIgc3Ryb2tlLWxpbmVqb2luPSJyb3VuZCI+PGNpcmNsZSBjeD0iMTIiIGN5PSIxMiIgcj0iMTAiLz48bGluZSB4MT0iMiIgeTE9IjEyIiB4Mj0iMjIiIHkyPSIxMiIvPjxwYXRoIGQ9Ik0xMiAyYTE1LjMgMTUuMyAwIDAgMSA0IDEwIDE1LjMgMTUuMyAwIDAgMS00IDEwIDE1LjMgMTUuMyAwIDAgMS00LTEwIDE1LjMgMTUuMyAwIDAgMSA0LTEweiIvPjwvc3ZnPg==)](https://over.world)
[![Discord](https://img.shields.io/badge/Discord-5865F2?logo=discord&logoColor=white)](https://discord.gg/overworld)
[![X](https://img.shields.io/badge/X-000000?logo=x&logoColor=white)](https://x.com/overworld_ai)
[![Windows](https://img.shields.io/badge/Windows-0078D6?logo=windows&logoColor=white)](https://github.com/Overworldai/Biome/releases)
[![Linux](https://img.shields.io/badge/Linux-FCC624?logo=linux&logoColor=black)](https://github.com/Overworldai/Biome/releases)
[![License: GPL v3](https://img.shields.io/badge/License-GPLv3-blue.svg)](https://www.gnu.org/licenses/gpl-3.0)

<img src="assets/launch_grid.gif" width="600">

**[Download the latest release](https://github.com/Overworldai/Biome/releases/latest)**
**[Download the latest release](https://github.com/Overworldai/Biome/releases/latest)**

</div>

Expand All @@ -31,7 +31,6 @@ Biome installs just like a video game — download, run the installer, and start
- Runs locally on your GPU via [World Engine](https://github.com/Overworldai/world_engine)
- Lightweight native desktop application


## Getting Started

Grab the installer from the [Releases](https://github.com/Overworldai/Biome/releases/latest) page and you're good to go.
Expand Down
4 changes: 4 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Binary file removed seeds/0_2.png
Binary file not shown.
Binary file removed seeds/default.png
Binary file not shown.
Binary file added seeds/sample_00000.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00001.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00002.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00003.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00004.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00005.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00006.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00007.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00008.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00009.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00010.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00011.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00012.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00013.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00014.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00015.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00016.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00017.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00018.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00019.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00020.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00021.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00022.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added seeds/sample_00023.png
Binary file added seeds/sample_00024.png
Binary file added seeds/sample_00025.png
Binary file added seeds/sample_00026.png
Binary file added seeds/sample_00027.png
Binary file added seeds/sample_00028.png
Binary file added seeds/sample_00029.png
Binary file added seeds/sample_00030.png
Binary file added seeds/sample_00031.png
Binary file added seeds/sample_00032.png
Binary file added seeds/sample_00033.png
Binary file added seeds/sample_00034.png
Binary file added seeds/sample_00035.png
Binary file added seeds/sample_00036.png
Binary file added seeds/sample_00037.png
Binary file added seeds/sample_00038.png
Binary file added seeds/sample_00039.png
Binary file added seeds/sample_00040.png
Binary file added seeds/sample_00041.png
Binary file added seeds/sample_00042.png
Binary file added seeds/sample_00043.png
Binary file added seeds/sample_00044.png
Binary file added seeds/sample_00045.png
Binary file added seeds/sample_00046.png
Binary file added seeds/sample_00047.png
Binary file added seeds/sample_00048.png
Binary file added seeds/sample_00049.png
Binary file added seeds/sample_00050.png
Binary file removed seeds/starter (1).png
Diff not rendered.
Binary file removed seeds/starter (11).png
Diff not rendered.
Binary file removed seeds/starter (12).png
Diff not rendered.
Binary file removed seeds/starter (14).png
Diff not rendered.
Binary file removed seeds/starter (15).png
Diff not rendered.
Binary file removed seeds/starter (16).png
Diff not rendered.
Binary file removed seeds/starter (17).png
Diff not rendered.
Binary file removed seeds/starter (18).png
Diff not rendered.
Binary file removed seeds/starter (19).png
Diff not rendered.
Binary file removed seeds/starter (2).png
Diff not rendered.
Binary file removed seeds/starter (20).png
Diff not rendered.
Binary file removed seeds/starter (21).png
Diff not rendered.
Binary file removed seeds/starter (22).png
Diff not rendered.
Binary file removed seeds/starter (23).png
Diff not rendered.
Binary file removed seeds/starter (24).png
Diff not rendered.
Binary file removed seeds/starter (26).png
Diff not rendered.
Binary file removed seeds/starter (27).png
Diff not rendered.
Binary file removed seeds/starter (3).png
Diff not rendered.
Binary file removed seeds/starter (4).png
Diff not rendered.
Binary file removed seeds/starter (5).png
Diff not rendered.
Binary file removed seeds/starter (6).png
Diff not rendered.
Binary file removed seeds/starter (9).png
Diff not rendered.
1 change: 1 addition & 0 deletions src-tauri/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion src-tauri/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,13 @@ tauri-plugin-opener = "2"
tauri-plugin-fs = "2"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
reqwest = { version = "0.12" }
reqwest = { version = "0.12", features = ["json"] }
zip = "2"
flate2 = "1"
tar = "0.4"
kill_tree = "0.2"
base64 = "0.22"
image = { version = "0.25", default-features = false, features = ["png", "jpeg"] }
ctrlc = "3"
log = "0.4"

266 changes: 266 additions & 0 deletions src-tauri/server-components/engine_manager.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,266 @@
"""
WorldEngine module - Handles AI world generation and frame streaming.

Extracted from monolithic server.py to provide clean separation of concerns.
"""

import asyncio
import base64
import io
import logging
import time
from dataclasses import dataclass

import torch
import torch.nn.functional as F
from PIL import Image

logger = logging.getLogger(__name__)

# ============================================================================
# Configuration
# ============================================================================

MODEL_URI = "Overworld/Waypoint-1-Small"
QUANT = None
N_FRAMES = 4096
DEVICE = "cuda"
JPEG_QUALITY = 85

BUTTON_CODES = {}
# A-Z keys
for i in range(65, 91):
BUTTON_CODES[chr(i)] = i
# 0-9 keys
for i in range(10):
BUTTON_CODES[str(i)] = ord(str(i))
# Special keys
BUTTON_CODES["UP"] = 0x26
BUTTON_CODES["DOWN"] = 0x28
BUTTON_CODES["LEFT"] = 0x25
BUTTON_CODES["RIGHT"] = 0x27
BUTTON_CODES["SHIFT"] = 0x10
BUTTON_CODES["CTRL"] = 0x11
BUTTON_CODES["SPACE"] = 0x20
BUTTON_CODES["TAB"] = 0x09
BUTTON_CODES["ENTER"] = 0x0D
BUTTON_CODES["MOUSE_LEFT"] = 0x01
BUTTON_CODES["MOUSE_RIGHT"] = 0x02
BUTTON_CODES["MOUSE_MIDDLE"] = 0x04

# Default prompt - describes the expected visual style
DEFAULT_PROMPT = (
"First-person shooter gameplay footage from a true POV perspective, "
"the camera locked to the player's eyes as assault rifles, carbines, "
"machine guns, laser-sighted firearms, bullet-fed weapons, magazines, "
"barrels, muzzles, tracers, ammo, and launchers dominate the frame, "
"with constant gun handling, recoil, muzzle flash, shell ejection, "
"and ballistic impacts. Continuous real-time FPS motion with no cuts, "
"weapon-centric framing, realistic gun physics, authentic firearm "
"materials, high-caliber ammunition, laser optics, iron sights, and "
"relentless gun-driven action, rendered in ultra-realistic 4K at 60fps."
)


# ============================================================================
# Session Management
# ============================================================================


@dataclass
class Session:
"""Tracks state for a single WebSocket connection."""

frame_count: int = 0
max_frames: int = N_FRAMES - 2


# ============================================================================
# WorldEngine Manager
# ============================================================================


class WorldEngineManager:
"""Manages WorldEngine state and operations."""

def __init__(self):
self.engine = None
self.seed_frame = None
self.CtrlInput = None
self.current_prompt = DEFAULT_PROMPT
self.engine_warmed_up = False

def load_seed_from_file(
self, file_path: str, target_size: tuple[int, int] = (360, 640)
) -> torch.Tensor:
"""Load a seed frame from a file path."""
try:
img = Image.open(file_path).convert("RGB")
import numpy as np

img_tensor = (
torch.from_numpy(np.array(img)).permute(2, 0, 1).unsqueeze(0).float()
)
frame = F.interpolate(
img_tensor, size=target_size, mode="bilinear", align_corners=False
)[0]
return (
frame.to(dtype=torch.uint8, device=DEVICE)
.permute(1, 2, 0)
.contiguous()
)
except Exception as e:
logger.error(f"Failed to load seed from file {file_path}: {e}")
return None

def load_seed_from_base64(
self, base64_data: str, target_size: tuple[int, int] = (360, 640)
) -> torch.Tensor:
"""Load a seed frame from base64 encoded data."""
try:
img_data = base64.b64decode(base64_data)
img = Image.open(io.BytesIO(img_data)).convert("RGB")
import numpy as np

img_tensor = (
torch.from_numpy(np.array(img)).permute(2, 0, 1).unsqueeze(0).float()
)
frame = F.interpolate(
img_tensor, size=target_size, mode="bilinear", align_corners=False
)[0]
return (
frame.to(dtype=torch.uint8, device=DEVICE)
.permute(1, 2, 0)
.contiguous()
)
except Exception as e:
logger.error(f"Failed to load seed from base64: {e}")
return None


async def load_engine(self):
"""Initialize the WorldEngine with configured model."""
logger.info("=" * 60)
logger.info("BIOME ENGINE STARTUP")
logger.info("=" * 60)

logger.info("[1/4] Importing WorldEngine...")
import_start = time.perf_counter()
from world_engine import CtrlInput as CI
from world_engine import WorldEngine

self.CtrlInput = CI
logger.info(
f"[1/4] WorldEngine imported in {time.perf_counter() - import_start:.2f}s"
)

logger.info(f"[2/4] Loading model: {MODEL_URI}")
logger.info(f" Quantization: {QUANT}")
logger.info(f" Device: {DEVICE}")
logger.info(f" N_FRAMES: {N_FRAMES}")
logger.info(f" Prompt: {self.current_prompt[:60]}...")

# Model config overrides
# scheduler_sigmas: diffusion denoising schedule (MUST end with 0.0)
# ae_uri: VAE model for encoding/decoding frames
model_start = time.perf_counter()
self.engine = WorldEngine(
MODEL_URI,
device=DEVICE,
model_config_overrides={
"n_frames": N_FRAMES,
"ae_uri": "OpenWorldLabs/owl_vae_f16_c16_distill_v0_nogan",
"scheduler_sigmas": [1.0, 0.8, 0.2, 0.0],
},
quant=QUANT,
dtype=torch.bfloat16,
)
logger.info(
f"[2/4] Model loaded in {time.perf_counter() - model_start:.2f}s"
)

# Seed frame will be provided by frontend via set_initial_seed message
logger.info(
"[3/4] Seed frame: waiting for client to provide initial seed via base64"
)
self.seed_frame = None

logger.info("[4/4] Engine initialization complete")
logger.info("=" * 60)
logger.info("SERVER READY - Waiting for WebSocket connections on /ws")
logger.info(" (Client must send set_initial_seed with base64 data)")
logger.info("=" * 60)

def frame_to_jpeg(self, frame: torch.Tensor, quality: int = JPEG_QUALITY) -> bytes:
"""Convert frame tensor to JPEG bytes."""
if frame.dtype != torch.uint8:
frame = frame.clamp(0, 255).to(torch.uint8)
img = Image.fromarray(frame.cpu().numpy(), mode="RGB")
buf = io.BytesIO()
img.save(buf, format="JPEG", quality=quality)
return buf.getvalue()

async def generate_frame(self, ctrl_input) -> torch.Tensor:
"""Generate next frame using WorldEngine."""
frame = await asyncio.to_thread(self.engine.gen_frame, ctrl=ctrl_input)
return frame

async def reset_state(self):
"""Reset engine state."""
await asyncio.to_thread(self.engine.reset)
await asyncio.to_thread(self.engine.append_frame, self.seed_frame)
await asyncio.to_thread(self.engine.set_prompt, self.current_prompt)

async def warmup(self):
"""Perform initial warmup to compile CUDA graphs."""

def do_warmup():
warmup_start = time.perf_counter()

logger.info("[5/5] Step 1: Resetting engine state...")
reset_start = time.perf_counter()
self.engine.reset()
logger.info(
f"[5/5] Step 1: Reset complete in {time.perf_counter() - reset_start:.2f}s"
)

logger.info("[5/5] Step 2: Appending seed frame...")
append_start = time.perf_counter()
self.engine.append_frame(self.seed_frame)
logger.info(
f"[5/5] Step 2: Seed frame appended in {time.perf_counter() - append_start:.2f}s"
)

logger.info("[5/5] Step 3: Setting prompt...")
prompt_start = time.perf_counter()
self.engine.set_prompt(self.current_prompt)
logger.info(
f"[5/5] Step 3: Prompt set in {time.perf_counter() - prompt_start:.2f}s"
)

logger.info(
"[5/5] Step 4: Generating first frame (compiling CUDA graphs)..."
)
gen_start = time.perf_counter()
_ = self.engine.gen_frame(
ctrl=self.CtrlInput(button=set(), mouse=(0.0, 0.0))
)
logger.info(
f"[5/5] Step 4: First frame generated in {time.perf_counter() - gen_start:.2f}s"
)

return time.perf_counter() - warmup_start

logger.info("=" * 60)
logger.info(
"[5/5] WARMUP - First client connected, initializing CUDA graphs..."
)
logger.info("=" * 60)

warmup_time = await asyncio.to_thread(do_warmup)

logger.info("=" * 60)
logger.info(f"[5/5] WARMUP COMPLETE - Total time: {warmup_time:.2f}s")
logger.info("=" * 60)

self.engine_warmed_up = True
Loading
Loading