bowerbot 1.5.0

AI-powered 3D scene assembly from natural language using OpenUSD

BowerBot

AI-powered OpenUSD scene assembly.

From empty scene to structured OpenUSD stage in seconds.

---

🐦 Meet BowerBot

In the rainforests of Australia and New Guinea lives one of nature's most remarkable architects: the bowerbird.

Instead of relying on appearance, the bowerbird collects, curates, and arranges objects from its environment into a carefully constructed 3D composition. Every object is chosen. Every placement is intentional.

BowerBot brings that same idea to 3D pipelines.

BowerBot is an AI-assisted OpenUSD scene assembly tool. It helps you go from an empty scene to a structured, usable layout in seconds by:

• finding assets from any connected source (Sketchfab, local disk, company DAM, or any custom provider)
• placing them with spatial awareness
• organizing them into a valid USD stage

It does not try to replace the artist. It accelerates the part that is repetitive, mechanical, and time-consuming.

You stay in control of composition, lighting, and final polish. BowerBot gets you there faster.

---

🎯 What BowerBot Is (and Is Not)

BowerBot is:

• A scene bootstrapper for OpenUSD pipelines
• A fast way to go from 0 → structured scene
• A conversational interface for asset search, placement, and layout
• A pipeline assistant that handles technical correctness (units, hierarchy, references)
• A pipeline guardian that catches asset issues before they reach production

BowerBot is NOT:

• A final scene generator
• A replacement for DCC tools like Maya or Omniverse
• A system that produces perfect composition or artistic layouts

Scenes generated by BowerBot are meant to be opened, reviewed, and refined in your DCC.

Think of it as:

"Block out the scene instantly, then refine like a pro."

Pipeline Quality Built In

BowerBot enforces ASWF USD standards at every step, not just placing assets. Fixable mismatches (non-canonical folder names, external dependencies) are auto-normalized on intake so the project copy is always self-contained. Unfixable violations (wrong root prim type, missing defaultPrim, incorrect metersPerUnit, circular references, missing dependencies) are caught at assembly time with a clear message about what's wrong and how to fix it.

This means problems that would normally surface weeks later in lighting, rendering, or review (broken references, unit mismatches, non-compliant asset structure) are caught the moment the asset enters the scene. Fix it once at the source, and every downstream department benefits.

"The cheapest bug to fix is the one you catch before it enters the pipeline."

Who It's For

BowerBot targets any team assembling OpenUSD scenes, including:

• 🏭 Digital twin: warehouses, retail floors, and factory scenes composed from vendor and internal assets
• 🤖 Robotics and simulation: USD environments for Isaac Sim, Isaac Lab, and custom sim stacks
• 🏛️ Architecture and AEC: bootstrapping interior and exterior USD scenes for visualization and design review
• 🥽 XR and spatial computing: lightweight .usdz bundles for Vision Pro, Quest, and ARKit
• 🎬 VFX and animation: shot-dressing and layout over ASWF-structured assets
• 🛠️ Pipeline TDs: a scriptable, LLM-driven scene assembler extendable with custom skills

---

✨ What It Does

$ bowerbot open coffee_shop

You: Search my Sketchfab for a round bistro table and download it
BowerBot: Found "Round Bistro Table". Downloaded to assets/Round_Bistro_Table.usdz

You: Place it at the center of the room
BowerBot: Placed at /Scene/Furniture/Round_Bistro_Table_01 (5.0, 0.0, 4.0)

You: Find an espresso cup in my local assets and place it on the table surface
BowerBot: Found espresso_cup.usdz. Table surface at Y=0.74.
         Placed at /Scene/Products/Espresso_Cup_01 (5.0, 0.74, 4.0)

You: Move the cup to the left side of the table
BowerBot: Moved to (4.7, 0.74, 4.0) (within table bounds).

You: Validate and package
BowerBot: All checks passed. Packaged to scenes/coffee_shop/scene.usdz

Open the .usda in Maya, usdview, Omniverse, or any USD-compatible DCC to refine composition, lighting, and materials.

BowerBot gets you to a clean, structured starting point—fast. From there, you take it to production quality.

Projects are persistent. Close the session, come back later, and continue where you left off.

---

✨ Features

• 📦 OpenUSD native: references, defaultPrim, metersPerUnit, upAxis, all correct out of the box
• 🏗️ ASWF-compliant asset folders: geometry, materials, and lighting split into a root + layer files, per the USD Working Group guidelines
• 🧳 Self-contained intake: non-canonical source folders are detected via USD composition, canonicalized (root.usd → <folder>.usda), and external dependencies (textures, sublayers) are localized into the asset folder so the project copy is always portable
• 🎨 Material binding: apply MaterialX or existing .usda materials to specific mesh parts
• 💡 Native USD lighting: sun, dome, point, area, disk, and tube lights at scene or asset level
• 🧩 Automatic unit handling: assets in cm, mm, or inches are scaled correctly at reference time
• 📐 Geometry-aware placement: bounding-box resolved positions for surface, above, below, or nested placements
• 🔌 Pluggable skills: connect any asset source (Sketchfab, PolyHaven, company DAM, or build your own)
• 🗣️ Conversational assembly: guide scene construction through natural language
• 🧠 Multi-LLM support: OpenAI, Anthropic, and any provider via litellm
• 📁 Project-based workflow: one folder per scene, resumable across sessions
• ✅ Scene validation: defaultPrim, units, up-axis, reference resolution, and material binding checks
• 📦 USDZ packaging: export for Apple Vision Pro, Omniverse, or any USD viewer
• 🏗️ Onboarding wizard: zero-config setup in 60 seconds

Built on OpenUSD, the ASWF USD Working Group standards, and the Alliance for OpenUSD (AOUSD) core spec driven by Pixar, Apple, NVIDIA, and others.

---

🔄 How It Works

BowerBot is conversational: you tell it what you want and it uses the right tools to build your scene. Behind the scenes, it manages asset discovery, USD composition, materials, lighting, and more.

Asset Discovery

BowerBot searches for assets across all connected sources, prioritizing what's already available:

1. Local assets first: BowerBot checks your local asset directory (assets_dir in config.json) for USD files (.usd, .usda, .usdc, .usdz). This includes anything you've exported from Maya, Houdini, Blender, or any DCC tool, as well as assets previously downloaded from cloud providers.

2. Cloud providers if needed: If the asset isn't found locally, BowerBot searches connected providers like Sketchfab, and downloads the asset to your local directory. Future skills will add support for PolyHaven, Fab, CGTrader, Objaverse, and custom company DAMs.

3. All downloads are cached locally: Once an asset is downloaded from any source, it lives in your assets_dir and is available for all future projects without re-downloading.

Scene Assembly

When you ask BowerBot to place an asset, it routes by what the source looks like and always produces a self-contained ASWF folder in the project:

• Folder with a detectable root (canonical wall/wall.usda, or non-canonical wall/root.usd + wall/geo.usd + wall/mtl.usd): the root is identified via USD composition (the file no sibling depends on), the folder is copied into the project, the root is canonicalized to <folder>.usda, sibling references are rewritten, and any externally-referenced textures or layers are localized into the folder so the output is portable.
• Loose USD geometry (.usd, .usda, .usdc from your DCC exports): wrapped in a fresh ASWF folder named after the file stem, producing <stem>/<stem>.usda + geo.usda.
• USDZ files (from Sketchfab, DAMs, etc.): placed as-is since they're already self-contained.

When an asset can't be safely intaken (missing external dependencies, or a folder with multiple independent USDs and no clear root), BowerBot refuses with a message naming the conflict instead of guessing.

Material Workflow

When you apply materials to an asset, BowerBot writes them into the asset folder's mtl.usda, not the scene file. The scene stays clean with only references:

You: Apply wood material to the table top
BowerBot: [searches local assets for "wood" materials]
         [discovers mesh parts: table top, legs, frame]
         [writes material definition + binding into assets/table/mtl.usda]
         Bound /table/mtl/wood_varnished to table top

The result is a production-ready asset folder:

assets/single_table/
  single_table.usda   <- root (references geo + mtl)
  geo.usda            <- geometry (untouched from source)
  mtl.usda            <- materials inline + bindings

Scene Output

The scene file (scene.usda) contains only references and lights: no material data, no geometry copies, no sublayers. Clean and readable:

def Xform "Scene" (kind = "assembly") {
    def Xform "Furniture" {
        def Xform "Table_01" {
            xformOp:translate = (5, 0, 4)
            xformOp:scale = (0.01, 0.01, 0.01)
            def Xform "asset" (
                references = @assets/single_table/single_table.usda@
            ) { }
        }
    }
    def Xform "Lighting" {
        def DistantLight "Sun_01" { ... }
        def DomeLight "Environment_01" { ... }
    }
}

Open it in Maya, Omniverse, usdview, or any USD-compatible tool to refine.

---

🚀 Quick Start

Requires uv (handles Python automatically).

# Clone
git clone https://github.com/binary-core-llc/bowerbot.git
cd bowerbot

# Install
uv sync

# First-time setup (creates ~/.bowerbot/config.json)
uv run bowerbot onboard

# Create a project and start building
uv run bowerbot new "Coffee Shop"
uv run bowerbot open coffee_shop

The onboard wizard asks for your LLM API key and the directories for your asset library and projects. Everything is stored in ~/.bowerbot/config.json. Skills are extension packages you install separately; after pip install bowerbot-skill-<name>, add the skill's config to the skills block of config.json.

---

📺 Tutorials

New to BowerBot? Watch the tutorial playlist on YouTube for setup walkthroughs, scene building demos, and tips for working with USD pipelines.

---

🛠️ CLI Commands

Command	Description
bowerbot new "name"	Create a new project
bowerbot open name	Open a project and start chatting
bowerbot list	Show all projects
bowerbot chat	Auto-detect project in current directory
bowerbot build "prompt"	Single-shot build (auto-creates project)
bowerbot skills	List scene builder tools and enabled skills
bowerbot info	Show current configuration
bowerbot onboard	First-time setup wizard

---

📁 Projects

Each project is a self-contained folder with metadata, scene, assets, and packaged output in one place:

scenes/coffee_shop/
  project.json    # Metadata: name, created_at, updated_at, scene_file
  scene.usda      # The USD stage (references only, clean and readable)
  scene.usdz      # Packaged output (Apple Vision Pro, Omniverse, etc.)
  assets/         # ASWF folders + self-contained USDZs used by this scene
  textures/       # Scene-level textures (HDRI maps for DomeLights, etc.)

Projects are resumable. Close the session, come back later, and continue where you left off:

$ bowerbot open coffee_shop
# Project: Coffee Shop
# Scene: scene.usda (5 object(s))

You: Show me the scene structure
BowerBot: Scene has 5 objects...

You: Remove Table_03
BowerBot: Removed /Scene/Furniture/Table_03

---

🔌 Skills

Skills extend BowerBot with external asset providers, DCC connectors, and simulation runtimes. Each skill is a separate Python package, discovered at runtime through Python entry points (bowerbot.skills). To add a provider, you pip install bowerbot-skill-<name> and BowerBot picks it up automatically. The skill SDK lives in bowerbot.skills; skills themselves ship and version on their own.

Scene Builder Tools

BowerBot's core tools for building USD scenes:

Tool	Description
create_stage	Initialize a new USD scene with standard hierarchy
place_asset	Add an asset (auto-creates ASWF folder for loose geometry)
place_asset_inside	Nest an asset inside an ASWF container's contents.usda
move_asset	Reposition an existing object without creating duplicates
compute_grid_layout	Calculate evenly spaced positions
list_scene	Show current scene with positions and bounding boxes
rename_prim	Move/rename objects in the hierarchy
remove_prim	Delete objects from the scene
create_light	Add native USD lights (sun, dome, point, area, disk, tube)
update_light	Modify an existing light's properties
remove_light	Delete a light from the scene or asset
create_material	Author a procedural MaterialX material and bind it to a prim
bind_material	Apply a material to a specific mesh part (writes into asset mtl.usda)
remove_material	Clear material binding from a prim
list_materials	Show all materials and their bindings
cleanup_unused_materials	Prune material definitions no prim binds to (per asset or project-wide)
list_prim_children	Discover mesh parts inside a referenced asset
list_project_assets	Show asset folders with scene usage status
delete_project_asset	Remove an asset folder (checks references first)
delete_project_texture	Remove a texture file (checks references first)
search_assets	Find USD assets in the user's library by keyword (geo, mtl, package)
list_assets	List every USD asset in the user's library, classified by category
search_textures	Find HDRIs and material maps in the asset library by keyword
list_textures	List every HDRI and material map in the asset library
validate_scene	Check for USD errors
package_scene	Bundle as .usdz

Extension Skills

Each skill ships as a separate pip package and is discovered at runtime via the bowerbot.skills entry-point group. A skill bundles its own Python module, a SKILL.md that teaches the LLM when and how to use it, and the four-folder layout BowerBot enforces (schemas/, services/, tools/, utils/).

Available:

• bowerbot-skill-sketchfab — searches and downloads models from your own Sketchfab account in USDZ format (your curated assets, not the public marketplace). Install with pip install bowerbot-skill-sketchfab.

More providers are planned (PolyHaven, Fab, CGTrader, Objaverse). You can write your own for any asset source, DCC, or simulation runtime. See CONTRIBUTING.md for the contract and a worked pyproject.toml example.

---

⚙️ Configuration

All settings live in ~/.bowerbot/config.json:

{
  "llm": {
    "model": "gpt-4.1",
    "api_key": "sk-...",
    "temperature": 0.1,
    "max_tokens": 4096,
    "context_window": null,
    "summarization_threshold": 0.75,
    "num_retries": 3,
    "request_timeout": 120.0,
    "max_tool_rounds": 25
  },
  "scene_defaults": {
    "meters_per_unit": 1.0,
    "up_axis": "Y",
    "default_room_bounds": [10.0, 3.0, 8.0]
  },
  "skills": {
    "sketchfab": {
      "enabled": true,
      "config": { "token": "your-sketchfab-token" }
    }
  },
  "assets_dir": "./assets",
  "projects_dir": "./scenes"
}

Switch models by changing one line:

{ "model": "gpt-4.1" }
{ "model": "anthropic/claude-sonnet-4-6" }
{ "model": "deepseek/deepseek-chat" }

Tested Models

Model	Tool Calling	Instruction Following	Recommended
gpt-4.1	Excellent	Excellent	Yes (default)
gpt-4.1-mini	Good	Good	Yes (budget)
gpt-4o	Poor	Poor	No (skips tool calls, ignores SKILL.md)
anthropic/claude-sonnet-4-6	Excellent	Excellent	Yes

BowerBot relies heavily on tool calling and SKILL.md instructions. Models that don't follow tool-calling patterns reliably will produce poor results.

Token Management

BowerBot automatically manages conversation context to stay within model limits. Two settings control this:

Setting	Default	Description
context_window	null	Context window size in tokens. null = auto-detect from the model.
summarization_threshold	0.75	Fraction of context budget that triggers history summarization.

Additional tuning options (usually don't need changing):

Setting	Default	Description
tool_result_age_threshold	2	User turns before old tool results are compressed.
min_keep_recent	6	Minimum recent messages always kept verbatim.
summary_max_tokens	512	Max tokens for the summarization LLM call.

Tool-Calling Loop

BowerBot runs a loop where the LLM requests tool calls, BowerBot executes them, and the results are fed back. Complex requests (e.g. binding materials to many mesh parts at once) can require many rounds.

Setting	Default	Description
max_tool_rounds	25	Maximum LLM ↔ tool exchange rounds per request. Increase if BowerBot stops with "Reached maximum tool-calling rounds" on legitimate workflows.

Error Recovery

BowerBot automatically handles transient API errors:

Setting	Default	Description
num_retries	3	Retries for rate limits and transient errors (429, 500, 503).
request_timeout	120.0	Seconds before a request times out.

• Rate limits and transient errors are retried automatically with exponential backoff.
• Validation errors are fed back to the LLM so it can auto-fix issues and re-validate.
• Permanent errors (bad API key, unknown model) show a clear message without crashing.

---

🏗️ Architecture

BowerBot is organized FastAPI-style:

• schemas/ describe data (pydantic models + enums)
• utils/ are pure-function primitives (no SceneState, no orchestration)
• services/ are state-aware orchestrators, one function per tool, signature (state, params), calls utils and other services freely, raises on errors
• tools/ are the LLM-facing surface, thin adapters that guard preconditions, call ONE service, wrap the result in ToolResult

Adding a feature is the same three-file change every time: schema, service, tool.

src/bowerbot/
  agent.py            # LLM tool-calling loop and prompt assembly
  cli.py              # Click CLI
  config.py           # Settings from ~/.bowerbot/config.json
  project.py          # Project lifecycle (create / load / resume)
  state.py            # SceneState: the context threaded through every tool handler
  dispatcher.py       # Aggregates tool defs + routes tool calls to handlers
  token_manager.py    # Conversation compression and summarization

  prompts/            # LLM instructions as markdown (editable without code changes)
    core.md
    scene_building.md

  schemas/            # Pydantic models and enums, grouped by domain
    assets.py         #   Asset formats, categories, ASWF layer names, metadata
    transforms.py     #   TransformParams, PositionMode, PlacementCategory, SceneObject
    lights.py         #   LightType, LightParams
    materials.py      #   MaterialXShaders, ProceduralMaterialParams
    textures.py       #   HDRI / image / texture-category enums
    validation.py     #   Severity, ValidationIssue, ValidationResult

  services/           # State-aware orchestrators, one per tools module
    stage_service.py       #   create_stage, list_scene, rename_prim, move_asset, ...
    asset_service.py       #   place_asset, place_asset_inside, list/delete_project_*
    library_service.py     #   list_assets, search_assets, find_package_for
    light_service.py       #   create_light, update_light, remove_light
    material_service.py    #   create_material, bind_material, list/remove/cleanup
    texture_service.py     #   list_textures, search_textures
    validation_service.py  #   validate_scene, package_scene

  tools/              # LLM-facing API layer (tool defs + thin handlers)
    _helpers.py            #   Precondition guards (require_stage / project / library)
    stage_tools.py         #   create_stage, list_scene, rename_prim, move_asset, ...
    asset_tools.py         #   place_asset, place_asset_inside, list/delete_project_*
    library_tools.py       #   search_assets, list_assets
    light_tools.py         #   create_light, update_light, remove_light
    material_tools.py      #   create_material, bind_material, list/remove_material
    texture_tools.py       #   search_textures, list_textures
    validation_tools.py    #   validate_scene, package_scene

  skills/             # Skill SDK. The contract every skill implements.
                      # Skills themselves ship as separate pip packages.
    base.py                #   Skill, SkillContext, SkillConfigError,
                           #   SkillCategory, Tool, ToolResult
    registry.py            #   Entry-point discovery and tool routing

  utils/              # Pure-function primitives shared by services
    stage_utils.py           #   Stage create/open/save, references, transforms, prims,
                             #   ref-path scanning, LIGHT_CLASSES
    asset_intake_utils.py    #   intake_folder, intake_usdz, create_asset_folder, ASWF
    asset_folder_utils.py    #   ASWF folder primitives (detect root, layer scopes,
                             #   resolve_asset_dir_for_prim)
    library_utils.py         #   scan_library, find_package_for
    light_utils.py           #   light_in_folder primitives, HDRI staging
    material_utils.py        #   material_in_folder primitives, find_first_material
    texture_utils.py         #   find_textures, copy_texture_to_project
    validation_utils.py      #   validate_stage, package_to_usdz
    geometry_utils.py        #   Bounds, unit conversion, layout math
    dependency_utils.py      #   USD dependency tree walker
    naming_utils.py          #   Name sanitization for files, prims, projects
  gateway/            # Future: FastAPI + MCP server

Design principles

• One tools module, one service: every service module backs exactly one tool surface, with one orchestrator per tool. Shared primitives live in utils/, called freely by any service.
• Functions only in tools / services / utils: classes live in schemas/ (pydantic models, enums) and a small set of state objects (SceneState, Project).
• Tools are thin: guard preconditions, call ONE service, wrap in ToolResult. No business logic, no util calls, no cross-service routing.
• Services own orchestration: take (state, params), do the cross-service and multi-util work, mutate state, raise on errors.
• Utils are pure primitives: no SceneState, no other services. Composable building blocks.
• State lives in one place: SceneState holds the open stage, the project binding, the asset library path, and the object counter; tool handlers thread it into service calls.
• All pxr is in services/ and utils/: the rest of the codebase never imports pxr directly.
• Prompts are content: editable .md files, not Python constants.
• Skills are external integrations: new asset providers ship as Python packages discovered via entry points.
• One config file: ~/.bowerbot/config.json, no .env.

---

📐 USD Compliance

Every scene follows OpenUSD best practices and the ASWF asset structure guidelines:

Scene level

• metersPerUnit = 1.0, upAxis = "Y", defaultPrim always set
• Standard hierarchy: /Scene/Architecture, /Scene/Furniture, /Scene/Products, /Scene/Lighting, /Scene/Props
• References only: no inline geometry, no scattered material sublayers
• Wrapper-prim pattern isolates scene-level transforms from asset-internal ones, so DCC export transforms (Maya pivots, rotations) stay untouched
• Pre-packaging validator checks defaultPrim, units, up-axis, reference resolution, and material bindings

Asset level

• References (not sublayers) per ASWF guidelines, for predictable opinion strength
• Materials inline in mtl.usda, lights inline in lgt.usda, nested references in contents.usda
• Automatic metersPerUnit conversion across composition boundaries

---

🗺️ Roadmap

What's next for BowerBot. Contributions welcome:

• USD Variant Sets: ASWF-compliant variants on asset root prims (materials, geometry, lighting, configurations, and more)
• Scene templates: JSON-driven scene assembly with asset resolution
• DCC exporter: Maya/Houdini tool to export scene layout as BowerBot JSON
• More asset providers: Fab, PolyHaven, Objaverse, CGTrader skills
• MCP Gateway: FastAPI server for web UI and external AI clients
• Web UI: chat panel + live 3D viewport
• BowerHub: community skill registry

---

🤝 Contributing

BowerBot is open source and welcomes contributions. The best way to start is writing a new skill for an asset provider, DCC, or simulation runtime you use. Skills ship as separate pip packages discovered through the bowerbot.skills entry-point group.

Read CONTRIBUTING.md for the skill contract, the required FastAPI internal layout, and a worked pyproject.toml example for a stand-alone skill package.

---

📄 License

Copyright 2026 Binary Core LLC

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

---

Built with 🐦 by Binary Core LLC

"The bowerbird doesn't have the flashiest feathers. It just builds the most compelling world."