Run it yourself

Everything below runs from a clone of the repository. Only the world-model calls and an optional GPU cost money; every paid stage resumes rather than resubmits.

Prerequisites

Python 3.12 (pipeline, OpenUSD conversion) and 3.13 (the bpy wheel for Blender), Node 22 for tests.
ffmpeg; COLMAP 3.9 for photoreal training.
A World Labs API key for the world-model stages, set as WLT_API_KEY. Never commit keys.
AWS credentials and the SAM CLI to deploy the backend.

Test and view

python -m pip install pytest && pytest -q tests
node --test tests/*.test.mjs
python -m http.server -d public 8000      # http://localhost:8000

Step 1: video to world model

python3.12 -m venv .venv
.venv/bin/pip install -r requirements.txt numpy scipy OpenEXR

# frames: pick a window of the tour, crop broadcast graphics (0 credits)
scripts/ingest_and_build.sh --video-file tour.mp4 --start 305 --interval 4 --crop-bottom 264 --ingest-only

# poses and metric depth (10 credits a frame), then local fusion (free)
.venv/bin/python scripts/atlas_twin.py pose  --input-dir data/ctfb-sorting/frames --work-dir build/atlas_sorting
.venv/bin/python scripts/atlas_twin.py fuse  --work-dir build/atlas_sorting \
    --mask-boxes data/ctfb-sorting/people_boxes.json --output build/atlas_sorting/out/zone.usdz
.venv/bin/python scripts/atlas_twin.py site  # places zones from data/site/layout.json

Step 1, free: photoreal splats

.venv/bin/pip install mediapipe opencv-python-headless scipy torch
sudo apt-get install colmap libgles2 libegl1
.venv/bin/python scripts/photoreal_twin.py frames --zone racking --video tour.mp4
for s in masks select fill pose; do .venv/bin/python scripts/photoreal_twin.py $s --zone racking; done
.venv/bin/python scripts/photoreal_twin.py train  --zone racking --iters 6000
.venv/bin/python scripts/photoreal_twin.py export --zone racking

Zones are racking, market, kitchen and sorting (sorting needs --matcher exhaustive --init-tri-angle 3 at the pose stage). On a GPU, notebooks/photoreal_gpu.ipynb runs the same stages on a free Colab or Kaggle T4.

Step 1b: Blender facility

python3.13 -m venv .venv-render && .venv-render/bin/pip install bpy pillow numpy
.venv-render/bin/python scripts/build_facility.py           # public/facility.glb + facility.usdz
.venv-render/bin/python scripts/build_facility.py --blend --render   # also a .blend and a Cycles still

The OpenUSD twin

pip install -r requirements-render.txt
python scripts/operations_to_usd.py      # public/operations.usda
python scripts/align_twin_usd.py         # public/twin.usda + twin_crop.usdc

Step 2: deploy to AWS

scripts/deploy.sh     # sam build + deploy (us-east-2 by default), syncs public/ to S3, prints URLs

Then set the repository variable TWIN_API_URL to the printed ApiBaseUrl and rerun the "Demo site" workflow; the viewer switches from "Simulated feed" to "Live from AWS". deploy.sh does both when gh is signed in.

Step 3: render

.venv-render/bin/python scripts/render_usd_splats.py public/site.usdz --shots docs/renders/shots.json --out renders/
blender -b -P scripts/blender_render_twin.py -- public/site.usdz --shots docs/renders/shots.json --out renders/blender

Omniverse: open public/twin.usda in a Kit app (USD Composer or USD Viewer from the Kit App Template, or Isaac Sim) on an RTX GPU. See Visualize.