Posed images to splats
Reconstruct refined Gaussian splats from posed RGBD images.
posedImages2Splats reconstructs Gaussian splats from posed RGBD images, then
refines them by rendering new views, cleaning them up, and training the splats
against them. The result is an SPZ file.
The task is asynchronous and currently available as a Marble 2 beta endpoint.
Each successful run costs:
- Refined (the default): 5,000 credits ($5).
- Draft, with
refine: false: 500 credits ($0.50).
Failed runs are free.
Prepare the capture
Provide 2 to 512 ordered posedFrames. Each frame combines an imageAsset
with its camera and its depth: a linear-depth EXR under depth.depthAsset,
as images2PosedRGBD returns for each image, or
atlasGenerate with returnDepth. An asset can be a
project-owned asset ID, inline base64 smaller than 10 MiB, or a public HTTPS
URL. Every frame needs depth. Keep each returned image, camera, and depth
together. Every camera grid must be one of 1280x720, 720x1280, 960x720, or
528x704 (width x height), and all frames must declare the same grid.
RGB images are resampled to that grid. The camera
intrinsics describe this grid, which can differ from the original image. See
Cameras and posed images.
Outputs from images2PosedRGBD and atlasGenerate at 1280x720, 960x720,
528x704, or 720x1280 can be passed directly. images2PosedRGBD returns one of
those grids for multi-view captures with camera translation; for a single image
or a capture without camera translation, set targetResolution to one of them.
The request accepts as few as two frames. For a single room, start with 8–16 well-distributed, overlapping views and add views to cover missing areas. Check that objects agree across views, including generated ones, before submitting. The Splat best practices guide covers suitable scenes, common artifacts, and capture examples.
For a worked example using generated views, follow Generate views and reconstruct splats. Inspect agreement across the inputs before reconstruction: refinement does not guarantee that inconsistent depths or conflicting object placements will be fixed.
Describe the scene
prompt is optional. Prompt enhancement is on by default: it combines your
description with the input images, or captions the images when you send no
prompt. Set enhancePrompt to false with a prepared prompt to skip
enhancement and reduce latency.
Wait for the result
A run usually takes 10 to 20 minutes, longer than a ?wait=true request stays
open. Submit without ?wait=true, persist the operation ID, and poll or call
:wait until done is true, or pass a webhookUrl to be notified when it
finishes. See Operations and Webhooks.
Example completed output
When the operation reports done: true, check for operation.error before
reading a successful result from operation.response. refinedSplats maps
each published splat count to its SPZ file; the count in this example is illustrative.
Pass the asset to splats2Mesh as its
splatAsset to extract a mesh. Open the
posedImages2Splats reference for the
exact fields, constraints, and response shape.