Posed RGBD generation
Generate posed views from image, depth, and camera context.
atlasGenerate generates posed RGB views at target cameras from one or more
posed RGBD context frames. Each context frame supplies an image, depth buffer,
and camera. Marble uses these views as references for the generated scene.
Account availability:
POST /api/v2/tasks:atlasGeneraterequires access to this task on your account.
Prepare RGBD context
Every contextFrames entry requires an imageAsset, depth, and camera.
Keep those three values aligned to the same view and coordinate system. Target
cameras determine how many frames are generated and their order in the
response. All context cameras must declare the same width and height; that
resolution may differ from the target cameras' resolution.
When using images2PosedRGBD output, check the returned image's grid and
framing against its depth and camera before supplying it as context. Omitting
targetResolution can retain the original RGB beside lower-resolution depth.
See Choose the output grid.
Keep each context image and its maps on the grid described by its camera.
Target cameras specify the generated output size. The aspect-ratio requirement
for returnDepth is described below.
depth takes the same two encodings as atlasChisel context depth: a
linear-depth EXR in depthAsset, or an 8- or 16-bit inverted log-depth PNG
in logDepthAsset with its zMin and zMax. Mark sky and other pixels with no
surface as 0 (or NaN or infinity in an EXR), not as a distant surface; any
finite positive depth is read as geometry. See
Encode context depth.
All target cameras must share one size: 1280 × 720, 960 × 720, 528 × 704, or
720 × 1280 (width × height). Marble selects the model; atlasGenerate does
not accept a model selector.
Context depth is required and supplies sparse depth references for generation.
Prompt enhancement is enabled by default. Atlas writes a scene description from
your prompt and context images before generating. Preserve promptUsed from the
response to audit the description, or reuse it with enhancePrompt: false on a
later request.
Omit modelParameters.numSteps to use the selected model's default number of
generation steps.
Generate a video or camera path
To start from recorded footage, prepare still-image context using Using video footage with Atlas. It also covers how to insert a frozen-time camera move between synchronized recordings.
Sample the camera path at the final frame rate and submit the complete ordered
trajectory as targetCameras in one atlasGenerate request. Atlas handles the
sequence internally, generating keyframes and using them as context for the
frames between them. Assemble the returned images in response order at that
frame rate. Set returnDepth: false when only video frames are needed.
Do not split a continuous trajectory into independent frame requests or small batches as a performance optimization. Separate requests do not share generated sequence context, and changing the target count can change model routing. Reusing the same seed does not make separate requests equivalent to one sequence; stitching their outputs can introduce flicker and layout jumps. If a trajectory exceeds the supported request limits, shorten it or plan separate shots rather than assuming independently rendered batches will join continuously.
Use sparse preview renders to inspect framing, then regenerate those camera positions as part of the complete final sequence. Do not splice preview outputs into the final video. If waiting times out, resume the existing operation rather than resubmitting its frames separately; see Operations.
Choose context views for consistent geometry and useful coverage of the path. One posed image is valid context. Add generated RGBD views when they reveal needed areas and remain consistent with the scene; do not automatically include every generated view. More context is not a guarantee of better continuity.
Request output depth
Set returnDepth: true to reconstruct depth for the generated views and attach
it to each returned frame. The depth scale comes from the camera centers, so a
request whose context and target cameras all share one center (a pure-rotation
rig) is rejected with HTTP 422 at submission. Translate at least one camera,
or set returnDepth to false.
The context frames anchor the depth reconstruction. Their aspect ratio must
match the target cameras' (a 1920 × 1080 context image
works with 1280 × 720 targets, a 1280 × 720 one does not work with 960 × 720
targets), and every target camera's principal point must be within 5% of the
image center, or the request is rejected with HTTP 422 at submission. A
context frame helps reconstruct depth only when its camera's principal point is
centered.
Returned depth can contain NaN where the reconstruction found no reliable
geometry; see
Missing values in returned depth.
Reconstruct splats from generated views
atlasGenerate returns images, cameras, and optional reconstructed depth for
individual views. Views can look plausible on their own while disagreeing on
surface depth or object placement. Combining those views into one 3D scene can
expose seams, doubled surfaces, or flicker as the viewing camera moves.
Use posedImages2Splats to reconstruct a Gaussian splat asset from consistent posed RGBD views. Reconstruction and refinement do not guarantee that conflicting geometry or objects across inputs will be fixed.
Generate and inspect the views
Use the TypeScript client setup. In this example, wl is
your configured client, contextFrames contains your posed RGBD references,
and targetCameras contains overlapping views of the scene at 1280 × 720.
For a single room, start with 8–16 well-distributed views. Include translated
camera positions so output depth can recover scale.
Before submitting the reconstruction, inspect the returned images and their aligned depth from neighboring cameras. Check that shared objects agree in position and shape, views overlap, and important surfaces have valid depth. Replace inconsistent views or improve the context coverage before proceeding. Keep each returned image, camera, and depth together in the same coordinate system and scale. See Splat best practices.
Submit the reconstruction
After inspecting the frames, pass their images, cameras, and linear depth to
posedImages2Splats. Both tasks must be available to your account, and each
successful task is billed separately; see splat pricing.
The result maps splat counts to downloadable SPZ assets. Save both operation IDs. If waiting times out, resume the existing operation using Operations rather than submitting the same task again.
Get the frames in the submission response
A request sent with ?wait=true and enhancePrompt set to false can come
back from its submission already done: true. By default that response names
each output file by assetId and url. Set base64Outputs to true to
receive each file in that response as a base64 data URL instead. The service
then skips storing the files before it answers, and you skip downloading them. A
later read of the operation names the files by assetId and url either way.
Frames answered at submission are JPEG; frames the operation generates later are
PNG. Depth answered at submission is half-float EXR, which reads any depth above
65504 as infinity; depth the operation produces later is float32 EXR. See
Operations.
Example completed output
After the operation reports done: true, its task-specific result is in
operation.response. This example requested output depth:
When returnDepth is false, each frame contains only its generated
imageAsset and camera. Open the
atlasGenerate reference for
the live request limits and complete schemas.