Skip to content
← All articles
Project introduction

MistubaVideo: rendering simulation recordings with Mitsuba

Listen to the audio story

Turning simulation data into video · 2:14 · AI-written script, synthetic voices

Transcript

Ava: A physics simulation produces numbers: where bodies moved, which patches of a surface changed. How do you turn that into a movie you can actually watch?

Andrew: I guess you need something that draws it.

Ava: That's MistubaVideo. It takes simulation exports and renders them into images and video with Mitsuba 3, a renderer that traces light paths for realistic pictures.

Andrew: Why separate the drawing from the simulation?

Ava: Because once you record the simulation, you can change the camera or the lighting without running it again. It's like filming a play once and then editing from every angle.

Andrew: So what goes in?

Ava: Two routes. A batch renderer handles a sequence of O B J mesh files, and FFmpeg stitches the frames into a video or a GIF. The second route is a helper called mitsuba-video, which replays recorded body poses and surface fields, such as wear on a material.

Andrew: What's a surface field?

Ava: A picture laid over a surface, mapped on a flat atlas, that says how much each region changed. The helper reads a manifest, a small file listing bodies, meshes, materials, poses and field images. Another simulator can write that same format.

Andrew: And then for each frame?

Ava: It applies the recorded poses, blends the base material with the affected one using the field, and renders the scene. It can add a heatmap, a panel showing the atlas, and a chart of covered area, then encodes an MP4.

Andrew: Is it easy to try?

Ava: Three commands. Inspect checks the input, preview checks the framing, and render makes the frames. There are quality presets named preview, draft and paper.

Andrew: What comes out?

Ava: Beauty images, composite frames, a C S V of field area, and a config file with the settings and timings, so a video can be reproduced later. Linear E X R frames are optional.

Andrew: And it's tested?

Ava: The package has pytest tests for transforms, material blending, the O B J and M T L readers, and the manifest.

Andrew: So a recording becomes a clear, repeatable video. The full write-up has the pipeline diagram.

Table of contents

MistubaVideo turns simulation exports into images and video with Mitsuba 3. Its batch renderer handles OBJ sequences; the mitsuba-video helper replays recorded body poses and surface fields such as wear.

1. Where rendering fits

A simulation computes motion and changes to surfaces. The renderer reads those outputs to show where bodies moved, which regions of the UV atlas changed, and how the material appearance evolves. Rendering a recording lets you change the camera or lighting without repeating the simulation.

python main.py multi --folder exports renders an OBJ sequence. Camera views, resolution and samples per pixel control the output; FFmpeg assembles the images into video or GIF. The field renderer adds per-frame transforms and textures.

A rough-conductor material rendered with Mitsuba.

Existing material illustration from the project report. It illustrates shading, without representing a wear-simulation result.

2. The render helper

StageData
InputMeshes, materials, recorded poses and surface fields
ProcessingSource adapter, material blend and Mitsuba rendering
OutputRendered frames, area CSV, settings and MP4

The mitsuba-video package accepts a versioned manifest or a TextureFriction recording through a source adapter. A manifest describes bodies, meshes, materials, poses and field images. ManifestWriter lets another simulator export this format.

For each selected frame, the helper applies recorded poses, blends base and affected materials using the surface field, and renders the scene. It can add a heatmap, a UV-atlas panel and a covered-area chart before encoding an MP4.

flowchart TD
  subgraph Inputs[Simulation output]
    R[TextureFriction recording]
    M[Manifest: meshes, poses and fields]
  end
  subgraph Helper[mitsuba-video helper]
    A[Source adapter]
    B[Apply poses and blend materials]
    C[Mitsuba 3 path tracing]
    D[Atlas panel and area chart]
  end
  R --> A
  M --> A
  A --> B --> C --> D
  D --> E[Frame images and MP4]

Start with inspect to check the input and preview to check framing. render produces the sequence; encode changes video assembly using existing frames. Quality presets include preview, draft and paper.

mitsuba-video inspect path/to/manifest.json
mitsuba-video preview path/to/manifest.json
mitsuba-video render path/to/manifest.json -q preview

3. Reading the output

Outputs include beauty images, composite frames, a field-area CSV and render_config.json with settings and timings. Linear EXR frames are optional. The chart measures the recorded field’s covered area. It does not measure the visible area of the rendered object.

Keep display settings with the render when comparing videos: changing the heatmap range can make the same field look different.

Mitsuba spotlight illustration from the project report.

Existing lighting illustration from the report. Scene lighting and the recorded surface field are separate inputs.

4. Sources and tests

The package documents pytest tests for transforms, material blending, OBJ and MTL readers, and the manifest schema and writer. A measured coverage percentage is not supplied here.

Render-helper documentation[1].

Implementation: source interface[2], material blending[3].

The project documentation provides the source link and terminology.

References

  1. ETSim. (). Render-helper documentation. [Source file or documentation] github.com.
  2. ETSim. (). source interface. [Source file or documentation] github.com.
  3. ETSim. (). material blending. [Source file or documentation] github.com.
  4. ETSim. (). MistubaVideo. [Software repository] github.com.

References use this article’s publication date.