ES3D adds component-level 3D editing via semantic embeddings

A paper on arXiv introduces ES3D, a framework for component-based editing of 3D assets. The authors propose embedding semantics directly into 3D space: multi-view semantic features are projected into the asset's voxelized space, after which the desired components are located by similarity to image or text query embeddings. Editing is performed by a pretrained generative 3D model with an inpainting mechanism, which modifies the found components based on reference images while preserving the rest of the geometry. The authors have not released the code. More details are in the paper on arXiv.

What it means

ES3D tackles a task that is absent from the fact sheets of tools in our reference. OpenLRM, Hunyuan3D 2.1, and TRELLIS are generators that create models from a single image (image-to-3d). They produce the asset as a whole and offer no mechanism for selecting an individual component by semantic query or for editing it in isolation while preserving the rest of the geometry. The closest in topic is TRELLIS: its description mentions structured 3D latents, which could potentially enable more controllable generation, but the fact sheet contains no information about component search or editing based on multiple references. Hunyuan3D 2.1 is positioned as a generator of highly detailed assets with PBR materials, yet nothing in the description covers semantic component labeling or their separate editing. OpenLRM is the simplest of the three models, focused on image-based reconstruction, with no component control mechanisms at all. For now, ES3D exists only as a research paper without open source, so a practical comparison with available tools is not possible. If the authors release an implementation, it could broaden the understanding of how 3D asset editing works in the AI tools for 3D section.