Shanghai AI Laboratory
Meshes generated by GetMesh. We focus on geometry generation, and materials of the meshes are generated by MATLABER.
Overview of the mesh autoencoder. Points are sampled from the surface of the input mesh and encoded to a varying number of latent points. The latent point representation is re-organized to the triplane representation by projecting the points to the triplane. DMTet is utilized to extract a coarse mesh from the triplane and a refinement module further refines the coarse mesh.
Mesh is a fundamental representation of 3D assets in various industrial applications, and is widely supported by professional softwares. However, due to its irregular structure, mesh creation and manipulation is often time-consuming and labor-intensive. In this paper, we propose a highly controllable generative model, GetMesh, for mesh generation and manipulation across different categories. By taking a varying number of points as the latent representation, and re-organizing them as triplane representation, GetMesh generates meshes with rich and sharp details, outperforming both single-category and multi-category counterparts. Moreover, it also enables fine-grained control over the generation process that previous mesh generative models cannot achieve, where changing global/local mesh topologies, adding/removing mesh parts, and combining mesh parts across categories can be intuitively, efficiently, and robustly accomplished by adjusting the number, positions or features of latent points.
GetMesh: A Controllable Model for High-quality Mesh Generation and Manipulation
Zhaoyang Lyu*, Ben Fei*, Jinyi Wang*, Xudong Xu, Ya Zhang, Weidong Yang, Bo Dai
GetMesh generates the geometry and then MATLABER generates the materials.
Turning a twin-engine passenger airplane into a four-engine transport airplane.
Turning a car into an airplane.
Combine the wings and tail of an airplane with a car.
Combine the legs of a chair with a bathtub.
Combine the armrests of a chair with another chair.
Combine a lamp with a table.
@misc{lyu2024getmesh,
title={GetMesh: A Controllable Model for High-quality Mesh Generation and Manipulation},
author={Zhaoyang Lyu and Ben Fei and Jinyi Wang and Xudong Xu and Ya Zhang and Weidong Yang and Bo Dai},
year={2024},
eprint={2403.11990},
archivePrefix={arXiv},
primaryClass={cs.CV}
}