OpenAI's Shap-E: Dall-E for 3D Objects
OpenAI is at it again with Shap-E, their newest model that generates 3D objects from text, much like Dall-E creates 2D images. According to OpenAI, this model is "a conditional generative model for 3D assets. Unlike recent work on 3D generative models which produce a single output representation, Shap-E directly generates the parameters of implicit functions that can be rendered as both textured meshes and neural radiance fields."
Shap-E is trained on a combination of mapping 3D assets and a conditional diffusion model. While this program is free to run, it is more challenging to install and set up than OpenAI's popular ChatGPT. The model can be downloaded on GitHub and accessed on Microsoft Paint 3D. It can also be converted into an STL file for 3D printing. However, technical knowledge may be required to install and run the model.
The editor-in-chief of Tom's Hardware tested out Shap-E and found that it took him eight hours to understand. Once he installed the model, he was able to test prompts with color-animated GIF files and monochrome PLY files, with the animated GIFs being favorable. Some prompts included a shark, a Minecraft creeper, and "an airplane that looks like a banana," all with varying levels of quality depending on their file type.
The Shap-E model requires a lot of system resources from a PC and is compatible only with Nvidia GPUs. High-performance CPUs are also necessary to render objects in a matter of minutes as opposed to hours.
