Learning Gradient Fields for Shape Generation
* github
* paper
In this work, we propose a novel technique to generate shapes from point cloud data. A point cloud can be viewed as samples from a distribution of 3D points whose density is concentrated near the surface of the shape. Point cloud generation thus amounts to moving randomly sampled points to high-density areas. We generate point clouds by performing stochastic gradient ascent on an unnormalized probability density, thereby moving sampled points toward the high-likelihood regions. Our model directly predicts the gradient of the log density field and can be trained with a simple objective adapted from score-based generative models. We show that our method can reach state-of-the-art performance for point cloud auto-encoding and generation, while also allowing for extraction of a high-quality implicit surface.
#3d #generative

GAN sculptures
#gan #machine_learning #3d #generative
https://studioanf.com/ai-sculptures/

Growing 3D Artefacts and Functional Machines with Neural Cellular Automata

* paper
* thread

We show that neural cellular automata can learn to grow not only complex 3D artifacts with over 3,000 blocks but also functional Minecraft machines that can regenerate when cut in half.

#generative #3D #nca

NeRF-VAE: A Geometry Aware 3D Scene Generative Model
* abs
* pdf

We propose NeRF-VAE, a 3D scene generative model that incorporates geometric structure via NeRF and differentiable volume rendering. In contrast to NeRF, our model takes into account shared structure across scenes, and is able to infer the structure of a novel scene -- without the need to re-train -- using amortized inference. Our model is a VAE that learns a distribution over radiance fields by conditioning them on a latent scene representation. We show that, once trained, NeRF-VAE is able to infer and render geometrically-consistent scenes from previously unseen 3D environments using very few input images. We further demonstrate that NeRF-VAE generalizes well to out-of-distribution cameras, while convolutional models do not.

#nerf #vae #generative