You May Like Getting NeRF-ed By NVIDIA, It Adds A New Dimension To Your 2D Photos

Turn A Series Of Photos Into A 3D Scene

Nvidia’s Instant NeRF is an evolution of the original version of their neural radiance field tech, a fancy way of describing how one of their high end cards can stich together a number of consecutive still shots into a 3D render of the subject.  It works best with limited or no moving content, as the movement will render the final product somewhat blurry.  This is in part because of how Nvidia’s Instant NeRF works.

It doesn’t try to fill in the blanks using the subjects in the photo, instead it uses their AI algorithms to model the light in the photos and how it reflects as well.  The software is capable of predicting the color of light radiating in the picture, from any location and any direction, even if that light is obscured by an object in the pictures.  The biggest hurdle they faced, as you might well expect, is the training of the neural net to process images.  In the past it took several days to train, now NVIDIA has managed to reduce it to a few seconds on a few dozen still photos.

While it is best run on HPC cards built on Ampere or Hopper but in theory a regular high end RTX card should be able to make a go of it.  It will take some work to set up if you are interested, and this GitHub repository is a good place to start.