🍏🍏 GAUDI: the Neural Architect 🍏🍏

👉Novel generative model for immersive 3D scenes from a moving camera

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Hundreds of thousands pics/scenes
✅Novel denoising optimization objective
✅New SOTA across multiple datasets
✅Un/conditional on images/text

More: https://bit.ly/3Bt65ye

🚜NeDDF: the NeRF evolution!🚜

👉Novel 3D representation that reciprocally constrains distance & density fields

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅NeRF provides no distance
✅Extending for arbitrary density
✅Density via dist-field & gradient
✅Alleviating the instability

More: https://bit.ly/3Bte8LC

🔥AND/OR: Composable Diffusion Models🔥

👉Novel neural compositional generation via Composable Diffusion Models

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅DM as energy-based models
✅Connecting diffusion models
✅Conjunction & negation, on top of DM
✅Zero-shot combinatorial generalization

More: https://bit.ly/3PYv1Cs

🔥 MobileNeRF is out -> Pure Fire! 🔥

👉MobileNeRF is out: the mobile evolution of NeRF via textured polygons.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Same quality, 10x faster than SNeRG
✅Memory-- by storing surface textures
✅Integrated GPUs: less memory/power
✅Suitable for browser & viewer is HTML

More: https://bit.ly/3PUKPWy

🔥🔥UPDATE🔥🔥

Code Released: https://github.com/andreas128/RePaint

🧣NeRF for Outdoor Scene Relighting🧣

👉NeRF-OSR: the first neural radiance fields approach for outdoor scene relighting

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅NeRF-method for outdoor relighting
✅Simultaneous illumination/viewpoint
✅Control over shading, shadow, albedo
✅Self-Supervised training from outdoor
✅Dataset: 3240 viewpoints, 110+ times

More: https://bit.ly/3vBiH2G

👩‍🦰 Real-Time Neural Hair 👩‍🦰

👉Accurate hair geometry & appearance from multi-pics

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Bonn, CMU and Reality Labs
✅Photorealistic Real-Time render
✅HQ strand geometry/appearance
✅Novel scalp texture description
✅Intuitive manipulation of 3D hair

More: https://bit.ly/3vBiH2G

🚀 #VR by NASA - 1985 🚀

👉Q: is #VR the technology that developed least in the last 40 years? 🤔

Let's talk: https://bit.ly/3JxDZ7i

🔥 MinVIS, a new SOTA is out 🔥

👉#Nvidia miniVIS: no video-based architectures nor training procedures🤯

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Video architecture/train not required
✅MinVIS outperforms the previous SOTA
✅Occluded VIS (OVIS): >10% improvement
✅1% of labeled frames >> fully-supervised

More: https://bit.ly/3pcYzk1

🔥🔥MultiNeRF: three NeRFs are out!🔥🔥

👉Google opens the code of three #cvpr2022 papers: Mip-NeRF 360, Ref-NeRF, RawNeRF

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Paper_1: Mip-NeRF 360
✅Paper_2: Ref-NeRF
✅Paper_3: NeRF in the Dark

More: https://bit.ly/3QjpRRc

☀️LocoProp: Neural Layers Composition☀️

👉Google AI unveils LocoProp: novel neural paradigm for modular composition of layers.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Backprop++ via Local Loss Optimization
✅Layer-based w-reg, target output, loss
✅Multiple local update via first-order opt.
✅Superior performance and efficiency

More: https://bit.ly/3Q40YJn

🔥PCVOS: clip-wise mask VOS🔥

👉PCVOS: new semi-supervised video object segmentation method

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Reformulating semi-supervised VOS
✅Novel per-clip inference perspective
✅Clip-wise operation on intra-clip
✅PCVOS: model for per-clip inference
✅New SOTA on multiple benchmarks

More: https://bit.ly/3vJtmbz

🍑 World-Object Detection via ViT 🍑

👉Google unveils OWL-ViT: open-vocabulary detector based on ViTs 🤯

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅ViTs for Open-World Localization
✅Img-level to open-vocabulary detection
✅SOTA one-shot (img.cond.) detection

More: https://bit.ly/3Sy3jOj

🎹🎹 Learning Piano in #AR 🎹🎹

👉PianoVision (on #META #Quest2) accelerates the piano learning via Passthrough #AR & hand tracking

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Sheet Insight to learn sight-read
✅MIDI keyboard connectivity
✅Air piano for no physical pianos
✅Multiplayer Music Instruction
✅PianoVision Music Hall in #VR

More: https://bit.ly/3zYvwGX

🧊EPro-PnP: Persp-n-Points Detection🧊

👉EPro-PnP: probabilistic PnP layer for general e2e pose estimation

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Probabilistic PnP for general e2e pose
✅Top-tier in 6DoF by inserting into CDPN
✅Deformable accurate detection
✅2D-3D corresp. learned from scratch

More: https://bit.ly/3BNPXYr

🥇#NVIDIA wins SIGGRAPH's Best Paper🥇

👉Instant #NeRF awarded as a best paper at SIGGRAPH 2022!

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Speed-up of several orders of magnitude
✅HQ neural primitives in a matter of secs
✅Render in tens of milliseconds at 1080p
✅Source code and resources available!

More: https://bit.ly/3Qt8c9D

🪰 EasyMocap: Open Neural Mocap 🪰

👉EasyMocap: open-source marker-less mocap with novel view synthesis from RGB

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬 (of last paper added):
✅Editable free-viewpoint video
✅Layered neural representation of humans
✅Multi-pax -> instances, weakly-supervised
✅HQ neural representation of the humans
✅Addressing camera error by human poses

More: https://bit.ly/3p6lUDO

🎰 Texturify: Neural Textures Generator 🎰

👉A step towards automated content creation. HQ textures directly on surface of 3D object

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅TUM + Max Planck + Apple 🍏
✅Realistic, HQ textures from 2D pics
✅3D shape geometry, no 3D supervision
✅3D-aware surface-based generation net

More: https://bit.ly/3BW7UUU

🍨 Scaling Neural Indoor Scene 🍨

👉Neural scene rendering for indoor: scalable in both training/rendering

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Neural scene rendering for indoor
✅#3D into tiles with MLPs to scale up
✅Parallel training of tile-based MLPs
✅View-indep. components (via surf-MLP)

More: https://bit.ly/3bH94IX

🔥🔥UPDATE🔥🔥

Code Released: https://github.com/google-research/jax3d/tree/main/jax3d/projects/mobilenerf