🟨 Lang<->Pics in 100+ Languages 🟨

👉#Google PaLI: unified lang-image #AI to perform tasks in 109 languages 🤯

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅PaLI: Pathways Lang & Image model
✅Answering, captioning, reasoning, etc
✅From Eng. to 109 lang. understanding
✅The new SOTA on several datasets

More: https://bit.ly/3QMslHC

🍐PeRFception: Largest IR Dataset🍐

👉#Nvidia, a new frontier in data collection via Plenoxels: same info, -96.4% in size.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅POSTECH + NVIDIA + Caltech = 🤯
✅Size: -96.4% from original dataset!
✅2D/3D image/object class/semantic
✅Ready-to-use pipeline for implicit dataset

More: https://bit.ly/3eW9hJA

🐸 CHARL-E: Stable Diffusion in 1 click 🐸

👉CHARL-E packages Stable Diffusion into a simple app.

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅No setup, dependencies, or internet
✅Images with 1-click on #macbook
✅Suitable only for M1/M2 processor
✅Source code under MIT license

More: https://bit.ly/3xv2z3G

🍋YOLOPv2: Better Driving Perception🍋

👉YOLOPv2: simultaneous object, road segmentation & lane detection

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅E2E perception net with better backbone
✅Efficient ELAN for reasonable memory
✅Stability for adapting to scenarios
✅SOTA on BDD100K, +50% faster!
✅Source code under MIT license

More: https://bit.ly/3LvYGBh

🍈SegNeXt: new SOTA in Semantic Seg.🍈

👉SOTA (by large margin) on ADE20K, Cityscapes, COCO-Stuff, Pascal VOC, Pascal Context, and iSAID 🤯

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Novel tailored network architecture
✅Spatial attention via multi-scale feats
✅Encoder + conv. better than transformers
✅SOTA on several datasets (ADE20K, etc.)

More: https://bit.ly/3UrZhrH

🦪StereoVoxelNet: RT Obstacles Detection🦪

👉Novel deep neural approach to detect occupancy from stereo images directly

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅Occupancy voxels via deep learning
✅RT on Jetson-TX2 (-98% CPU of SOTA)
✅Optimization via octrees / sparse conv.
✅Real-world stereo in/outdoor dataset

More: https://bit.ly/3BylAn3

🚜 NeRF-Factory: a NeRF collection 🚜

👉PyTorch-reimplemented NeRF library with 7 popular models/implementations & 7 datasets

𝐇𝐢𝐠𝐡𝐥𝐢𝐠𝐡𝐭𝐬:
✅NeRF: Project | Paper | Code
✅NeRF++: Paper | Code
✅DVGO: Project | Paper v1/v2 | Code
✅Plenoxels: Project | Paper | Code
✅Mip-NeRF: Project | Paper | Code
✅Mip-NeRF360: Project | Paper | Code
✅Ref-NeRF: Project | Paper | Code

More: https://bit.ly/3qUgmgC

🥶 Lumos by #Nvidia: Relighting Portrait 🥶

👉The new SOTA in relighting without requiring a light stage

😎Review https://bit.ly/3dCH9ej
😎Project deepimagination.cc/Lumos
😎Paper arxiv.org/pdf/2209.10510.pdf
😎Demo http://imaginaire.cc/Lumos/

🍜 SURF-GAN: NeRF - >StyleGAN 🍜

👉 Editable portraits by injecting the NeRF's prior into StyleGAN

😎Review https://bit.ly/3SohEw3
😎Project jgkwak95.github.io/surfgan
😎Paper arxiv.org/pdf/2207.10257.pdf
😎Code github.com/jgkwak95/SURF-GAN

🔥#Google just announced "TensorStore"🔥

👉Novel open-source C++ / #Python library for storage/manipulation of high-dim data

😎Review https://bit.ly/3DLwbha
😎Project https://bit.ly/3C4T2TR
😎Code github.com/google/tensorstore

🦠 Motion Transformer for #selfdriving 🦠

👉The 1st place solution for 2022 #waymo "motion prediction" challenge

😎Review https://bit.ly/3f8G4LD
😎Paper arxiv.org/pdf/2209.10033.pdf
😎Code github.com/sshaoshuai/MTR

💹 Image Synthesis @160+ FPS! 💹

👉Super-fast, 3D-Aware Image Synthesis with Sparse Voxels -> up to 167 FPS!

😎Review https://bit.ly/3r3ZNij
😎Paper arxiv.org/pdf/2206.07695.pdf
😎Project katjaschwarz.github.io/voxgraf

👛 #Nvidia GET3D: #3D generative #AI 👛

👉AI-based Textured 3D meshes with complex topology, rich geometry & hi-fi textures

😎Review https://bit.ly/3SgnT5h
😎Code github.com/nv-tlabs/GET3D
😎Project nv-tlabs.github.io/GET3D/
😎Paper nv-tlabs.github.io/GET3D/assets/paper.pdf

🔥🔥 IDE-3D: source code is out! 🔥🔥

👉Novel, photorealistic, 3D-aware facial generator: source code just released!

😎Review https://bit.ly/3BNrO2C
😎Project mrtornado24.github.io/IDE-3D/
😎Code github.com/MrTornado24/IDE-3D
😎Paper arxiv.org/pdf/2205.15517.pdf

🔥Diffusion Model of Neural Checkpoints🔥

👉Conditional diffusion model on Millions of checkpoints of a given task/architecture 🤯

😎Review https://bit.ly/3SBR4Qb
😎Project www.wpeebles.com/Gpt
😎Code github.com/wpeebles/G.pt
😎Paper arxiv.org/pdf/2209.12892.pdf

🔥 Semantic VISOR dataset is out! 🔥

👉Segmenting hands / active objects in egocentric video (millions masks)

😎Review https://bit.ly/3LOBLBv
😎Project epic-kitchens.github.io/VISOR/
😎Paper arxiv.org/pdf/2209.13064.pdf

🥇🥇 Olympic Games in 2028? 🥇🥇

👉 In a few years, the fastest runner on earth will not be a human 🥶

😎Review https://bit.ly/3Rme3O3

🔥 SOTA ALERT: new Text-to-Video #AI 🔥

👉#META unveils a novel Text-to-Video (T2V) generation #AI

😎Review https://bit.ly/3E1ZDzG
😎Project https://makeavideo.studio/
😎Paper makeavideo.studio/Make-A-Video.pdf

🔥DreamFusion: Text-to-3D via Diffusion🔥

👉DeepDream-like procedure to create #3D assets just from a given text

😎Review https://bit.ly/3BYY5nu
😎Paper arxiv.org/pdf/2209.14988.pdf
😎Project dreamfusion3d.github.io/gallery.html

🧪 Light Field Neural Rendering 🧪

👉Two-stage transformer capable of non-Lambertian effects (reflection, refraction, translucency)

😎Review https://bit.ly/3CpIFdm
😎Paper arxiv.org/pdf/2112.09687.pdf
😎Project light-field-neural-rendering.github.io
😎Code github.com/google-research/google-research/tree/master/light_field_neural_rendering