Hacker News
23.8K subscribers
117K links
Top stories from https://news.ycombinator.com (with 100+ score)
Contribute to the development here: https://github.com/phil-r/hackernewsbot
Also check https://t.me/designer_news

Contacts: @philr
Download Telegram
Show HN: Lemon Slice Live – Have a video call with a transformer model (Score: 150+ in 17 hours)

Link: https://readhacker.news/c/6tk5N

Hey HN, this is Lina, Andrew, and Sidney from Lemon Slice. We’ve trained a custom diffusion transformer (DiT) model that achieves video streaming at 25fps and wrapped it into a demo that allows anyone to turn a photo into a real-time, talking avatar. Here’s an example conversation from co-founder Andrew: https://www.youtube.com/watch?v=CeYp5xQMFZY. Try it for yourself at: https://lemonslice.com/live.
(Btw, we used to be called Infinity AI and did a Show HN under that name last year: https://news.ycombinator.com/item?id=41467704.)
Unlike existing avatar video chat platforms like HeyGen, Tolan, or Apple Memoji filters, we do not require training custom models, rigging a character ahead of time, or having a human drive the avatar. Our tech allows users to create and immediately video-call a custom character by uploading a single image. The character image can be any style - from photorealistic to cartoons, paintings, and more.
To achieve this demo, we had to do the following (among other things! but these were the hardest):
1. Training a fast DiT model. To make our video generation fast, we had to both design a model that made the right trade-offs between speed and quality, and use standard distillation approaches. We first trained a custom video diffusion transformer (DiT) from scratch that achieves excellent lip and facial expression sync to audio. To further optimize the model for speed, we applied teacher-student distillation. The distilled model achieves 25fps video generation at 256-px resolution. Purpose-built transformer ASICs will eventually allow us to stream our video model at 4k resolution.
2. Solving the infinite video problem. Most video DiT models (Sora, Runway, Kling) generate 5-second chunks. They can iteratively extend it by another 5sec by feeding the end of the 1st chunk into the start of the 2nd in an autoregressive manner. Unfortunately the models experience quality degradation after multiple extensions due to accumulation of generation errors. We developed a temporal consistency preservation technique that maintains visual coherence across long sequences. Our technique significantly reduces artifact accumulation and allows us to generate indefinitely-long videos.
3. A complex streaming architecture with minimal latency. Enabling an end-to-end avatar zoom call requires several building blocks, including voice transcription, LLM inference, and text-to-speech generation in addition to video generation. We use Deepgram as our AI voice partner. Modal as the end-to-end compute platform. And Daily.co and Pipecat to help build a parallel processing pipeline that orchestrates everything via continuously streaming chunks. Our system achieves end-to-end latency of 3-6 seconds from user input to avatar response. Our target is <2 second latency.
More technical details here: https://lemonslice.com/live/technical-report.
Current limitations that we want to solve include: (1) enabling whole-body and background motions (we’re training a next-gen model for this), (2) reducing delays and improving resolution (purpose-built ASICs will help), (3) training a model on dyadic conversations so that avatars learn to listen naturally, and (4) allowing the character to “see you” and respond to what they see to create a more natural and engaging conversation.
We believe that generative video will usher in a new media type centered around interactivity: TV shows, movies, ads, and online courses will stop and talk to us. Our entertainment will be a mixture of passive and active experiences depending on what we’re in the mood for. Well, prediction is hard, especially about the future, but that’s how we see it anyway!
We’d love for you to try out the demo and let us know what you think! Post your characters and/or conversation recordings below.
Notation as a Tool of Thought (1979) (Score: 158+ in 9 hours)

Link: https://readhacker.news/s/6tmx3
Comments: https://readhacker.news/c/6tmx3
FBI arrests Wisconsin judge on charges of obstructing immigrant arrest (🔥 Score: 163+ in 34 minutes)

Link: https://readhacker.news/s/6tp82
Comments: https://readhacker.news/c/6tp82
GCC 15.1 (Score: 150+ in 10 hours)

Link: https://readhacker.news/s/6tnns
Comments: https://readhacker.news/c/6tnns
Wikipedia’s nonprofit status questioned by D.C. U.S. attorney (🔥 Score: 157+ in 2 hours)

Link: https://readhacker.news/s/6tqCq
Comments: https://readhacker.news/c/6tqCq
Show HN: I used OpenAI's new image API for a personalized coloring book service (Score: 152+ in 19 hours)

Link: https://readhacker.news/s/6tnhS
Comments: https://readhacker.news/c/6tnhS

I've had an idea for a long time to generate a cute coloring book based on family photos, send it to a printing service, and then deliver it to people.
Last month, when OpenAI's Sora was released for public use I (foolishly) thought I'd manually drag-and-drop each order’s photos into Sora's UI and copy the resulting images back into my system. This took way too much time (about an hour for each of the few books I made and tested with family and friends). It clearly wasn't possible to release this version because I’d be losing a huge amount of time on every order. So instead, I decided I'd finish off the project as best I could, put it "on ice," and wait for the API release.
The API is now released (quicker than I thought it'd be, too!) and I integrated it last night. I'd love your feedback on any and all aspects.
The market is mostly family-based, but from my testing of the physical book I've found that both adults and kids enjoy coloring them in (it's surprisingly cathartic and creative). If you would like to order one you can get 10% off by tapping the total price line item five times.