Tencent just released HunyuanPortrait

https://redd.it/1kwklhj
@rStableDiffusion

Veo3 + Flux + Hunyuan3D + Wan with VAce

https://redd.it/1kwl75r
@rStableDiffusion

How To Create Consistent & Realistic AI Character with Dzine
https://youtu.be/qPNPaek2sbE

https://redd.it/1kwmm9z
@rStableDiffusion

ComfyUI Tutorial Series Ep 49: Master txt2video, img2video & video2video with Wan 2.1 VACE

https://youtu.be/KGq96ag7FiY?si=ax3uzkV4z6M3IkmZ

https://redd.it/1kwpn8o
@rStableDiffusion

New SkyReels-V2-VACE-GGUFs 🚀🚀🚀

https://huggingface.co/QuantStack/SkyReels-V2-T2V-14B-720P-VACE-GGUF

This is a GGUF version of SkyReels V2 with additional VACE addon, that works in native workflows!

For those who dont know, SkyReels V2 is a wan2.1 model that got finetuned in 24fps (in this case 720p)

VACE allows to use control videos, just like controlnets for image generation models. These GGUFs are the combination of both.

A basic workflow is here:

https://huggingface.co/QuantStack/Wan2.1-VACE-14B-GGUF/blob/main/vace\_v2v\_example\_workflow.json

If you wanna see what VACE does go here:

https://www.reddit.com/r/StableDiffusion/comments/1koefcg/new\_wan21vace14bggufs/

https://redd.it/1kwpq1s
@rStableDiffusion

Wan 2.1 video of a woman in a black outfit and black mask, getting into a yellow sports car. Image to video Wan 2.1

https://redd.it/1kwqigk
@rStableDiffusion

Found Footage - [FLUX LORA]

https://redd.it/1kwp5tf
@rStableDiffusion

The CivitAI backup site with torrents and comment section

Since Civit AI started removing models, a lot of people have been calling for another alternative, and we have seen quite a few in the past few weeks. But after reading through all the comments, I decided to come up with my own solution which hopefully covers all the essential functionality mentioned .


Current Function includes:

Login, including google and github
you can also setup your own profile picture
Model showcase with Image + description
A working comment section
basic image filter to check if an image is sfw
search functionality
filter model based on type, and base model
torrent (but this is inconsistent since someone needs to actively seed it , and most cloud provider does not allow torrenting, i set up half of the backend already, if someone has any good suggestion please comment down there )

I plan to make everything as transparent as possible, and this would purely be model hosting and sharing.

The model and image are stored to r2 bucket directly, which can hopefully help with reducing cost.

So please check out what I made here : https://miyukiai.com/, if enough people join then we can create a P2P network to share the ai models.

https://redd.it/1kwyi1g
@rStableDiffusion

I wrote software to create my diffusion models from scratch. Watching it learn is terrifying.
https://i.imgur.com/TawoHWY.png

https://redd.it/1kwzp0m
@rStableDiffusion

WAN i2v and VACE for low VRAM, here's your guide.

Over the past couple weeks I've seen the same posts over and over, and the questions are all the same, because most people aren't getting the results of these showcase videos. I have nothing against Youtubers, and I have learned a LOT from various channels, but let's be honest, they sometimes click-bait their titles to make it seem like all you have to do is load one node or lora and you can produce magic videos in seconds. I have a tiny RTX 3070 (8GB VRAM) and getting WAN or VACE to give good results can be tough on low VRAM. This guide is for you 8GB folks.

I do 80% I2V and 20% V2V, and rarely use T2V. I generate an image with JuggernautXL or Chroma, then feed it to WAN. I get a lot of extra control over details, initial poses and can use loras to get the results I want. Yes, there's some n$fw content which will not be further discussed here due to rules, but know that type of content is some of the hardest content to produce. I suggest you start with "A woman walks through a park past a fountain", or something you know the models will produce to get a good workflow, then tweak for more difficult things.

I'm not going to cover the basics of ComfyUI, but I'll post my workflow so you can see which nodes I use. I always try to use native ComfyUI nodes when possible, and load as few custom nodes as possible. KJNodes are awesome even if not using WanVideoWrapper. VideoHelperSuite, Crystools, also great nodes to have. You will want ComfyUI Manager, not even a choice really.

Models and Nodes:
There are ComfyUI "Native" nodes, and KJNodes (aka WanVideoWrapper) for WAN2.1. KJNodes in my humble opinion are for advanced users and more difficult to use, though CAN be more powerful and CAN cause you a lot of strife. They also have more example workflows, none of which I need. Do not mix and match WanVideoWrapper with "Native WAN" nodes, pick one or the other. Non-WAN KJNodes are awesome and I use them a lot, but for WAN I use Native nodes.

I use the WAN "Repackaged" models, they have example workflows in the repo. Do not mix and match models, VAEs and Text encoders. You actually CAN do this, but 10% of the time you'll get poor results because you're using a finetune version you got somewhere else and forgot, and you won't know why your results are crappy, but everything kinda still works.

Referring to the model: wan2.1_t2v_1.3B_bf16.safetensors, this means T2V, and 1.3B parameters. More parameters means better quality, but needs more memory and runs slower. I use the 14B model with my 3070, I'll explain how to get around the memory issues later on. If there's a resolution on the model, match it up. The wan2.1_i2v_480p_14B_fp8_e4m2fn.safetensors model is 480p, so use 480x480 or 512x512 or something close (384x512), that's divisible by 16. For low VRAM, use a low resolution (I use 480x480) then upscale (more on that later). It's a LOT faster and gives pretty much the same results. Forget about all these workflows that are doing 2K before upscaling, your 8GB VRAM can only do that for 10 frames before it craps.

For the CLIP, use the umt5_xxl_fp8_e4m2fn.safetensors and offload to the CPU (by selecting the "device" in the node, or use --lowvram starting ComfyUI), unless you run into prompt adherence problems, then you can try the FP16 version, which I rarely need to use.

Memory Management:
You have a tiny VRAM, it happens to the best of us. If you start ComfyUI with "--lowvram" AND you use the Native nodes, several things happen, including offloading most things that can be offloaded to CPU automatically (like CLIP) and using the "Smart Memory Management" features, which seamlessly offload chunks of WAN to "Shared VRAM". This is the same as the KJ Blockswap node, but it's automatic. Open up your task manager in Windows and go to the Performance tab, at the bottom you'll see Dedicated GPU Memory (8GB for me) and Shared GPU Memory, which is that seamless smart memory I was

talking about. WAN will not fit into your 8GB VRAM, but if you have enough system RAM, it will run (but much slower) by sharing your system RAM with the GPU.

https://preview.redd.it/ok68l3mk4e3f1.png?width=853&format=png&auto=webp&s=5940f6644884cd44202e23685538ecf694c6a60a

I have 128GB of RAM, so it loads all of WAN in my VRAM then the remainder spills into RAM, which is not ideal, but workable. WAN (14B 480p) takes about 16GB plus another 8-16GB for the video generation on my system total. If your RAM is at 100% when you run the workflow, you're using your Swap file to soak up the rest of the model, which sits on your HDD, which is SSSLLLLLLOOOOOWWWWWW. If that's the case, buy more RAM. It's cheap, just do it.

WAN (81 frames 480x480) on a 3090 24GB VRAM (fits mostly in VRAM) typically runs 6s/it (so I've heard).

WAN with 8GB VRAM and plenty of "Shared VRAM" aka RAM, runs around 20-30s/it.

WAN while Swapping to disk runs around 750-2500s/it with a fast SSD. I'll say it again, buy enough RAM. 32GB is workable, but I'd go higher just because the cost is so low. On a side note, you can put in a registry entry in Windows to use more RAM for file cache (Google or ChatGPT it). Since I have 128GB, I did this and saw a big performance boost all across the board in Windows.

Loras typically increase these iteration times. Leave your batch size at "1". You don't have enough VRAM for anything higher.
I can generate a 81 frame video (5 seconds at 16fps) at 480x480 in about 10-15 minutes with 2x upscaling and 2x interpolation.
WAN keeps all frames in memory, and for each step, touches each frame in sequence. So, more frames means more memory. More steps does not increase memory though. Higher resolution means more memory. More loras (typically) means more memory. Bigger CLIP model, means more memory (unless offloaded to CPU, but still needs system RAM). You have limited VRAM, so pick your battles.

I'll be honest, I don't fully understand GGUF, but with my experimentation GGUF does not increase speed, and in most cases I tried, actually slowed down generation. YMMV.

Use-Cases:
If you want to do T2V, WAN2.1 is great, use the T2V example workflow in the repo above and you really can't screw that one up, use the default settings, 480p and 81 frames, a RTX 3070 will handle it.

If you want to do I2V, WAN2.1 is great, use the I2V example, 480p, 81 frames, 20 Steps, 4-6 CFG and that's it. You really don't need ModelSamplingSD3, CFGZeroStar, or anything else. Those CAN help, but most problems can be solved with more Steps, or adjusted CFG. The WanImageToVideo node is easy to use.

Lower CFG allows the model to "day dream" more, so it doesn't stick to the prompt as well, but tends to create a more coherent image. Higher CFG sticks to the prompt better, but sometimes at the cost of quality. More steps will always create a better video, until it doesn't. There's a point where it just won't get any better, but you want to use as few steps as possible, because more steps means more generation time. 20 Steps is a good starting point for WAN. Go into ComfyUI Manager (install if if you don't have it, trust me) and turn on "Preview Method: Auto". This shows a preview as the video is processed in KSampler and you'll get a better understanding of how the video is created.

If you want to do V2V, you have choices.

WanFUNControlToVideo (Uses the WAN Fun control model) does great by taking the action from a video, and a start image and animating the start image. I won't go into this too much since this guide is about getting WAN working on low VRAM, not all the neat things WAN can do.
You can add in IPSampler and ControlNet (OpenPose, Depthanything, Canny, etc.) to add to the control you have for poses and action.

The second choice for V2V is VACE. It's kinda like a swiss army knife of use-cases for WAN. Check their web site for the features. It takes more memory, runs slower, but you

can do some really neat things like inserting characters, costume changes, inserting logos, face swap, V2V action just like Fun Control, or for stubborn cases where WAN just won't follow your prompt. It can also use ControlNet if you need. Once again, advanced material, not going into it. Just know you should stick to the most simple solution you can for your use-case.

With either of these, just keep an eye on your VRAM and RAM. If you're Swapping to Disk, drop your resolution, number of frames, whatever to get everything to fit in Shared GPU Memory.

UpScaling and Interpolation:
I'm only covering this because of memory constraints. Always create your videos at low resolution then upscale (if you have low VRAM). You get the same quality (mostly), but 10x faster. I upscale with the "Upscale Image (using Model)" node and the "RealESRGAN 2x" model. Upscaling the image (instead of the latent) gives better results for details and sharpness. I also like to interpolate the video using "FILM VFI", which increases the number of frames from 16fps to 32fps, making the video smoother (usually). Interpolate before you upscale, it's 10x faster.

If you are doing upscaling and interpolation in the same workflow as your generation, you're going to need "VAE Decode (Tiled)" instead of the normal VAE Decode. This breaks the video down into pieces so your VRAM/RAM doesn't explode. Just cut the first three default values in half for 8GB VRAM (256, 32, 32, 8)

It's TOO slow:
Now you want to know how to make things faster. First, check your VRAM and RAM in Task Manager while a workflow is running. Make sure you're not Swapping to disk. 128GB of RAM for my system was $200. A new GPU is $2K. Do the math, buy the RAM.

If that's not a problem, you can try out CausVid. It's a lora that reduces the number of steps needed to generate a video. In my experience, it's really good for T2V, and garbage for I2V. It literally says T2V in the Lora name, so this might explain it. Maybe I'm an idiot, who knows. You load the lora (Lora Loader Model Only), set it for 0.3 to 0.8 strength (I've tried them all), set your CFG to 1, and steps to 4-6. I've got pretty crap results from it, so if someone else wants to chime in, please do so. I think the issue is that when starting from a text prompt, it will easily generate things it can do well, and if it doesn't know something you ask for, it simply ignores it and makes a nice looking video of something you didn't necessarily want. But when starting from an image, if it doesn't know that subject matter, it does the best it can, which turns out to be sloppy garbage. I've heard you can fix issues with CausVid by decreasing the lora strength and increasing the CFG, but then you need more steps. YMMV.

If you want to speed things up a little more, you can try Sage Attention and Triton. I won't go into how these work, but Triton (TorchCompileModel node) doesn't play nice with CausVid or most Loras, but can speed up video generation by 30% IF most or all of the model is in VRAM, otherwise your memory is still the bottleneck and not the GPU processing time, but you still get a little boost regardless. Sage Attention (Patch Sage Attention KJ node) is the same (less performance boost though), but plays nice with most things. "--use-sage-attention" can enable this without using the node (maybe??). You can use both of these together.

Installing Sage Attention isn't horrible, Triton is a dumpster fire on Windows. I used this install script on a clean copy of ComfyUI_Portable and it worked without issue. I will not help you install this. It's a nightmare.

Workflows:

The example workflows work fine. 20 Steps, 4-6 CFG, uni_pc/simple. Typically use the lowest CFG you can get away with, and as few steps as are necessary. I've gone as low as 14 Steps/2CFG and got good results. This is my i2v workflow with some of the junk cut out. Just drag this picture into your ComfyUI.

E: Well, apparently Reddit

strips the metadata from the images, so the workflow is here: https://pastebin.com/RBduvanM


https://preview.redd.it/fp5ep7xaee3f1.png?width=960&format=png&auto=webp&s=37a6e32bf12d74321f38477cd8eb2b7af58906bc

Long Videos:
At 480x480, you can do 113 frames (7 seconds) and upscale, but interpolation sometimes errors out. The best way to do videos longer than 5-7 seconds is to create a bunch of short ones and string them together using the last frame of one video as the first frame of the next. You can use the "Load Video" nodes from VHS, set the frame_load_cap to 1, set skip_first_frames to 1 less than the total frames (WAN always adds an extra blank frame apparently, 80 or 160 depending if you did interpolation), then save the output, which will be the last frame of the video. The VHS nodes will tell you how many frames are in your video, and other interesting stats. Then use your favorite video editing tool to combine the videos. I like Divinci Resolv. It's free and easy to use. ffmpeg can also do it pretty easily.

https://preview.redd.it/jhhhamoume3f1.png?width=806&format=png&auto=webp&s=dc01386e95187e0d60a1c96afcefd9d9b90c2a6b




https://redd.it/1kx0ly2
@rStableDiffusion

Hunyuan Video Avatar is now released!

It uses I2V, is audio-driven, and support multiple characters.
Open source is now one step closer to Veo3 standard.

HF page

Github page

https://redd.it/1kx6p8y
@rStableDiffusion

Love at First Bite: Animating a Dark Cat-Pig Tale with WAN 2.1 in ComfyUI

https://redd.it/1kx75ii
@rStableDiffusion

Looking for Lip Sync Models — Anything Better Than LatentSync?

https://redd.it/1kx862k
@rStableDiffusion

🤯 FOSS Gemini/GPT Challenger? Meet BAGEL AI - Now on ComfyUI! 🥯
https://youtu.be/C9qgKNuaRTQ

https://redd.it/1kxbz0c
@rStableDiffusion

Stable Diffusion Cage Match: Miley vs the Machines [API and Local]

https://redd.it/1kxdao4
@rStableDiffusion