Taking Krita AI Diffusion and ComfyUI to 24K (it’s about time)
In the past year or so, we have seen countless advances in the generative imaging field, with ComfyUI taking a firm lead among Stable Diffusion-based open source, locally generating tools. One area where this platform, with all its frontends, is lagging behind is high resolution image processing. By which I mean, really high (also called ultra) resolution - from 8K and up. About a year ago, I posted a tutorial article on the SD subreddit on creative upscaling of images of 16K size and beyond with Forge webui, which in total attracted more than 300K views, so I am surely not breaking any new ground with this idea. Amazingly enough, Comfy still has made no progress whatsoever in this area - its output image resolution is basically limited to 8K (the capping which is most often mentioned by users), as it was back then. In this article post, I will shed some light on technical aspects of the situation and outline ways to break this barrier without sacrificing the quality.
At-a-glance summary of the topics discussed in this article:
\- The basics of the upscale routine and main components used
\- The image size cappings to remove
\- The I/O methods and protocols to improve
\- Upscaling and refining with Krita AI Hires, the only one that can handle 24K
\- What are use cases for ultra high resolution imagery?
\- Examples of ultra high resolution images
I believe this article should be of interest not only for SD artists and designers keen on ultra hires upscaling or working with a large digital canvas, but also for Comfy back- and front-end developers looking to improve their tools (sections 2. and 3. are meant mainly for them). And I just hope that my message doesn’t get lost amidst the constant flood of new, and newer yet models being added to the platform, keeping them very busy indeed.
1. The basics of the upscale routine and main components used
This article is about reaching ultra high resolutions with Comfy and its frontends, so I will just pick up from the stage where you already have a generated image with all its content as desired but are still at what I call mid-res - that is, around 3-4K resolution. (To get there, Hiresfix, a popular SD technique to generate quality images of up to 4K in one go, is often used, but, since it’s been well described before, I will skip it here.)
To go any further, you will have to switch to the img2img mode and process the image in a tiled fashion, which you do by engaging a tiling component such as the commonly used Ultimate SD Upscale. Without breaking the image into tiles when doing img2img, the output will be plagued by distortions or blurriness or both, and the processing time will grow exponentially. In my upscale routine, I use another popular tiling component, Tiled Diffusion, which I found to be much more graceful when dealing with tile seams (a major artifact associated with tiling) and a bit more creative in denoising than the alternatives.
Another known drawback of the tiling process is the visual dissolution of the output into separate tiles when using a high denoise factor. To prevent that from happening and to keep as much detail in the output as possible, another important component is used, the Tile ControlNet (sometimes called Unblur).
At this (3-4K) point, most other frequently used components like IP adapters or regional prompters may cease to be working properly, mainly for the reason that they were tested or fine-tuned for basic resolutions only. They may also exhibit issues when used in the tiled mode. Using other ControlNets also becomes a hit and miss game. Processing images with masks can be also problematic. So, what you do from here on, all the way to 24K (and beyond), is a progressive upscale
In the past year or so, we have seen countless advances in the generative imaging field, with ComfyUI taking a firm lead among Stable Diffusion-based open source, locally generating tools. One area where this platform, with all its frontends, is lagging behind is high resolution image processing. By which I mean, really high (also called ultra) resolution - from 8K and up. About a year ago, I posted a tutorial article on the SD subreddit on creative upscaling of images of 16K size and beyond with Forge webui, which in total attracted more than 300K views, so I am surely not breaking any new ground with this idea. Amazingly enough, Comfy still has made no progress whatsoever in this area - its output image resolution is basically limited to 8K (the capping which is most often mentioned by users), as it was back then. In this article post, I will shed some light on technical aspects of the situation and outline ways to break this barrier without sacrificing the quality.
At-a-glance summary of the topics discussed in this article:
\- The basics of the upscale routine and main components used
\- The image size cappings to remove
\- The I/O methods and protocols to improve
\- Upscaling and refining with Krita AI Hires, the only one that can handle 24K
\- What are use cases for ultra high resolution imagery?
\- Examples of ultra high resolution images
I believe this article should be of interest not only for SD artists and designers keen on ultra hires upscaling or working with a large digital canvas, but also for Comfy back- and front-end developers looking to improve their tools (sections 2. and 3. are meant mainly for them). And I just hope that my message doesn’t get lost amidst the constant flood of new, and newer yet models being added to the platform, keeping them very busy indeed.
1. The basics of the upscale routine and main components used
This article is about reaching ultra high resolutions with Comfy and its frontends, so I will just pick up from the stage where you already have a generated image with all its content as desired but are still at what I call mid-res - that is, around 3-4K resolution. (To get there, Hiresfix, a popular SD technique to generate quality images of up to 4K in one go, is often used, but, since it’s been well described before, I will skip it here.)
To go any further, you will have to switch to the img2img mode and process the image in a tiled fashion, which you do by engaging a tiling component such as the commonly used Ultimate SD Upscale. Without breaking the image into tiles when doing img2img, the output will be plagued by distortions or blurriness or both, and the processing time will grow exponentially. In my upscale routine, I use another popular tiling component, Tiled Diffusion, which I found to be much more graceful when dealing with tile seams (a major artifact associated with tiling) and a bit more creative in denoising than the alternatives.
Another known drawback of the tiling process is the visual dissolution of the output into separate tiles when using a high denoise factor. To prevent that from happening and to keep as much detail in the output as possible, another important component is used, the Tile ControlNet (sometimes called Unblur).
At this (3-4K) point, most other frequently used components like IP adapters or regional prompters may cease to be working properly, mainly for the reason that they were tested or fine-tuned for basic resolutions only. They may also exhibit issues when used in the tiled mode. Using other ControlNets also becomes a hit and miss game. Processing images with masks can be also problematic. So, what you do from here on, all the way to 24K (and beyond), is a progressive upscale
Reddit
From the StableDiffusion community on Reddit: Creative upscaling all the way to 16K (and beyond) with WebUI Forge, a comprehensive…
Explore this post and more from the StableDiffusion community