there are some other nuances, we will look at them next time, but, in general, this information is enough to start working on training your own models.
👍1
an interesting feature of model training for flux is that captions in the dataset are not necessary at all, just a folder with images is enough. the reason for this is the t5 encoder, which is somewhat llm (large language model), and can make captions itself. you may have noticed that t5 is also used in comfyui in flux algorithms along with the usual clip (familiar from sdxl and earlier versions of sd) , there it is needed to interpret your prompt more accurately, and in training to interpret captions or even make them.
#comfyui #lora #flux
#comfyui #lora #flux
❤3
let's compare: these are renders with exactly the same prompts, seeds and all parameters, only lora is different. lora models were also trained on the same seeds and the same parameters, but for the first image lora is used, where in the dataset were captions, and for the second were only images (the images themselves are exactly the same as in the first dataset).
and here is another example. remember that the dataset was photos of Coop Himmelb(l)au projects. a conclusion can be that if a dataset with captions was used, elements of real buildings from the company's portfolio are literally borrowed. this can of course be leveled out by reducing the weight of lora when rendering or mixing it with other lora models. if there are no captions, then the general architectural style remains, but without precisely following the details. both can be needed in different situations.
This media is not supported in your browser
VIEW IN TELEGRAM
Can Chat GPT be used to change information in a Revit model? Check out this video that Anatoly Razinkov prepared for our upcoming workshop on integrating AI into BIM.
This workshop will be in Russian, but if you are interested in the English version, leave a comment
This workshop will be in Russian, but if you are interested in the English version, leave a comment
👍4❤1
Let's talk about LORA training based on the Schnell model. in general, it's kind of impossible to do this because the model itself is not suitable for this task, but the developers from Ostris have prepared a special adapter that solves the problem. in the Google Colab code you need to additionally write a link to the model and the adapter, as well as replace the parameters for the test rendering. i've already done it, here's the link.
#comfyui #lora #flux
#comfyui #lora #flux
👍3❤1
I trained on the same dataset and with the same parameter variations as when using Dev model. in general, of course, the renders are a bit simpler, but Schnell, and everything you produced with it, can be used freely for commercial purposes. in addition, this model is noticeably less demanding on computer resources
👍3❤1
The last thing I'd like to tell you about training LORA for Flux is an attempt to train something on top of a very curious model called OpenFLUX.1. The idea behind this model is that it is a fine-tuned version of Schnell, reconfigured to produce Flux Dev quality. Technically it's possible to train LORA on top of this model, but I wouldn't say I've been able to find good parameters. Here's the Colab that I've configured, if you experiment with it, please let me know what you find.
#comfyui #lora #flux
#comfyui #lora #flux