🔥 unsloth - fine tuning Llama 3, Mistral, Phi and Gemma LLM, reducing memory consumption by 80%, speeding up 2-5 times

conda create --name unsloth_env python=3.10
conda activate unsloth_env

conda install pytorch-cuda=<12.1/11.8> pytorch cudatoolkit xformers -c pytorch -c nvidia -c xformers

pip install "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"

pip install --no-deps trl peft accelerate bitsandbytes

Some unsloth benchmarks are given here

🖥 GitHub
🟡 Wiki

https://t.me/DataScienceT ✅

☄️ Qwen2 is the coolest release of open LLMs since Llama 3!

Alibaba has just released their new family of multilingual models that outperform the Llama 3 in many ways.

🤯 Qwen2 is released in 5 sizes, trained in 29 languages!

5️⃣ Sizes: 0.5B, 1.5B, 7B, 57B-14B (MoE), 72B.
✔️ Context: 32k for 0.5B & 1.5B, 64k for 57B MoE, 128k for 7B & 72B
✅ Supports 29 languages.
🟢 Released under the Apache 2.0 license, except for version 72B.

🔖 BLOG: https://qwenlm.github.io/blog/qwen2/

🤗 HF collection: https://huggingface.co/collections/Qwen/qwen2-6659360b33528ced941e557f

🤖 https://modelscope.cn/organization/qwen

👨‍💻 GitHub: https://github.com/QwenLM/Qwen2

https://t.me/DataScienceT ✅

Flash Diffusion - acceleration of any diffusion model for image generation in a few steps

This is the name of the work recently published by Clément Chadebec, Onur Tasar and their colleagues.
This is a distillation method for creating fast diffusion patterns.
When training the model using this method on the COCO2014 and COCO2017 datasets, the FID and CLIP-Score achieve good values; it requires only a few hours of training on the GPU and fewer model parameters than existing methods require.

🟡 Flash Diffusion page
🖥 GitHub

🤗 Flash SD
🤗 Flash SDXL

https://t.me/DataScienceT ✅

🚀 AgentGym: Evolving Large Language Model-based Agents across Diverse Environments

🖥 Github: https://github.com/woooodyy/agentgym

🔖 Paper: https://arxiv.org/abs/2406.04151v1

☄️ Project: https://agentgym.github.io/

✔️ Model (AgentEvol-7B): https://huggingface.co/AgentGym/AgentEvol-7B

https://t.me/DataScienceT ✅