🔥 Trending Repository: LMCache

📝 Description: Supercharge Your LLM with the Fastest KV Cache Layer

🔗 Repository URL: https://github.com/LMCache/LMCache

🌐 Website: https://lmcache.ai/

📖 Readme: https://github.com/LMCache/LMCache#readme

📊 Statistics:
🌟 Stars: 4.3K stars
👀 Watchers: 24
🍴 Forks: 485 forks

💻 Programming Languages: Python - Cuda - Shell

🏷️ Related Topics:

#fast #amd #cuda #inference #pytorch #speed #rocm #kv_cache #llm #vllm

==================================
🧠 By: https://t.me/DataScienceM

🔥 Trending Repository: vllm

📝 Description: A high-throughput and memory-efficient inference and serving engine for LLMs

🔗 Repository URL: https://github.com/vllm-project/vllm

🌐 Website: https://docs.vllm.ai

📖 Readme: https://github.com/vllm-project/vllm#readme

📊 Statistics:
🌟 Stars: 55.5K stars
👀 Watchers: 428
🍴 Forks: 9.4K forks

💻 Programming Languages: Python - Cuda - C++ - Shell - C - CMake

🏷️ Related Topics:

#amd #cuda #inference #pytorch #transformer #llama #gpt #rocm #model_serving #tpu #hpu #mlops #xpu #llm #inferentia #llmops #llm_serving #qwen #deepseek #trainium

==================================
🧠 By: https://t.me/DataScienceM