dipampaul17/KVSplit
Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.
Language: Python
#apple_silicon #generative_ai #kv_cache #llama_cpp #llm #m1 #m2 #m3 #memory_optimization #metal #optimization #quantization
Stars: 222 Issues: 1 Forks: 5
https://github.com/dipampaul17/KVSplit

Olow304/memvid
Video-based AI memory library. Store millions of text chunks in MP4 files with lightning-fast semantic search. No database needed.
Language: Python
#ai #context #embedded #faiss #knowledge_base #knowledge_graph #llm #machine_learning #memory #nlp #offline_first #opencv #python #rag #retrieval_augmented_generation #semantic_search #vector_database #video_processing
Stars: 252 Issues: 2 Forks: 25
https://github.com/Olow304/memvid

MemTensor/MemOS
MemOS (Preview) | Intelligence Begins with Memory
Language: Python
#agent #kv_cache #language_model #llm #llm_memory #long_term_memory #lora #memcube #memory #memory_management #memory_operating_system #memory_retrieval #memory_scheduling #memos #neo4j #rag #retrieval_augmented_generation #tree
Stars: 512 Issues: 14 Forks: 35
https://github.com/MemTensor/MemOS