dipampaul17/KVSplit
Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.
Language: Python
#apple_silicon #generative_ai #kv_cache #llama_cpp #llm #m1 #m2 #m3 #memory_optimization #metal #optimization #quantization
Stars: 222 Issues: 1 Forks: 5
https://github.com/dipampaul17/KVSplit
Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.
Language: Python
#apple_silicon #generative_ai #kv_cache #llama_cpp #llm #m1 #m2 #m3 #memory_optimization #metal #optimization #quantization
Stars: 222 Issues: 1 Forks: 5
https://github.com/dipampaul17/KVSplit
GitHub
GitHub - dipampaul17/KVSplit: Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache…
Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with &am...
Olow304/memvid
Video-based AI memory library. Store millions of text chunks in MP4 files with lightning-fast semantic search. No database needed.
Language: Python
#ai #context #embedded #faiss #knowledge_base #knowledge_graph #llm #machine_learning #memory #nlp #offline_first #opencv #python #rag #retrieval_augmented_generation #semantic_search #vector_database #video_processing
Stars: 252 Issues: 2 Forks: 25
https://github.com/Olow304/memvid
Video-based AI memory library. Store millions of text chunks in MP4 files with lightning-fast semantic search. No database needed.
Language: Python
#ai #context #embedded #faiss #knowledge_base #knowledge_graph #llm #machine_learning #memory #nlp #offline_first #opencv #python #rag #retrieval_augmented_generation #semantic_search #vector_database #video_processing
Stars: 252 Issues: 2 Forks: 25
https://github.com/Olow304/memvid
GitHub
GitHub - Olow304/memvid: Video-based AI memory library. Store millions of text chunks in MP4 files with lightning-fast semantic…
Video-based AI memory library. Store millions of text chunks in MP4 files with lightning-fast semantic search. No database needed. - Olow304/memvid
MemTensor/MemOS
MemOS (Preview) | Intelligence Begins with Memory
Language: Python
#agent #kv_cache #language_model #llm #llm_memory #long_term_memory #lora #memcube #memory #memory_management #memory_operating_system #memory_retrieval #memory_scheduling #memos #neo4j #rag #retrieval_augmented_generation #tree
Stars: 512 Issues: 14 Forks: 35
https://github.com/MemTensor/MemOS
MemOS (Preview) | Intelligence Begins with Memory
Language: Python
#agent #kv_cache #language_model #llm #llm_memory #long_term_memory #lora #memcube #memory #memory_management #memory_operating_system #memory_retrieval #memory_scheduling #memos #neo4j #rag #retrieval_augmented_generation #tree
Stars: 512 Issues: 14 Forks: 35
https://github.com/MemTensor/MemOS
GitHub
GitHub - MemTensor/MemOS: MemOS (Preview) | Intelligence Begins with Memory
MemOS (Preview) | Intelligence Begins with Memory. Contribute to MemTensor/MemOS development by creating an account on GitHub.