janhq/nitro
A fast, lightweight, embeddable inference engine to supercharge your apps with local AI. OpenAI-compatible API
Language:C++
Total stars: 362
Stars trend:
#cplusplus
#accelerated, #ai, #cuda, #gguf, #inferenceengine, #llama, #llama2, #llamacpp, #llm, #llms, #openaiapi, #stablediffusion, #tensorrtllm
A fast, lightweight, embeddable inference engine to supercharge your apps with local AI. OpenAI-compatible API
Language:C++
Total stars: 362
Stars trend:
6 Jan 2024
1am ▏ +1
2am ██▉ +23
3am █████▎ +42
4am ████▍ +35
5am █▍ +11
6am █▋ +13
7am ▌ +4
8am ▍ +3
9am █▎ +10
10am ▏ +1
11am ▌ +4
12pm ▊ +6#cplusplus
#accelerated, #ai, #cuda, #gguf, #inferenceengine, #llama, #llama2, #llamacpp, #llm, #llms, #openaiapi, #stablediffusion, #tensorrtllm
zhihu/ZhiLight
A highly optimized inference acceleration engine for Llama and its variants.
Language:C++
Total stars: 135
Stars trend:
#cplusplus
#cuda, #gpt, #inferenceengine, #llama, #llm, #llmserving, #pytorch
A highly optimized inference acceleration engine for Llama and its variants.
Language:C++
Total stars: 135
Stars trend:
9 Dec 2024
9pm ▏ +1
10pm +0
11pm +0
10 Dec 2024
12am ▏ +1
1am █▎ +10
2am ██▏ +17
3am █▋ +13
4am █▏ +9
5am █▎ +10
6am █ +8
7am ▉ +7#cplusplus
#cuda, #gpt, #inferenceengine, #llama, #llm, #llmserving, #pytorch