microsoft/aici
AICI: Prompts as (Wasm) Programs
Language:Rust
Total stars: 213
Stars trend:
11 Mar 2024
6pm ██▏ +17
7pm ███▎ +26
8pm ████▊ +38
#rust
#ai, #inference, #languagemodel, #llm, #llmframework, #llminference, #llmserving, #llmops, #modelserving, #rust, #transformer, #wasm, #wasmtime

cncf/llm-in-action
🤖 Discover how to apply your LLM app skills on Kubernetes!
Language:Python
Total stars: 106
Stars trend:
20 Mar 2024
8am █████████████ +104
#python
#cloudnative, #inference, #llm

huggingface/text-generation-inference
Large Language Model Text Generation Inference
Language:Python
Total stars: 7640
Stars trend:
9 Apr 2024
9pm ▊ +6
10pm ▊ +6
11pm ▍ +3
10 Apr 2024
12am ▋ +5
1am ▌ +4
2am █▎ +10
3am █▌ +12
4am ▉ +7
5am ▊ +6
6am ▋ +5
7am ▊ +6
8am ▉ +7
#python
#bloom, #deeplearning, #falcon, #gpt, #inference, #nlp, #pytorch, #starcoder, #transformer

NVIDIA/TensorRT
NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.
Language:C++
Total stars: 9485
Stars trend:
14 Jun 2024
10am ▌ +4
11am ███▊ +30
12pm ████▉ +39
1pm ████▋ +37
#cplusplus
#deeplearning, #gpuacceleration, #inference, #nvidia, #tensorrt

zml/zml
High performance AI inference stack. Built for production. @ziglang / @openxla / MLIR / @bazelbuild
Language:Zig
Total stars: 178
Stars trend:
17 Sep 2024
9am ███▍ +27
10am █▉ +15
11am ██▊ +22
12pm ██▋ +21
1pm ███████ +56
#zig
#ai, #bazel, #hpc, #inference, #xla, #zig

AgibotTech/agibot_x1_infer
The inference module for AgiBot X1.
Language:C++
Total stars: 107
Stars trend:
24 Oct 2024
2am ▏ +1
3am ▉ +7
4am ██ +16
5am ██▏ +17
6am ██▏ +17
7am ███▉ +31
#cplusplus
#inference, #opensource, #robotics

adithya-s-k/AI-Engineering.academy
Mastering Applied AI, One Concept at a Time
Language:Jupyter Notebook
Total stars: 262
Stars trend:
3 Dec 2024
7pm ▎ +2
8pm ▍ +3
9pm +0
10pm ▏ +1
11pm ▍ +3
4 Dec 2024
12am ▊ +6
1am █▏ +9
2am █▉ +15
3am █▏ +9
4am █▍ +11
5am █▍ +11
6am █▍ +11
#jupyternotebook
#finetuning, #finetuning, #finetuningllms, #inference, #largelanguagemodels, #llm, #python, #quantization

vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Language:Python
Total stars: 34142
Stars trend:
20 Jan 2025
8pm ▎ +2
9pm ▎ +2
10pm █▏ +9
11pm ▍ +3
21 Jan 2025
12am ▎ +2
1am █ +8
2am ▉ +7
3am ▉ +7
4am █▌ +12
5am ▊ +6
6am █▎ +10
7am █▍ +11
#python
#amd, #cuda, #gpt, #hpu, #inference, #inferentia, #llama, #llm, #llmserving, #llmops, #mlops, #modelserving, #pytorch, #rocm, #tpu, #trainium, #transformer, #xpu

sgl-project/sglang
SGLang is a fast serving framework for large language models and vision language models.
Language:Python
Total stars: 8931
Stars trend:
7 Feb 2025
10pm ▊ +6
11pm ▊ +6
8 Feb 2025
12am ▍ +3
1am █ +8
2am █▋ +13
3am █▎ +10
4am ▌ +4
5am █▊ +14
6am █▋ +13
7am █ +8
8am █▊ +14
9am █▊ +14
#python
#cuda, #deepseek, #deepseekllm, #deepseekv3, #inference, #llama, #llama2, #llama3, #llama31, #llava, #llm, #llmserving, #moe, #pytorch, #transformer, #vlm

GetSoloTech/solo-server
Platform for Hardware Aware Inference
Language:Python
Total stars: 136
Stars trend:
6 Mar 2025
3am ▏ +1
4am +0
5am ███▊ +30
6am █████▍ +43
7am ▎ +2
#python
#ai, #computervision, #deeplearning, #edgeai, #inference, #llm, #llmops, #machinelearning, #mlops, #modeldeployment, #ondevice, #python3

huggingface/huggingface.js
Utilities to use the Hugging Face Hub API
Language:TypeScript
Total stars: 1819
Stars trend:
22 May 2025
10am ▎ +2
11am ▉ +7
12pm ▉ +7
1pm █▎ +10
2pm ▉ +7
3pm █▏ +9
4pm ▊ +6
5pm ▋ +5
6pm ▊ +6
7pm ▊ +6
8pm █▏ +9
9pm █▏ +9
#typescript
#apiclient, #hub, #huggingface, #inference, #machinelearning

trymirai/uzu
A high-performance inference engine for AI models
Language:Rust
Total stars: 241
Stars trend:
15 Jul 2025
12pm ▏ +1
1pm ▋ +5
2pm ███ +24
3pm ███▏ +25
4pm ██▍ +19
5pm ██▉ +23
6pm █▍ +11
#rust
#ai, #highperformance, #inference, #llm, #metal, #rust