GitHub repo leaderboard by stars, growth rate and activity.

• vllm-project

  #1vllm-project/vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  Python91,383 starslast pushed 23 hours ago
• bentoml

  #2bentoml/BentoML

  The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

  Python8,832 starslast pushed 3 days ago
• vllm-project

  #3vllm-project/vllm-omni

  A framework for efficient model inference with omni-modality models

  Python6,743 starslast pushed 23 hours ago
• beclab

  #4beclab/Olares

  Open-Source Personal Cloud OS for Always-On Agents

  Go5,263 starslast pushed Yesterday
• FedML-AI

  #5FedML-AI/FedML

  FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

  Python4,065 starslast pushed 11 months ago
• vllm-project

  #6vllm-project/vllm-ascend

  Community maintained hardware plugin for vLLM on Ascend

  C++2,786 starslast pushed 23 hours ago
• kitops-ml

  #7kitops-ml/kitops

  An open source DevOps tool from the CNCF for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI Artifact.

  Go1,412 starslast pushed 3 days ago
• truespar

  #8truespar/paddock

  Native Rust inference server for open models on NVIDIA GPUs. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio

  Rust75 starslast pushed 2 days ago

#	Repo	Language	Stars	30-day trend	Last updated
1	vllm-project vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs	Python	91,383		last pushed 23 hours ago
2	bentoml bentoml/BentoML The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!	Python	8,832		last pushed 3 days ago
3	vllm-project vllm-project/vllm-omni A framework for efficient model inference with omni-modality models	Python	6,743		last pushed 23 hours ago
4	beclab beclab/Olares Open-Source Personal Cloud OS for Always-On Agents	Go	5,263		last pushed Yesterday
5	FedML-AI FedML-AI/FedML FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.	Python	4,065		last pushed 11 months ago
6	vllm-project vllm-project/vllm-ascend Community maintained hardware plugin for vLLM on Ascend	C++	2,786		last pushed 23 hours ago
7	kitops-ml kitops-ml/kitops An open source DevOps tool from the CNCF for packaging and versioning AI/ML models, datasets, code, and configuration into an OCI Artifact.	Go	1,412		last pushed 3 days ago
8	truespar truespar/paddock Native Rust inference server for open models on NVIDIA GPUs. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio	Rust	75		last pushed 2 days ago

All · 11,474