GitHub repo leaderboard by stars, growth rate and activity.

• vllm-project

  #1vllm-project/vllm

  A high-throughput and memory-efficient inference and serving engine for LLMs

  Python91,383 starslast pushed 22 hours ago
• sgl-project

  #2sgl-project/sglang

  SGLang is a high-performance serving framework for large language models and multimodal models.

  Python35,728 starslast pushed 23 hours ago
• NVIDIA

  #3NVIDIA/TensorRT-LLM

  TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

  Python14,580 starslast pushed 22 hours ago
• lightseekorg

  #4lightseekorg/tokenspeed

  TokenSpeed is a speed-of-light LLM inference engine.

  Python2,112 starslast pushed 22 hours ago
• truespar

  #5truespar/paddock

  Native Rust inference server for open models on NVIDIA GPUs. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio

  Rust75 starslast pushed 2 days ago

#	Repo	Language	Stars	30-day trend	Last updated
1	vllm-project vllm-project/vllm A high-throughput and memory-efficient inference and serving engine for LLMs	Python	91,383		last pushed 22 hours ago
2	sgl-project sgl-project/sglang SGLang is a high-performance serving framework for large language models and multimodal models.	Python	35,728		last pushed 23 hours ago
3	NVIDIA NVIDIA/TensorRT-LLM TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.	Python	14,580		last pushed 22 hours ago
4	lightseekorg lightseekorg/tokenspeed TokenSpeed is a speed-of-light LLM inference engine.	Python	2,112		last pushed 22 hours ago
5	truespar truespar/paddock Native Rust inference server for open models on NVIDIA GPUs. OpenAI- and Anthropic-compatible APIs, GGUF + safetensors, FP8/NVFP4/MXFP4/Q8/Q4, built-in Studio	Rust	75		last pushed 2 days ago

All · 11,474