GitHub repo leaderboard by stars, growth rate and activity.

• LMCache

  #1LMCache/LMCache

  LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

  Python11,749 starslast pushed 43 minutes ago
• syv-ai

  #2syv-ai/qwen38-27b-rtx3090

  Qwen3.8-27B on a single RTX 3090 with vLLM: ~1,000 tok/s at 64 concurrent (int8 tensor-core GEMMs, fp16 DeltaNet state), ~114 tok/s single-user at default sampling / ~124 greedy (MTP drafts, own-output draft vocab, calibrated int4 lm_head, split-KV verify attention), 150k-262k context; patches, requant scripts, benchmarks

  Python1,266 starslast pushed 20 hours ago
• datawhalechina

  #3datawhalechina/zero-to-sglang

  Official SGLang × Datawhale course on LLM inference (中英双语): understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. 《从零手搓SGLang》：读懂推理，手搓 mini-sglang，吃透 SGLang 源码。

  Python694 starslast pushed 14 hours ago
• chadhurley25075-png

  #4chadhurley25075-png/pd-bridge

  Heterogeneous prefill/decode for DeepSeek-V4-Flash: CUDA prefill (DGX Spark, vLLM) -> Metal decode (Mac Studio, oMLX) over plain 10GbE

  Python108 starslast pushed 2 days ago

#	Repo	Language	Stars	30-day trend	Last updated
1	LMCache LMCache/LMCache LMCache: Supercharge Your LLM with the Fastest KV Cache Layer	Python	11,749		last pushed 43 minutes ago
2	syv-ai syv-ai/qwen38-27b-rtx3090 Qwen3.8-27B on a single RTX 3090 with vLLM: ~1,000 tok/s at 64 concurrent (int8 tensor-core GEMMs, fp16 DeltaNet state), ~114 tok/s single-user at default sampling / ~124 greedy (MTP drafts, own-output draft vocab, calibrated int4 lm_head, split-KV verify attention), 150k-262k context; patches, requant scripts, benchmarks	Python	1,266		last pushed 20 hours ago
3	datawhalechina datawhalechina/zero-to-sglang Official SGLang × Datawhale course on LLM inference (中英双语): understand inference, build a mini-sglang from scratch, then read the real SGLang source and land your first PR. 《从零手搓SGLang》：读懂推理，手搓 mini-sglang，吃透 SGLang 源码。	Python	694		last pushed 14 hours ago
4	chadhurley25075-png chadhurley25075-png/pd-bridge Heterogeneous prefill/decode for DeepSeek-V4-Flash: CUDA prefill (DGX Spark, vLLM) -> Metal decode (Mac Studio, oMLX) over plain 10GbE	Python	108		last pushed 2 days ago

All · 11,658