huggingface/tokenizers · MMO ALL

MMO ALLBuild · Share · Earn

Home Community Projects Following GitHub Insight Opportunity Radar

/

Build · Share · Earn

Pages

About
Rules
Guide

Connect

© 2026 MMO ALL. All rights reserved.

huggingface/tokenizers · MMO ALL

← Back to Insight

huggingface

huggingface

huggingface/tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Open on GitHub Project homepage

11,027

Stars

1,195

Forks

260

Open issues

11,027

Watchers

Apache-2.0

License

Rust

Language

Nov 1, 2019

Created

Sep 10, 2026

Last pushed

+2

Star growth · 1 day

Not enough data yet

Star growth · 7 days

Not enough data yet

Star growth · 30 days

Topics

bert gpt language-model natural-language-processing natural-language-understanding nlp transformers

Stars over the last 90 days