huggingface/tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

https://github.com/huggingface/tokenizers ↗

★ 10.9k · forks 1.1k · watchers 121 · open issues 235

License: Apache-2.0 · Default: main · Topics: bert, gpt, language-model, natural-language-processing, natural-language-understanding, nlp, transformers

github · bert · 0 reviews · updated 19 days ago

Critics score

—

Audience score

—

Scores hidden until this repo has at least 5 reviews.

Wiki, Report & Thread

Wiki page · AI Report (premium) · Discussion thread

Your review

Recent reviews

No reviews yet. Sign in to be the first to review this repo.