huggingface/tokenizers

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

https://github.com/huggingface/tokenizers ↗

★ 11.2k · forks 1.2k · watchers 120 · open issues 188

License: Apache-2.0 · Default: main · Topics: bert, gpt, language-model, natural-language-processing, natural-language-understanding, nlp, transformers

github · bert · 0 reviews · updated 10 days ago

Critics score

—

Audience score

—

Scores hidden until this repo has at least 5 reviews.

Wiki, Report & Thread

Wiki page · AI Report (premium) · Discussion thread

Your review

Recent reviews

No reviews yet. Sign in to be the first to review this repo.