tiktoken
CuratedFast BPE tokenizer library for use with OpenAI's models.
Curated listing — added by OSSDrop, not the maker.
Sign in to claimGet your badge
Add this to your site or README. It shows your live OSSDrop standing and links back — no account needed.
HTML
<a href="https://ossdrop.com/tool/tiktoken"><img src="https://ossdrop.com/badge/tiktoken" width="250" height="56" alt="tiktoken on OSSDrop" /></a>Markdown
[](https://ossdrop.com/tool/tiktoken)Frequently asked
What is tiktoken?
tiktoken is an open-source developer tools & cli tool — Fast BPE tokenizer library for use with OpenAI's models. It is written in Python.
Is tiktoken free and open source?
Yes. tiktoken is free and open source, released under the MIT license. You can view and contribute to the source at https://github.com/openai/tiktoken.
What language is tiktoken written in?
tiktoken is primarily written in Python.
What are open-source alternatives to tiktoken?
Other open-source developer tools & cli tools on OSSDrop include VS Code, Oh My Zsh, MarkItDown, Go.
Discussion (0)
Sign in to comment.
No comments yet.