OpenAI Tokenizer (Q2799): Difference between revisions
(Added [en] alias: Tokenizer) |
(Changed label, description and/or aliases in de, and other parts: Added german label and description) |
||
| label / de | label / de | ||
OpenAI Tokenizer | |||
| description / de | description / de | ||
Ein Tool, das zeigt, wie ein großes Sprachmodell einen Text tokenisiert | |||
Latest revision as of 11:52, 18 June 2026
A tool to show the how a large language model tokenizes a text
- Tokenizer
| Language | Label | Description | Also known as |
|---|---|---|---|
| English | OpenAI Tokenizer |
A tool to show the how a large language model tokenizes a text |
|
Statements
OpenAI
0 references