AP[lab]
Courses
Tools
EN
RU
Sign In
Text Tokenizer Visualization
Visualizes different text tokenization approaches with interactive controls.
Text Tokenizer
Input Text:
Natural language processing is fascinating!
Tokenization Method:
Word
Character
Subword-bpe
Visualization Style:
Token Blocks
Highlighted Text
Bracketed Text
Tokenized Text:
Natural
language
processing
is
fascinating
!
Token Information:
Total tokens: 6