Articles → NLP → Introduction To Natural Language Processing And Tokens In Spacy

Introduction To Natural Language Processing And Tokens In Spacy

What Is Natural Language Processing?

Tokenization In NLP

Example Of Tokenization

import spacy

nlp = spacy.load("en_core_web_sm")
doc = nlp(u"This is a sentence.")  # each word is token

for token in doc:
    print(token.text)

Output