Articles → Spacy → Introduction To Natural Language Processing And Tokens In Spacy

Introduction To Natural Language Processing And Tokens In Spacy

In this article, we will provide introduction to Natural Language Processing and Tokens in Spacy.

What Is Natural Language Processing?

Natural Language Processing or NLP is the branch of computer science that deals with processing and analysis of text data.

Tokenization is the process of breaking up the original text into components pieces called tokens.

Here is the example

import spacy

nlp = spacy.load("en_core_web_sm")
doc = nlp(u"This is a sentence.")  # each word is token

for token in doc:
    print(token.text)

Once you run the above code, you get the following output

Query/Feedback