Articles → NLP → Bag Of Words In NLP

Bag Of Words In NLP

This article describes the Bag of Words in NLP.

What Is A Bag Of Words?

Bag of Words is a text representation technique used in NLP and machine learning. It converts text (a sentence, paragraph, or an entire document) into a fixed-size numeric vector.

It ignores grammar, which means that it does not care about the sentence structure. It ignores word order. It keeps a count of the number of occurrences of the word.

Sparse Matrix

A sparse matrix is a matrix in which most of the elements are zero, and only a few are non-zero. There is no single universal threshold, but in practice, if the number of zero elements exceeds 50%, then the matrix is considered sparse.

Consider a 3 X 2 matrix, i.e., the matrix has six elements. If the number of zeros is four, then it is considered a sparse matrix.

How Does A Bag Of Words Work?

Build a vocabulary

To begin, collect all unique words from the dataset. Consider the following dataset: -

Sentence 1: "I like NLP"
Sentence 2: "I like machine learning"

Vocabulary = {I, like, NLP, machine, learning}

Create a count vectorization for each document

Count how many times each word appears in the sentence. The order of words in the vector is fixed according to the vocabulary.

In count vectorization, we create a matrix called a sparse matrix, where the documents are specified in rows and the vocabulary words in columns.

"I like NLP" → [1, 1, 1, 0, 0]  
"I like machine learning" → [1, 1, 0, 1, 1]

Example

import nltk
from sklearn.feature_extraction.text import CountVectorizer

# Example corpus
documents = [
    "I like NLP",
    "I like machine learning",
    "NLP is fun and powerful"
]

# Initialize CountVectorizer
vectorizer = CountVectorizer(
    lowercase=True,
    stop_words='english'   # remove English stopwords
)

# Fit and transform documents
X = vectorizer.fit_transform(documents)

# Get vocabulary
vocab = vectorizer.get_feature_names_out()

print("Vocabulary:", vocab)

print("\nBag of Words Matrix:")
print(X.toarray())

Output

How Is Bags Of Words Different From The Count Vectorization?

Bags of Words is the theoretical model/idea. Whereas the Count Vectorization is the practical implementation of Bags of Words that outputs a sparse matrix.

Bag Of Words And Semantic Meaning

In Bag of Words (BoW), a text is represented only by: -

Which words appear?
How many times do they appear?

It completely ignores word order, context, meaning, and the relationships between words. Consider the following two sentences: -

I love this movie
I do not love this movie

The sparse matrix would be: -

Sentence	I	love	this	movie	do	not
I love this movie	1	1	1	1	0	0
I do not love this movie	1	1	1	1	1	1

Both sentences look very similar, even though the meanings are opposite.

Same Meaning Of Different Words

Consider the following sentences: -

The car is fast
The vehicle is quick

BoW treats car ≠ vehicle, and fast ≠ quick, even though they mean almost the same thing.

Posted By -	Karan Gupta

Posted On -	Thursday, September 11, 2025

Updated On -	Tuesday, February 10, 2026

Query/Feedback

Your Email Id		**

Subject		*

Query/Feedback	Characters remaining 250	**