Articles → MONGODB → Default Language Of Text Search Indexing In Mongodb

Default Language Of Text Search Indexing In Mongodb

In this article, we will set the default language of text search indexing in MongoDB.

Tokenization And Stemming

While performing the search, we use tokenization and stemming.

This is the process of breaking a text into individual words or tokens. For example, the sentence The quick brown fox jumps over the lazy dog would be tokenized into separate words like The, quick, brown, fox, jumps, over, the, lazy, and dog.

This is the process of reducing words to their root or base form. For example, words like running, runs, and ran might all be reduced to the base form run.

Are Tokenization And Stemming Different For Different Languages?

As Stemming involves reducing the words to their root form, the grammar rules are applied. Different languages have different grammar rules, so stemming is different for different languages.

In tokenization as well, there are different rules for breaking text. Consider the following example: -

Picture showing the sample language sentence

As there is no space between the words, the tokenization of the above text is: -

Picture showing the tokenization of string

So, the tokenization of the Thai language is different from English.

How To Set The Default Language?

At the time of creating an index, we can set the default language.

db.collection.createIndex(
   {[field]: "text" },
   { default_language: "[language]" }
)

Picture showing setting the default language while creating the index

Posted By -	Karan Gupta

Posted On -	Tuesday, May 14, 2024

Query/Feedback

Your Email Id		**

Subject		*

Query/Feedback	Characters remaining 250	**