Articles → NLP → Word2vec (Skip-Gram Version) In NLP

Word2vec (Skip-Gram Version) In NLP

This article explains Word2Vec (skip-gram version) in NLP.

How Skip-Gram Is Different From CBOW?

In CBOW, we predict the center word from the surrounding words. But in skip-gram, predict the surrounding word from the center word.

To implement Word2Vec (Skip-gram version), we will work on these steps: -

Consider the following example: -

I love working with AI

The tokens would be: -

[I, love, working, with, AI]

Window size is the hyperparameter. For this example, let us set the window size to 1. So, each target word uses 1 word before and 1 word after.

Skip-gram asks: Given a word, what words appear around it?

love = [0 1 0 0 0]

Input  = working
Output = love
Input  = working
Output = with

Query/Feedback