Articles → NLP → Word2vec (Skip-Gram Version) In NLP

Word2vec (Skip-Gram Version) In NLP

This article explains Word2Vec (skip-gram version) in NLP.

How Skip-Gram Is Different From CBOW?

In CBOW, we predict the center word from the surrounding words. But in skip-gram, predict the surrounding word from the center word.

To implement Word2Vec (Skip-gram version), we will work on these steps: -

Tokenization

Choose window size

Create skip-gram training pairs

Convert words to vectors (one-hot encoding)

Passing data to a neural network

Consider the following example: -

I love working with AI

The tokens would be: -

[I, love, working, with, AI]

Window size is the hyperparameter. For this example, let us set the window size to 1. So, each target word uses 1 word before and 1 word after.

Skip-gram asks: Given a word, what words appear around it?

love = [0 1 0 0 0]

Input  = workingOutput = loveInput  = workingOutput = with

Query/Feedback