Introduction to Natural Language Processing

When it comes to text processing or when you send or receive some sort of data in any text format whether it is a mail or a message or anything else then just think about it that how you are going to apply machine learning on it. A lot of applications are using NLP like Apple Siri, Windows Cortona, Amazon Alex, etc.

In this blog we are going to cover :

• Usage of NLP
• Text Processing Approach
• Convert Text document to vector form
• Implementation using Python and NLTK

Usage of NLP

Let’s understand the use of NLP with the help of a few examples :

• Gmail (Spam Classification)

When you receive a mail on Gmail then how Gmail finds out whether this mail will go to your inbox or the spam box.

• Sentiment Analysis

When you watch a movie and review it on any website then how my machine learning model will predict that the review is positive or negative. This kind of feature is used on online shopping websites or Google Play Store. When we write a review for any product or an app then it is classified as positive or negative based on the text or words I have used in my review.

Similarly, Gmail keeps track of each mail that what sort of text or words are used. For example :

• Chatbots

Chatbots have killed a lot of jobs and introduced a new way of tackling the customers. When you call at customer care centers than there are 90% chances that you will be talking to a chatbot rather than a human. With the help of chatbots, AI has become more powerful. Earlier it was a job of humans to listen to the complaints or queries and then process them and they took a salary for that. But now the same task is done by bots without any salary. A Lot of companies like Google, Apple, Microsoft, or Facebook are using these kinds of chatbots.

So, these were the few examples of NLP that how companies are using this technique to process the text data and apply machine learning to it. We need to understand how to make our machines understand the text and convert it into vector form.

Text Processing Approach

Now, this is the approach towards the implementation of text data. There is a process that we need to understand and accordingly, we will get the text data converted into vectorized form. Let’s understand this approach one by one.

Tokenization

Tokenization means splitting your text data into individual tokens like this :

Sentence = "Hello John, how are you? let’s meet tomorrow for a movie, I bought two tickets"

Tokens = ["hello", "john", "how", "are", "you", "?", "let’s", "meet", "tomorrow", "for", "a", "movie", "I", "bought", "two", "tickets"]

Removing Stopwords

After splitting your data into tokens, we need to remove the words like is, am, are, the, that, and so on. These words are known as stopwords. We also need to remove punctuations like $, %, ^, &, *,?

Tokens = ["hello", "john", "how", "are", "you", "?", "let’s", "meet", "tomorrow", "for", "a", "movie", "I", "bought", "two", "tickets"]

Updated_tokens = ["hello", "john", "let’s", "meet", "tomorrow", "movie", "bought", "two", "tickets"]

Read Full Articel Here – Introduction to Natural Language Processing

Brain Mentors Pvt. Ltd. started with a mission to link the IT industry and educational institutions. We aim to transform our every student into an IT professional who is ready to be employed in the industry.