What is Natural Language Processing?

What is Natural Language Processing?

Natural language processing is basically the process of computer analysis of inputs provided in human language and conversion of these inputs into a useful form of representation. The field of NLP is primarily concerned with getting computers perform some interesting tasks with human Language. It secondarily concerned with getting us come to a better understanding of human language. In simple terms, we humans and machines should talk and interact with each other. You have probably seen this interaction between Iron Man and Jarvis. Right?

The inputs/outputs of an NLP system can be:-

-        Written text

-        Speech

Here, the difficulty is more with spoken language. It needs everything required to process written text plus the challenges of speech recognition and speech synthesis.

There are two components of NLP:-

1.     Natural Language Understanding

It deals with mapping the given input in the natural language to some useful representation. Basically, machines to understand our language.

2.     Natural Language Generation

It is producing the output in natural language from some internal representation. Basically, the machine should respond to us in human language.

Here Natural language understanding is much more difficult than natural language generation. Because the human speeches have different forms, structures, and reference and thus here lies ambiguity.

How does it work?

When we speak something, the first step is to convert our speech into written text. This is generally done by converting our natural language into the artificial language which machines could understand.

We use speech recognition system to convert the spoken language into written text. Mostly speech recognition system today is based on Hidden Markov Models(HMMs). These are statistical models that convert our speech into written text using some mathematical calculations.

HMM, does this by breaking what we said into smaller units. Then comparing these units with pre-recorded phonemes. Phonemes are smallest units of speech. Then it looks at the series of phonemes and tries to map most likely words that suit what we said.

After converting our speech into written text. The next step is to understand the meaning. Most NLP Systems has lexicon(Vocabulary) and set of grammar rules. Firstly, it identifies whether the word is noun or verb, whether it is in present tense or past tense. NLP algorithms apply all these pre-fed things and use statistical machine learning technique to find out what is the meaning of the sentence and then uses more or less the same techniques to revert back. Just reverses the first step to text-to-speech and finally speaks the result loudly. 

How will it help us?

Natural language processing is already helping us in different ways.

You talk with the assistants provided by Google and Apple. It simply understands our language and redirects us to our result.

NLP is also used in spam filters. It uses techniques to understand which emails are good and which are spam. They do this by scanning the text of the emails that we receive and interprets whether it is a spam or not.

 Auto-Correction in our Keypads also work using the techniques of NLP. It understands the sentence that we type and map the incorrect word in it with the most suitable word present in its database.

 NLP can do much better things than we would have ever thought of in the near future.

I Hope this information helps. Finally, Thank you.