Gain Unprecedented Insight Into Your Business, Customers and Products Using Natural Language Process

Companies everywhere have been generating tons of unstructured data after going paperless. Everything that’s happened in the paperless history of a company has some kind of digital record. Altercations between customers and sales teams are reflected in old email threads. The journey of a new product as it was developed and brought to market is trapped inside a collection of PDFs, presentations, emails, search histories, ticketing systems, etc. How can these stories be told using these systems?

Information in email messages, website content, PDF documents, presentations, recorded telephone calls, text messages, recorded video footage, word docs, and social media posts are examples of dark, or unstructured, data. And companies have a ton of it. In fact, It is estimated that 80 percent of company data is in this form. Unstructured data is usually scattered across various applications, computers, storage spaces, etc., and are isolated from each other. If that’s not bad enough, consider this: It’s not easy to query the unstructured data, so the valuable information trapped in these repositories cannot be effectively used for analytical purposes.

Structured data, on the contrary, is a form of information that resides inside database systems that is easily accessible and can be utilized for running queries. Mining the information inside structured data repositories and databases helps businesses make key decisions based on facts, not intuition.

Traditionally, it’s been impossible to automatically mine large amounts of unstructured data, and the only viable option has been human resources who find, read, and make sense of this type of information. It is simply cost-prohibitive and very exhausting for humans to try and process massive amounts of data.

It’s remarkable to note that not only is it expensive to generate dark data, but it also costs a company quite a bit to retain this data onsite or to store it in offsite storage systems. Nowadays, companies save dark data mostly for compliance purposes and not necessarily for future analysis. It’s just too much work.

Or was too much work until Artificial Intelligence, machine learning, and language processing changed everything.

Natural language processing is a field of computer science and artificial intelligence concerned with the interaction between computers and human language. Traditionally, the interaction between computers and humans has been through machine language. People have had to learn how to "talk" to/interact with computer systems and software applications through rigid computer commands and user interfaces. NLP is shifting the mindset to having machines learn how to interact with humans, eliminating the need for training the human element. This interaction happens in the form of written messages. Other forms of interactions, such as spoken words, need to be converted to text and fed to an NLP engine for processing.

Natural language processing consists of two main components: Natural Language Understanding and Natural Language Generation. Think of NLU as the ability to read and NLG as the ability to write. In this article, we will mostly focus on Natural Language Understanding and its application in the world of business.

How can NLP understand the meaning of text?

Here is a gross oversimplification in layman terms: Natural Language Understanding processes written text by breaking it down into basic building blocks and removing the most common words that make up the skeleton of the sentence and don’t participate in the intention and meaning. What’s left reflects the meaning or intention of the sentence. NLP compares the key words against a training set to make sense of the context of the sentence, regardless of the exact words used or their position in the sentence. NLP is trained to process text sentence by sentence and to categorize each sentence as it matches the intention according to predetermined categories attached to phrases and words. These are then handed over to systems that can perform actions based on the intended context. The actions vary and may include saving the detected information in structured databases, connecting to a third party application to take actions, or providing intelligent answers to questions. In a chatbot conversation scenario, the NLP intercepts messages from customers, interprets intention, then provides meaningful responses to help them with what they need.

The following example will help better understand the concept:

A customer contacts an HVAC company to ask for help with an air conditioning system that is in need of service. The customer sends a message to the company that says: "Hello, my air conditioning unit is not working. I need help as soon as possible, please." Note that the chatbot in this scenario has already been trained for industry-related keywords and phrases and can execute tasks on behalf of the customer. The NLP receives the message from the contact channel and understands that this message is from an existing customer directed to customer support, and that the object in question is an "air conditioning unit." It also understands that the help is needed immediately. Now, the customer may ask the question using many different words and phrases, but the NLP reacts in the same way nonetheless. This is because the NLP is scanning the message for more than just words; it works to derive the context to understand the meaning of the message. Equipped with the ability to understand customers` needs, the company in our example can handle basic customer inquiries around the clock without human involvement. A chatbot trained to handle customer service related issues and setup service calls can help this HVAC company do more with less resources.

NLP works in conjunction with Artificial Intelligence and Machine Learning to improve its training and increase the intent-matching success rate. With the ability to match intents derived from text, Natural Language Understanding can make sense of unstructured text and categorize it to produce structured data that can be saved in databases to be accessed by other software applications for a variety of different purposes.

A good example of unstructured data is information in email correspondence. It happens all the time. Employees scan through emails and read them only to find that a large percentage of incoming emails are noise or not really intended for them and need to be forwarded to others. An NLP based system can automatically filter noise and even route emails to intended recipients to save precious time.

If you remember back in the day, spam emails were a major problem for businesses that fell victim to lost productivity due to malware and virus infection. Fighting spam emails was a huge business challenge companies had to deal with. Then the spam filter techniques were born. What these spam filters did was go through every single incoming email, scan the content, and categorize them based on a detection logic to automatically flag them as spam.

Today, email filtration technologies use Natural Language Understanding to detect the intentions of email messages, then organize, categorize, delegate, or even respond to the emails, saving employees time.

Natural Language Processing can analyze the content of emails, text messages, word documents, PDF files, and social media posts to make sense of it all for the purposes of tagging, segmenting, normalizing, and summarizing the dark unstructured data into actionable structured data.

Without NLP, the only alternative is to use human cognitive abilities to process unstructured data. This alternative is not scalable, since structuring dark data requires immense cognitive processing power and a ton of time, which are scarce resources for any functioning business.

As mentioned before, 80% of data at the disposal of companies is unstructured. Therefore, there is a huge benefit for every organization to use some form of NLP to automate and make use of gathered data. Applications that use Natural Language Processing to automate business processes are on the rise. Here are some areas of business that are being transformed by this technology:

Sentiment Analysis:

For businesses, monitoring the reputation of their brands and products on social media is vital. Manually searching for clues in an ocean of information is an impossible task. Billions of lines of text need to be processed and analyzed, while new information is constantly being added. Older technologies relied on gathering samples of data to be analyzed by statistical models. The results were slightly better, but still not good enough.

Social monitoring applications equipped with Natural Language Understanding can scan the ocean of information for posts related to a specific company, brand, or product to accurately assess public opinion. When negatively trending information on social media outlets is found early on, company managers can be alerted to act on it immediately before small problems become big crises.

Text summarization

Another application of Natural Language Understanding is text summarization. Coupled with Artificial Intelligence, large amounts of text are processed and organized before a brief version of the original text is generated. This is a great tool for people who want to be informed about certain matters but don’t have the time to read through large amounts of text. CEOs are supposed to read 4–5 books a month to stay informed about various topics, but time is always hard to find. The text summarization technique can give readers the gist of the overall subject matter while saving them lots of time.

Text Translation

NLP systems support lots of different languages. For a list of languages supported by Google NLP, refer to the footnote of this article.

Natural Language Generation, a branch of NLP, is the concept of creating understandable sentences from structured data. It’s pretty much the reverse of the concept of Natural Language Understanding.

For example, if you ask your Smart speaker about the weather, it will check your location and the weather to find the answer, but instead of yelling out a single word like "70," it puts some context around it to present an appropriate response. It might say, "Right now, the weather is very nice in Arlington, Virginia, around 70 degrees fahrenheit. Would you like to know the weather for the weekend?"

That’s much nicer than just the word "70," isn’t it?

So we alluded to the fact that Natural Language Understanding and Natural Language Generation can understand and generate communication in a variety of different languages. I think it’s pretty clear where we are going with this: Real-time translation from one language to another.

Businesses that serve customers who speak different languages can use the power of NLP to effectively communicate with them. For example, a patient who does not speak English may not receive proper care when communicating with a practice in the U.S. due to language barriers. Medical organizations usually hire outsourced live translation services in advance to treat patients who do not speak the local language.

With Natural Language Processing, the patient can speak in their own language and the doctor can communicate back to the patient without the need for a human translator.

Virtual Assistants

Natural Language Processing is the building block of many interesting applications that have advanced in recent years, including virtual customer assistance and virtual employee assistance.

Answering telephone calls is a labor-intensive operation for call centers. Extending call center hours is extremely expensive. So much so that many companies outsource their call center operations to other countries to manage costs. Although offshore call center agents cost a lot less, they often fail to provide the customer experience demanded by callers due to culture clash and subpar language skills.

Natural Language Processing can change all that. According to Gardner, 25 percent of all first-level customer-to-business engagement will be handled by machines by the end of 2020. With proper training, conversational systems can conduct human-like voice conversations with callers and understand their requests to help them with marketing, sales, and customer service-related issues. Recent advancements in text-to-speech technology has reached a stage where many people cannot tell a virtual agent from a real person. Listen to the following example:

Call centers are on the verge of undergoing radical changes when companies embrace technologies that can mimic and, in some cases, even outperform humans.

It’s important to note that the idea is not to force humans out of their jobs. It’s about relieving human resources of monotonous tasks and shifting their responsibilities to deliver more gratifying work experiences. Call center agents who do not have to perform level 1 agent tasks can work in conjunction with other agents in teams to elevate customer experience. Participating at this level is an interesting and desirable job for humans who want to use their brains and creativity to produce value. For businesses, the extracted value from a human employee rises while the bandwidth of the call center increases.

Take customer experience to a new level

VCA technology enables companies to give personalized one-on-one engagement experiences to every single customer. This is huge, because if the company relied solely on human teams, such ambitions would be impossible. There are always fewer available agents than callers, resulting in queues where people have to wait before they can be served.

At the same time, customer experience is never consistent when interacting with human agents because they treat customers differently according to their own skill levels, emotions, and demeanors. VCAs can theoretically handle an unlimited number of inbound inquiries and provide a consistent customer experience, no matter the channel of contact. Customers never have to wait in queues to be served and always receive the same level of service.

Equipped with Artificial Intelligence, Machine Learning and Natural Language Processing, companies will be doing interesting things in the near future to further appease customers. Using NLP and artificial intelligence chatbot technologies, it is possible for companies’ fictional commercial ambassadors to take center stage when it comes to customer interaction. Sound crazy? It’s true!

Chatbots bridge the real-time isolated channels of communication between every single caller and the business. Chatbots provide customers with as much assistance as they need. No wait time, no limited hours. There will be personalized one-on-one help available no matter the time of day or day of the week.

The personalization of customer engagement adds a wow factor to the customer experience that people love to receive.

Flo, star of commercials aired by Progressive insurance, is very popular. People calling Progressive after seeing Progressive commercials would love for Flo to answer the phone and help them shop for insurance policies. Flo knows how to save you money and is a trustworthy character, and probably the reason why people decide to contact Progressive in the first place. When Flo takes your call, you’re likely to give Progressive your business. With the new advancement in text-to-speech and voice synthesis, we are approaching the point where it is impossible to tell a machine from a human in this context. So, it might just be possible in the near future to create a Flo avatar that can answer every customer’s call! This way, companies can give the true VIP treatment to every single one of their customers.

So, how might NLP play a role in medicine?

Telemedicine is without a doubt the future of receiving non-catastrophic care. In the near future, patients will contact cyber doctors that use NLP to communicate. Apart from the convenience factors such as no commute and no wait time, a cyber doctor can provide unlimited assistance to a patient. Moreover, it has access to unlimited resources, such as journals, research papers, medical data, and test results from a variety of different sources around the world and in different languages. Combining all existing experiences about a peculiar condition can result in the best possible course of treatment backed up by proven data.

NLP in your business

So how can you use Natural Language Processing technology in your business?

For starters, you can use NLP to make use of all that dark data, turning them into accessible, minable, and shareable knowledge. This is what we call a knowledge base, which is a vital component of Virtual Assistance technologies. You may not have started contemplating deploying conversational platform technologies in your business yet, but when the time comes to deploy a website live chat or a virtual customer service bot to handle your customers after hours, the knowledge base becomes a necessary element.

Use Social Listening to find what people want.

Use NLP to analyze social media platforms as you hunt for new product ideas. With social listening, you can gather data-backed evidence about problem areas. If you can act swiftly and take your product to market in time, you will have much better chances of success. Knowing that a product will be a hit ahead of time is unimaginable power.

Use chatbots to make your customers’ lives easier.

Back in the day, people loved the idea of hearing Angela’s soft and familiar voice on the phone to get help. They didn’t even mind waiting to be greeted by Angela. Today, waiting on the phone to speak to Angela is considered outrageous for many people who want to be served in real time and move on with their lives. When the market and customer demands change, businesses have to change along with them to adapt to the new normal. Failing to adapt in time may bring heavy consequences for modern businesses. Chatbot technology helps businesses of all sizes augment their human teams and empowers them to work smarter while providing the best possible customer experience.

Interested in learning how you can implement conversational technologies into your business? Take the next step and schedule a free consultation with a Zebyl expert to explore what conversational technologies can do for you and your business.

Author: Pejman Rajabian

Chief Technology Adviser

Zebyl Inc.

Footnote:

List of all languages supported by Dialogflow.

https://cloud.google.com/dialogflow/docs/reference/language

Zebyl is a conversational communications platform for businesses of all sizes. Zebyl makes it quicker, easier, and cheaper for businesses to interact with prospects and customers on the channels they prefer – like text messaging over landlines.