Natural Language Processing (NLP) in computing started over 40 years ago with a series of hard if-then rules that defined how text should be interpreted. Traditionally, NLP is thought of as analyzing large volumes of text to achiee a goal. Some of these include but are not limited to:
- Basic speech recognition and parsing
- Relational analysis
- Meaning, intent and sentiment
- Summarization
Natural Language Search (NLS) was introduced through search engines (anyone else an AltaVista lover?) and perfected by Google. Today, you can type the same phrases in Google and Bing and get very similar results.
There’s a lot of processing going on behind these, and the overall models are quite similar. Both Google and Bing crawl the web to create a corpus of data to search (“the data”). They presumably index this data and great some contextual mapping. When a user types a question, there’s a whole host of models Google and Microsoft (along with other search platforms) are using to return a contextual, accurate answer. While Google does not confirm which models they use for Search, they sometimes publish when they’re making updates. See here.
There’s probably some sort of neural network running for context and relationships between concepts, a basic parser that helps to understand sentence syntax and structure, a distance model that helps deal with misspellings and similar words and more.
The point being, the end results are generally the same for all search engines. They’re all indexing the same data, understanding the questions you type and providing the same answers within reason (or even suggesting the same things).
In a way, Google, Bing and the rest of the search providers have it easy. They have huge datasets available to them by serving consumers directly, and they index billions of websites represented with a collective size over 100,000,000 gigabytes.
This leads us to the intersection of NLS and the Great B2B Whale.
NLS in Enterprises
First time founders often build B2C, while second time founders build B2B products. Why is that? Businesses have money and they’re willing to spend it at scale! So it makes sense that natural language search built for consumers and made pervasive by Google would be extended to the B2B market.
NLS for Analytics (NLSa) is the intersection of data and NLP at the scale of the individual enterprise. ThoughtSpot was founded in 2012, IBM had natural language search in their Watson line in the middle of the decade, and PowerBI/Tableau/others followed by 2018/2019. So what has the last 9 years of NLS in Enterprises given us?
“In the long run, everything is a toaster” — Bruce Greenwald
Turns out, natural language search for business isn’t all that different to natural language search for consumers. The only difference is the scope and type of data that is exposed. Further, the intent of natural language search for enterprises is geared towards analytics — data that is structured as opposed to unstructured webpage listings. This should be easier to build rules that are customizable to individual enterprises.
Unlike the consumer market, Natural Language Search has not been adopted for analytics within most organizations. If you ask the average analyst how they consume analytics today, they would tell you it is via canned reports or dashboards.
So, is NLSa a commodity?
At first glance, it may appear to be a commodity. Look at these similar user interfaces:
Commoditizing a product means to render it widely available and interchangeable with other products in it’s class. Typically, widely available implies there is both supply and demand. By this definition, NLSa is not yet a commodity product for two related reasons: demand (aka adoption) and product maturity.
Adoption
While adoption is accelerating, the vast majority of companies do not use NLSa today. This simple measure of usage is a testament to why NLSa is not yet a commodity. The reason why adoption has lagged is due to product maturity
Product maturity
Unlike it’s consumer-based cousin, NLSa is not at the point that it effectively saves users significant time. While the interface has coopted the best, most recognizable part of Google (typing a question and tying that to some data), it is currently missing complementary automation features.
In your typical NLSa platform today, a user has a business problem. Let’s take the following example:
Problem: Performance is down for sales of alcoholic beverages
Step 1: User queries “show me sales of alcoholic beverages”, tool returns a chart
Step 2: They ask the same question, with a different twist, 100 times. It’s very similar to navigating a dashboard with filters.
Natural Language Search for consumers is a mature product with an entire ecosystem built around it to allow the user to solve their entire problem. Take the following example of a user’s experience in Google Search:
Problem: User’s vehicle is broken
Step 1: User Googles vehicle symptoms, Google returns likely cause
Step 2: User Googles how to fix likely cause, Google returns a step-by-step Youtube video
The interface is the same (typing a question), but the results are staggeringly different. NLSa effectively has coopted the best, most portable parts of natural language search: understanding individual words and the context behind them relative to “the data”. Today, NLSa vendors have not done a good job of providing tools that help the user to truly solve their problem.
To drive adoption of NLSa, vendors should look towards their consumer compatriots.
So what’s next?
We can get a glimpse of what is next by looking at what happened to Google Search. After they built a great Search tool, they started building complementary automation services that address the most frequent things that people are searching for:
- Flights (Google Flights)
- Translations (Google Translate)
- Directions (Google Maps)
- Things to buy (Google Shopping)
- Entertainment/instructions/self-help (Youtube)
They saw what their customers were searching for, and they started to build turnkey solutions that would help customers solve their problems. All of a sudden, Search was funneling customers to automation based on the questions they were asking.
The same blueprint is following in NLSa. Vendors who take a look at the questions that business users, analysts and data scientists are asking and build automation to solve those problems will succeed. Natural language search is not the destination — it is the funnel towards solving the problem.
Picture a world where a business user or analyst can easily answer complex why questions through automation. When an analyst for an eCommerce company asks
Why is my profit increasing for laptops
the answer that is returned will not be a simple visualization — it will be a complex, nuanced summary presented in natural language, driven by applied ML and statistics. This summary should be a turnkey analysis for typical performance questions.
As consumers use Google Search as their jumping off point to learn all about a restaurant’s menu, reviews, directions, booking a reservation and a ride, business users will use NLSa to as a jumping off point for measuring, understanding and making an impact on their organizations powered by data.
Automated answers adjacent to search is where the market is going, and Tellius is spearheading this vision with our Automated Insights. To learn more about what we’re doing to help analysts work faster, check out Tellius.
Note: for all of the Google vs Bing examples in this article I used Incognito mode.
