Consumer Insights

Product Design Case Study
Note: details omitted to protect company IP.

Problem Statement

Given Natural Language Processing (NLP) technology that can "read" news articles, how can we automatically detect consumer insights in news articles?

Currently, the customer manually searches for consumer insights or reads through briefings produced by marketing research firms. This process takes a lot of time, and they still miss important new consumer trends.

Approach

1. Talk to the customer about what kind of "consumer insights" they're looking for and how these insights are used today. Ask them to provide examples of what they consider to be "good" consumer insights in news articles.

2. Given their examples, manually gather a set of news articles that appear to contain "good consumer insights" and check understanding with customer.

3. Determine and record what makes each example a "good consumer insight." Evaluate for commonalities and patterns and synthesize into working definition of a "good consumer insight."

A good consumer insight contains a:

a who: consumer group (e.g. millennials, Americans, American women)
a what: market segment (e.g. plant-based meat alternatives, dairy, smartwatches)
a change: a change in consumer behavior
- how much: magnitude of the change (e.g. risen 50%, up $4 million from last year)
- time frame: over what time frame did the change occur (e.g. since last year, in the past decade)
a why: ideally, some indication as to why the change has occurred

https://www.usatoday.com/story/money/2017/07/03/americans-eat-more-beef-and-meat-trend-thats-expected-continue/435331001/

4. Work with engineers to identify these elements within news articles and score each document for the appearance and prevalence of these elements.

5. Evaluate the results and iterate on the formula. Factor in other important attributes, like credibility and content category.

Regarding the credibility of news articles, we curated a list of priority sources internally and with our customer that would boost the importance of a particular insight. We also created a blacklist of untrusted sources that the customer never wanted to see.
Regarding content coverage, we initially were focused broadly on "retail." Per our customer's interest, we added specific categories, like grocery and electronics, in which to search for consumer insights.

6. Present automated results in format suitable to the customer (a chatbot). Set up automatic daily update so the customer can discover and interact with new insights daily.

Results

The customer uses the bot everyday to surface consumer insights and inform their 200 person team about new or interesting consumer trends.

​Consumer Insights

Consumer Insights