No One Ever Went Broke by Underestimating the Intelligence of Typical Sentiment Analysis


Your next great product innovation is rooted in understanding consumer behavior in a novel way that the market does not understand it at the moment. This will be your leg up on the competition. This will be how your next great product is born. Towards those ends, your understanding of a consumer needs to be deeper than an emoji analysis.

The problem is that consumers do weird stuff. It’s hard to understand why consumers do what they do even if you have newfangled big data software that can chew on big data and spit out who is happy and who is not. But, your true insight lies beyond typical sentiment analysis.

To illustrate this point, let us revisit what I believe is a famous case of consumer behavior frustration — now almost 100 years in the past!

Legendary journalist, satirist, cultural critic, and scholar, H.L. Mencken, famously stated, “No one in this world, so far as I know — and I have searched the records for years, and employed agents to help me — has ever lost money by underestimating the intelligence of the great masses of the plain people. Nor has anyone ever lost public office thereby.”

This quote has been tweaked, re-spun, and tossed about like salad, ever since. Over time, the original quote evolved, most notably into the disparaging phrase, “No one ever went broke underestimating the intelligence of the American public.”

I believe that at the heart of of Mencken’s sentiment is a frustration born of struggling to understand consumer behavior. After all, Mencken wanted his paper, the Chicago Daily Tribune, to sell more newspapers. However, Mencken was rankled that the tabloids had several advantages over the Chicago Daily Tribune.

In his signature piece, Notes on Journalism, published September 19, 1926, he excoriated readers of tabloid papers. He also lamented about the advantages tabloids had over newspapers like the Chicago Daily Tribune: Tabloids were lighter and less bulky than daily newspapers; they had two or three sections and weighed only a pound or more; they could be distributed more quickly than the larger papers. He went on explain that a critical mistake publishers of tabloids often made was that once they were successful and profitable, they tried to appeal to more refined readers by increasing the quality of their publication, which never worked.

What does Mencken’s frustration tell us? For one thing, it tells us that consumer behavior back then was a hard thing to understand. It still is today.

Modern sentiment analysis technology wasn’t available to H.L. Mencken, but even if it was, would it really reveal anything other than Mencken’s frustration? Would it simply confirm his notion that the “near illiterates” (ouch) love tabloids?

That is the problem with sentiment analysis. Typical sentiment analysis far too often leaves out the context of how sentiment analysis can be applied to a consumer’s unarticulated needs and compensating behaviors.

An unarticulated need is one which a consumer doesn’t even know they have. A compensating behavior is one where a consumer does something with an existing product in a way in which it was not necessarily designed to do. Think of the myriad of ways one can use duct tape. Think of how people use mason jars to drink beer. Think of how creative parents package lunch snacks in plastic Easter eggs. When you understand those aspects of consumer behavior, you begin to understand the consumers themselves.

With a deeper understanding of the consumer, could the Chicago Daily Tribune way back then have made their papers lighter? Could they publish, under the fold of their hallowed publication, content that appealed to the, shall we say, more lowbrow nature, perhaps that dwells in all consumers?

What would Mencken think of today’s popular online publications? After all, many presumably smart intellectual consumers, perhaps just like Mencken, might likely read an article about economics, the news of the day, and some new technological breakthrough first. However, afterwards, their eyes might wander, ever so slowly, down to the bottom part of the website — where “sponsored” or “promoted” content lives.

Perhaps once Mencken had his intellectual fill, his reading eyes might gander at the “Controversial ‘Brain Pill’ Billionaires Use”; or perhaps, he might find himself inexplicably drawn to the link for “210 Kate Upton GIFs You Just Have To see”. Mencken, so far as we can tell from historical photos, might then hide the view to his monitor from nearby editors, while he secretly perused, “The Botox Alternative That Doctors Don’t Want You To Know About.”

That is all hypothetical of course. After all, we know very little about H.L. Mencken’s consumer behavior, other than his well chronicled frustrations with tabloid readers. However, we do know one thing: No one ever went broke by underestimating the value of typical sentiment analysis.

Learn More