Man versus machine – Why text analytics alone is not the solution

Man versus machine – Why text analytics alone is not the solution


Apr 01, 2015

Share It

“The technology doesn’t work. It doesn’t give me the answers.”

This tends to be the viewpoint of many with high hopes for a technology tool to deliver on the hyped promises seen in the marketing literature.

Many times, we view technology as a substitute for human labor – we hope that we can get the answers we need without having to do any work.  Yet, Peter Thiel (co-founder of Paypal), in his book Zero to One: Notes on Startups, or How to Build the Future points out that technology tools, such as machine learning and text analytics, should not be viewed as a replacement or substitute for human labor, but as a complement.

In his book, Mr. Thiel gives the example of a quickly emerging fraud problem during his days at Paypal, and how is perspective was informed:

“In mid-2000, we had survived the dot-com crash and we were growing fast, but we faced one huge problem: we were losing upwards of $10 million to credit card fraud every month.  Since we were processing hundreds or even thousands of transactions per minute, we couldn’t possibly review each one—no human quality control team could work that fast.”

“So we did what any group of engineers would do: we tried to automate a solution.  First, Max Levchin assembled an elite team of mathematicians to study the fraudulent transfers in detail.  Then we took what we learned and wrote software to automatically identify and cancel bogus transactions in real time.  But it quickly became clear that this approach wouldn’t work either: after an hour or two, the thieves would catch on and change their tactics.  We were dealing with an adaptive enemy, and our software couldn’t adapt in response.”

“The fraudsters’ adaptive evasions fooled our automatic detection algorithms, but we found that they didn’t fool our human analysts as easily.  So Max and his engineers rewrote the software to take a hybrid approach: the computer would flag the most suspicious transactions on a well-designed user interface, and human operators would make the final judgment as to their legitimacy.”

Since, Peter Thiel has taken this same approach in launching his latest venture, big data mining software company, Palantir Technologies.  He rationalized, “if humans and computers could achieve dramatically better results than either could alone, what other valuable businesses could be built on this core principle?”

The same is true of text analytics technologies.  While these software applications will never supplant the need for expert human analysis, the technology is key to identifying patterns undetectable to the human eye and interesting places to look. One of the major complementary benefits of text analytics is it offsets human biases.  Humans tend to see what we want to see in the data.  Text analytics takes the subjectivity out of reading large data sets and drawing incorrect conclusions which sometimes happens when a key phrase or concept jumps off the page to the human reader.

From there, it is the human analyst that has to make meaning and derive context from the results. This strategy, starting with unsupervised text analytics and finishing with an analyst identifying, monitoring, and assigning meaning to the critical insights, is far superior to either method on its own. It removes bias yet leaves space for continuous, nuanced learning tailored to one’s goals and objectives. Technology tools like text analytics make trained analysts more valuable and scalable in their functions of extracting meaning from unstructured data sets.

One of the drawbacks of investing in technology tools like text analytics is the expense.  Unless you have vast amounts of unstructured data for analysis and want to invest in the expert human capital to effectively use these applications, the cost-benefit may not justify the investment.

Our clients appreciate the “crawl, walk, run” strategy that we use regarding text analytics. Whether they’re coming off of an annual contract with a text analytics tool that they had independently licensed or just beginning to integrate text analytics into their consumer feedback strategies, they find it helpful to get started slowly, understand what’s possible, and then expand use cases from there. They value the fact that we help bring them up the learning curve and avoid many of the pitfalls and exceptional costs companies experience when they don’t have sufficient understanding of all that is entailed in implementation and maintenance.

As Peter Thiel points out, “humans and technology can achieve dramatically better results than either could alone.”

New Call-to-Action

  • Andrew Fu

    Great article and great story from Thiel – as much as we want to believe that computers will one day think like humans, it’s nice to know we’re not expendable just yet.

iModerate’s online qualitative interviews have been enormously helpful to us during the concept testing phase of research. iModerate provides us with invaluable feedback from a nationally representative group of Americans within a very short time frame. Not only do we get this data quickly, but it is also high quality. iModerate’s moderators are skilled at asking questions that yield useful responses. iModerate reports provide information that’s more than interesting, it’s actionable.

Sara Bamossy, Senior Strategic Planner, Saatchi & Saatchi LA