By Bill Paarlberg—As automation becomes more common in communications measurement, algorithms are doing more data analysis. Can we depend on computers to find insight in our data? The answer is both yes and no; read on…
Standardization sets measurement free to provide insight
Today, the measurement industry is making welcome strides toward standardization. As a result, measurement data are becoming more consistent and readily available, and measurement procedures more valid, reliable, and replicable.
These are good things, because now our industry can concentrate on generating insight. Which is, after all, what we really get hired for. Although we call ourselves “measurement,” the service we provide is improving communications through insight derived from data. Or, as Mark Weiner has said, “the value of data is in the actionable insights it generates rather than in the data itself.”
So the future looks bright for all you measurement insight types out there. Freed from the confusion of unreliable data and procedures, you’ll be able to shine. Your insights will generate dramatic improvements in communications. Welcome to The Golden Age of Measurement Insight. Champagne all around.
Not so fast: Your insight is just not that, well, insightful
The unfortunate problem with insight is that humans are not that good at it. Or at least not as good as an algorithm. Measurement pros pride themselves on their ability to wrestle knowledge from piles of data. And rightly so. Today, human insight is vital for understanding the nuances of the data. But computers are getting smarter all the time, and, sad to say, experience shows that humans are not as smart as machines at making many types of data-based judgments.
Paul Meehl demonstrated this way back in 1954 in his book “Clinical vs. Statistical Prediction: A Theoretical Analysis and a Review of the Evidence.” This classic of the psychology canon provides plenty of evidence that statistical models almost always yield better predictions and diagnoses than the judgment of trained professionals. That is, experts are inferior to algorithms in making decisions based on complex information. Intuition, keen judgment, and decades of experience? No match for automation.
This result has been replicated many times during the 60 years since Meehl’s book, to the chagrin of proud professionals in a wide variety of disciplines. It is no doubt a measure of how unwelcome this finding is that is not more widely known. For more on the cognitive science of this, see Daniel Kahneman’s “Thinking, Fast and Slow” for a discussion.
An algorithm is going to take your job—for some it already has
As for your job in measurement insight, the writing is already on the cubicle wall. Companies aren’t shy at all about replacing human professionals with algorithms. In the U.S. employment space, for instance, some 72% of CVs are never seen by human eyes. Remember that next time you apply for a job: your first test is to make it past the robot gatekeeper.
Of course, communications measurement is a lot more complex than reviewing a pile of job applications, but automation has already changed the measurement landscape. Computers are more and more often doing media data analysis. SaaS (software as a service) measurement companies already provide some automated analysis. Although their products as yet have serious limitations, they point to the future.
Wait—there’s hope for you yet
So does this mean the Golden Age of Measurement will have more to do with decision-making algorithms than with the expert judgment of measurement pros? Well, yes and no. It may well be true that algorithms rule our working lives, but they still make mistakes. Computers are now smart enough to beat humans at chess and Go, but they are still only as good as the humans who design them. And they are not smart enough to just run out and play on their own. Remember the financial crisis of 2008?
Facebook is the latest poster child for sheer algorithmic stupidity. It thought it would remove the possibility of bias in its Trending Topics feature by replacing its human editors with an anonymous algorithm. Oops; the algorithm turned out to be not smart enough to tell real news from fake.
Herding data with algorithms
Algorithm-driven automation has dramatically improved our ability to collect and analyze impossibly large amounts of raw media monitoring data. But there are times when algorithms just can’t do the job. Humans still need to do the tricky stuff. There are times when humans are more accurate and/or less expensive than computers. Algorithms will get smarter and smarter, but humans will always have to design them, check their work, and try to improve on their results.
And so you can relax about the robots and communications measurement; your communications insight job is safe. Well, more or less. You’ll just be herding data with more and more help from algorithms. Heck, if you set up search strings or filters for your media monitoring, then you are already doing that. An algorithm is only going to take part of your job. In the best of all possible worlds, it will take the obvious and boring parts, and leave the really clever and exciting insights to you.
Latest posts by Bill Paarlberg (see all)
- The Growing Demand for Online Privacy: Will It Hamper Communications Measurement? - July 21, 2017
- What Is the Meaning of Measurement, Part 2 - July 21, 2017
- Great Minds on Measurement: John Cage - July 18, 2017