Department of Controversy
A Standard for Internet Measurement
at Last?

Summary: Techniques for estimating the number of readers of a Web page are discussed. It is suggested that comScore's methodology might make a standard acceptable to most.

by Katie Paine

A long, long time ago in a city far, far away, the forces of consistency gathered to discuss what was at the time a thought too nascent to be considered an idea... a gleam, perhaps, in the eyes of a few pioneers: How would we measure the Internet? In those days, cookies were still something you bought from Mrs. Fields, a hit was something that you heard on the radio and the evil genius who dreamed up pop-ups was probably still in diapers.

The Internet was just coming into commercial usage, and it was clear to those of us in the measurement business that this would be a form of communication that would be highly measurable at some point, even though back then we didn’t even have a common language to describe what we were talking about.

At the time, if you’d been doing something with the Internet for six or seven months, you were considered “an established expert.” Well, we at The Delahaye Group, Inc. started analyzing Web traffic and the content of discussion groups, and so a year after that initial conference, I was being introduced as “the Granny” of Internet measurement. I was glad to be counted as an expert, but the fact of the matter was that we could only track appearances; we had no clue as to how many eyeballs actually saw the mention.

Nearly a decade later, there are now industry coalitions, ongoing research studies and many, many more people trying to agree on the same things we were struggling with in that very first meeting. While there does seem to be general consensus that “hits” stands for “How Idiots Track Success,” and visits are a preferable measure, we still don’t have an answer to that eyeball question. We are, however, definitely getting closer.

First, a very basic explanation:

Essentially, there are two established methodologies for determining how many eyeballs saw your Web site, or, for PR people, how many people saw your story on an online news site.

The first approach is to actually audit the Web site in question by examining its log files and publishing the results. The leader in this audit arena is iPRO, a firm that virtually started Internet measurement with its launch in 1994 and, no doubt, is the standard in auditing.

The second approach takes its cue from Nielsen’s television ratings. A research firm lines up a random sample of the public and puts a device on their computers that tracks where they go and what they do on the Internet. Recent consolidations have narrowed the players in that arena to comScore Networks (which recently bought MediaMetrix) and Nielsen Net Ratings.

comScore boasts 1.5 million members on its panel, including some 50,000 from outside the U.S., 35,000 students and another 35,000 professionals at work. Nielsen claims to be the “gold standard” and “the worldwide standard in audience measurement,” but their Web site doesn’t give the actual size of its panel. Published reports list 41,000 users for its workforce panel and 65,000 overall.

The Advertising Research Foundation investigated comScore’s approach and reported:

In summary, netScore represents a bold departure from traditional media measurement in that it endeavors to measure both audience behavior and transactional behavior in the same respondents at a very fine-grained level. In scope as well as in scale, this undertaking holds significant promise for the research industry as a contributor to online media and commerce… The sample size involved in netScore is more like that found in marketing databases than it is in research panels. As such, netScore holds promise to take media measurement the 'final mile'—from media exposure to advertising exposure, and even to linking that exposure to behavioral response.

One would think that an endorsement by the highly respected Advertising Research Foundation would be all it would take to achieve “standard” status, but not in this field. There’s an organization called the Alliance for Online Media Measurement that has been set up to establish standards. Ironically, the AOMM doesn’t have a Web site, and the ARF and ESOMAR (ARF’s International equivalent) have very publicly opted out of the Alliance. Now the ARF and ESOMAR are setting up “an exploratory investigation with respect to standards of performance for online measurement and analysis.” Whatever that means, they promise us yet another paper on the topic by the end of the year. (Are you confused enough yet?)

Regardless of what other alliances and associations have to say on the topic, one PR measurement vendor has decided that comScore is the way to go. PRtrak has long used audited circulation data for print and Nielsen data for broadcast in its database. To keep up with the Internet, they needed similarly authoritative numbers for Internet media coverage. Thus their choice of comScore’s data.

While not endorsed by any major PR organization, a group of measurement professionals did get a presentation on the thought behind comScore’s methodology and, since nobody got up and screamed or threw rotten tomatoes at the idea, the conclusion could be drawn that comScore’s data is acceptable to most of us. We can only hope that this will end the debate so we can go back to actually doing measurement rather than debating how many hits fit on the head of a bar chart.