The Measurement Standard

April 06, 2010

Twitter Ranking Redux: We road test TweetLevel, Twitalyzer, and MyRealTwitterScore.

 Twittericon5plus3100
 by Katie Delahaye Paine

Summary: The author tested three Twitter profile analysis tools by comparing herself to five peer bloggers. Her relative ranking to her peers came out roughly the same for all tools. However, some of the numbers differed so wildly as to make no sense. Recommendations are given for measuring one's own influence and for measuring the relative influence of a blogger.
 

Nary a day goes by when a client doesn’t ask us here at KDPaine & Partners to determine the “Top 50” Twitterers covering their industry. So we spend a lot of time thinking about the best way to do this. Last June, our then Director of Research Chris Near put various Twitter influencer rating systems through their paces, including Twinfluence, TwitterAnalyzer, Twitter Grader, Twitterholic, and  TwitterScore. (Read his results in "Which Twitter Profile Analysis Tool Rules the Nest?")
 

TwitterGrader and TwitterAnalyzer came out on top in Chris’ study. But, as Chris said then, there’s a new tool coming along every week, so we thought it was time to revisit the topic. For this update we added TweetLevel, Twitalyzer, and MyRealTwitterScore. We dropped Twinfluence, TwitterAnalyzer, and TwitterScore from consideration because we couldn’t get them to work. And when we tried MyRealTwitterScore there was insufficient data in their system to rate three of the seven Twitter names in the study.


Even though all these systems crunch the same basic Twitter stats, each uses a different algorithm and each weighs different numbers in different ways. As such, each offers different insights into your program and is good for different things.  I’ll let you pick which one is right for you. Here’s how we conducted the study: 

Methodology:
 
Step 1. Establish a competitive benchmark set  
For the purposes of this analysis I picked 7 Twitterers that I considered my peers: 

  • Avinash Kaushik (@avinashkaushik)
  • Eric T. Peterson (@erictpeterson)
  • Johna Burke (@gojohnab)
  • Richard Bagnall (@richardbagnall) 
  • Shonali Burke (@shonali)
  • Amber Naslund (@ambercadabra)


Step 2: Decide what is important 
Some of these systems (TweetLevel, TwitterGrader, TwitterHolic and MyRealTwitterScore) give you one number to rate a person by, which makes life much easier when you’re in a hurry. Others offer various ranking metrics, including "Influence," "Popularity," "Engagement," "Trust," "Velocity" and "Clout." These metrics have a general but unspecific relationship to the standard meanings of their names. They all combine the various stats that are available in Twitter, including number of tweets, number followed, number of followers, number of retweets, and number of references. Sometimes the metric is expressed as a percentage of all people on Twitter, in other cases it’s a single number. 


For the purposes of ranking the top people, I went with the overall numbers plus the Influence scores that combine the references and likelihood of retweets. Frankly, because of the various ways now available to game the system, I consider the number of followers irrelevant.

The Results: 
 Twitterraters3-1-10

Conclusions:

Some numbers didn't seem to make sense, so I created a peer average to better understand the landscape. The next thing I realized was that while it was nice to know that I scored high on Engagement, and low on Influence, there really wasn’t much I was going to do about it. (And you can’t help but wonder how I managed to score better on most Twitalyzer grades than that guy who invented the system.)


Tweetlevel and Twitalyzer are definitely the fastest. While Twitter Grader entertains you with cute sayings while you wait for it to generate a report, this is only amusing for the first one or two tries. If you’re trying to judge the relative influence of a half a dozen bloggers it will drive you nuts. Also, if you have a long list of names to check, you may find it easier to use a single-number system, since the multiple criteria may make things too confusing. Or if you’re using Twitalyzer or Tweetlevel decide on one number -- Impact, Clout, or Influence -- and track just that. 


And the winner is...

What I really wanted to know was, based on a basic gut check, which analysis system made the most sense given my knowledge of my own behavior and my peers. To that end, Tweetlevel seemed to make the most sense, no doubt because it uses such a broad combination of numbers and algorithms.

  • If you’re measuring your own level of influence, then focus on the increase in your score, not the absolute number. Now that we’ve done this exercise once, I’m planning on going back and rechecking my scores once a month to see if I’m improving and hitting my goals. I’ve got the data in a basic Excel spread sheet, so I can easily generate charts to note the trend. Twitalyzer has a fabulous guide that tells you exactly how to improve your scores on their system. 
  • If you’re trying to figure out the relative influence of a blogger that is following you, look at their ranking relative to other bloggers. tan box

Comments

KDPaine

April 11, 2010

thanks! and I totally agree. it’s what you do with these metrics that count. I think as a very simple way to determine if someone if worth paying attention to or if its safe to ignore.

Reply
Add Comment Register



Leave a new comment

Your email address will not be published.