"Rocketboom is watched by 350,000 people a day."
April 21, 2006: Rolling Stone magazine (link)

"[Rocketboom's] audience is up about 100,000 in the last month, to about 400,000 a day."
August 10, 2006: Dow Jones MarketWatch (link)

Q: "Since Amanda left, have you seen any change in the audience?"
A: "It's grown a bit."
October 11, 2006: Future of PR Conference (link)

"...over 300,000 downloads per day."
October 24, 2006 blog post (link)
Sure, his numbers aren't consistent. And the Alexa chart above shows that Rocketboom's audience does not appear to be "up about 100,000 in the [July-Aug 10] month".

Don't trust Alexa? Andrew used Alexa during the time period in question to demonstrate that Rocketboom was more popular than CBS (link). You can't use Alexa when you're hot and then claim it's worthless when you're not.

But Alexa is not the problem. The real issue is that nobody knows how to determine the value of a web property or web audience given loose concepts like "page views" and "complete downloads." Although Andrew's handling of this complex issue leaves quite a bit to be desired, the situation he's found himself in is not entirely his fault.
business is booming
MSN, YouTube and other sites are also Rocketbooming. They're all desperate to come across as popular, so they all present aggressive background statistics. On some gallery sites just browsing a page full of thumbnails increases the view count for all the items on that page. Other times you click a blurry video, immediately leave, but it still counts as a view.

Web browsers download web sites that you don't visit, just in case you wind up there. TiVo downloads shows that you don't watch, just in case you get the urge. iTunes will download a podcast long after you've given up on it. Everybody's reaching for the sky and rounding up, up, up.

Andrew's caught in the web of these fuzzy numbers. To him, "complete download" not only means a "completely watched show" but also a "unique audience member." He also touts his large subscriber base and brags how a number of software products download his show by default after they're installed -- without even asking the user if he's interested or wants that to happen.

Those aren't viewers, they're nickels. Grab a thousand; swipe $50 from an advertiser.
everybody's rocketbooming
Let's face it, everybody does it. It's only natural to exaggerate a bit when you really care about something:

     "I've got the most beautiful baby in the whole world."
     "Wagner is the smartest Golden Retriever ever."
     "400,000 people watch my videoblog every day."

Even I got bit: Newsweek published that 100,000 people watch my show every day.

According to my video host Revver, viewers downloaded my videos well over 100,000 times.
A report from Revver dated 8/24 shows my 08-04-06 video with over 170,000 downloads.

So what's wrong with that? If you go to the Revver site and click that video today, Revver now says it's been viewed only 27,000 times. Revver fundamentally changed the way it calculates views, for all videos from all producers. Oops, sorry Newsweek.

"To me, the most important stat is not the hits, and not the bandwidth but the page views."
June 25, 2005: Andrew Baron blog post (link)

"The core of the matter is not hits or page views... it's how many completed videos were served."
October 24, 2006: Andrew Baron blog post (link)
revver 1.0
But if so many people in the metrics game feed each other inflated statistics, why would Revver actually decrease previously published numbers?

For all its flaws, Revver provides one very useful statistic: complete views. Not downloads, not hits, but views. Because they serve a custom ad at the end of each video playback, they know how many people watch each video all the way to the end. These stats are available (on good days) for all to see on Revver's website.

That said, I'm not sure I'd hire anyone from Revver's accounting department to do my taxes.

But why the stat change? Let's imagine you made $170 in Revver ads across 170,000 views. That equals $1 CPM which is well below what Google pays for text links. By changing the metric from "downloads" to "completed views" now you make $170 on 27,000 views, or $6.30 CPM.

By that same logic, any site that's trying to sell advertising based on "downloads" is unlikely to have much success.

I did a quick analysis: across my videos you can take the number of complete downloads, divide it by 6, and that's a rough estimate of how many people watched to the end. This is a very rough approximation: autoplay, the speed of your site, the speed of your visitor's connection, and whether there's an actual human clicking a video or just a sleeping robot pulling down a file affects the Rocketbooming Coefficient.
Should we even care about eyeballs? I don't. I care about my audience, but my show ends on March 17th, 2007 whether I have one eyeball or a million. Given the current state of web metrics, talking about eyeballs seems to create more risk than value anyway.

Rocketboom might very well have more eyeballs than Ask A Ninja, Amanda Across America, or any of the currently popular video shows. But it's important to question not only what those numbers are and where they're coming from, but also whether the eyeballs these shows claim to have are even open.

This is the interesting part of the conversation and I hope the web dialogue continues in the direction of answers and solutions. Producers cannot make intelligent creative or business decisions about their work without meaningful measurements. In the absence of sane metrics, we're already repeating the mistakes that turned television into what it is today.

posted October 26, 2006

leave a comment