Craig Morrison of Funcom Anarchy Online and Age of Conan fame has put out two interesting articles About 'the numbers game'
parts I and
II.
In the
first article his conclusion is that it's hard to get numbers out because it's hard to collect them and also hard to use the same numbers. But if someone else would only give him an "official" measuring stick, he'd use it.
Hmmm...
The
second article is about how subscription numbers are not really indicative of whether a game is successful or not because, you know, there's this return on investment thingy. Actually he gives three reasons, but dwells the longest on the effect we all see. How we all try the new shiny
MMO and drop it within one or two months. Real success should be measured by whether the subscriber numbers (which they're unable to figure out) are in the ballpark of what they were aiming for. How you aim for a number that's apparently so hard to determine beats me.
Actually, I don't understand why Craig claims they're so difficult. In the first article he blithely wishes for something along the Nielsen TV ratings for
MMO's.
Erm... What? Nielsen ratings are at best a wild guesstimate, a clutch for an industry that has no truly meaningful data available. It's like the fuzzy statistics wizardry used for advertisement in magazines and radio.
New Media, like the
Interwebz and
MMO's being wholly digital are a rather different breed of animal though. So the statement that meaningful numbers are hard to come by appear... odd... at best.
Why would Craig use the least accurate media rating system as an example of the kind of
MMO rating system he'd like to get behind? I mean, it's not like there aren't better alternatives available in New Media.
Other branches don't do it, do they?:Internet advertisement has grown into different breed of animal from paper magazine advertisement because, well because
page impressions, views, unique visitors,
click-throughs and what not can all be turned into discrete, analysable data. Unlike the fuzzy
wuzzy "No idea if people actually see the ads on page 7 anymore or if the stick with the pictures on page 4"
paper mag advertisement.
TV ratings are more diffuse than those even.
But that's really still to close to 'old media'.
MMO's are something different, something new and never seen before. Or are they? What's the business model again? You purchase a "physical" product initially. A viewer, which enables you to make use of a
service. For this service you pay a monthly fee.
Hmmmm..... I don't think I have far to go from TV ratings and stay within the medium to find several companies who are actually capable of whipping up those subscriber numbers.
But of course,
MMO's are a lot more "difficult", there's people who use time cards you know. And it's not like there's
pay-per view schemes in Television we could figure out how they do their measurements on right? Oh wait there are.
But let's look at a different branch.
Mobile phones. You buy a viewer, or
Jobsian status symbol, which enables you to make use of the services which (in its basic subscription form) costs you a
monthly fee *plus* pay-per-view. There's alternate forms like
pre-paid cards (time cards actually use pretty much the *same* technology) and partial flat fee models. There's deals and hundreds of models of 'viewer' too. Where a
Funcom MMO will typically have a
convoluted subscription system, it's nothing like the plethora of options you get at your local
Phonehouse store. Somehow mobile phone companies are capable of generating useful, meaningful and very, very
data-minable user
statistics.
In fact a former colleague at
Spellborn, who used to work for one of those companies in their marketing department, explained to me how their data mining gave them an
80% accuracy estimate on when you'd switch subscription plans or want a new model phone and even what way you'd be likely to jump. They had 95% accuracy over a time span of a month. He was bemoaning the fact that privacy laws prevented his company to do targeted advertisement based on that data. They're limited to using the data anonymously (in the Netherlands and EU anyway) so have to play with global campaigns, rather than individually targeted ones.
I'm taking my usually rumbling long way to get to a point here. And it's that the technology required to get meaningful user data about things like
MMO players has been in existence, for longer than
MMO's exist. So the argument it's hard to get them is poppycock.
Counting bottle-caps is hard you know:Let's take a look at some reasonably
standardisable numbers which I don't for a second believe Craig doesn't have access to:
Concurrent users: This means the
nr of concurrently logged on users at time X. You can poll this once an hour on every server, aggregate that and collect the data over time so you can make some nice graphs showing Server load over time for each server. Any
MMO company which doesn't at least have the ability to
collect this data from logs doesn't deserve to be in business. You don't just gather this data in order to produce nice fluffy "Active User" numbers for your Marketing department to throw around, but your
Live Operations team kinda needs to know this as well.
This is all the information you need to produce a number called:
Active Players This Day. From there it's real easy to go to Week, Month, Year. Just use the AVG formula in Excel.
Log on/off session data: Similarly to concurrent user data which you're storing for technical use, logging on and off fires a
loggable event. The same for Session time outs. Collecting average in-game time from this is a breeze. Heck,
EQ-II easily collected a ton more than this and academics are having lots of fun with it. Excel, or a decent data analysis tool can get you nice graphs with average in-game time and the argument that in-game time doesn't necessarily mean play time is moot. Players know this. They play
MMO's and have a good idea of how intensively they really use their online time.
So we have two numbers that can easily be fabricated out of data any
MMO company worth their salt is collecting anyway... In fact, if you're willing to have a developer spend a few days setting it up you can buy
turn-key third party solutions compelte with
auto-generated graphs on a nice little web-server or in PDF format mailed to you at configured intervals.
An apple is not a pear, but who cares if you're counting fruit:Craig also intimated that there's some difference between the definitions of player, user and subscriber and of course there's those pesky time-cards to obfuscate things. But wait. In a subscription based game, isn't a Time-card equal to a month's subscription? Or three months, depending on how many different cards you got? So would it be safe to put a user who bought and activated (you have this data!) a one month time-card be comparable to someone who used their credit card to pay non-recurring for a month's subscription?
Yes you can. You can also keep them segregated in your monthly report which your payment services provider is legally bound to provide you with at each month.
Gasp! What? You mean, subscriber numbers are actually readily and transparently provided to
MMO company CFO's?
Yes, yes they are.Any
MMO publisher not able to whip up a report breaking down paid for subscriptions into Different subscription and payment plans will not have a clear fix on their revenue streams. Since Investors and shareholders alike take a dim view of that kind of thing, you can bet your rosy butt they do have this information. I've seen these types of reports first hand so I know they exist, and are not that hard to break down into simple numbers.
Of course there's the problem that not everyone measures in the same way. This makes it less transparent.
Blizzard's Active Subscribers may mean something other than
Funcom's Active Users. Oddly enough this hasn't brought the industry to a halt. The key is
transparency. Both Blizzard and
Funcom have to provide transparency to these numbers to their Investors/Shareholders. Of
course they try to spin, massage and otherwise obfuscate their numbers towards these people as well. It's second nature to them. The computer Branch (hard- soft- and serviceware) has been notoriously lousy at using branch-wide measurement numbers. The reaosn for that is the same reaosn that Craig would rather use something as inaccurate as a
Nielsen's rating for
MMO Ratings than provide the public with accurate and transparent numbers. It's not in their best interest to provide these numbers.
I can't blame him for that really. He makes some good arguments on how success means something different for each project. Project goals generally aren't publicly announced and it's the meeting of those goals which define success, not an arbitrary number like subscribers. Why he doesn't come out and say so in the first article but but rather pretents his and other companies are incompitent and generating numbers is hard I don't know.
So while his second article actually makes sense, it's undermined by the poppycock arguments of the first article.