fame has put out two interesting articles About 'the numbers game' parts I and II.
In the first article his conclusion is that it's hard to get numbers out because it's hard to collect them and also hard to use the same numbers. But if someone else would only give him an "official" measuring stick, he'd use it.
Hmmm...
The second article is about how subscription numbers are not really indicative of whether a game is successful or not because, you know, there's this return on investment thingy. Actually he gives three reasons, but dwells the longest on the effect we all see. How we all try the new shiny MMO and drop it within one or two months. Real success should be measured by whether the subscriber numbers (which they're unable to figure out) are in the ballpark of what they were aiming for. How you aim for a number that's apparently so hard to determine beats me.
Actually, I don't understand why Craig claims they're so difficult. In the first article he blithely wishes for something along the Nielsen TV ratings for MMO's. Erm... What? Nielsen ratings are at best a wild guesstimate, a clutch for an industry that has no truly meaningful data available. It's like the fuzzy statistics wizardry used for advertisement in magazines and radio.
New Media, like the Interwebz and MMO's being wholly digital are a rather different breed of animal though. So the statement that meaningful numbers are hard to come by appear... odd... at best.
Why would Craig use the least accurate media rating system as an example of the kind of MMO rating system he'd like to get behind? I mean, it's not like there aren't better alternatives available in New Media.
Other branches don't do it, do they?:
Internet advertisement has grown into different breed of animal from paper magazine advertisement because, well because page impressions, views, unique visitors, click-throughs and what not can all be turned into discrete, analysable data. Unlike the fuzzy wuzzy "No idea if people actually see the ads on page 7 anymore or if the stick with the pictures on page 4" paper mag advertisement. TV ratings are more diffuse than those even.
But that's really still to close to 'old media'. MMO's are something different, something new and never seen before. Or are they? What's the business model again? You purchase a "physical" product initially. A viewer, which enables you to make use of a service. For this service you pay a monthly fee. Hmmmm..... I don't think I have far to go from TV ratings and stay within the medium to find several companies who are actually capable of whipping up those subscriber numbers.
But of course, MMO's are a lot more "difficult", there's people who use time cards you know. And it's not like there's pay-per view schemes in Television we could figure out how they do their measurements on right? Oh wait there are.
But let's look at a different branch. Mobile phones. You buy a viewer, or Jobsian status symbol, which enables you to make use of the services which (in its basic subscription form) costs you a monthly fee *plus* pay-per-view. There's alternate forms like pre-paid cards (time cards actually use pretty much the *same* technology) and partial flat fee models. There's deals and hundreds of models of 'viewer' too. Where a Funcom MMO will typically have a convoluted subscription system, it's nothing like the plethora of options you get at your local Phonehouse store. Somehow mobile phone companies are capable of generating useful, meaningful and very, very data-minable user statistics.
In fact a former colleague at Spellborn, who used to work for one of those companies in their marketing department, explained to me how their data mining gave them an 80% accuracy estimate on when you'd switch subscription plans or want a new model phone and even what way you'd be likely to jump. They had 95% accuracy over a time span of a month. He was bemoaning the fact that privacy laws prevented his company to do targeted advertisement based on that data. They're limited to using the data anonymously (in the Netherlands and EU anyway) so have to play with global campaigns, rather than individually targeted ones.
I'm taking my usually rumbling long way to get to a point here. And it's that the technology required to get meaningful user data about things like MMO players has been in existence, for longer than MMO's exist. So the argument it's hard to get them is poppycock.
Counting bottle-caps is hard you know:
Let's take a look at some reasonably standardisable numbers which I don't for a second believe Craig doesn't have access to:
Concurrent users: This means the nr of concurrently logged on users at time X. You can poll this once an hour on every server, aggregate that and collect the data over time so you can make some nice graphs showing Server load over time for each server. Any MMO company which doesn't at least have the ability to collect this data from logs doesn't deserve to be in business. You don't just gather this data in order to produce nice fluffy "Active User" numbers for your Marketing department to throw around, but your Live Operations team kinda needs to know this as well.
This is all the information you need to produce a number called: Active Players This Day. From there it's real easy to go to Week, Month, Year. Just use the AVG formula in Excel.
Log on/off session data: Similarly to concurrent user data which you're storing for technical use, logging on and off fires a loggable event. The same for Session time outs. Collecting average in-game time from this is a breeze. Heck, EQ-II easily collected a ton more than this and academics are having lots of fun with it. Excel, or a decent data analysis tool can get you nice graphs with average in-game time and the argument that in-game time doesn't necessarily mean play time is moot. Players know this. They play MMO's and have a good idea of how intensively they really use their online time.
So we have two numbers that can easily be fabricated out of data any MMO company worth their salt is collecting anyway... In fact, if you're willing to have a developer spend a few days setting it up you can buy turn-key third party solutions compelte with auto-generated graphs on a nice little web-server or in PDF format mailed to you at configured intervals.
An apple is not a pear, but who cares if you're counting fruit:
Craig also intimated that there's some difference between the definitions of player, user and subscriber and of course there's those pesky time-cards to obfuscate things. But wait. In a subscription based game, isn't a Time-card equal to a month's subscription? Or three months, depending on how many different cards you got? So would it be safe to put a user who bought and activated (you have this data!) a one month time-card be comparable to someone who used their credit card to pay non-recurring for a month's subscription? Yes you can. You can also keep them segregated in your monthly report which your payment services provider is legally bound to provide you with at each month.
Gasp! What? You mean, subscriber numbers are actually readily and transparently provided to MMO company CFO's?
Yes, yes they are.
Any MMO publisher not able to whip up a report breaking down paid for subscriptions into Different subscription and payment plans will not have a clear fix on their revenue streams. Since Investors and shareholders alike take a dim view of that kind of thing, you can bet your rosy butt they do have this information. I've seen these types of reports first hand so I know they exist, and are not that hard to break down into simple numbers.
Of course there's the problem that not everyone measures in the same way. This makes it less transparent. Blizzard's Active Subscribers may mean something other than Funcom's Active Users. Oddly enough this hasn't brought the industry to a halt. The key is transparency. Both Blizzard and Funcom have to provide transparency to these numbers to their Investors/Shareholders. Ofcourse they try to spin, massage and otherwise obfuscate their numbers towards these people as well. It's second nature to them. The computer Branch (hard- soft- and serviceware) has been notoriously lousy at using branch-wide measurement numbers. The reaosn for that is the same reaosn that Craig would rather use something as inaccurate as a Nielsen's rating for MMO Ratings than provide the public with accurate and transparent numbers. It's not in their best interest to provide these numbers.
I can't blame him for that really. He makes some good arguments on how success means something different for each project. Project goals generally aren't publicly announced and it's the meeting of those goals which define success, not an arbitrary number like subscribers. Why he doesn't come out and say so in the first article but but rather pretents his and other companies are incompitent and generating numbers is hard I don't know.
So while his second article actually makes sense, it's undermined by the poppycock arguments of the first article.
Sunday, September 26
Subscribe to:
Post Comments (Atom)
Nicely written Lani, glad you can share some insight in to how these things do work. It's clearly preposterous that they can't gather those figures reliably. Another shot in the foot for AoC *sigh*.
ReplyDeleteYour welcome Geek :-)
ReplyDeleteI should point out that things like concurrent user measurement usually isn't often done after Beta / first weeks of launch.
Once you have an idea of the relation between concurrent suers and server load as well as peak times e.t.c it's more efficeint to guestimate concurrent users from load. Parsing login logs once an hour to get a decent indication works too.
There's dozens of different ways to measure this. This does add to the apples & pears argument, but that's debunked by the transparancy point. Something that's always nice.
I really don't mind them not wanting to give out active subscriber numbers because people will just use those numbers and run every which way with them. But please have the balls to just say it, as he almost did by tiptoeing around it in Part II.
Of course they know. The data is there. Maybe they should invest in a proper data mining person to get some more data, but even the basic stuff is known to them for sure.
ReplyDeleteI can imagine though that the developer doesn't get insight into all the data. I know I don't get the juicy details from my company's dealings. But I think in this case he was asked by his marketing/sales department to ramble out some smoke and mirrors. I am not sure exactly why. Maybe the numbers are worse than they hoped and this is an interlude to that.
Or maybe this is to lure in ignorant investors for Secret World. They can point to this fluff when they ask for data when TSW is not bringing in the big bucks.
"I can imagine though that the developer doesn't get insight into all the data."
ReplyDeleteTrue, a code monkey would not. He'd be interested in the Concurrent user / Server load data analysis though. But certainly doesn't want/need a running tally.
Craig isn't a code monkey though. He's the man brought in to 'save' Age of Conan like he did with Anarchy Online (mainly by hiring Playerbase Solutions to tell them what to do).
If anyone in the Funcom:AoC team has these numbers, it's him.
As to the why, I suspect it's simply the not altogether incorrect view that if you give out real numbers to the "Sky-is-Falling crowd" is a recipe for disaster. That's what he was too "subtle" in Part II to say out loud.
Economic success is a matter of income and expenses. Most companies will go bankrupt before personell begins to leak usually wrong information regarding expenses, so we'll never see that side of the equation. Not from anyone. Just showing the player numbers side is giving a slanted view that's only useful if you're sure you got the Biggest Number, i.e. Blizzard/Activision.
In addition to the always unfavourable comparison to WOW's numbers Craig's fear is that when an MMO startts out with 900K subs at launch and drops to 450K after the first month is out the drop will be substantially worse would the numbers be publicly available due to people panicking.
Whether or not the piublicly stated goal was 300-500K subscribers or "We're the WOW-Killer" is a moot point.
Point in case, when Wow's numbers begin to slide. As I suspect they'll begin to do in the coming years, at some point Blizzard will not be touting their numbers anymore. ArenaNet still touts their 5 Milion boxes sold. They're very careful not to point out they hit the 4 Million mark at the first expansion box and by implication the second and third installments together made for a "mere" 1 Million boxes sold.
"ArenaNet still touts their 5 Milion boxes sold. They're very careful not to point out they hit the 4 Million mark at the first expansion box and by implication the second and third installments together made for a "mere" 1 Million boxes sold."
ReplyDeleteSept 2005 saw the first million landmark, very close to the release of the original chapter. Dec 2006 saw 3,000,000 (by which time Nightfall and Factions had been released), it currently stands as 6,000,000 units sold.
Ah, I stand corrected :-)
ReplyDeleteThanks for pointing that out.
However... I'm actually pretty sure the 3 Milion mark was reached between First and second expansion as I remember discussing the numbers on Gameamp and I wasn't posting at Gameamp after the second expansion.
I came back sometime after the third expansion and EotN had been released but tidn't discuss box nr's in the forums during that time.
I'm fuzzy on 3 or 4 Million though, so I'll grant you that :-)
Still, it's a moot point. It does illustrate the publisher's gut instinct to obfuscate numbers though. I'd love to see the exact nr's on Prophecies, Factions, Nightfall EotN (am I missing something?)
Because total units sold and active players are two different things. Knowing the break down of how many people bought each expansion gives a better view of a declining player base. Still pretty favourable as no doubt people returned to GW for a month or two at each expansion, the same as with WOW. Blizzard tends to broadcast subscription nrs shortly after an expansion rather than those of two months before.
Factions was released in April 2006, 2,000,000 units sold by june 2006. All of this is easily available from the guildwars.com news archives, I had to check to make sure my memory wasn't playing tricks, it often does.
ReplyDeleteThe current total will also include the one expansion PLUS the edition that contains all three chapters, that's five different boxes (counting special editions and collector editions as the same campaign).
The growth from the first chapter to the third chapter remained pretty consistent with a million units extra soon after the release of each.
It's obvious they would have had a steady decline from the announcement of GW2 (back in 2007) and the subsequent lack of considerable new content to the original. I would be surprised if they have current players as a 10th of box sales, and I know I'm being optimistic.
The beauty though is that their business model allows people to pop in and out so freely.
Quest lines have recently appeared that have started to give background for the lore in GW2 and when the announcement of what exactly your Hall of Monuments will bestow in GW2 hits I suspect they'll see a lot of returning players and new ones, I know the buzz for GW2 has already attracted new players to the original game, I'd be interested in their sales figures post the first hit of real news we had about GW2.
For some incomprehensible reasong guildwars.com is blocked from my work location and the site doesn't play nice with Glype 2.0 so I have to rely on my tricks playing memory and your assertions and of the two I'm more than half inclined to believe you :-)
ReplyDeleteThe main reason it's so easy to pop in to GW regardless of playerbase is that it's A: perfectly scalable due to heavy instancing and B: the campaigns are 90% very soloable anyway. Especially with both heroes and henchman at your disposal.
ReplyDeleteThe only cons are for a returning player, not a new one. Being A: the suspicion you'll suck ass at the higher 'level' gameplay and B: ageing engine. Not sure I could handle no jumping anymore :-)
I know right, what is it about the ability to jump? I really miss it when it's not there. FFXIV like GW, becomes completely fenced in by it, not being able to clamber over the smallest height is so immersion breaking. GW also has those horrible invisible walls everywhere, yet it doesn't bother me so much as it does in games like AoC when invisible walls appeared that previously didn't exist.
ReplyDeleteGW was my first online game so I wasn't spoiled by the ability to jump and roam around in open worlds. When I go back it's still the inability to jump and not the fenced in world that grates the most.
Aye, and the funny thing is you can go for days without jumping in games where you can :-)
ReplyDeleteIt's part of the uncanny valley for games is what.
When we talk of the uncanny valley we tend to think only of the aspect where avatars become so pixel-perfect that any discrepancy begins to stand out. But being stuck behind a 2" wall you can't jump over is not a prblem in 8 Bit console land, but incredibly immersion breaking in this 64 bits DX11 day and age.
With GW it was the simplest way to make sure you wouldn't end up in places where you'd see the back of the meshes for instance. With FF XIV it's probably due to the completely Top Down decision making in Japanese game design.
Just going back to those figures and a trip down memory lane, Factions caused one hell of a hullabaloo on the forums, you could not progress except in a very linear manner, explorers were f****d, they had to remove some barriers to calm the masses, I'm sure they must have lost players from the original game because of that and how claustrophobic the starter city was. I always felt the city was supposed to feel like that, your first forays out of it provided a deep sigh of relief at open space, it added to my immersion in the world.
ReplyDeleteI didn't mind the claustophobic feel of the city so much. I felt the same way you did. I did get fed up rather quickly with the inability to press forward for more than 3-4 seconds without hitting another mob. The fact you hardly ever had a breather to cam around and enjoy the view is what spoiled that city for me.
ReplyDeleteThe biggest let down for me in Fashions was the 45 second end battle.