June 25th, 2009 by adam

Really? O, RLY?

Well, no, probably not - but this is the kind of opening statement I often make at industry-conference parties. In this rare case, at LOGIN this year, I was showing something on my laptop at the time and happened to *type* my opening salvo, rather than just say it.
(more…)

May 18th, 2009 by adam

Last week at the LOGIN conference I sat on a panel with three far more smart/successful/famous people than myself entitled “Online Games 2014: Twelve Spoilers for the Future” (I think I was there as “the argumentative one” ;)). The real value of the panel was the four of us arguing^H^H^H^H^H^H^Hdiscussing each other’s predictions, and the audience suggestions afterwards, but the predictions themselves were pretty interesting alone, just to compare and contrast.

I couldn’t liveblog this session (obviously) and it looks like no-one else did, so - until the slides go up on the conference website - here’s what I can remember of the predictions (I may get some of these wrong, apologies!):

  1. There will be tonnes of cheap rackspace; anything that uses Cloud Computing will be very successful thanks to low cost base
  2. iPhone will become the dominant gaming platform
  3. We’re heading into a big recession that may do well for the MMO industry
  4. Browser-based MMOs will disappear in favour of iPhone/SmartPhone-based MMOs
  5. South Korean MMO Publishers will vanish as a major player in the MMO industry, eclipsed and swallowed by Chinese and SEA MMO Operators (”non-publishing” backgrounds)
  6. Europe will get its first successful Europe-wide MMO publisher, and that company will quickly rise to dominance over the more fractured USA MMO publisher market
  7. Advertiser-sponsored Virtual Worlds will be huge in number and variety
  8. A small percentage of advertiser-sponsored VWs will succeed - but will dominate the mainstream MMO market, since for them “profit is optional”
  9. Traditional game developers will be blindsided by the advertiser-sponsored MMOs
  10. Most PC MMOs (IIRC “90% or more”) will become F2P
  11. Console development-studios will become dominant in the MMO market since they are best at “polish” and very high quality user-experiences
  12. …and one more I’ve forgotten (!). Actually, some of the above I suspect I’ve misinterpreted - have to wait for the slides to be posted to check…

I’ve never before engaged in these kinds of generic future predictions, because I have so little confidence in either my own ability to describe them, or in my ability to understand other people’s ones in a useful fashion. I joined this session because the opportunity to argue them against other people was a lot more interesting. As stated above, I think our conversations on the panel were a lot more valuable than the actual predictions themselves.

Of course, when it comes to more narrow, specific predictions, well … if I really knew the answers there, I wouldn’t be telling you, I’d be making billions out of knowing :). And anyway, at that point you’re effectively asking me what the precise strategy is of my current employer (whoever that may be), which I’m generally not going to be able to reveal :).

FYI the speakers on the panel:

  1. David Edery, previously Worldwide Games Portfolio Manager for XBLA
  2. Charlie Stross, author of Halting State, Accelerando, etc
  3. Tarrney Williams, previously General Manager of Relic Entertainment
  4. + me, of course
May 10th, 2009 by adam

Reviewing video games is hard. In some ways, it’s an impossible mission: a reviewer has too many conflicting interests:

  1. please the publishers or else be denied access to the materials they seek to review
  2. please their editor or else don’t get paid; but the editor’s primary source of capital is often advertising … from the publishers
  3. answer the consumer’s main question in a way that earns their trust: should they purchase this game or not?
  4. stand out from the crowd of a million game players who decide to write about their hobby

Who’s your Daddy?

This has been a problem for as long as I can remember (20+ years of game playing and reading game reviews); the consumer *believes* that the reviewer is answerable to them - but it has been a very long time (10 years now?) since consumers were the paymaster of reviewers; nowadays, it’s advertisers (which usually means: game-publishers).

Of course, consumers still wield huge power. The virtuous value circle - the only circle that matters - is driven by consumers:

  • A reviewer has a “readership” of consumers who are influenced in their purchasing decisions by those reviews
  • Publishers therefore court the reviewer to try and curry favour with the consumers and increase sales of the publishers’ products (to those readers, and anyone they themselves influence - friends, family, colleagues, etc)
  • Reviewers earn more money, and get deeper access to development teams (courtesy of the publishers), so produce more reviews

But that power is - clearly - both indirect and hard to quantify. A consumer - even many of them - threatening to “stop reading a reviewer’s reviews” is not particularly effective.

Publications like Edge helped along the indirection of consumer-power when they decided to go out of their way to obscure the identities of their individual reviewers, turning reviews into as much of a crap-shoot as buying games was in the first place. Since the web rose to prominence, it’s been eroded at the other end - there’s now so many reviewers around that, well … who has the time to remember who any individual reviewer is?

Qui custodet custodes?

But if journalists/reviewers are supposedly there as a watchdog on the publishers’ marketing depts, supposedly helping the consumer determine which are the (non-refundable) purchases they ought to be making, then who’s checking that the journalists themselves are honest?

No-one, really. And that’s where the rot begins. The storms of outraged public opinion are nothing new: examples of journalists writing reviews of games (reviews both scathing and rejoicing) they hadn’t even played go way back into the 1980’s.

A case study in lies, damn lies, and video game journalism

In case you hadn’t heard, this week a “staff writer” from Eurogamer (a games review / news site) ripped to pieces one of the most recently-released MMOs - Darkfall. At which point Aventurine, the developer of Darkfall, responded with increasing anger and dismay.

But the really interesting thing here is that Aventurine didn’t merely rant “you bastards! Our game is Teh Awesum!!!111! STFU, Beotch!” (well, they did that as well) … no, they dropped a little A-bomb in the middle of their reply:

“We checked the logs for the 2 accounts we gave Eurogamer and we found that one of them had around 3 minutes playtime, and the other had less than 2 hours spread out in 13 sessions. Most of these 2 hours were spent in the character creator”

Pwned. MMO developers *actually know whether your journalist played the game before reviewing it*. What’s more … they have proof…

The EG reviewer (whose “references and background are immaculate”, according to the editor - but from reading his only two EG reviews, I’m afraid it does rather sound like he knows little about MMOs), responded (via his editor) with the claim:

“the logs miss out two crucial days and understate others, … and he insists he played the game for at least nine hours”

It would seem that someone is lying (and it could be either party). Worse, someone is being particularly stupid. Because the journalist is claiming “your computers lie”, and the developer is claiming “your journalist is a lier”; either way, it’s not a subtle, small, mistake - whoever is wrong, if they get discovered, they’re going to create themself a good amount of long-term trouble (bad reputation).

Lots of MMO developers write shitty server code, and honestly don’t know what the hell is going-on inside their own game-world (but fondly imagine that they do - and proudly boast to the press (in the vaguest terms) that they do). But the rule of thumb is that devs who don’t know … don’t even know what it is they ought to be claiming that they know. The specificity of Aventurine’s claims suggests that they do have the stats, and those stats are mostly correct.

(I say “mostly” because there is a bit of vagueness about what - precisely - the reviewer was doing in-game. That reeks of holes in their metrics/logging. They clearly know when the player was logged-in, and what they did/said in chat, and how many characters were created - but apparently not what they were doing in the client, e.g. how long did they spend in character creation? Implicitly: unlogged; unknown)

Whereas it’s quite likely that a non-knowledgeable journalist, accustomed to buggy games, would assume that they could safely claim “your server is buggy, those figures are wrong”.

Unfortunately for any such journalist, server logs are generally either correct, or absent entirely - there’s rarely any middle-ground. If he knew a bit more about MMO tech he might know this; very few journos (any of them?) know that much about the games they review, though.

So … based on nothing but casual observation and intimate knowledge of the tech issues (and several decades of reading game reviews…), I’m leaning in favour of Adventurine and against Ed Zelton. My guess (pure *guess*) is that he’s been caught out being either incompetent or perhaps a bit lazy as a reviewer, and he’s thought he could get away with blaming it on buggy code. From reading the review, I get the impression he wishes he were Ben “Yahtzee” Croshaw (from Zero Punctuation) - although he clearly isn’t funny enough - but he seems to like saying “it’s shit; you’re shit; you’re all shit; STFU” instead of reviewing the game, and seems to think that’s good enough. As an MMO player, my feeling was that the review was, well … useless - without even playing the game, there is so much more I would want to hear in a review, and so much of his wanky whining that I couldn’t care less about. As an MMO developer, it felt downright insulting, as if he’d made no effort at all to play the game as a game. Actually, it felt like he’d hardly played MMOs in his life, and didn’t really know what they were.

(NB, from the review: his apparent ignorance of some of the most important *and best-selling* RPG + MMORPG games of all time - the Ultima series - suggests that he really isn’t much good as a game reviewer. YMMV.)

Reviewing the reviewers

Up-front I’m going to point out that I don’t believe all MMO developers are currently capable of doing this - many people would be amazed to discover the true state of metrics collection in this industry - although *all* modern MMO developers ought to, and it’s not too hard to add-on later (add it to the list of “things MMO developers ought to do as standard practice, but many of them don’t do”). But it’s a general thing that I think we should move towards.

MMO developers (well, actually, the Operators, but that’s getting pedantic) are in an excellent position to help guard journalistic honesty, in a way that traditional game developers have never been able to. I would like to start seeing the following published by *every* MMO developer each time their game is reviewed:

  1. What level the account(s) started at
  2. What level the account(s) peaked at
  3. How many hours the reviewer spent at the lowest levels, levelling-up manually
  4. How many hours the reviewer spent at the highest levels
  5. What percentage of time was spent on each of the different primary character classes and factions
  6. Which areas of the game / aspects the reviewer actually engaged in (hours of combat, hours of crafting, hours of chat, etc)

…but, honestly, this isn’t so much about “journalistic honesty” (I used that phrase tongue-in-cheek above) as it is about starting a virtuous cycle of developers being more cognizant of what, actually, players “do” in their games - preferably *before* gold launch. In particular, if publishers (developers) started supplementing reviews with this info (as a matter of course), I think we’d see a sea-change in industry staff appreciating three key things about metrics:

  1. How little metrics they’re actually collecting compared to how much they think they’re collecting
  2. What metrics actually matter, and/or are useful?
  3. How players actually play the game; by extension: how fun is the game, really, and which parts suck horribly?

Does this work / matter?

At NCsoft, I got into the habit of asking prospective partners, hires/employees, and external studios which MMO’s they played (fair enough) … and how many characters they’d got to the level-cap with / what level their characters had reached. It started as an innocent question, but I quickly noticed how often it gave early warning of failures of honesty among individuals, and how much it presaged the problems they would have in the future.

The two worst problems were “complete ignorance of the MMO industry (either of pre-existing design practices, or tech practices)” and “personal self-deceit about what the person knows, and what they don’t know”. The latter tended to be a far worse problem: when someone is deceiving *themself*, it’s doubly hard to re-educate them, because first you have to get them to accept their own deception.

Of course, it turned out to lead to a lot of defensive responses and a spew of self-justification, which made us both uncomfortable. In those situations, it can easily lead to making assumptions that certain people’s opinions are “worth less” because, say, you know for a fact they’ve never really played an MMO - at least, not in the way that most of that MMO’s players would/will/do play it. I hate that tendency, since it’s part of a snobbishness that lies at the root of a lot of oyster-like, head-in-sand behaviour in our industry. On the other hand, it’s important and useful to know when someone’s ideas are random conjecture and when they’re based on fact (and very few people in a design meeting or publisher/developer meeting will honestly tell you their ideas are conjecture :)).

On the whole, though, it turned out to be a really useful line of questioning - even bearing in mind the additional (smaller) problems it created. There are obvious problems that come from the statistical supplementing of free-form prose game-reviews - but I’m confident that these will be outweighed by the advantages (and the problems that will be shrunk).

PS:

Despite the TLC of good friends, I’m still weak and sapped of all energy from my month of illness. I’m triaging like mad to deal with urgent issues, but there’s plenty of highly important stuff that’s been pending on me for a while that I still haven’t had the time + energy to deal with. So, if you’re still waiting … I’m sorry.

March 25th, 2009 by adam

Alex Evans, Media Molecule
Mark Healey, Media Molecule

Summary

The MM guys are funny as ever, although Alex’s “I made it myself on the way here” presentation tool would perhaps have been more usefully replaced with something like Presi (or whatever it’s called - the “interactive” presentation tool that is like Alex’s thing, but on steroids. Ask Jussi, he’s a fan of it).

The overall impression I got is: here’s another studio that has “by trial and error and cunning and talent” independently discovered something very similar to Scrum. They don’t do Scrum, and I’m sure a lot of people will scream at me for even saying it, but … I went through similar “find a process that worked for game development” (not carried so far, and on much smaller projects), and I recognize a lot of the lessons they learnt and things they incorporated in their processes and approaches. And from my experience, I think they’d find it relatively easy to switch over to Scrum, and that they’d get a lot of benefit from having a more polished version of their processes. Not to say that Scrum is universally better - there’d be losses too - but for people considering their own processes to use - or trying to “understand” Scrum - you’d do well to read this liveblog and try to internalize some of the lessons and attitudes. And then consider this and scrum as alternative to each other, but both near-relatives. And … if you are *not* MM, and don’t have all the details of precisely how they work, you’d probably find it much easier and more effective to adopt the well-documented Scrum instead.
(more…)

March 22nd, 2009 by adam

What’s the biggest single challenge to a Studio Director? Or to the VP of Development / Studios who oversees a handful of publisher-owned studios?

Recruitment

In the games industry there are no raw materials of variable quality, there is no variety of base services to build upon; everything that distinguishes one company (and set of products) from another comes solely from the people they hire.

In the games industry there are no raw materials to pay for, there are no service charges. There are only salaries and employee-support costs.

Recruitment is where the studio heads find their hardest problems, and see their biggest successes/failures as the studio grows in size. Eventually, all their own experience and ability at design, marketing, sales, programming, art, etc become subsumed by their ability to attract, recruit, retain, lead, and motivate their people.

Recession

…is the best thing for new game studios to happen in the past 5 years. It’s achieved four things:

  1. Removed lots and lots of people from their comfortable jobs, by force
  2. …simultaneously…
  3. …indiscriminately w.r.t. quality of personnel…
  4. …and made even the supposedly “secure” games companies (EA, Microsoft, Sony) suddenly look as fragile and short-term as the riskiest of startups

The VCs have been blogging about the benefits to startups wrought by this recession, and I’ve put it to a couple of them now that, for the game industry, this one - recruitment - is the biggest by far, and each time met with straight agreement. Our industry is very like Management Consultancy: it’s driven by the people. Nothing else matters.

Culture

I’ve worked with a lot of experienced managers who’ve been adamant that “no-one leaves their job because of (too little) salary”. Also with slightly fewer who were convinced that “no-one accepts a job based on salary” (more often, that was rephrased with a rider to be: “no-one good accepts a job based on salary alone“).

In that case, why do people accept / leave a job?

“Culture” is the catch-all term that describes not just the direct environment which people experience each day in the office, but also the emotional and psychological experiences that they go through while there.

It describes how their colleagues think and act - and how those actions effect the individual. But it also describes how the “teams” within the organization think and act, which can often be very different from the people within them. You often see teams of smart people “acting dumb”, or teams of nice people act like assholes when taken collectively. Group think is powerful, very powerful.

But it’s hard, very hard, to really see the culture of a company until you’ve worked there for a couple of years, and in a couple of different divisions, and perhaps a dozen different departments. Which is not an option for most of us. You can work somewhere for just a few months and pick up the culture if you know what you’re doing and really work at it - but even that requires skill and dedication, and can only be done AFTER accepting a job offer.

(this is one of the reasons I posted my Manifesto for a Game Studio online - you can get a strong taste of the culture of my next startup, and decide if you want to work with us, without having to sacrifice a year of working there first)

Reputation

Game industry staff often worry about reputation. The companies (as represented by the senior management) themselves often don’t.

The former care how their organization is perceived, and assume everyone else does too. They assume that a “better reputation” will lead to “more sales”.

The latter have access to the actual sales figures, and have convinced themselves that this is a nice idea but simply not borne out by fact (in some cases this is true, in some it isn’t - but it’s much easier to look at the figures on paper and believe it’s true than to see the flaws in that logic).

But the truth is that it IS important, very important. It’s the external reflection of the internal culture. As such, it’s what most people use to make a decision about whether they want to work there.

Obviously, it varies. The older and more experienced you are, the more you come to use a company’s reputation as a barometer of its culture - and the more heavily you weight this in your decision about accepting a job. The younger, more ignorant staff generally haven’t been burnt by terrible culture, or haven’t yet learned what to look for / avoid in their next employer.

Back to the issue of Recruitment: the biggest successes/failures are going to be from the more experienced people you hire (and, remember - hiring a “bad” person into a senior position is not just a loss, it can easily cause negative productivity, by screwing up lots of other staff who were doing their jobs better before that person arrived and started interfering / roadblocking them / etc).

So … you probably should care about your reputation, somewhat in proportion to the size of your company.

Blizzard

Pre-WoW, Blizzard had an exceptional reputation, for a handful of common reasons (amongst others):

  1. Never shipped a game that wasn’t really good fun
  2. Frequently invented + defined large sub-genres with their games (Warcraft was one of the first RTS’s, Starcraft created the “truly strategic” RTS genre, Diablo re-invented the hack-and-slash RPG, etc)
  3. Publicly talked about “finishing” their games, and then deliberately deciding to spend another whole year (or similar) working on them before shipping, to make sure they were really polished
  4. All of their games were best-sellers - i.e. they didn’t just make cool stuff, they made cool stuff that the market appreciated and paid for, too

Now, I’m not so sure. If a recruiter called me tomorrow with an “amazing, once-in-a-lifetime opportunity” to work at Blizzard, my first reaction would be hesitation: would I really want to work at the place that Blizzard has become?

While people have queued up to defend them, the history of their actions against Glider, and now this absurd crackdown on World of Warcraft add-on authors, have left me with a sour taste in the mouth.

In my opinion, using the law to beat over the head people who discover flaws in your basic business model / acumen is the last refuge of those who recognize their own incompetence but would rather not go to the effort of raising their own quality bar. Blizzard seems to be making a habit of it. That’s not encouraging. Ten million paying players for one MMO is great, but … the sales figures of their games were only ONE of those bullets I cited above about Blizzard’s reputation traditionally. Money buys a lot of forgiveness, but not infinitely so.

March 4th, 2009 by adam

http://www.gamasutra.com/php-bin/news_index.php?story=22528

“It is probably safe to say that, despite decades of ever more spectacular Hollywood visions of extra-terrestial domination, humanity in its worst nightmares never imagined it would have to contend with spawn-camping aliens.”

(also … If that article is accurate, sad but unsurprising to hear that (apparently) the underpowered server tech for TR yet again managed to make a misery of gameplay, even at the very end. If that article is accurate, then well done to the ops for managing to get some instancing sorted out, but note to self: never let this happen with future twitch-based / FPS MMOs)

March 4th, 2009 by adam

European Online Game Operators infamous for being overly secretive announce new association:

“The founding members have set three major goals for PEOGA:

* Networking. PEOGA is the meeting point for European companies from the online game publishing industry.
* Improvement of the Public opinion. PEOGA shall improve the image of online games by providing realistic information to the public.
* Self Regulation of the Online Game Industry. The online game industry has responsibilities towards their users. PEOGA will assist to set general rules and regulations for the industry.”

…launch website…

http://www.peoga.eu/

…and add an idiotic javascript hack that tries to disable the right-mouse-button + context menu in your web browser…

The page at http://www.peoga.eu/ says:

Copyright - Pan-European-Online Games Association PEOGA

So, let’s get this straight:

  1. None of the developers of the top 10 online games in Europe are in your org
  2. None of the publishers of the top 10 online games in Europe are in your org
  3. You have a President. But in case he gets lost on the way to meetings, fortunately you have someone else who is the “also President”. Nice.
  4. You can’t even make a website

Guys, I have to say: this isn’t looking great so far. Let’s hope this was just an unlucky start (BTW: you might want to get GOA on board - they know all about rocky starts, they’re probably the most experienced company in the *world* when it comes to bungling a launch).

PS: OMG I HAXX0RED UR SEKURITY - look, I “stole” the URL to your image - HHAHAHHAA111!!!!11!11
(please. It’s not 1996 any more. Get rid of the childish javascript hacks. It’s … embarassing. For all of us)

Jeans. In Europe, everyone wears jeans. From the same company, too.

Jeans. In Europe, everyone wears jeans. From the same supplier.

March 2nd, 2009 by adam

Here are the founding principles of my next startup. It’s incomplete and imperfect, but for where I want to go … it’s a start. Incidentally, if you share them, and want to work with me, you should get in touch (adam.m.s.martin at gmail.com). I don’t have any spare money to pay salaries right now, but I’m sure we can find a way to work together.

EDIT: if you’re interested in these ideas, have ideas of your own you want to discuss, or are just looking for other like-minded people … I’ve set up a Google Group for this at: http://groups.google.com/group/game-studio-manifesto

30 hour working week

  1. 4 days a week, 7.5 hours a day
  2. Salaries are 20% below the going rates; we aim to employ 20% more staff than usual for a given project size; cost is the same, output is the same (modulo an output-reduction/cost-increase due to increased overheads/inefficiencies for larger team size)
  3. everyone takes the same day off. I’m thinking Friday. Friday sound good? Let’s make it Friday. You know that if you’re in the office, so is everyone else (modulo normal holidays, off-sites, illness, etc).
  4. if we ever have to “crunch” and work unpaid overtime, I’m afraid we’ll all have to start coming in 5 days a week. I know, it’s tough.

Self ownership

  1. you own your work: no-one will chase you; informing people of your status, and of delays, is *your* job
  2. you own your project: everything works on Scrum (where the “team” owns the entire process - no producers, no project managers); NB: if you claim to “know Scrum” or be a Scrum Master, or “have used Scrum” and you don’t understand/believe this team ownership thing, here’s a big fat hint: YOU MISSED THE POINT
  3. you own the company: everyone has vested equity (not options) in the company

Mentors not managers

  1. increased organizational power is based on your ability to bring others up to your level. It’s based on your contributions to the other individuals. It’s not based on your organizational prowess
  2. if you cannot mentor, cannot explain complex/new things simply and clearly … you will not advance in the management chain (you should become a Domain Expert instead!)

Your value is what you are paid

  1. This is an implicit assumption in all salary negotiations and performance reviews.
  2. It will also be required to be *stated* explicitly in all negotations and reviews
  3. If your manager believes you’ve got better, they have to increase your pay
  4. If they do not increase your pay, they’re not allowed to give you a positive performance review
  5. There is no point in your career where “Becoming a Manager” is a requirement to get your salary any higher; the only benchmark is “can you further increase your usefulness to the company?”

You have a duty to become the best you can be

  1. Playing games, during company time, is an expected part of most jobs, since we are a “game development company” and you *need* to know what our competitors are doing
  2. Learning new skills, during company time, is an expected part of most jobs, since we’ll always be looking to make use of any “better” new technologies and tools that become available
  3. Not going on paid training courses, not increasing your understanding of our industry, allowing your personal skill progression to plateau … makes you sink behind what your peers in other companies are doing, people who would like your job. Get too far behind and we’ll give it to them. You owe it to yourself, as well as all the rest of us, to make sure YOU, as an individual, are constantly getting better, and learning new things
  4. The structure of the company is explicitly designed to support as many people as possible to become the best they can be. If in doubt, or in difficult situations where no alternative is “easy”, we will err on the side of helping people to improve themselves.

100% Organization-level transparency

  1. knowing what is happening in the organization is a right, not a priviledge
  2. knowing the reasoning behind organization decisions is a right, not a priviledge … from the reasons behind a marketing campaign being run the way it is, to the reasons for the product strategy, to the reasons that one particular tech is being used rather than another
  3. being informed of the progress of ongoing processes / issues is an expectation, not a priviledge … that means that people working on things are expected to proatively inform the rest of the company what they’re up to
  4. transparency overrides privacy (unless forced otherwise by explicit legal requirements)
  5. e.g. the salary someone earns is a personal and private matter - but the salary the company pays to each of its staff is not, and every member of the company has full free right to see that info. The company knows additional things - e.g. thanks to tax law, companies may know of other income their staff are receiving - but those are not part of the company, hence they are not part of the transparency

The buck stops with the directors

  1. any issue that necessarily has to be handled by an individual, that can’t be handled by the “team ownerships” etc, or e.g. is “sensitive” or a private personnel issue, WILL be handled by a named director instead
  2. no manager can accrete decision-making power, unless they are a company director
  3. e.g. if too much power is taken away from teams by directors, by accident or device, the directors will become overworked and will have obvious incentive to push decision power back to the teams

Google 20% time

  1. Problem: it’s either half a day, 12.5% time, or 1 day, 25% time. I’m not happy with either - one whole day makes things much easier mentally for the person to switch, but I’m afraid that converting it to 25% time and having people available only 3 days in every 7 would be too destructive?
  2. As per Google, this is not a right, it’s a priviledge
  3. all 15% time projects require sign-off by the person’s direct manager (with appeal to a director)
  4. all 15% time projects require monthly status presentations to show what’s been achieved, and the manager has to approve or deny continued work on the project

Team budgets - food; drink

  1. every project team has a weekly budget for food, and one for drink, and is expected to on average have one team lunch a week, and one team evening social (with free alcohol) per week

Healthy food; healthy environments

  1. the office will not have Cola vending machines, or ChocolateBar vending machines. *If* it has any vending machines, they’ll be majority subsidised - free, or practically free
  2. the office will have a surfeit of fresh fruit, renewed every day, starting at or before anyone gets into the office
  3. any meeting called before 11am will have some free small fresh food with substantial sugar content (for anyone who missed breakfast. Until they recharge their blood sugar, they’re probably cranky and irritable - and irrational - or simply silent and unthinking, like a robot, and make everyone else suffer because of it)
  4. choosing to hold meetings physically outside the office, e.g. in local cafes, and having the company pay for coffees and snacks, is a right, not a priviledge, for all employees

Remote working, and Online Working

  1. at any given time, we aim to have a substantial minority of staff working remotely / telecommuting, e.g. around 20%
  2. remote workers can expect slightly lower salaries than their full-time equivalents; the company gets more value out of people who are co-located - but it makes all of us work better to have a mix of co-local and remote colleagues, so we welcome the presence of remote workers
  3. all employees are required to be online and available *and reactive* on IM during all working hours
  4. all development systems and tools will support remote working by default (e.g. remote compilation, remote builds, remote deployment, remote access for all internal systems). This is one of the ways that having remote staff makes our overall operations better: better tested, more robust, more adaptable
  5. all employees will have their own password-protected SSH keys stored on a free USB key; all company systems will work on SSH key-based auth; all workstations will be configured to do single-sign-on using the individual’s SSH key - no passwords required

Guards against the unscrupulous

  1. all ownership is only part-vested, tied to time served AND ALSO personal performance targets. This will take substantial time to invent/negotiate on a per-person basis (I know, I’ve tried. Sometimes, I’ve given up on it, because it was so much effort. But … in the short and long term, its worth it)
  2. managers have more time to look for problematic individuals, as they’re freed of some of their normal duties in other companies
  3. teams, being self-owning, have the power and the incentive to reject any failing members. Over time, failing individuals will either change, find teams that do welcome them, or find themselves conspicuously under-employed, making them an easy target for management attention (this does not imply “firing”, it’s up to the management what action they take, but they clearly now have staff they’re paying for and getting nothing from)
  4. directors have a lot of burdens of responsibility under this system; they also have a lot more visibility into the company’s status than in a standard company, so more chance to fulfil their responsibilities
  5. most of the processes are designed to be self-healing/recovering when encounting unforseen problems: the teams and individuals that do the bulk of the actual *work* are self-owning, the managers whose roles are mostly shepherding are largely disempowered to break anything, any unusual problems fall into the laps of the Directors who already have total legal power to enact whatever is needed anyway, etc.

Next steps

Please help me debug this thing … add your own suggestions, or highlight the flaws in what I’ve written, or point to evidence both for and against the realities of what might work … etc, etc, etc.

February 26th, 2009 by adam

(4 posts in one day? Yeah! Too busy to be regular right now. GDC is on the way)

At the end of my commentary on the formation of NC West, I added almost as an afterthought a little comment about Europe and the MMO industry:

there’s a gaping hole in the MMO sphere. For someone bold enough to step into that hole, you could “own” Europe’s online gaming industry for the next decade.

Since I wrote that, surprisingly many people have commented on it both by email and in person, either along the lines of “look who else missed the opportunity” (Jagex, Ankama, Bigpoint, Gameforge), or along the lines of “yeah!” (count me in) … or both.

So, I wonder: what *would* it take, right now? If you have ideas, post them here. I’ll be at GDC in 3 weeks time, and I’d be more than happy to pitch this as a credible plan, if you come up with a comprhensive-enough plan. It’s a lot more expensive than any of my own humble plans, but in many ways I find it a lot easier to justify, financially. And I’m not even trying (it’s just a game for me right now, idly imagining what I’d do, if I were Jagex, or Ankama, or Gameforge, etc).

Wishlist time. Over to you, Dear Readers…

February 20th, 2009 by adam

I’m sitting in the Departures Lounge at Helsinki airport, which now has end to end free wifi (I can see 3 or 4 different wifi stations here, on two channels). It’s the “open a web browser window first and hit a button to say “yes, I agree to your terms and conditions”" variety - took me a couple of attempts to check email until I woke up (it’s not yet dawn here!) and guessed what I’d need to do.

But the interesting thing is quite how much benefit the airport gets.

Modern airports, as entities, get a huge amount of their revenue from the shops inside them. I’m from the UK, where Heathrow (and to a lesser extent Gatwick) have taken this to extremes for decades, but it’s spread over most of Europe and much of the USA by now too.

Advising passengers that they must arrive 3 hours before a flight leaves is one way to make them spend lots of money. Cancelling their flights is another (the branch of the WHSmith’s newsagent inside Heathrow airport made vastly more profit than any other branch in 2007 thanks to the plane cancellations that year). Making the airport experience a pleasant one, so that people *don’t mind* coming early is yet another. Facilitating people “working” at the airport too.

And free wifi supports not one but two of those. Making it hassle-free and ubiquitous is the difference between me wiliingly turning up more than an hour before my flight, and what I would normally do (aim to arrive 30-45 minutes before an international flight, and waste as little time as possible).

This is a model of “free” that I feel is still under-explored in the game space: Free as driver of larger secondary monetized activity.

February 12th, 2009 by adam

Yesterday’s announcements of layoffs at NCsoft (both in USA and Europe) caught many people by surprise, judging from the number of emails and conversations I’ve had where people have brought it up. I think it’s interesting to try to understand why this is happening, and given Scott’s point that this is really not the right way to do layoffs (bits and pieces at different times), then to look at how it could in this situation be done better (if it could; that may not be possible).

But in terms of the surpise? No. I find there is no surprise here. There’s IMHO two major things going on.

1. Clean up the mess created last summer in Europe

From yesterday’s announcement:

“The European office is transitioning to have a stronger focus in marketing and sales”

Last Summer, they made redundant the entire Development division of NCsoft Europe. Traditionally, in games, you have Development and Publishing. In online games you have a third major wing: Operations/Support.

Publishing and Ops have to be / should be local to the country(ies) where they are being sold - it makes things much cheaper, and it makes things more successful, as the staff are actually immersed in the culture and timezone of the people they’re selling to or serving.

I was a little surprised when the Dev division was cut that it wasn’t done cleanly. As Scott points out “Hey, management? You’re doing it wrong”. If you get rid of Dev, then certain other things HAVE to happen:

  1. Get rid of all Dev-sub-depts in their entirety - including things like QA, that “can” have a foothold on the side of both Ops and to a lesser extent Publishing. If you’re actively *cutting* dev, then that QA dept (as an example) is an abandoned outpost that will get left to rot, politically speaking, and you can guarantee it will be starved and eventually killed (or die of hunger)
  2. Get rid of a large chunk of Publishing and Ops that are *not* part of Dev, but are co-supporting of them. If you had a Dev division, you would have built up extra resource in those areas; that resource is now under-utilized. If you’ve had to do something as brutal as destroying your Dev division, you clearly are desperate enough that you need to be making those cuts as well

NCsoft Europe did *some* of the above - but clearly not all. I’m not expecting you to tell from the headcounts (that would take some effort with LinkedIn, or buying beers for a few people after work) - there’s an easier way: look at what “departments” still had staff. Once the redundancies had completed, there should have been *no-one left* in a bunch of departments that - in fact - were left with a handful of lost, abandoned, individuals.

Going back to that press release, what it really meant was:

“The European office has finally implemented the strategic plan from last Summer, so is transitioning toeffecting immediately have a strongerpure focus in marketing and sales”

Cutting Development in 2008 meant one thing: NCsoft Europe was now purely an off-shore Publishing division (coincidentally, back to its historic roots). In the games industry that means you are (in decreasing order of importance): Sales, Marketing, Localization. You’ll be lucky to keep anything in teams like QA because there’s no need for QA to work hand in hand with sales teams - they could be located anywhere (unlike QA + dev, which really need to be colocated). In some companies, with the number of people remaining relatively small, the CEO’s would have left the abandoned people to sit in their jobs not being very useful, while the management got on with bigger issues of trying to do whatever the strategic plan was that they were doing. But that couldn’t happen for NCsoft, for reason 2 (see below).

PS: I like to believe that the reason it took so long for this week’s cuts (in Europe) to happen is largely that the exec team in Brighton were trying hard to keep as many good people within the company as possible. I don’t know Geoff Heath (the CEO) well, but he’s always come across as genuine and proactive in his concern for his staff. The rest of the exec team also - whatever their faults and failings - have tended to put a lot of effort into “looking after” their people, whether or not it’s worked.

2. Convert the overall company to “how it should have been run back in 2001″

Last summer’s re-organization in NCsoft America was all about giving total control of the non-Asian subsidiaries to ArenaNet. Reading news articles etc, I do occasionally wonder how many people grokked what had happened. A quick summary…

All this waffle about becoming “a unified organization under NC West”, and the reporting by bloggers and journalists that this was “consolidating” the subsidiaries and offices (they were already consolidated, you know) … what a load of crap. Follow the money, guys: who has the power now, and what unites those people? And if the answer is “nothing”, then ask yourself: who stands to benefit from an exec team comprised of individuals that are likely to be in conflict?

Look at the Directors of NC West:

  1. Jeff Strain - Co-founder of Arena.Net, Director of Arena.net
  2. Chris Chung - Director of Arena.Net
  3. Pat Wyatt - Co-founder of Arena.Net, Director of Arena.net
  4. David Reid - only started working at NCsoft 2 months ago

Notice a pattern?

So NCWest was simply a handing over of the reins of power from the OSI Mafia (ex-Origin people such as: Robert Garriott, Richard Garriott, Peter Jarvis, Starr Long, etc) to the Arena.Net Directors (the only one who stayed behind was Mike O’Brien, who now runs Arena.Net).

Remember that Korea acquired not one but two studios early on in North America: the first was Destination Games, which developed the tragically failed Tabula Rasa, and the second was Arena.Net, which quickly (note) developed Guild Wars, then a bunch of expansions, and is now well on the way to shipping Guild Wars 2.

This is “acquire experienced and skilled American game-developer Directors, and get them to run our non-Asia subsidiaries … attempt 2″.

Which should also point out something pretty obvious (to me at least): Chris Chung has a heck of a lot riding on the success of the revamped NCWest. ArenaNet’s top team has to show that it can do what the Origin team failed to do. They’ve been waiting in the wings all this time, implicitly saying “we could do better than that”, and now they have to prove it.

He’s / they’re re-arranging the entire company to fit with “how we would have done it in the first place if we’d been given the chance” (or something like that).

Doing it Better

I have two criticisms of what’s going on, and neither seems to be shared by the general press. Which either suggests I’m very wrong, or I’m very right. Your choice. Guess which one I’m going for :).

1. Too slow

If you’re reforming a company, do it lightning fast. If you’ve been at that company, playing the politics, for 5 years, you ought to have a battle plan in mind well in advance of being “officially” given the reins. There are always reasons that you “cannot”, from the operational to the legal.

But I’m sure that’s what they said to Lou Gerstner at IBM, and he proved wrong, when he fired the entire middle management, worldwide. I bring up this piece of history regularly, because it’s an excellent reference point: if one of the biggest, most bureaucratic companies in the world can do “the unthinkable” then what excuse does everyone else have left for not going far enough themselves? The redundancy pay-outs cost IBM so much money they booked a sudden loss that year greater than the GDP of entire nations. But they did it.

The “what would we do if could break the rules…?” game was one I played at NCsoft quite a bit; I needed to second-guess what would happen, given the long lead times of any development, organizational, and tech issues, if/when failing teams, projects, and managers got cancelled (as they did). Lots of other people were playing it too. It’s much scarier to actually have to put your thoughts into practice and risk being wrong, so some “more serious” prep may be needed when push comes to shove, and some paralysis is understandable (but still not acceptable). But with all the time we had, the extra due diligence shouldn’t have been necessary. Courage of convictions and all that. I’ve become a fan of moving as fast as possible (although even at NCsoft I still had crises of confidence and over-analysed some of the risky situations, and was fortunate to work with better people who simply said “stop worrying, run with what you’ve got, it’s planned more than well enough already”).

2. NC Europe is screwed

NCsoft had the opportunity to create a giant of the MMO publishing world in Europe; Europe is screaming out for it and just needs a banner to rally behind - and a visionary exec team to say “we’re going to turn Europe into an online gaming powerhouse”.

Europe is a bigger market than the USA (by some 30% or more).

Europe has no multi-title successful MMO developer or publisher.

The UK alone has a lesser but comparable level of mainstream game developers and output of titles to the US (UK currently 4th in the world behind Canada).

So … the development industry is here, less so the publishing industry (although there’s a lot of mid-sized publishers spread through Europe), but there’s a gaping hole in the MMO sphere. For someone bold enough to step into that hole, you could “own” Europe’s online gaming industry for the next decade.

Missed opportunity? Hell yeah.

At the end of the day, I eventually realised that NCsoft won’t do it for one simple reason: Korea still probably doesn’t quite understand how they managed to go so badly wrong with the Garriott brothers as the founders and owners of NCsoft North America, and wouldn’t dare risk another, independent, self-managing, ambitious subsidiary *anywhere* in the world. The Asian subsidiaries are all kept on a very short leash and get practically no independence from the Mothership (in Seoul) at all - the whole western conceptualization of subsidiaries is already anathema to the Koreans.

If anyone out there is interested in taking over Europe like this, drop me a line. I’d love to join in.

January 16th, 2009 by adam

In the online games industry, if we keep quiet about the causes, the hopes, the fears, the successes, and the failures of the best part of $100million burnt on a single project, then what hope is there for us to avoid making the same mistakes again?
(more…)

January 13th, 2009 by adam

In the best tradition of ignoring 100 years of the Scientific Method and the concept of a Control Group, the FCC Commissioner has been talking about American students dropping out because of computer games, MMOs especially.
(more…)

December 29th, 2008 by adam

Here’s a question about increasing the profitability and decreasing the development cost of any MMO, although probably no-one except the web-people will recognise it as such (and even some of them won’t get it):

How do you improve the customer support for an existing MMO?
[where do you start, and what do you target?]

(more…)

December 19th, 2008 by adam

Bienvenue au blog, M. Bidaux!

“There were many reasons, but mainly, we decided against it because we knew that we would be very busy and the blog was always going to be left as a “when we have time” thing, and that always translate into in a “if we have time” thing.

The main issue was the commitment a good blog requires. There is nothing as sad as a blog you really like that gets updated irregularly. We will solve this right now, in this first post, just by managing the readers expectation: we won’t commit to have regular updates and features on this blog.

It’s OK, Thomas - we won’t stop loving you if the blog updates are irregular. I think it adds to the charm.

Fortunately for us, Thomas and Diane forged ahead anyway, and the ICO Partners blog is now open for business, featuring “in an approximate English and at irregular intervals”:

* news from the online game sector
* views on common and uncommon problems we encountered working on online games
* news on ICO Partners activities

December 16th, 2008 by adam

…could be that the “beat your staff with a stick, and if that doesn’t work … beat them harder” style of management was de rigeur for the Norwegian games industry:

AoC, in the words of an (alleged? ex?) employee:

the problem with Age of conan is that the game was in “crunch” for almost 3 years.

…who had *great fun* working for a company called Funcom:

then i was sick for a week, after having worked so intensely. i’ve never been that sick before, says “theodor”.

after having researched if the workers rights are after the work environment laws, and talked to his colleagues about this, he was asked in by the management. there he got a lucrative quitter-package if he stopped working the same day, which he agreed to.

…and Anarchy Online:

Keskin tells that he chose to leave Funcom because he was treated very badly by person in management. as he was being laid off, he claims that lies were spread about him from the management to his earlier co-workers.

i worked on anarchy online, and played that game for several years. it was a joy to work on the game, but if you ask questions, either about what they say to the public, or about something ethical– there’s a lot of strange things going on there– they turn around very quickly.
even if the whole thing is about if you want to do improve projects you’re working on, keskin says.

Of course, it could all be a big misunderstanding (mistranslation), since I don’t speak Norwegian, and I have no idea where any of my Norwegian friends are these days to ask for a second opinion on the translation (PS: Bjorn, if you ever read this blog, get in touch :)).

December 11th, 2008 by adam

(a.k.a. “How to invest in MMO development … profitably”)

The world is full of games companies that blow stupid amounts of money on making online games (typically “massively multiplayer online games” (MMO)). It’s time to put a stop to this madness; honestly, I thought everyone learnt their lesson about 5 years ago when we had the last wave of “everyone’s making an MMO … oh god, these things are TEN TIMES as expensive and ONE HUNDRED TIMES as difficult as we thought … Run away!”. Apparently not.

I think there’s two ways you can learn for yourself how to make a profit from developing online games:
(more…)

November 26th, 2008 by adam

Your possible answers include:

1. Because … Apple engineers have never heard of the concept of a “patch”, and require you to re-download the *entire IDE*, with all libraries, all documentation, all binary code - everything - when they release an update? So the current “SDK” for iPhone (hint for Apple: when most people say “SDK” they don’t mean “plus a copy of a bloody operating system”, they just mean “the few custom bits that are specific to that app”) is a whopping 1.6Gb?

[NB: actually in general I think that's a good thing - avoids a lot of mis-configuration / version mismatch problems - but as an MMO developer the idea of *not* patching gigabyte-sized packages horrifies me, and avoiding those problems actually isn't THAT hard (it's been solved many times by now!) these days. Writing (or buying) a good patcher is one of the first steps you do in MMO dev projects...]

2. Because … Apple didn’t think to split The Behemoth into multiple files, perhaps make them something reasonable, like a few hundred meg each?

3. Because … Apple decided to put this monster behind an authentication check on their website, presumably for legal reasons, and there is no other “official” mirror (all the ones you find on google are technically-illegal torrents or else, ultimately, redirect you back to the apple.com link), and their authenticated sessions TIMEOUT after 1 hour of “not fetching any new pages from the site” (completely ignoring whether you have any transfers in progress!), and refuse to send you data once your authenticated session runs out?

4. All the above?

NB: I wasn’t brave enough to try resuming the downoad without first re-authenticating and loading at least one web page from the apple developer site to prove I was logged in. I suspect (*suspect*) that the web browser would receive an HTTP 300 redirect to the login page, at which point most browsers are going to delete the partial download. Ha. Haha. HAHAHAHAHAHAHAHAHAHAHAAARRRRRRGHHH!.

Expect to see some comments/tutorials/advice on iPhone game development here at some point in the near future. If I can ever get the download to complete…

November 23rd, 2008 by adam

Now that I no longer work for a large MMO publisher, I no longer have access to all the juicy numerical goodness, research, and stats that they had on their games and everyone else’s. A chance email recently suggesting I take a look at Xfire’s gamestats led to some quick experiments that came out surprisingly well. It’s given me a new predictor for player numbers for any MMO that’s available in English which is sufficiently accurate that I’m going to use it going forwards. Take it or leave it :).

(this is rather the opposite end of interpretation to “Over 1 billion people play online games” - and make sure you read Raph Koster’s thoughts before trying to interpret these figures)

What are these used for

Even though there is NO audited, trustable source for these figures, we already know that the public “guesstimates” like MMOGchart.com are routinely used:

  • in audited (!) company annual reports as a reference point (especially in China and South Korea)
  • by publishers, when deciding which game projects to fund (used directly in projections of potential market-size - and hence how much cash funding to provide!)

These numbers are *seriously important* to the industry (like it or not!).

What’s out there - official figures

There are three types of official figures for player numbers for online games:

  1. Very precise figures included in the quarterly or annual audited company accounts, and legally-required to be accurate
  2. Detailed figures included in press-releases and/or conference presentations
  3. Vague figures cited in public interviews

Public companies whose primary business is online games are often expected (required, perhaps?) to publish precise figures (a side-effect of the rules on what they have to stick in their annual reports). Not all do (?), but noteworthy examples include:

  • NCsoft (one of the best-known publishers to do this, and the one with most “global” data, covering USA, Europe, and Asia)
  • CJ Internet (South-Korea + Asia only)
  • Giant Interactive (China only)
  • NetEase (China only)
  • Shanda (China only)

You … may well note a trend there. These figures are useful, and aid businesses operating in Asia, but by comparison life is somewhat harder for anyone wanting to sell into America or Europe. In all fairness, there are American and European companies that chose to (usually irregularly) make official statements via Press Releases, but this is an order of magnitude less detailed and usually less accurate than what would go in an annual report for a public company.

(NB: IMHO, the American and European economies and industries suffer for this lack of transparency - business models are more fragile, staff are less well-informed, decision-making is weaker, etc).

What’s out there - estimated figures

  1. Bruce Woodcock’s MMOGchart.com - guestimates extrapolated from superficially similar games with official figres
  2. mmogdata.voig.com - guestimates from a private methodology
  3. Vague figures cited in public interviews
  4. Independently measured figures

Bruce started out by taking as many of the official figures as he could find, modelling graph-based trends, and then re-applying those trends to missing data to try and extrapolate or interpolate the missing items. Where a game has never had ANY official figures, he took estimates based on a wide variety of inputs, everything from unsubstantiated rumours through to unofficial figures “leaked” by employees of the companies that were running the games.

Good points: (mostly) documented estimation process, started with accurate data, includes data for many games, includes detailed writeups explaining which figures are “accuate” and which are “guesses”, and ascribes an estimate of the amount of error in each individual estimate
Criticisms: assumes all games behave similarly in growth/shrinkage, updated very infrequently (every 4-12 months)

Phil’s VOIG was started apparently in frustration with the slowness of updates to Bruce’s figures (originally he updated frequently, but over time updates got less and less frequent). Phil doesn’t divulge his methodology, and you cannot download their figures (although you could read the website visually and type down each individual number. Umm. No, thanks).

Good points: *still* more frequently updated than Bruce even though Bruce has tried to speed up again
Criticisms: unknown methodology, unknown error-margins, poor data format, no download of figures available

Lots of games industry staff believe in sharing their figures more openly than their managers are willing to. On top of that, it’s often difficult or very difficult to answer a journalist’s question in an interview - or to explain a decision made during a post-mortem or conference talk - whent the audience have no idea what the underlying figures are. So, we often see individuals from games companies making public statements as to player figures for various of their games.

Good points: effectively these are “official” figures
Criticisms: not just vague as to numbers (usually they are only quoted to 2 sig.figs) but also vague as to *meaning* (registered players? active? paying?), very irregular publication times, often non-specific about what *date* they apply to (and people often quote figures that are a year or more out of date!)

A few organizations try to independently measure figures. It has long (ten years) been a complaint in the industry that no organization of high reputation in the traditional Media sphere (e.g. ABC for printed publication circulations) has started auditing online games. Recently, there have been huge efforts by a handful of companies to measure website traffic specifically - e.g. Quantcast, Compete, comScore - and for some online games those figures are often extremely good (games where people have to use a website each time they play the game, for instance).

Good points: stringent accounting standards (they hope to become ABC equivalents), strong expertise with web properties generally (so accustomed to the many tricks that black-hat website owners use to try and inflate their figures), very frequently updated (in some cases as frequently as per-day, taking them almost into real-time status)
Criticisms: mostly useless for non-web games

…but this final type - independently-measured figures - is the one we need more of. Because we need something that:

  • updates frequently, giving us “up to date” figures whenever we consult the source
  • uses a common reporting standard across ALL games (doesn’t compare “registered” from one game against “active” from another)
  • requires little effort to maintain (likely to stick around long term and become a reliable resource)
  • uses an open algorithm that is easily verfiable by anyone (the maintainers cannot deliberately write-up or write-down individual games without detection)

Xfire

Xfire is one of several companies trying to make “a social network for video game players” by creating a custom chat client that you keep open while playing the game. This allows them to track who is playing what games, when, for how long. For some time now they’ve been publishing (openly, for free), stats on how many hours each game is being played for per day in total. That figure gives some idea of the total “attention” that particular games are receiving, both individually and comparitively, but it’s useless for anything else.

I’d looked at the Xfire stats before, but only used them for very high-level comparitive judgements, since in most cases I work with games that have wildly varying “average number of hours of play per player per month”, and so the Xfire stats could not be used to judge games.

I had an email from one of the Xfire guys, suggesting I look at the stats again, and I noticed that they currently have a “number of Xfire users playing each game” stat too. Interesting…

A stupidly simple Methodology

Xfire has far too few users for those users-playing-today figures to be even close to the actual Concurrent Users figures, let alone number of players.

But I have a lot of high quality data on a wide variety of games (through official and unofficial channels), and I have most of the “official” figures, so I wondered what would happen if I tried using some well-known and accurate figures to look for a correlation with the daily users figures on Xfire. Pretty obvious. NCsoft sells directly into US and Europe and has established subs games in both western-developed MMORPG (City of Heroes/Villains (CoH/CoV) - known as “CoX”) and eastern-developed MMORPG imported into USA/Europe (Lineage 2 - known as L2).

I chose these two games because:

  • They’re from the same publisher, so counting algorithm OUGHT to be about as similar as we’ll ever get for different games
  • They’re both subscription based, so we get a relatively non-ambiguous figure
  • (most important of all) NCsoft releases precise figures for both these games *every single quarter*

The ratio of “Xfire activity” : “actual subs” is very different for those two games - but I wondered how well they predict the ratios for other games I had the figures for? I tried classifying each game simple as “eastern import” or “western”.

In each case, I looked for the following success / fail / anomaly criteria:

  • (any game), L2 and CoX are approximately equal multiples of known figures = fail
  • (any game, true figure unknown), L2 and CoX are both much bigger or much smaller than the estimated figure = anomaly
  • Eastern game, L2 is a smaller multiple of the known figure than CoX = success
  • Western game, CoX is a smaller multiple of the known figure than L2 = success

The “anomaly” result allowed me to run this against all the games where we only have “generally-accepted estimates”, and then decide in each case whether it was a breakdown in the methodology, or if it pointed to the “generally-accepted estimate” being wrong.

I had 4 types of number to compare against, FYI:

  • Official figures
  • Personal estimate (sometimes based on insider-knowledge, sometimes based on industry “common knowledge”, sometimes on odd bits of public data that indirectly confirms or predicts for a particular game)
  • Public estimates
  • Private official figures

Because Bruce gives you a downloadable spreadsheet of his data - and because you can read his own commentary on how (in)accurate each individual figure is - I used his data as the “public estimate” figures.

East vs West - Some example data









Name Official/trusted MMOGchart Best-Guess Xfire Xf-v-NC-CoX % NC-CoX Xf-v-NC-L2 % NC-L2
2Moons     n/a 448 74,567 n/a 314,633 n/a
9Dragons     n/a 211 35,120

n/a

148,187 n/a
Age of Conan 415000   415000 1032 171,771 41.39% 724,780

174.65%

Anarchy Online   12000 12000 164 27,297 227.47%

115,178

959.82%
Archlord     n/a 1330 221,372 n/a

934,067

n/a
Audition     n/a 473 78,728 n/a

332,191

n/a
City of Heroes / Villains 125000 136250 125000 751

125,000

100.00% 527,432 421.95%
Dance Online     n/a 107

17,810

n/a 75,147 n/a
Dark Age of Camelot   45000 45000

144

23,968 53.26% 101,132 224.74%
Dofus 10000000

452000

10000000 1433 238,515 2.39% 1,006,405 10.06%

Dungeon Runners

    n/a 95 15,812 n/a 66,719 n/a

Dungeons & Dragons Online

  45000 45000 159 26,465 58.81% 111,667 248.15%
EVE Online 250000 236510 250000 3429 570,739 228.30%

2,408,208

963.28%
EverQuest   175000 175000 109 18,142

10.37%

76,551 43.74%
EverQuest II   200000 200000 440

73,236

36.62% 309,015 154.51%
Exteel     n/a 202

33,622

n/a 141,866 n/a
Final Fantasy XI   500000 500000

509

84,720 16.94% 357,474 71.49%
Granado Espada     n/a

222

36,951 n/a 155,912 n/a
Hellgate: London     n/a

542

90,213 n/a 380,650 n/a
Hero Online     n/a

269

44,774 n/a 188,920 n/a
Horizons   5000

5000

58 9,654 193.08% 40,734 814.68%
Kal Online    

n/a

207 34,454 n/a 145,377 n/a
Kart Rider    

n/a

10 1,664 n/a 7,023 n/a
Legends of Mir    

n/a

0 0 n/a 0 n/a
Legends of Mir 2    

n/a

0 0 n/a 0 n/a
Legends of Mir 3    

n/a

0 0 n/a 0 n/a
Lineage

1100000

1100000 2 333 0.03% 1,405 0.13%

Lineage II

1005000 1006556 1005000 1431 238,182 23.70% 1,005,000 100.00%
MapleStory 15000000   15000000 4042 672,770 4.49% 2,838,721

18.92%

Mu Online     n/a 56 9,321 n/a 39,329

n/a

Neopets     n/a   0 n/a 0 n/a
Perfect World     n/a 1472 245,007 n/a 1,033,795 n/a
Pirates of the Burning Sea   65000 65000 64 10,652 16.39% 44,948

69.15%

Pirates of the Caribbean Online   10000 10000 443 73,735 737.35%

311,122

3111.22%
Ragnarok Online     n/a 173 28,795 n/a

121,499

n/a
Regnum Online     n/a 236 39,281 n/a

165,744

n/a
RF Online     n/a 347 57,756 n/a

243,700

n/a
ROSE Online     n/a 83 13,815 n/a

58,291

n/a
RuneScape 6000000 1200000 6000000 2535

421,937

7.03% 1,780,346 29.67%
Seafight     n/a 151

25,133

n/a 106,048 n/a
Second Life   91531 91531

4398

732,024 799.76% 3,088,742 3374.53%
Secret Online 10000000  

10000000

  0 0.00% 0 0.00%
Silkroad Online     n/a

4980

828,895 n/a 3,497,484 n/a
Special Force     n/a  

0

n/a 0 n/a
Star Wars Galaxies   100000 100000

644

107,190 107.19% 452,285 452.29%
Tabula Rasa   75000

75000

184 30,626 40.83% 129,224 172.30%
The Lord of the Rings Online  

150000

150000 2282 379,827 253.22% 1,602,662 1068.44%

Toontown Online

  100000 100000 172 28,628 28.63% 120,797 120.80%
Twelve Sky     n/a 797 132,656 n/a 559,738 n/a
Ultima Online   75000 75000 192 31,957 42.61% 134,843 179.79%
Vanguard: Saga of Heroes   40000 40000 583 97,037 242.59% 409,444

1023.61%

Warhammer Online 800000   800000 5621 935,586 116.95%

3,947,662

493.46%
Wonderland Online     n/a 202 33,622 n/a

141,866

n/a
World of Warcraft 12000000 10000000 12000000 112784

18,772,304

156.44% 79,208,889 660.07%
World War II Online   12000 12000

50

8,322 69.35% 35,115 292.63%
Yohoho! Puzzle Pirates 200000

34000

200000   0 0.00% 0 0.00%


East vs West - Does this work?

It’s not as bad as I thought it would be - there’s at least *some* correlation here :). It’s good enough that if you assume an inherent error margin of +/- 20% you can feel confident you’re getting good numbers.

It also works quite well with the private figures I have which (because of their sources) I consider to be pretty good.

Ah, but … statistically, does it work?

Well, running some simple Pearson correlation tests over the public numbers, I get a small increase in correlation (about 0.6 instead of 0.53) for using this method instead of just using a single comparator. That’s actually pretty good although I’d hoped for better. It does get a little better if I add in some private figures and/or replace some of the public estimates with private info I have.

I’d like to get hold of more data, either more things tracked by Xfire or more “public, official” figures, to check the correlation better. At the moment, there are a *lot* of holes in the public “Best Guess” column :(.

East vs. West: Interesting correlations and anomalies

9Dragons, 2Moons, Dance Online - The L2 predictor would put these at 150k, 310k, and 75k respectively. Acclaim has announced that they have a total of 750k active players across their 9 games, of which these three have been running for Acclaim longest and are the most mainstream of their games. At a total of 535k between them via L2 predictor, that fits reasonably well with the published figure.

Age of Conan - methodology breaks here. CoX predicts a mere 40% of the figure that Funcom has “officially unofficially officially” released recently. Sorry.

Anarchy Online - my estimate is 27k players; Bruce’s estimate is probably only counting subscribers, whereas the game has been F2P for several years now.

Archlord - L2 predicts a whopping 1 million players. Unless this is a truly huge hit in Asia, this has got to be wrong: even though the game has gone F2P in the west, the number of players in the west are generally thought to be in the region of less than 25k.

Audition Online - L2 predicts 300k, I believe it’s more like 3 million. c.f. the Maple Story notes.

DAoC - I’m happy with the CoX estimate of 25k players

Dofus - RS prediction is 1/3 of the reported number by Ankama. However … Ankama’s number *appears* to be “registered accounts” (the number I’ve used for RS is an estimate of active monthlys - the RS registered accounts figure is twice as big).

Also, Dofus is very much a French-language product. Although they’ve internationalized, their French start is still clear in that 30% of the playerbase is French (according to Ankama).

Therefore, I suspect that this may ALSO be being negatively affected by Xfire’s bias against non-English-language users (non English speakers tend to avoid English products if they can get equivalent native products).

Proportion of French users for internationalized American MMOs normally runs to around 30% of the European players, who normally represent around 50%-100% of the American players, which would suggest that Dofus has between twice as many and four times as many French players as an English-language MMO. Given the relative lack of interest advertising it in America, I’ll go for the four times.

Therefore, Xfire would only be counting around 75% of the playerbase it ought to be counting, and we’d get (via the RS predictor) around 4.5m players. I’m happy with that (until I get a phone call from Ankama. Salut?)

Dungeons and Dragons Online - again, a big drop, not so big as EQ2, but then I’ve heard that D&DO has had some uptick thanks to cross-selling to Lord of the Rings Online players.

Who knows? I can certainly *believe* the 25k predicted by CoX, but I’d ask some Turbine people if I were you, see if you can ferret out some more precise info…

EVE Online - methodology breaks here. The 250k figure comes from CCP themselves (give or take up to 10k). Based on the unique game-design and marketing, maybe EVE is just special (yeah, I know - I’m just making excuses here :)).

Everquest 2 - it’s a big drop from Bruce’s last-reported figure, but I think the CoX predictor of around 80k players is probably closer than Bruce’s.

FF XI - A while back I’d have thought the L2 estimate of 350k players was about right - nowadays I’m not sure, it’s been a long time since I looked into FF XI numbers in detail?

Horizons - CoX predictor says twice Bruce’s last estimate. Believable, but unconvincing.

Lineage - Epic fail. I can only guess: Xfire isn’t tracking Lineage 1 players. Off the top of my head, it’s hardly played outside Asia - Lineage 2 gets all the marketing love etc in the west. c.f. notes on other Asia-only games.

Lord of the Rings Online - CoX predicts a HUGE increase vs. Bruce’s estimate here. Turbine have never officially released figures IIRC, so maybe Bruce’s estimate should be treated as a shot in the dark anyway. Given that there’s been no new servers added to LOTRO since they were making noises about hoping to reach circa 500k - but also no server-merges - I could accept the 380k estimate from CoX predictor. But it’s just a guess.

Maple Story - L2 predictor breaks, suggesting just 2.8m players, less than a fifth of the best estimate I could find. This leads to a suggestion for a more precise prediction routine, see below…

Perfect World - L2 predicts 1m. Looking at companies with similar revenues to Perfect World, “active players” come in at approximately 1m-3m. Thoughts?

Pirates of the Burning Sea - CoX is predicting a mere 10k players - one sixth of the Bruce estimate. I believe it, because 7 months ago they shut down 7 of their 11 servers, leaving only 4 servers with a maximum concurrent playerbase of around 8k between them. In practice, I would have estimated around 15k-20k players based simply on the number of servers they’re running, but I suspect they may have kept extra around to keep some variety in the server populations, and because running just 1 or 2 extra servers is not that expensive, but gives you good fast response for all players.

Pirates of the Caribbean Online - Again, CoX predicts a whopping 7 times as many players as Bruce’s estimates. But then this game is one of those that went F2P, so, again, I can believe the CoX predictor when it says 75k players.

Seafight - CoX predicts 25k players, but this is way short of my own estimates based on the publisher’s aggregate player numbers. c.f. the notes on Maple Story, and the new analysis below.

Silkroad Online - L2 predictor comes close to my personal rough estimate based on taking their quoted number of registered players and dividing by 4, but its still short by about 30%. Again, see notes on MapleStory.

SL - isn’t a game, I’ve included it simply because both sources were counting it. The fact that it bears no resemblance to ANY of the games is no surprise considering it really has very little in common with them.

Star Wars: Galaxies - CoX’s predictor is within 10% of Bruce’s number. Cool.

Tabula Rasa - CoX predicts 30k. Estimates by industry consultants like Jessica Mulligan (see the comments) put it at around 30k.

Toontown Online - methodology broken; I’m sure the CoX estimate is wrong, and that TTO’s audience (young children) doesn’t overlap with Xfire’s audience.

Vanguard - The CoX predictor is suggesting more than twice the number of players that Bruce estimates. I have no idea what the correct number is - I haven’t bothered tracking Vanguard since its inexcusably poor launch. I’d love some independent confirmation of one number or the other being closer?

Warhammer Online / WAR - CoX predicts 935k, EA recently stated 800k. Not bad…

World of Warcraft - CoX predictor gets it wrong by a factor of 1.5 … you could take that as an indicator of the amount of error in the predictor :).

WW2 Online - I’m happy with the CoX estimate of 8k players

Some notable MIA games

Habbo Hotel - definitely millions of active players, but not tracked by Xfire. c.f. notes on ToonTown Online w.r.t. Xfire’s poor demographic tracking.

Neopets - definitely millions of active players, but not tracked by Xfire. c.f. notes on ToonTown Online w.r.t. Xfire’s poor demographic tracking.

Puzzle Pirates - 200k active players from the last number they put out publically, IIRC, but not tracked by Xfire.

Secret Online - 10 million players (”active”, IIRC) in China, announced in US/EU 7 months ago, not tracked by Xfire.

Special Force - not tracked by Xfire.

East vs. West - Problems

So we see three major problems here:

  • Xfire doesn’t appear to track non-western players at all, tracks European-but-primarily-non-English players (Dofus, Seafight) noticeably poorly
  • Xfire doesn’t appear to track younger users at all (all the games for young children / parents come out “untracked”)
  • Basing all eastern game estimates off a subscription-only game (L2) works for a lot of things, but for the few really massive F2P (free to play) eastern games it fails

Can’t do anything about the first two problems, since those are flaws in Xfire itself, but I thought I’d have a quick look at the third problem and see if adding additional predictors (specifically for F2P games, both east and west) would help.

Analysis: Subscription vs F2P (Free to play)

The eastern F2P games fail dramatically when judged purely by the sucess of Lineage 2. Unfortunately, the mighty NCsoft “doesn’t do” F2P games, so we’re going to have to look at other sources of comparison.

Maple Story (eastern, localized), Runescape (Western) seem like good starting points, although in both cases there is only mediocre “official” data.

Subs vs. F2P: notable failures from East vs. West

Seafight - RS gets only 25% over my guesstimate based on the number of games Bigpoint publishes and the total number of active users they have. Serious guesswork - although Seafight is one of the slightly more popular of the BP games, so I would expect it to have more than that many users, closer to the RS predictor. But I suspect it could be MUCH higher, as much as 3-5 times higher, since we have no data on how much overlap there is between BP players of different games, and this assumes zero overlap.

Audition - Even using MS as a predictor, we get barely half of my last estimate for Audition’s playerbase. I’ve heard rumours it has been eroding a great deal in the past few years (it is an old game now - and with little or no design updates, it shows!), so I guess this is possible?

Subs vs. F2P - Does this work?

I’d have to say … no. And at that point, you start getting into using dozens of different predictors, split by genre + revenue model + country of origin + age of game, etc … and there aren’t enough MMOs in the world for that level of detail to be worth it (you’re into fantasy land by that point - and it’s too much effort :)).

Analysis: WoW

NB: this one I don’t take seriously, it’s just for fun; I think it’s meaningless until/unless I get hold of an Asian equivalent of Xfire, and come up with a WoW equivalent from Asia (probably Maple Story - huge locally, and large globally), and we can do the WoW-western-based-global vs. MapleStory-eastern-based-global comparison.

It would be interesting to see how well that worked as a predictor - does the “global success” dominate, or does the “subscription vs. F2P” dominate?

Follow-up ideas

1: Correlate “hours played”

Xfire’s preferred stat is “hours played per day” not “number of people playing per day”. This stat varies massively by Genre in fairly obvious ways. Doing a similar correlation to the above one for mapping “hours played + genre” to “number of people playing per day” would be relatively easy and possibly even more valuable.

2: Cross-correlate “hours played” with the above-inferred “number of players”

Especially useful would be to take the results of 1 above, and combine them with the work done in this article.

That would give you a basis for inferring “number of players” directly from Xfire’s primary free published-statistic.

3: Xfire to penetrate Asia

Well, we can wish…

4: Xfire to track younger children / older parents

My guess is that they too would love it if they could do this…

Exclusions

Kart Rider got closed down in USA, and although it’s still one of the most played games in the world, Xfire shows a mere “10 people playing” - so I guess those are the few who’ve braved non-localized versions?

Legend of Mir - Xfire is tracking it, but saying 0 for all games. IIRC they shut down the old LoM games as they open new ones - has a LoM 4 just come out?

PS….

Wordpress is *still* corrupting raw HTML source - if you see big blocks of whitespace in this blog, it’s something odd in WP not liking inline style declarations. Sorry.

November 18th, 2008 by adam

(from Nic Brisbourne’s blog)

NB: I can’t actually try it, of course - Microsoft is still subscribing to the classic anti-Web 2.0 ideal of making it “zero information from our walled-garden until you pass a detailed user-verification process; visitors will be shot; guests are not welcome here”.

The interesting piece for me is that they are inferring the social graph from Instant Messenger. It has long seemed sensible to me that building from existing social graphs (email, IM, phone records etc.) is a better way to go than building a new one from scratch as we have all been doing on Facebook, Myspace, LinkedIn etc., although there are many tricky issues around service design. Google and Microsoft think the same way according to this Techcrunch post of a year ago, although we have yet to see thought translated into action.

Yeah … we wanted to do the exact same thing with MMO publisher data when I was at NCsoft. Given how much you know about subscribers, you can infer some extremely valuable stuff (that is much harder for people like Google/Microsoft/etc to piece together). Turned out there were a lot of internal political problems in the way (something I’ll be talking about at GDC 2009: “How to sell social networking to your boss / publisher”) that really came down to a handful of extremely powerful people not getting it / not caring. It was an … interesting … journey learning what they didn’t get, and why, and how to dance around that.

(PS: all the above is assuming “without breaking privacy / data-protection laws”; if you’re reasonably well-moralled, you can provide a lot of value while being well inside the law; I have little sympathy for organizations that run roughshod all over the Data Protection stuff - IME you really don’t need to)