We need to talk about Tabula Rasa; when will we talk about Tabula Rasa?

In the online games industry, if we keep quiet about the causes, the hopes, the fears, the successes, and the failures of the best part of $100million burnt on a single project, then what hope is there for us to avoid making the same mistakes again?

Unlike Scott, I actually (superficially speaking) agree with this statement as to why Tabula Rasa, Age of Conan, Pirates of the Burning Sea, and Hellgate:London failed (TR, AoC, PotBS, and HL from now now…)

“No, these games failed because their developers let it happen.”

• Funcom *should have* learned enough lessons with Anarchy Online not to make the mistakes they did with AoC; not the precise same mistakes, but the same “class” of mistakes were made, suggesting that they tried to fix only the symptoms and failed to understand the causes
• Destination Games knew a long long time before TR went to beta that it wasn’t (going to be) ready even for beta, let alone launch. IMHO NCsoft collectively knew very well that TR wasn’t ready for launch, but went ahead and launched it anyway
• Bill Roper went on record to say that no-one understood their sales/revenue model, from the start. As I’ve mentioned before, that pretty much guarantees failure, and it’s not rocket-science to understand why!
• Pirates … I have no idea, actually. It’s the one that I have never played (although I really wanted to) nor even *seen* (which is unusual). I’m not going to talk about PotBS any more, since I really know nothing about it

NB: I don’t happen to agree with anything else in that post. I’ve got nothing against it, I just didn’t find anything interesting or new about the games themselves in the post, and IMHO the list of “why this happened” is too shallow and derivative to be worth saying in 2009 – the same has been said many times over the last ten years by many people, and ain’t particularly insightful in the first place. Sorry, dude.

How do you “let” an MMO fail, pre-launch?

Anyway, back to the interesting bit. This is why I find the statement particularly interesting: the choice of phrasing, that the developers “let it” fail.

The implication being that they didn’t do anything wrong, perhaps, but that they stood by and watched the train rolling slowly towards the brick wall and didn’t try (hard enough) to stop the collision.

TR was in development for 7 years (give or take a bit, and arguably just half that depending on whether you count the bit before there was an official “resetting” of the project (and re-shuffling of staff)).

When did they first ship a playable that people found fun?

Ah. Hmm. Um. Well. Now, *I* certainly wasn’t around during all this, so I can’t authoritatively answer that. However, I well remember the large number of people remarking that the beta – just before launch – was starting to be “actually a lot of fun to play”. 6+ years to get to the first fun version, eh? Hmm. Traditionally, you start with something fun, then you build a game around it, not the other way around.

Rating Tabula Rasa, in Alpha

While people are busily shooting me down in flames for such disloyalty :), I’m going to make a confession: I played TR in the alpha, and (to the great amusement – and in many cases total disbelief – of my colleagues) I actually enjoyed it. It certainly wasn’t a lot of fun to play, but there were nice elements that I could really see how they would develop (could be developed) into a great game, if we started development of the game at that point. I was accustomed at the time to making forward-looking evaluations of games, and reading between the lines and guess at how the final product would look.

i.e. it was a good pre-production prototype, and if I’d been asked “this studio wants us to fund them to turn this into a full game, should we do it?” I’d have said “well, modulo some small but fundamental changes that are needed, and the fact we need to explain to them a small number of basic mistakes they’re making with misunderstanding the MMO market, and they obviously will want a *lot* more money to build it up, flesh it out, and implement what they’ve barely sketched out in outline at this point … definitely YES. This will make a great game, probably”.

(IIRC … I was asked this question at the end of my second playsession and gave pretty much that answer. At which point he said something like “Yes. We’ve been telling them that for a while. They’re not going to do any of it”)

Which maybe sounds very negative? To me it wasn’t. At that time I was (one of many people) doing due-diligence reviews and milestone reviews for games being published by NCsoft. IMHO no game (and no studio) is perfect in pre-production. Every studio worth their salt pushes the boundaries, and – critically important – their own personal comfort zones. That guarantees that they’re not experts at producing the new game-type when they start. There are always warts; that’s partly why they make the prototype – to show to many other people, get second opinions, find out the flaws other people see, and then go “ah! yes! and actually … now you’ve said that, we’ve just thought of a much better way of doing this!”.

(incidentally, I found Valve’s post-mortem discussions of Team Fortress 2 fascinating, revealing that even for a studio with a perfect record of successes making new IPs they still screwed-up the pre-production work on TF2, and only late into the project did they really turn it around into the shining piece of awesomeness that they finally shipped. I can’t find a google link to the frank interviews Valve did on the subject, but here’s an interesting one on how much they still had left to do post launch – 53 updates (!) – even AFTER rescuing the project halfway through by fundamentally rethinking it)

Time-out: who are you, again?

I…:

1. …was the European CTO at the time
2. …was not on the TR dev team (wasn’t even in the same office)
3. …started playing TR as soon as I joined the company, and played Alpha, Beta, and some of Live
4. …unlike many outside the TR team (I suspect: “the vast majority”) I voluntarily played TR during my free time
5. …got locked-out when the game launched, and it took many months for my official corporate free account to get allowed back in, thanks to some stupid bugs in NCsoft North America’s account management systems
6. …was on a lot of the internal development mailing lists. Particularly interesting ones were the bugs list, the internal playsessions list (both for the dev teams and for other internal players), and the producers list (especially the scrum-masters list when the team eventually switched to Scrum).

I want to be clear about this: I had nothing to do with the development of TR. But, like many people who can say that, I was heavily exposed to it – both the project, and the game, and the politics. TR had a massive effect on the company at the time, everyone was touched by it. Anyone in development – anywhere – got affected a lot more than most; it affected budgets, management structures, technology investments, publishing strategy, investment strategy, etc. I’m not claiming to know what was going on in there for certain, but given my position I had the luxury of a lot of insights that other people wouldn’t have had.

So … although I know a lot about what happened, please view this post (it was going to be about the 4 games generally, but it appears to have warped into a TR-centric story) as an outsider’s view. And don’t read it as all true, I may accidentally report some rumours (and, you know, I don’t want to get sued if someone takes this as 100% literal truth) though I’ll try hard not to. TR team members may well find some big mistakes in what’s here – and I’d welcome their corrections and counter-arguments.

(I know my name appears in at least one set of the credits due to an NCsoft policy of “crediting all staff who were employed on the launch day of the product”, especially ironic since I think I’ve ended up being “officially” credited only on the game I did *not* work on :).)

Back to the topic:

The difficulty of …. Timing

It wasn’t ready for beta. I said so. Many others said so. How privately they said it, in many cases I don’t know. However, I am aware of plenty of people that said it pretty loudly internally at NCsoft (I saw the emails, or sat in the meetings).

NB: I said “wasn’t ready for beta“. We’re not even discussing “launch” yet.

But it was never going to be as easy as simply saying “hey, I’m not that busy for the next fortnight; howabout we launch TR next week? Or do you want to wait another year or two?”. On a project that had already burnt through tens of millions of dollars with almost nothing concrete to show for it (not necessarily a “fair” judgement; but if you were *literal* about it, which by that point many people were, then technically there was “nothing” to show), and had on the order of 100 people employed full time working on it every day, there was a lot of money at stake even just delaying launch by a single week.

(do the math; you’re already counting in the “hundreds of thousands of dollars” each time you prolong development by a single week there)

And then there were the political issues, for instance the fact that NCsoft North America had never developed a game internally in their long years of existence (all the internal games were developed by studios that NCsoft acquired during development). That means that the core business for the USA wasn’t making any revenue *at all* (publishing and development are usually seen as different divisions). Again, I’m not defending this perspective, or claiming it’s fair – but it was technically true, and was mentioned a lot.

Such things tend to scare stakeholders, especially shareholders. Especially directors of a public company who are trying to keep shareholders happy. Especially directors in a foreign country who may or may not even speak the same language as you. (I’m not trying to make veiled accusations against individuals here, nor against the different national divisions within NCsoft – I’m simply pointing out basic facts of life when it comes to large multinational companies, and observing that there was *inevitable* pressure along those lines, independently of whether or not anyone deliberately applied it).

And there were other issues. With that many people working on one project? Some of them for more than 5 years? Well. There’s plenty of dirty laundry on a project that size. But I don’t feel that anyone except the people directly involved get to decide whether its fair and reasonable to air it (because, frankly, no-one else is going to have much insight into what really happened).

So. It was hard, surely, to make any decision on launch dates.

There are no easy decisions in such situations, no “obviously, the best solution is X” (although to many different people such obvious answers seem to exist, the “easy” answers tend to screw-over several other teams). Making any decision was hard, but the decisions that were taken were considered inarguably “wrong” by many people, immediately that they were made.

They may not have known what the “best” solution was, but they certainly recognized (or felt they did) one of the “worst” ones.

And they were vocal about it. A survey was taken, internally, asking what people thought. The results were never published – so no-one (apart from the survey takers) knows exactly what the results were, but we were told that the *company* knew.

TR: The stage is set

To summarise so far (I know, I know – this is a long post, sorry)

1. TR went off the rails on some meandering journeys into research & development for many years burning through lots of cash (this is not necessarily something to castigate them for – many hit games did exactly the same; if you can afford the cash + the chance of failure (and NCsoft is a billion-dollar company, so let’s face it: they could), then it’s perfectly reasonable to decide on this course of action / allow it to continue)
2. Very late, they eventually hit upon a good formula, a good core game
3. Before they could actually make that game, a difficult decision was taken to push the team to the wall and force an early beta test
4. …and then the even more difficult decision taken to push them even harder to do an insanely early live launch. Certain alignments of astrological constellations in the Marketing department (also known as “tenth anniversary of the launch date of the last MMO that the core members of this team shipped”) may or may not have had something to do with this
5. The choice made was widely (if not universally) regarded as “very bad”
6. The company was made aware of the volume of people holding opinions along those lines

…and as they say on the Quiz shows: What Happened Next?

And this is where we come back to the point that interests me: did we, collectively as an organization, “allow it to happen”?

Personally, I believe the answer is an unqualified: “Yes”. Because many people worked *really hard* from that point on to make the game a success; many who had been working very hard already pushed themselves to work harder. And yet, in parallel, while working their asses off to make sure the train was big and beautiful, no-one stopped the train-wreck from happening.

There are excellent mitigating excuses for why individuals allowed this, many of them related to “not wanting to lose my job”. Many others relate to “life is too short to kill myself (with stress) over continuing to bash my head against this particular brick wall” (many people had complained long and hard about TR in the months and years running up to that point). Some people had just given up hope, and decided it was less painful simply to stop caring. Others were relishing the impending catastrophe, and I can think of some individuals who I personally believe were deliberately planning to get maximum political advantage for themselves out of the death of TR.

Re-reading this as I go, I remember there was another big excuse that I never gave credence to: “I’ll bury my head in real work and make *my* parts as good as I possibly can, and hope if everyone else does the same, it will All Come Together In The End”. This one wasn’t voiced so much, but is simply what people did, in some cases.

Hope is not a strategy. Whenever your attempt to avoid disaster revolves around the H-word instead of a concrete averting action, you are doomed.

But since I’m not trying to blame anyone here, it doesn’t matter whether or not we had good reasons for allowing it. What I’m really interested in is “how to make games better”, so the important point is that – collectively – we did allow it.

What would Jesus do?

What should we have done, not as an organization, but as individuals? How often does anyone talk about this?

% of player forums posts that blame developers for launching early
% of player forums posts that explain what an individual employee should do about it

It’s very easy to “work harder” when doom is impending, but as I’ve mentioned above this achieves nothing.

There are plenty of cases in the games industry where disasters are forseen, and the people involved charge into them with gusto, screaming “YOU’RE ALL WRONG! DON’T BE A HATER!”, and deserve what they get. But in those situations where people know it’s wrong, what can they do?

One colleague attempted various things and ultimately ended up trying to deal with the “root” of the problems by bringing about wholesale organizational change. Not in terms of who was employed, who was in charge, etc, but in terms of the basic attitudes and beliefs of the people turning up to work each day. He tried to remove the cultures of secrecy and fear (*) and replace them with cultures of actively seeking constructive criticism and actively supporting naysayers, so long as they adhered to rules of “decency” when it came to how criticism was provided.

(*) – (tens of millions of dollars spent and the game doesnt work but is going to beta/live? Fear was ever-present. Maybe (I heard rumours, and saw some … strange … stuff, but I don’t know for sure) for other reasons too. c.f. my comments about dirty laundry above)

In the end, I suspect if he’d started his campaign a year earlier, it *might* have worked. It certainly seemed to be having some surprisingly impressive results towards the end.

Among other things, the team itself tried adopting Scrum, with some tremendous results IMHO. Incidentally, I’m hoping one of the talks at GDC this year will be someone from the TR team (perhaps Andy Bruncke or April Burba?) on their experiences adopting Scrum with a team of 50-100 people at the end of development of a > $50million failed AAA title.

(If that talk does happen, I’ll be the one at the back of the room during the Q&A session at the end sticking my hand up to ask: In your opinion, if the team had adopted Scrum 12 months earlier, might it have saved TR? I would be very interested to hear the team’s thoughts on that)

Of course, both of those paths – and some of the other things people tried – were probably too little too late. TR didn’t have the luxury of time – the (contested) decisions being made were by definition time-critical.

So … what should we have done? Both as individuals, and as members of an organization that we each believed in?

What would you do?

Not long after, for unrelated reasons, my manager resigned. And shortly after that, so did several other people, myself included. TR didn’t (I believe) cause any of us to leave – none of them were on the TR team itself – but some of the problems it exposed within the company did come up often in people’s informal (down the pub) complaints about leaving.

When the organization disempowers you, and nothing you do seems able to make a diference, but – in your opinion – the impending event is an “extinction-level” disaster, is resignation the only valid response? Surely not?

Final Note

To my knowledge, NCsoft never admitted that TR was a failure, internally.

In June 2008, when I left the company, the CEO had gone, the lead designer had gone, and the rest of the directors were about to get axed in the pending re-shuffle (which hadn’t been announced even internally yet) – but still no admission in sight.

I used to gently point out that until we admitted the failure, we would fail to fully respond to it, and to fully adjust and improve – so that we were almost certainly doomed to repeat it. As far as I know, the painful admission has still never been made, even internally. The subject was danced around many times, but no-one would come out and say it publically (internally); it was always oblique references, and statements such as “Tabula Rasa is doing very well, although not as well as we hoped” – eliciting mirth, disbelief, looks of remembered pain, or simply blank looks of “wanting to forget it ever happened” etc among the various people in the company.

Personally, for each of the senior management at the company at the time, I shall never forget that you guys did not make that happen. To me, this one thing was symptomatic of, and encapsulates, the institutional failure to respond to the failings of the project.

Privately, reasons were cited to me varying from “it doesn’t matter any more, everyone knows its over” to “I don’t want to hurt anyone more than they’ve already been hurt” to “just basic tact” to “let’s not rock the boat” to “we should look on the bright side and get on with the other games we’re making as a company, and not get mired in history / water under the bridge”.

But as one of my friends said at the time: what’s it got to do with hurting people? we just want to use the experience to learn to make better games. And how the hell are we going to do that when you people won’t even admit we were wrong?