Category Archives: MMOG development

Everything related to MMOG development

#Gamification, #StackOverflow: How they create new sites from the community while blocking spam

If you’re not a StackOverflow user …

  • It’s rapidly become the go-to place for answers to precise technical questions
  • It has a bold, “points-based reputation controls everything”, moderation system (e.g. get upvoted enough and you become a moderator – there is no human intervention!)
  • It worked so well they expanded to infinitely many clone sites, for any topic you can think of, collectively termed “StackExchange”
  • …and the process for creating a new clone is itself points-based (not human moderated)

Historically this kind of setup has been a recipe for disaster – too easily gamed (taken advantage of), both by selfish users and purely malicious griefers. SO has had, and still has, many problems – some of the design choices that worked early on caused more problems than they solved as it scaled up in size. But overall it worked, and continues to work, very well.

The main site is easy to understand – by the time you’ve earned 10,000 reputation you are probably so enmeshed in the community it’s probably safe to give you moderator tools. (Note “probably” – it causes serious damage in a large minority of cases, but overall it works well, and it’s very cheap. This generally means no corporate sponsors/advertisers/subscriptions are needed)

But the process for creating new sites is less obvious, more convoluted, and – for the brand – potentially a lot more dangerous. I’m involved in two sites going through the process right now, and it’s interesting to compare them.

Site creation overview

  1. A special site – Area51 – allows you to create new sites, with this FAQ
  2. Your new site goes into phase 1: “Definition”
  3. If it passes, it goes to phase 2: “Commitment”
  4. Once it’s live, it’s monitored for a while, and if the site was a bad idea / disaster / failure, it gets shutdown
  5. If it passes the probationary period, the site is live

SO is worried about a bunch of things. From game-design and community-management perspective, I’d expect them to focus on e.g.

  • Will anyone use the site? (ask questions)
  • Are there any experts around to answer the questions?
  • Is the community large enough to be self-sustaining?


This checks the first worry – will anyone ask any questions?

To prove this programmatically, they force you to ask 40 “Good” questions.

To determine if a question is good, it has to receive 10 more upvotes than downvotes. This is arbitrary, but it means e.g. 10 different people felt it was a good question, and no-one thought it bad (or more for both sides, so long as there’s 10 more up than down votes).

Riiight … so you write 40 questions, get 9 friends to upvote all of them, and away you go … right? Easily gamed/abused.

Wrong. Area51 uses a modified version of SO’s voting. Each user is limited to asking 5 questions – which is fair and harsh. Fair because: if there’s really a community ready to go, it will have questions from many people (at least 8 people will have to pose good questions to get the 40 needed). Harsh because: most people will struggle to think up more than 1 or 2 good questions.

Still easily gameable, but now enough to dissuade idle / bored people.

Similarly each user can only upvote 5 questions. Choosing to up/downvote is very easy, so this isn’t “harsh” at all. It’s (almost) equally gameable: other users (anyone, anywhere) can register and counter-game by downvoting bad questions, forcing the collaborators to work harder.

Sadly, this scheme has a non-obvious element – the need to get 40 questions to 10 upvotes – that MOST users fail to understand, and SO has done nothing to fix. On, more upvotes is always better; on Area51 the 11th upvote (and all afterwards) are not only worthless, but actively delay the proposal because they squander upvotes that the user could have used on other questions.

So, for instance, the Computer Science Educators proposal was popular but spent many months failing to pass this phase because people arrived, upvoted the top questions, felt they’d helped … and left. Because of the design-flaw, not only were they “not helping” but the surge of high voted questions above the fold encouraged the next wave of newcomers to do the same. #facepalm.

By comparison, the IMHO less valuable Microbit proposal, for a politically driven educational tool that distracts from CS education, appears to have been supported by people with better understanding of the rules, and got through much more quickly. I don’t mind a microbit SE site, but … not if something so niche and political gets through at the cost of the CS Educators site (because it diverts attention away).

Solution: SO should change Area51’s visual design so that any question with 11 or more upvotes is displayed as “accepted” instead of a number, and the total number for each question is shown in smaller type somewhere else on the page.


Guarding against the second concern is phase 2. With 80 people signed up as “committed” to microbit, and only 65 “committed” to CS Educators, microbit has now overtaken the older proposal. Right? Wrong.

Microbit proposal CS Educators proposal
Screen Shot 2016-10-05 at 12.33.01 Screen Shot 2016-10-05 at 12.32.53

Clicking the link at bottom right of each info panel shows that SO has a different approach at this stage. They use not one but three measurements to pass – it’s got harder.

Microbit would be winning here, except … SO judges you on the weakest link in the chain. And two of the three criteria are biased against people who don’t use SO much:

Microbit proposal CS Educators proposal

Screen Shot 2016-10-05 at 12.06.03

Screen Shot 2016-10-05 at 12.05.59

Measure 1: Total number of committers

Raw score of “how many people have clicked a button to say they believe this is a good site worth adding to the web”. Very much in the vein of SO.

EXCEPT: I see no “anti-commitment”; true SO ideals would suggest that we have a way to say “no, I don’t think this deserves a site” – and have it cost you some of your positive influence on committing to sites you do like. This is the essence of what made SO successful, and it’s intriguing that they’ve dropped it here.

I suspect (guess) that the relative infrequency of people proposing/committing to new sites (a few a year) vs voting/asking/answering SO questions (hundreds a year) means that the danger of people putting in negative votes at no effective personal cost was considered too likely. Or the cost to individuals of gaining enough positive commitments to “earn” the right to anti-commit was too high.

Measure 2: Require committers with > 200 reputation elsewhere

This is the classic gating strategy SO doubled-down on when they became too successful / too big: a requirement for minimum amount of positive reputation before enabling basic features that spammers tried to (Ab)use.

Earning 200 rep on “any” site is much too easy, and I think they’ve made a mistake here. It ought to be something like “earn 200 rep on 2 different sites, or 500 rep on one site”. I say this because over time SO rep has continual inflation problems, just like real-world currencies in live economies. A 200 rep barrier on one site is no barrier to people gaming the system – there are now so many obscure SE sites that it’s easy to find a gameable one. But finding multiple gameable ones would be substantially harder.

Measure 3: fudge factor

…this is the solution to the problems with Measure 2. I’d prefer a better Measure 2, but I can see value in having Measure 2 be very simple to describe, and then to fix the problems later.

Official stance on measure 3

The one piece of data we have that tells us a lot and is hard to game is a user’s reputation on the existing sites.

If you have a lot of reputation, you’re much more likely to actively use the site, because you’ve shown that you actively use similar sites

If you have a significant amount of reputation across multiple sites, you’re even more likely to actively use the site, because you’ve shown that you actively use many such sites

On the other hand, if you’re some random person off the internet with no reputation, you’re very hard to quantify but there’s a good chance that you won’t contribute very much

Here’s the formula we have right now. It’s almost certainly wrong and we’ll be tweaking it as we go:

Writing a #unity3d mini-game … to help you design a AAA game

One of my hobby projects is a testament to the Ultima and Elder Scrolls games – a massive open-world, where everything you do impacts the world around you. This has long been promised by commercial games – and it sucks from a gameplay perspective; it’s just not fun. But a very old ASCII-graphics game, Omega, showed a couple of ways of doing it that delivered on that promise while still being “fun” – very fun! That’s what I’m working towards.

It’s a test-bed for the high-performance Entity System I’m writing for Unity (more info here). I’m aiming for “gameplay considerably richer than Skyrim” with “graphics about the level of Oblivion”. To be clear: Oblivion was released almost 10 years ago! To reproduce those graphics today is fairly easy.

Challenge: Believable Cities that evolve over time

One of the things I have yet to see in any FPS RPG is believable cities, and more: cities that actually evolve. The nearest I’ve seen is the beautiful “large towns” of the Assassin’s Creed series (they represent cities, but the distances are way too small).

That doesn’t cover the “evolve” part though – even the AC cities are 100% static and unchanging. Hand-crafted, and because they cannot change (technical design choice) the player is incapable of altering them. You can’t burn down blocks, or build a new bridge. You can’t buy two houses and knock them together.

Hey, RPG industry, you can do a lot better!

This is one of the main new gameplay elements in my design. If all I managed was to take an off-the-shelf-RPG mod and add living, growing cities … I’d probably be content. What does this mean, though?
Continue reading

Debugging odd problems when writing game servers

Cas (@PuppyGames) (of PuppyGames fame – Titan Attacks, Revenge of the Titans, etc) is working on an awesome new MMO RTS … thing.

He had a weird networking problem, and was looking for suggestions on possible causes. I used to do a lot of MMO dev, so we ran through the “typical” problems. This isn’t exhaustive, but as a quick-n-dirty checklist, I figured it could help some other indie gamedevs doing multiplayer/network code.


  1. Java, Windows 7
  2. DataInputStream.readFully() from a socket… a local socket at that… taking 4.5 seconds to read the first bytes
  3. and I’ve already read a few bits and bobs out of it

Initial thoughts

First thought: in networking, everything has a realm of durations. The speed of light means that a network connection from New York to London takes a few milliseconds (unavoidably) – everything is measured in ms, unless you’re on a local machine, in which case: the OS internally works in microseconds (almost too small to measure). Since Cas’s problem is measured in SECONDS – but is on localhost – it’s probably something external to the OS, external to the networking itself.

Gut feel would be something like Nagle’s algorithm, although I’m sure everyone’s already configured that appropriately ;).

That, or a routing table that’s changing dynamically – e.g. first packet is triggering a “let me start the dialup connection for you”, causing everything to pause (note that “4 seconds” is in the realm of time it takes for modems to connect; it’s outside the realm of ethernet delays, as already noted)

My general advice for “bizarre server-networking bugs”: start doing the things you know are “wrong” from a performance view, and prove that each makes it worse. If one does not, that de facto that one is somehow not configured as intended.

Common causes / things to check

1. Contention: waht’s the CPU + C libraries doing in background? If system has any significant (but small) load, you could be contended on I/O, CPU mgiht not be interrupting to shunt data from hardware up to software (kernel), up to software (OS user), up to JVM

Classic example: MySQL DB running on a system can block I/O even with small CPU load

2. file-level locking in OS FS. Check using lsof which files your JVM is accessing, and what other processes in the machine may have same files open.

(NB: there’s a bunch of tools like “lsof” which let you see which files are in use by which processes in a unix/linux system. As a network programmer, you should learn them all – they can save a lot of time in development. As a network programmer, you need to be a competent SysAdmin!)

Classic problem: something unexpected has an outstanding read (or a pre-emptive write lock) which is causing silliness. I remember several times having a text editor open, or etc, that was inadvertently slowing access to local files (doh).

3. can you snoop the traffic? (i.e. its socket, not pipe)

Classic problem: some unrelated service is bombarding the loopback with crap. eg. Windows networking (samba on linux) going nuts

Try running Wireshark, filtered to show only 127. and see what’s going through at same time

Also: this is a good way to check where the delay is happening. Is it at point of send, or point of receive? … both?

There’s “net traffic” and there’s “net traffic”. Wireshark often shows traffic that OS normally filters out from monitoring apps…

4. Check your routing table? AND: Check it’s not changing before / aftter the attempted read?

5. Try enabling Nagle, see if it has any effect?

My point is: use this as a check: it ought to make things worse. If not … perhaps the disabling wasn’t working?

6. Have you done ANY traffic shaping (or firewalling) on this machine at any time in the past?

Linux: in particular, check the iptables output. Might be an old iptables rule still stuck in there – or even a firewall rule.

Linux + Windows: disable all firewalls, completely.

7. Similarly: do you have any Anti-Virus software?

Disconnect your test machines from the internet, and remove all AV.

AV software can even corrupt your files when they mistakenly think that the binary files used by the compiler/linker are “suspicious” (IIRC, that happened to us with early PlayStation3 devkits).

8. On a related note: security / IPS tools installed? They will often insert artificial delays silently.

CHECK YOUR SYSTEM LOG FILES! …whatever is causing the delay is quite possibly reporting at least a “warning” of some kind.

9. (in this case, Cas’s socket is using SSL): Perhaps something is looking up a certificate remotely over the web?

…checked your certificate chain? (if there’s an unknown / suspicious cert in the chain, your OS might be trying to check / resolve it before it allows the connection)

10. (in this case, Cas is using custom SSL code in Java to “hack” it): Get a real SSL cert from somewhere, see if it behaves any different

(I think it would be very reasonable for your OS to detect any odd SSL stuff and delay it, as part of anti-malware / virus protection!)

11. “Is it cos I is custom?” – Setup an off-the-shelf webserver on the same machine and check if YOUR READING CODE can read from that SSL localhost fast?

(it often helps to get an answer to “is it the writer, the reader, both … is it the hacked SSL, or all SSL?” etc)

12. (getting specific now, as the general ideas didn’t help): if you bind to the local IP address instead of 127 ?

13. Howabout reverse-DNS? (again, 4 seconds is in the realm of a failed DNS lookup)

Might be a reverse DNS issue … e.g. something in OS (kernel or virus protection), or in JVM SSL library, that’s trying to reverse-lookup the 127 address in order to resolve some aspect of SSL.

I’ve had to put fake entries into hosts file in the past to speed-up this stuff, some libs just need a quick answer


Turns out we were able to stop here…

Cas: “Well, whaddya know. It was a hosts lookup after all – because I’m using and it was doing a hostname lookup. I added those to hosts and all is solved

Me: “(dance)” (yeah. It was skype. The dance smiley comes in handy)

How middleware (and open source) downloads ought to work – Unity3D

While upgrading Unity, I noticed the current download page is a great example of how it SHOULD be done:

Unity 4 has some … issues … with backwards compatibility – but at least they made the “need an older version?” link prominent. And how many old versions can you download?


(it goes on right back to unity 3.0)

Old versions? Who cares!

Well, that backwards compatibility thing is a *****. If you work on a project with other people, and they’re using Unity 3.5 … you SHOULD NOT (must not?) use Unity 4 (there be Dragons).

But it’s fine; Unity makes it trivial for anyone joining such a project to get exactly the version they need.

Some games middleware *cough*Hansoft*cough* companies declare that everyone must use the latest version, even if it is buggy and breaks existing projects. Or if it requires staff retraining. You must retrain EVERYONE! NOW!

(Hansoft has probably changed by now – maybe unfair to single them out. But for a long time they only allowed you to download the “latest” version, and actively deleted everything else. As soon as a new version existed, BOOM! Everything else got wiped. A happy customer I was not)


So, here we have a piece of middleware, with a download page:

  • Lives at an obvious, permanent URL:
  • Makes it very easy to find the download link (many open-source projects: shame on you)
  • Uncluttered webpage, and makes it easy to understand which download you want ( shame on you)
  • Every version has its release notes right there, for you to click on! (Apple (every product), and Firefox: shame on you)
  • Every version has BOTH the windows AND the mac downloads (computers today are cheaper than they’ve ever been. Many people have a laptop thats Mac, and desktop that’s Windows, or vice versa. You can’t assume that the browser they’re using dictates the desktop they’ll be working from)

Designing a website to look simple is certainly a difficult and non-trivial task.

But in the case of a download page – where almost everyone has the same needs, and there are many examples to copy (plagiarise) from – it doesn’t take much. More projects (and companies) should at least try to do this.

All Game Developers should read Pat Wyatt’s blog…

I noticed a few months back that Pat Wyatt has been blogging rgularly and in a lot of detail last year. This (IMHO) is big news: Pat is an awesome developer who held key positions in the teams behind many of the bestselling computer games (e.g.: Diablo 1 + 2, Starcraft, Warcraft) and went on to co-found Arena.Net (creators of Guild Wars).

I worked with him briefly in the past, and he’s friendly and full of advice and knowledge – but while he was happy to share, IIRC it was rarely in published form.

I’ve had a tough few months, but I’ve been dipping into his blog a few times, and it delivers in spades. Here’s a few hilights:

Assertions: enable them in live builds

(I’ve always felt this was the “right” way to do it for servers – where you don’t have to worry so much about frame-time, and assertions are more valuable at runtime because they help with the hardest-to-trace bugs … but it’s hard to get broad data on what the performance cost is)

“The bug was easily fixed by upgrading the build server, but in the end we decided to leave assertions enabled even for live builds. The anticipated cost-savings in CPU utilization (or more correctly, the anticipated savings from being able to purchase fewer computers in the future) were lost due to the programming effort required to identify the bug, so we felt it better to avoid similar issues in future.”

…and a great rule of thumb for any Programmer:

“After my experience reporting a non-bug to the folks at Microsoft, I was notably more shy about suggesting that bugs might be caused by anything other than the code I or one of my teammates wrote.”

Some bugs are due to … user’s broken hardware

“Mike O’Brien, one of the co-founders and a crack programmer, eventually came up with the idea that they were related to computer hardware failures rather than programming failures. More importantly he had the bright idea for how to test that hypothesis, which is the mark of an excellent scientist.

He wrote a module (“OsStress”) which would allocate a block of memory, perform calculations in that memory block, and then compare the results of the calculation to a table of known answers. He encoded this stress-test into the main game loop so that the computer would perform this verification step about 30-50 times per second.

On a properly functioning computer this stress test should never fail, but surprisingly we discovered that on about 1% of the computers being used to play Guild Wars it did fail! One percent might not sound like a big deal, but when one million gamers play the game on any given day that means 10,000 would have at least one crash bug. Our programming team could spend weeks researching the bugs for just one day at that rate!”

AI cheats to improve game balance in RTS’s, starting with Warcraft/Starcraft

In most Warcraft missions the enemy computer players are given entire cities and armies to start with when battling human players. Moreover, Warcraft contains several asymmetric rules which make it easier for the AI player to compete, though these rules would perhaps be called outright cheating by most players.

One rule we created to help the computer AI was to reduce the amount of gold removed from gold mines to prevent them from being mined-out. When a human player’s workers emerge from a gold mine those workers remove 100 units of ore from the mine and deliver it back to the player’s town hall on each trip, and eventually the gold mine is exhausted by these mining efforts. However, when an AI-controlled worker makes the same trip, the worker only remove 8 units of ore from the mine, while still delivering 100 units into the AI treasury.

This asymmetric rule actually makes the game more fun in two respects: it prevents humans from “turtling”, which is to say building an unassailable defense and using their superior strategic skills to overcome the computer AI. Turtling is a doomed strategy against computer AIs because the human player’s gold-mines will run dry long before those of the computer.

Secondarily, when the human player eventually does destroy the computer encampment there will still be gold left for the player to harvest, which makes the game run faster and is more fun than grinding out a victory with limited resources.”

Reaction to CoH (City of Heroes) community, and NCsoft’s response

(background: after 8 years as one of the world’s mid-tier MMO games, City of Heroes (+ City of Villains) is being shut down. The community banded together to ask if they could take over running the world that meant so much to them; NCsoft (the publisher, and a company I used to work for) said: no)

“No means no”

NCsoft is basically saying: “Please. We love you, but … you just *don’t understand*. It’s more complex than you could possibly imagine!”

That’s not a dialogue; it reads like a “this conversation ends when I stop talking” monologue.

“Why on earth wouldn’t you say yes?”

Lots of people wondering that. Obviously, being a public company, no-one’s going to answer that in public. We can only guess. But hear’s a few (over the top) suggestions…

If the community succeeds … then THE FEAR IS: some Executive(s), somewhere, are going to look like bad (I’m not accusing; I’m just saying that in corporates I’ve worked at, this kind of *fear* is common). A lot of the work they do is guess-work. That’s fine, they’re paid to make the best decision they can, while never truly know if they made the right one.

But if a bunch of inexperienced, eager novices come along and offer to do it for free. And – the worst possible outcome – they succeed … that could make someone look really bad.

Another thing I’ve seen in corporate politics at this level is a lot of “horse-trading”. i.e. sacrificing one project (that someone else resents, or has been snubbed by) in return for that person helping out out with a problem on a separate project, that you’re trying to rescue.

Who (individually or collectively) made the decision, and what did they stand to gain or lose? (they are probably worried about / aiming for / trying to win … something bigger than this single game. c.f. my 2009 post on why NCsoft is so huge a company gains nothing from “profitable” games, they need “mega profitable” games)

“Software is software”


Has anyone found out yet what format(s) the data is in? Imagine the most insane, unwieldy, incomprehensible, inconsistent, unusable format that bears no relationship *at all* to the game itself … and you’re probably half way there.

This game was written *8 years ago*.

Read the biographies of the people involved. Were they non-game developers … academics with decades of expertise in distributed systems and real-time transaction messaging? … or … were they a bunch of smart guys trying to catch up with the academic research in the space of months, just enough to build and ship a major new computer game? And … most importantly … to make it “fun” before they ran out of budget.

I’ve not yet found an MMO where the people who made it feel – with hindsight – they had any idea what they were doing at the start. When they started, of course, many of them thought they’d covered all the bases, and were “well prepared”. Everyone tries their best up-front (or fails completely); but everyone finds it much harder than expected.

What should we/they do?

Looking at it analytically and logically, I’d give the community a very high chance of failing dismally if they were given the game. But … the eagerness, the excitement, the sheer determination: I’d give them a small chance of succeeding despite everything. Simply because: when you see this much determination, it often wins out and overcomes the obstacles in its way.

So, I say: Go for it.

They know the game they’re trying to (re-)create. The difficulty is simple: whenever you try to re-create a game, the temptation is always there to “improve” it … and 99 times in 100, you find you slightly misunderstood what you were “improving”.

Dropbox tech scaling – some great, some not-so-great

“I was in charge of scaling Dropbox … from roughly 4,000 to 40,000,000 users. … Here are some suggestions on scaling”

The first section is a WTF – the guy advocates deliberately over-taxing your servers, without a good explanation. I’ve got some guesses at why they did it – but in the general case I’d say: never do this. Only do it when it’s obvious, because you have a specific reason to do so (and you really know what you’re doing).

The rest is a lot clearer, good advice there.

Also, IMHO worth reading for this part alone:

“Let’s say you’re trying to debug something in your webserver, and you want to know if maybe there’s been a spike of activity recently and all you have are the logs. Having graphing for that webserver would be great,

[ADAM: but … often you don’t have the right set of graphs set up, and it takes a while to do that – no use if the server is in trouble *right now*]

Apr 8 2012 14:33:59 POST …
Apr 8 2012 14:34:00 GET …
Apr 8 2012 14:34:00 GET …
Apr 8 2012 14:34:01 POST …

You could use your shell like this:

cut -d’ ’ -f1-4 log.txt | xargs -L1 -I_ date +%s -d_ | uniq -c | (echo “plot ‘-’ using 2:1 with lines”; cat) | gnuplot

Boom! Very quickly you have a nice graph of what’s going on, and you can tailor it easily (select only one URL, certain time frames, change to a histogram, etc.).”

LinkedIn (maybe) just leaked your password, won’t tell you; change it now

I’ve posted a few times over the years the … disappointing … state of LinkedIn’s engineering. But this takes the biscuit: it appears they were storing deliberately insecure passwords, and someone leaked the list:

(that page has links + info on how to check if your own password is in the mega list)

How bad is this?

  1. Many people have checked their personal, unique, passwords, that they claim to have only ever used on – and they’ve hit matches in the file.
  2. LinkedIn hasn’t told its users about the possible leak, more than 24 hours after it happened
  3. Many users re-use their passwords on other sites; any hackers could easily have stolen many accounts on other sites by now

How unlucky is LinkedIn?

This file is unsalted. That’s about as smart as locking your front door and then leaving the key under the mat – on the outside.

  1. Every tutorial, book, “newbie guide”, etc about using databases and writing login pages tells you never ever to do what was done here
  2. For any tech team, it is easy to check if this is what you’re doing, and tell your boss “uh, we need to fix that”
  3. It only takes a few *minutes* to prevent this problem, permanently. It’s not difficult

If LinkedIn were a small site, with a few hundred thousands users, I’d accuse them of laziness. But with 165million users, and a public company, you’d be looking at stunning incompetence by the tech wing of the company (the CIO and CTO never bothered to audit their own security?), or wilful negligence (no-one knew? really?).

Here’s hoping it’s a hoax…

Concepts of “object identity” in game programming…

Hume just posted his Lessons Learned from the warmup for Ludum Dare 23 (48 hours to write a game from scratch – starts this weekend!) – and his positive experience using an Entity System.

In his epic comment (sparked by a different Adam – not me, honest), is this gem:

“Using the entity system for the first time was unreal to me. It’s like polymorphic code. I did really weird things on the fly. For example:

– In the health processor, if the enemy was just destroyed, set a flag in the lifecycle component.
– In the lifecycle processor, if the fresh kill flag is set, extract its loot component and put that into a new entity with a small randomized velocity component and a gravity component so that the loot drops; then, remove most of the other components from the entity and add an explosion component.

The “enemy” still has the same entity ID — any other components that are looking for that entity will still find it (e.g. missiles homing in on the wreckage, or score processors looking for slain entities) — but by swapping one set of data with another, its implementation has changed from an enemy to some kind of non-interactive effect object.”

(emphasis mine)

Identity. It’s important.

(Quick sidenote: for all the people asking questions like “but … which variables do I put in Component A as opposed to Component B? How do I manage Events in an Entity System? … etc” – Hume’s approach above is a good concrete example of the first-draft, easy-to-write way of doing things. Copy it.)

Identity in games

This is one of those things that newbie game programmers seem to underestimate, frequently.

And when I say “newbie” I include “experienced, skilled programmers with 10+ years of coding experience – but who haven’t yet shipped a game of their *own*”.

(e.g. I’ve seen a couple of studios that started as Digital Agencies, or as Animation Studios, etc – that then transitioned to writing their own games. This is the kind of thing that they often struggle with. Not for lack of skill or general programming experience, but for lack of the domain-specific experience of game coding)

Examples of Identity in games, off the top of my head – all of these are independent, and interact in complex ways with each other :

  1. Game-role: e.g. … “enemy”, “powerup”, “start location”
  2. Code ‘object’ (in OOP terms): e.g. … “the sprite you are drawing at position (4,5) is part of Object X. X is coloured THIS colour”
  3. Gameplay ‘object’: e.g. … “the sprite at (4,5) represents a Tank. If a Tank sprite ever touches a Glass sprite, we need to play the Broken Glass noise”
  4. Physics element: e.g. … “5 milliseconds ago, our Physics Engine thought this thing was THERE. Now it’s over HERE. Don’t confuse the Physics Engine! Make sure it ‘knows’ they are the same object – not two separate objects”
  5. Network “master/clone”: e.g. … in multiplayer, there are N copies of my avatar: one per computer in the game. One of those N is the original – and changes to the original are constantly used to overwrite the clones; changes to clones are LOCAL ONLY and are discarded. Which is original? What do we do with incoming “changes” – which local Code Object do we apply them to? (your Code Object will be different from my Code Object – but they’ll both be the same identical Network Object, save mine is flagged “clone”)
  6. Proper Noun object: e.g. … “The Player’s Tank” is a very specific tank out of all the Tanks in the game. Many lines of game code don’t care about anything except finding and operating on that specific tank.
  7. Game-Over representation: e.g. … after the player has killed all the enemies, and they see a Game Over (won/lost/etc) screen, and you want to list all the enemies they killed … how do you do that? The enemies – by definition – no longer exist. They got killed, removed from the screen, removed from memory. You could store just the absolute numbers – but what if you want to draw them, or replay the death animations?
  8. …etc

Identity in Entity Systems

ES’s traditionally give you a SINGLE concept of Identity: the Entity (usually implemented as a single Integer). Hmm. That sounds worryingly bad, given what I wrote above. One identity cannot – by definition – encompass multiple, independent, interrelated identities.

But we’re being a bit too literal here. ES’s give you one PRIMARY identity, but they also give you a bunch of SECONDARY identities. So, in practice…

Secondary Identities in an ES

In OOP, the Object is atomic, and the Class is atomic. You cannot “split” an Object, nor a Class, without re-defining it (usually: re-compile).

In ES, the Entity is atomic, and the Component is atomic. But the equivalent of an OOP Object – i.e. “an Entity plus zero or more Components” – is *not* atomic. It can be split.

And from there comes the secondary identities…

A Primary Identity: e.g. “The Player’s Tank” (specific)
A Primary Identity: e.g. “a Gun Component” (generic)

A Secondary Identity: e.g. “The Gun component … of the Player’s Tank Entity” (specific)

Revisiting my ad-hoc list of Game Identities above, I hope it’s clear that you can easily re-write most of those in terms of secondary identity.

And – bonus! – suddenly the relationships between them start to become (a little) clearer and cleaner. Easier for humans to reason about. Easier *for you to debug*. Easier *for you to design new features*.

Global Identity vs. Local Identity

Noticeably, the network-related Identities are still hard to deal with.

On *my* computer, I can’t reference entities on *your* computer. I cannot store: “The Gun component … of YOUR player’s tank”, because your “Player’s Tank” only exists in the memory of your computer – not mine.

There are (trivially) obvious solutions / approaches here, not least: make your Entity integers global. e.g. split the 64bit Integer into 2 32bit Integers: first Integer is the computer that an Entity lives on, the second is the local Entity Integer. Combined, they are a “global Entity ID”.

(I’m grossly over-simplifying there – if you’re interested in this, google for “globally unique identifiers” – the problems and solutions have been around for decades. Don’t re-invent the wheel)

But … at this point, they also offer you the chance to consider your game’s network architecture. Are you peer-to-peer, or client-server?

For instance, P2P architectures practically beg for unique Global entity numbers. But C/S architectures can happily live off non-global. For instance:

  • On each client, there are ONLY local Entity numbers
  • When the client receives data from the server, it generates new, local, Entities
  • …and adds a “ServerGenerated” component to each one, so it’s easy to see that they are “special” in some ways. That component could hold info like “the time in milliseconds that we last received an update on this object” – which is very useful for doing dead-reckoning, to make your remote objects appear to move smoothly on the local screen
  • The server *does* partition all entities from all machines. But none of the clients need to know that

Or, to take it further, if your network arch is any good at all for high-paced gaming:

  • The server differentiates between:
    1. The entity that the game-rules are operating on
    2. The entity that client 1 *believes* is current
    3. …ditto for client 2, client 3 … etc (each has their own one)
    4. The entity that the game-rules accept (e.g. if a hacked client has injected false info, the game-rules may override / rewrite some data in the local object)
  • The server also tags all the entities for a single in-game object as being “perspectives on the same thing”, so that it can keep them in synch with each other
  • The server does Clever Stuff, e.g.:
    • Every 2 milliseconds, it looks at the “current entity”, and compares it to the “client’s belief of that entity”. If it finds any differences, it sends a network message to the client, telling it that “you’re wrong, my friend: that entity’s components have changed their data. They are now X, Y and Z”

… or something like that. Again, I’m grossly over-simplifying – if you want to write decent network code, Google is your friend. But the fastest / lowest latency multiplayer code tends to work something like that.


Ah, well.

What do you think?

(hint: you can do wonders using Reflection/Introspection on your entity / components. By their nature, they’re easy to write generic code for.

But you WILL need some extra metadata – to the extent that you may wish to ‘upgrade’ your Entity System into a SuperEntity System – something with a bit more expressive power, to handle the concept of multiple simultaneous *different* versions of the same Entity. Ouch)

Yeah, I’m bailing on you here. Too little time to write much right now – and it’s been a *long* time since I’ve implemented this level of network code for an ES. So, I’m going to have to think hard about it, refresh my memory, re-think what I think I knew. Will take some time…

Entity Systems: integrating Box2D with Artemis

Thanks to Mike Leahy for spotting this:

…a short blog post (with code) on how a team is integrating Box2D (a very well known open source physics lib) with Artemis (a java implementation of Entity Systems which starts from the same point as my Entity Systems posts, but fleshes it out)

Realm of the Mad God – a great game, interesting monetization

I’ve recently been playing the excellent Realm of the Mad God – a very fast-paced 2d co-operative shooter. My feeling is that it’s going to be one of the most important games of 2011/2012, as it continues to grow in popularity. Typical experience of this game is that within 30 seconds of being dumped into the main level, you’re surrounded by monsters, and then surrounded by other players, all on the same screen as you, blasting away in a rainbow of colours.

Sounds good. As if that’s not enough … it’s the guys who’ve been working with AmitP (Amit J Patel) (Amit maintains one of the best up-to-date collections of links and algorithms for indie game-developers). If you haven’t seen Amit’s pages, I recommend browsing through the blog – his links collection is OK, but his blog posts on algorithm design are excellent. If you get as far as the posts on “how I auto-generated a realistic 3D-world”, you may notice a striking similarity to the 2D worlds used in RotMG …

Anyway, it parallels an idea for an MMO shooter I’ve had kicking around for a long, long time. For me, it’s been a delight to see what works (and doesn’t) about the core ideas. The RotMG authors have done a great job of making a fast, simple, quick, easy-to-grasp game.

The Good

Fast. No barriers to play

This is how MMO-shooters should be: fast, furious, permadeath – but very quick to get back into the fray. You should expect to die tens of times every hour.

Permadeath – but paralled with some perma-advancement

Your progress is split evenly between items (which can be banked) and avatar stats (which are lost forever upon death).

Perma-advancement increases variety, unlocks new features

With 10 classes, there’s plenty of variety – and each class can only be unlocked by achieving a minimum level of progress with one or more other classes.

NB: this part could be improved and expanded IMHO. In particular, the classes are wonderfully varied – but merely unlocking classes isn’t enough these days. Plenty of games have shown that permanent-unlocks work best when there’s a variety of game-features in there. Also, the classes themselves would work better if there were some cross-effects (c.f. Diablo 2’s Lord of Destruction expansion, which had abilities in one class improve abilities in your older classes, re-vitalizing them for re-play)

Free game, paying is optional – payment kicks in when you’re most bought-in to the design

Free players get a tiny storage for permanent items (1/50th of what paying players get – it’s not enough! … so pay!), and are only allowed 1 class “alive” at once.

In a delicious twist, if you don’t pay for the game, the only way to take advantage of a newly-unlocked class is … to commit suicide … since you’re only allowed a single character per account (unless you pay)

You can ONLY benefit from other players, never suffer

(there’s actually a case where you can suffer, sadly – Thief’s get killed as a secondary-effect of other players teleporting to them, since the game doesn’t have a “prevent people from teleporting to me” flag)

This is the one that should have most wannabe-MMO-designers sitting up and paying attention. If you group-up with other players:

  1. You all get the same experience-points as if you’d single-handedly killed every monster
  2. You get the points just for being nearby – no need to score hits just to “tag” it for yourself
  3. Mob strength is constant, but player damage is multiplicative on number of players present

Net effect:

Every player is willing and eager (*) to collaborate with every other player, without words being exchanged, without fear of being ripped-off.

In a game that’s fast paced and frantic, you don’t have to keep pausing to negotiate. Other players can ONLY benefit you, so … run with them.

(*) – or just ignores the other players. Their presence doesn’t provide negative impact on you. It’s only their absence that is negative (in game-design terms).

Interesting design choices for lag

As a real-time game with dozens of players on screen at once, lag is guaranteed to effect gameplay. We’re always saying “try to work around lag through game-design changes”, so here’s the decisions they made:

  1. When packets are lost, everything moves in exactly the same direction it was going, at exactly the same speed, forever
  2. “Speed” used above is the “on-screen speed, including any rubber-banding effect”
    • FAIL: this means monsters and players often move MANY times faster than they are allowed to – so that when the lag stops, the side-effects are magnified
  3. Your avatar can’t be hit NOR damaged while it’s missing packets from the server
    • For the early parts of the game, this *almost* completely fixes lag problems
  4. Projectiles (bullets etc) that your client didn’t receive are queued up and sent to you all in one go once the packet-dropping stops
    • FAIL: this multiplies the damage output of enemies (NB: not players!), breaking all the designed-in balance in the game
    • In mid to late game, this ruins the gameplay – players end up running around never seeing a single enemy, because if you’re close enough to see it, a single flicker of lag will cause you to receive ZERO damage initially, followed by MORE damage than the monster is capable of – delivered instantaneously
  5. The client is authoritative on player liveness/death
    • MILD FAIL: in effect, coupled with the other features of the game, and the lack of lag compensation … this means you CAN and SHOULD (and, for some cases, effectively: MUST) cheat. You can run a bot on your machine – and if the network is less than perfect, you have to, in order to play the game properly.

Overall, apart from the massive security flaw (where anyone could write a bot to be invulnerable – and the developers are encouraging them to do so), it seems very close to a good solution for an MMO shooter.

I’m surprised by the way they approached the “save up the enemy bullets, then unleash them all at once”. My guess is that it wasn’t designed, it was just an accident: maybe they took a slightly lazy approach to compensating for lag (they don’t), and the net effect is this. It looks very much like what you get when using TCP for game-data packets (I really hope they’re not using TCP; if so, most of the lag is the developers’ own fault)

The Bad

I’ve unlocked half the classes, and looked at what classes other people play (and which classes rise high on the leaderboards). There’s good variety, and almost all the classes get used – even the beginning class, the one you get for free, works well.

Except one.

Unfortunately, at around 50% through unlocking the special classes, one of the classes is horribly unbalanced. The Assassin (which is supposedly an upgraded Thief – but is a massive downgrade) is almost impossible to play. The special ability fails completely when there’s lag (which is frequent in this game), and the class is the weakest, lowest-range of the lot. Looking around, you rarely see Assassins (I suspect: you only see them when people are desperately trying to upgrade them, to get past this dull and frustrating point in the upgrade tree).

Worse, because the *only* way to unlock the higher level classes is to reach the level cap with this class … you’re forced to play it. Over and over again. Watching the bad game-design … over and over … and over … again.

Every time the assassin dies, it’s like another twist of the knife:

We know you don’t enjoy this

We know that a mistake in our game design has you stuck here

(and our overall game design makes that “mistake” into “a disaster”)

We know that this whole process is turning “a class that wasn’t much fun” into “a class you hate”

And there’s nothing you can do about it!

So, single-handedly, it’s driven me to *not* purchase any game credit. I’d enjoyed the game enough to that point that I’d already decided to buy it – and if this had been on iTunes, I’d have paid already. But since it’s not an iPhone/iPad game, and paying for it is a bit more difficult, I hadn’t made the payment yet.

As it stands, I’m still playing occasionally, but now it’s for research rather than for enjoyment, which is a great pity.

Monetization: money thrown away

I think the developers are missing an ENORMOUSLY successful way to make money from this game. In fact, it’s so big, I suspect they could increase their revenues by a substantial multiplier.

With a permadeath game, there really is no need to actually delete the dead character. If the player isn’t paying, they are forced to kill their character sooner or later to change characters.

Taking a leaf from Flickr’s book, why not keep ONE single character in storage, with a tempting “buy now, for goodies *and to have this one returned to you, ready to play*”?

I’d set it so that when you change class (if you’re a free player), only the last character of the PREVIOUS class is retained. If you switch from Warrior to Knight, you can die many times as a Knight, and your Warrior remains on ice. But if you then grow tired of the Knight and switch to a Rogue … the Warrior is tipped out, and replaced with the last Knight you had.

i.e. you setup the exact flow of decision-making and options and “safety” that the player would have had, if only they’d purchased sooner, and allow them to benefit from it retroactively – if only they make the decision to pay.

Of course, it’s a very limited “retroactively” – it’s a sampler, to let you see the benefits of paying.

(*) – Flickr’s early promise was “upload your photos in highest resolution, you can view them for free – but only low-res versions. HOWEVER, we keep the high-res versions for you – forever, for free – until you decide to purchase a subscription. At which point, not just your new photos, but ALL your photos, become magically available at highest res”. It was a great way of simultaneously offering a high-value to paying customers, while making non-paying customers feel they weren’t committing themselves to loss. It reassured a lot of potential customers at a time when Flickr wasn’t yet famous, and most people weren’t yet “hooked” – it bridged that gap.

Analyse this

So, the interesting question is: how common is this problem?

Are the dev team correlating “players who pay” and “the point at which they pay”?

More importantly, are they correlating “players who DON’T pay” and “how their experience differed from the average”?

The last time I saw an MMO with a level-based kick in the teeth this bad … was in Tabula Rasa. We had a point where poor signposting by the quest designers meant many players were given quests that were many levels too hard for them, and effectively impossible to do. Those players died over and over and over again in a short time – and they hated it.

The dev team knew “you’re not supposed to do that quest”, but often they (randomly) gave it to new players as the first quest. I wasn’t privy to the arguments over whether this needed to be changed (and there were definitely arguments), but I did see the analytics that eventually got produced. They showed an almost perfectly smooth, averaged, graph of player behaviour – bar a big notch at this particular location. It stuck out like Rudolph’s nose on a snowy day.

I wonder if there’s a similar notch in RotMG? For a game that’s almost *designed* to drive people to rage-quit … what stats do they see on “what the last thing was before a player stopped playing forever?” … and what stats do they see on “…stopped playing for a long time, but eventually came back”?

Mongo DB is WebScale. MySQL is not WebScale.

There’s good reasons for adopting Mongo, I’m unconvinced (but open-minded) that performance is one of them. Here’s a ROFLMAO viewpoint on it:

“If your write fails, you’re ****ed”

Obviously, MySQL’s not perfect, but in most cases I’ve seen, it’s been lack of competence on the developer side, and the lack of basic DBA skills – not problems with MySQL itself – that’s broken scalability. In which case, I’m a little suspicious that a company that fails to scale MySQL will equally fail to write their code correctly on Mongo. In many ways, throwing away SQL makes it much easier to prevent scalability…

Using an Entity System with jMonkeyEngine (mythruna)

If you’re interested in using an ES on indie projects, and you’re craving concrete examples, you might want to look at the comments (page 1 and page 2) on Mythruna:

“Since this is a developers forum, I’ll describe a little bit about what went on behind the scenes.

During the early development for this phase, I read about an architectural pattern called an entity system.

I had (pre Entity Systems) a plan for how I was going to store the placement of objects in the world but this past weekend when I actually got to implementing it, I couldn’t make that original plan work and came to the point where I needed to solve the general problem of world state storage. This is the kind of thing that Entity Systems make relatively straight forward… and by Saturday, Mythruna had an embedded SQL database that autogenerates tables based on the components stored. (HyperSQLdb for the win.)

So at some point I swapped out the in-memory version of the entity system with the SQL version… and suddenly objects were persistent. It was so easy I had to double check a few different ways that it was actually working. :)”

…but of course, also: use the Entity Systems wiki – put your questions there, put your ideas there, and (most of all) if you have an ES project or ES source code you want to share, please add it to the wiki!

Richard Bartle: 10 Games you Should Have Played

Normally, I don’t allow “guest posts”, but I’m making an exception for my “10 Games You Should Have Played” series. I’ve been asking other games-industry people to write up their own lists + explanations, and that’s not always compatible with their personal/work/etc blog. When that happens, I’m happy to post them here instead.

So, here’s Richard Bartle‘s take (“co-creator of MUD1 (the first MUD) and the author of the seminal Designing Virtual Worlds” – but if you read this blog, you should already know who he is ;).

I think it’s a great list. I asked him to define in his own way what he meant by “should” (why are we saying “should”? who’s the audience? etc) and to run with it, which he did …

Come up with your own rules for a top-10, define it clearly, and share your list.

“OK, well modulo all the usual complaints about lists of 10, here we go.

I don’t have any rules per se, but I am sort of assuming that this is for people who play games or design games or want to know more about games.

Also, I’m going to go with categories rather than individual games (except in the last case). This is because it’s not the games themselves that are necessarily important so much as what you get from playing them. I will, however, give an example of a game in each category that I myself have tried.

1) A game you have bought but haven’t played yet.

You should always have a game ready to play. I don’t care what it is, but unless there is one you’re never going to expand your gaming horizons.

For me, right now that game is “Victoria II”, which I’ve installed and read the manual for but haven’t actually started to play. The reason I haven’t started it is because I bought “Mount & Blade: with Fire and Sword” and snuck that in front of it in the queue. Once I do start it, I’ll be looking for another game to play when it’s completed – hopefully not another damned sequel…

2) An abstract game.

Games can be many things, but unless they have gameplay they’re not games. An abstract game only has gameplay. To understand games, whether to design, play, or study them, you need to understand gameplay; an abstract game shows you the game mechanics with everything else stripped away.

You need to play one. You may, if you’re keen, try think of a skin for it, but that’s not essential.

In my case, I guess the game would be “Chess”. I captained my primary school Chess Club, but my interest in the game waned when I realised that the openings were always the same and that people who were less good at the fun, thinking part could win by doing the boring, memorise-the-openings part. That came straight from an appraisal of the clear-for-all-to-see mechanics.

That said, I’d also like to give a shout-out to the altogether more obscure “Besikovitch’s Game”. Now that’s a mechanic with potential…

3) A tabletop role-playing game.

Everyone thinks they know why they want to play games, but they also need to know why everyone else plays games. They’re not going to get that unless they understand what it means to be part of the game. In a tabletop role-playing game, with the other players right there next to you, there’s no escape: you have to participate, you have to involve yourself, you have to become part of the game, part of the narrative. In short, you have to live the game. Unless you’ve lived a game, how can you ever hope to understand what’s gamingly possible?

For me, the hours I spent playing “D&D” with my friends in my late teens were some of the best gaming experiences I ever hard. I wish I’d been able to get a “Call of Cthulhu” group going, mind you, but it came out too late for me.

4) A spectator sport.

If a game is good enough that people will pay to watch it played, you need to understand what it’s like to play it. This gives you an insight into the theatrical aspects of games that you wouldn’t easily get from merely observing the performance. You don’t actually have to be any good at the game, and the game itself doesn’t have to be all that good either (in my case, “Snooker” fits both those categories); the important thing is to understand what gives a game presence. I don’t care whether it’s high
skill, clever strategy, viscerality, physicality – if you don’t play it, you won’t appreciate it.

In my own case, I played “Association Football” (yeah, soccer) at school (attacking midfielder if you must know); I was good, but we were never taught any skills or anything and most games descended into kicking matches. I nevertheless found out what made it “the beautiful game”, though.

5) A game in which you can lose actual money.

There is a dark side to games, and gambling gives people a chance to sense it. Personally, I don’t like playing games for money at all; however, a lot of people love it. Everyone has their limits, though.

For some, gambling games are at their best when the amounts involved actually hurt if you lose them; for others, it’s the amounts that can be won that make the difference. The point of playing a gambling game from the perspective of this list is to gain an appreciation of the morality of games. When something stops being “just a game” and starts to take over the player’s life, that’s potentially a bad thing. Unless
you’ve seen it (or something close to it), you’re never going to understand that fully. Gambling games let you do that. Warning: you run a big risk with this if it turns out you’re the one who gets hooked…

For me, I used to play “Poker” with my friends over lunch when I was 17 or 18. We played for Tic-Tac mints. This was before “Texas Hold ‘Em” got big, so we’d play mainly “Draw Poker”, “5-Card Stud”, “7-Card Stud” or, occasionally, “Montana Red Dog”. We stopped playing when one of my friends, who consistently lost, had to borrow money to buy more Tic-Tacs; I decided things had gone far enough, and called
the lunchtime sessions off. From that point on, no way would I design a game that deliberately tried to addict someone to it.

6) A game released in the year you were born.

Most games are built on the foundations of games that went before them, and an appreciation of their history means you appreciate the games themselves more. Games have a very long history (indeed, they go back into prehistory), but a modern game is unlikely to quote directly from ancient archetypes. They’re more probably going to quote from games from the generation before them. You therefore need to
play a bunch of old games to see where the advances were made. Unfortunately, “old” is a relative term: what you think is old might, to me, seem fairly new. What’s old enough for both of us is something from the year we were born in (or a year close to that). Play a game from back then and see how things have (or haven’t) changed. Bonus: you’re almost guaranteed to notice the gameplay more than you do in a (what currently looks) slick, modern game.

For me, the old game would be “Diplomacy”, which was released commercially in 1959 (the year before my birth, but that’s near enough). Ah, what a game! It’s trapped in its time, because it needs 7 players and could only really be played by post. Play-by-email is even more of a niche than play-by-mail was, so it’s not a game that is played a lot nowadays. Lovely mechanics, though!

7) A really bad game.

Some games are just BAD. The mechanics are all wrong, they’re unfun, or no fun, or the rules are ambiguous, or they drag on and on, or there’s a dominant strategy, or … well, the list continues. If you play such a game, you can ascertain what it is that’s bad about it; this will enable you to avoid similar games in future and to avoid
making similar mistakes in any games you design yourself (see next point). The more you understand about games, the more you’ll be able to find the games that are right for you.

For me, tempting though it is to nominate “Trivial Pursuit” as the game that laid waste to the British board games industry, I didn’t actually play that. However, my personal pick is one that I’m sure many other people will share, too: “Monopoly”…

8) A game you wrote yourself that no-one else has played.

Game design is actually quite hard to do well. You’re not going to know quite how hard unless you try it yourself. In the attempt, you’ll come to understand more about games and what makes them tick – but only if you actually play the game (if
it needs more than one player, play it against yourself). If you actually are a game designer, this is something you will have done many, many times before, of course; just make sure you keep on doing it.

For me, well, all game designers have a corpus of games in various stages of completion that they have never shared with anyone else, simply because doing the design itself was the fun part.

I’m a bit low on computer games in this list, so I’ll go for one I did called “Mombasa” about the exploration of Africa. It’s not all that good, but the point is that I wouldn’t know that if I hadn’t played it…


This is because I co-wrote the first virtual world, and therefore the more people who play these, the higher my kudos rises.

I’ll list the last MMO I played through up to the level cap as my example here: “Rift”. I came away not so much impressed by the game itself but by its developer, Trion Worlds, which is more understanding of its players than any other developer I’ve
come across except perhaps CCP.

10) “Mornington Crescent”.

It’s actually called “Finchley Central”, but I’ll go with the version that’s best known. This is a very simple game, the rules of which, in their entirety, are as follows: players take it in turns to name London Underground stations, and the first to say
Mornington Crescent wins. This is a game everyone should play, because it gets to the heart of what a game is: what happens when you freely and knowingly bound your behaviour according to a set of rules in the hope of gaining some benefit that you might not get. You play it for just so long as it’s fun, with people who also play for just so long as it’s fun. It’s the Magic Circle incarnate.

So those are my top 10 games that people should play. If you already played them, my apologies for having wasted your time with this list. If you haven’t played them, I envy you the treasure trove that lies ahead.”

“by running a spy network I am griefing”

If you’re an MMO designer, and you *still* don’t grok the griefer-mindset, or you somehow hope/believe that “one day, there will be no griefers”, then maybe this RPS interview with the always-fun-to-watch Goonswarm will help you:

MT: We are griefers. If nothing is going to happen then we’re going to try to find something that screams and bleeds and poke at it.

RPS: Griefing is something goons are known for doing, but now I’m talking to you it’s not something I can imagine you personally doing.

MT: Technically speaking, by running a spy network I am griefing.

RPS: But would you go out and aggravate other players for the Hell of it if you were a lower ranking member of Goonswarm?

MT: Well, most lower ranked Goons make their money by doing that. Scamming people is a very quick way of making money in Eve. Rather than making an honest buck, you take that buck from somebody else.

and, much further down, maybe this will help you see how griefers often serve just as positive and valuable a role as all your “preferred” player-types:

RPS: For my money, Eve might be the most fascinating game in existence today. But that doesn’t stop it from being interminably boring as well.

MT: Right. I mean most Eve players are stuck in high security space mining, and a lot of the core PvE in Eve has you sitting there are watching three grey bars slowly turn red.

Goonfleet is a socialist alliance. We give people ships so that rather than being forced to rat [fight low-powered AI NPCs] they can take part in PvP, we teach them how to scam so that they don’t have to mine, we teach them how to make ISK most effectively, we give them a lot of ISK and we reimburse their losses. This way they can focus on the fun aspects of the game, like griefing and warfare, so they’re not forced to endure derp-derp-ing around high sec.

If they play your game, you should be glad; if they grief, you should be asking yourself why – and if you’re a commercial operation, you should probably be asking:

“are they fixing a problem for us?

can we afford to leave them to it, part of our unpaid workforce?


is it worth our time trying to fix the problem itself, or should we accept their help and move on down our never-ending list of pending fixes?”

Entity Systems: updates to source code

I’ve just done a round of fixes for the source-examples of ES’s. Github projects updated on this page:


  1. Added a complete Java implementation of the most basic ES example
  2. Fixed some minor bugs in the Objective-C basic ES example; added some missing classes
  3. Added a missing class method from the documentation (System.

The nature of a Tech Director in games … and the evils of DevOps

Spotted this (the notion “DevOps”) courtesy of Matthew Weigel, a term I’d fortunately missed-out on.

It seems to come down to: Software Developers (programmers who write apps that a company sells) and Ops people (sysadmins who manage servers) don’t talk enough and don’t respect each other; this cause problems when they need to work together. Good start.

But I was feeling a gut feel of “you’ve spotted a problem, but this is a real ugly way to solve it”, and feeling guilty for thinking that, when I got down to this line in Wikipedia’s article:

“Developers apply configuration changes manually to their workstations and do not document each necessary step”

WTF? What kind of amateur morons are you hiring as “developers”? Your problem here is *nothing* to do with “DevOps” – it’s that you have a hiring manager (maybe your CTO / Tech Director?) who’s been promoted way above their competency and is allowing people to do the kind of practices that would get them fired from many of the good programming teams.

Fix the right problem, guys :).

Incidentally – and this will be a long tangent about the nature of a TD / Tech Director – … my “gut feel” negativity about the whole thing came from my experience that any TD working in large-scale “online” games *must be* a qualified SysAdmin. If they’re not, they’re not a TD – they’re a technical developer who hasn’t (yet) enough experience to be elevated to a TD role; they are incapable (through no fault of their own – simply lack of training / experience) of fulfilling the essential needs of a TD. They cannot provide the over-arching technical caretaking, because they don’t understand one enormous chunk of the problem.

I say this from personal experience in MMO dev, where people with no sysadmin experience stuck out like a sore thumb. Many network programmers on game-teams had no sysadmin experience (which in the long term is unforgivable – any network coder should be urgently scrambling to learn + practice sysadmin as fast as they can, since it’s essential to so much of the code they write) – and it showed, every time. In the short term, of course, a network coder may be 4 months away from having practiced enough sysadmin. In the medium term, maybe they’ve done “some” but not enough to be an expert on it – normally they’re fine, but sometimes they make a stupid mistake (e.g. being unaware of just how much memcached can do for you).

And that’s where the TD-who-knows-sysadmin is needed. Just like the TD is supposed to do in all situations – be the shallow expert of many trades, able to hilight problems no-one else has noticed, or use their usually out-dated yet still useful experience to suggest old ways of solving new problems that current methods fail to fix. And at least be able to point people in the right direction.

…but, of course, I was once (long ago) trained in this at IBM, and later spent many years in hardcore sysadmin both paid and unpaid (at the most extreme, tracking and logging bugs against the linux kernel) so I’m biased. But I’ve found it enormously helpful in MMO development that I know exactly how these servers will *actually* run – and the many tricks available to shortcut weeks or months of code that you don’t have to write.