Entity Systems are the future of MMOG development – Part 1

A few years ago, entity systems (or component systems) were a hot topic. In particular, Scott Bilas gave a great GDC talk (http://scottbilas.com/files/2002/gdc_san_jose/game_objects_slides.pdf – updated link thanks to @junkdogAP) on using them in the development of Dungeon Siege. The main advantages to entity systems were:

  • No programmer required for designers to modify game logic
  • Circumvents the “impossible” problem of hard-coding all entity relationships at start of project
  • Allows for easy implementation of game-design ideas that cross-cut traditional OOP objects
  • Much faster compile/test/debug cycles
  • Much more agile way to develop code

I first started using entity systems in anger back in 2001-2003, when I was working on MMOG server middleware. We were targetting the few most painful problems in MMOG development, one of which was the difficulty of constantly changing your game logic after launch, which led us to entity systems. I learnt then that entity systems were an almost perfect solution to massively speeding up the development time for most MMOG’s, and for also allowing almost unrestrained re-writing of fundamental game features post-launch with very little effort.

That last issue is critical to the success of an MMOG: once an MMOG is launched successfully, its long term success or failure depends more upon the ability of the dev team to evolve that game into a better game month after month than upon anything else.

I learned a lot from that first run-in with them, more about the things that go wrong and what makes developing with entity systems particularly hard than about the things that went right. We also discovered that performance could easily become a major issue – although they are very flexible and dynamic, the lack of pre-compiled lookups and optimizations can make runtime performance disappointingly (unacceptably) poor. Why? Mainly because of the amount of indirection and checks needed to run even a single method call (but I’ll go into detail on that problem, and how to fix it, a bit later).

I moved on, and didn’t think about them again, until last year. In 2006 I joined the Operation Flashpoint 2 team as the lead network programmer, where we trying to make an MMO-FPS on an unprecedented scale, and I discovered that the programming team was considering an entity-system to drive the whole game. The attractions for the OFP2 team were different – mainly around the improvements to memory management they could get from it, and the stream-oriented coding (which is essential for PS3 development) – but it turned out to be something of a silver bullet for the “MM” and “O” parts of the MMOFPS. As the network programmer, discovering that an entity system was going to be the interconnect for all other subsystems was a huge relief: the entity system meant I could implement the complex latency hiding and prediction techniques with minimal interference with the coding of all the rest of the game systems. When you’ve got 20+ programmers on a team, you really don’t want to be placing yourself in a position where you’re going to have to redesign code for all of them just to make it “network friendly”.

whilst I was working on OFP2, I got in touch with Scott, and exchanged ideas on what the future might hold for entity systems, including additional uses for them, and on how to share this knowledge more widely. He encouraged me to start a blog (and, actually, a year later that was one of the main reasons I even started this blog, T=Machine), so I figured now was a good time to start (finally) writing about those entities :).

And to blame Scott for the existence of this blog…

Since then, I’ve been thinking a lot off and on about entity systems and their appropriateness for use in MMOG development (again!). Only this time around – thanks to OFP2 and some of the issues it threw up – I had a much better idea how to use them to increase performance rather than reduce it, and had come across some other major problems that they conveniently solve. Most obviously, they work wonders for PS3 development. Knowing how PS3’s fundamental architecture works, I can’t immediately see how you’d want to use anything other than a full entity system for game development. There’s so much horsepower in that beast that you can certainly write games many different ways, but it’s particularly well-suited to this approach.

As it stands, I’m beginning to think that next-generation MMOG’s are going to be practically impossible to develop unless based heavily around a core entity-system. “practically” being the key word – unless you’re willing to spend $100 million for your development…

“Anything is possible”, of course, so there’ll be many exceptions to that – but I think any team that doesn’t go this route is going to suffer a lot because of it.

I think most people would agree. But … recent discussions on game-industry mailing lists – and the experiences I had within games companies with programmers who were much better at programming than I was – made me realise that there’s a lot of ignorance over these systems, and many people aren’t getting anywhere near the full potential of them. So, if you’re interested, read on…

I’ll do this in a series of posts (it’s going to take some time to write it all up!), but it’ll go something approximately like this:

ADDENDUM:

EDIT, December 2007 – You may also want to take a look at Mick West’s introduction to entity systems for game development, it’s a shorter read than my posts, but narrower in scope.

NB: as I post the other parts, I’ll update the list above to link to them.

Did this post help you?

Support me on Patreon, writing about Entity Systems and sharing tech demos and code examples


78 thoughts on “Entity Systems are the future of MMOG development – Part 1

  1. Fiordean Dacian

    Hi,

    I’m interested in networking and databases for games (I do this in my spare time). I just went over Scott’s paper from GDC on entity systems last year when looking for an efficient way to organize gama data. I didn’t go in depth with them. Anyway, I’m just curious (as you’ll continue your posts with parts 2 to 5) if databases can help building these kind of systems efficiently. Scott mentions storage (INI files, XML, etc.), but is there a place for SQL engines in there? Hopefully, we’ll see that :)

    Dacian

  2. adam Post author

    Relational DB’s are a major part of this – even if you don’t choose to use them. One of the interesting things that comes out of trying to work with entity systems is that they behave a lot more like that than they do like OOP – so, not only are they data driven and you’d *really like* to store and retrieve them using SQL , but they turn out to be well-suited for that kind of serialization.

    And, of course, these days there are many variants of SQL-based DB, from the traditional mega-servers down to the single-user, local-only DB’s extremely well suited for shipping as part of your game client, or the in-memory-only DB’s…

  3. Pingback: T=Machine » Entity Systems are the future of MMOG development - Part 2

  4. Pingback: T=Machine » Entity Systems are the future of MMOG development - part 3

  5. Pingback: T=Machine » Entity Systems are the Future of MMOs Part 5

  6. Sander van Rossen

    When are you going to post the next post on this topic?
    Really love what you wrote up so far.

  7. Pingback: A Technical Overview « Engine Of Evermore

  8. Pingback: Dino Interactive » Game Development Chronicles: Episode #1

  9. Aengus

    Hi.

    I have read all 4 posts about the entity system and i can’t wait for the next one. I am thinking of trying to use this for a game engine and for the implementation side i started looking at the article in game programming gems 5. I would like to know what you think of that one.

    Thanks

  10. Pingback: Start of feature list « Beef Machine Games

  11. Pingback: Entity Systems « chrislunsford.com

  12. Pingback: T=Machine » Entity Systems are the Future of MMOs Part 5

  13. Pingback: Das Gamesystem-Modul « Zeitalter3 – Browsergames Entwicklerblog

  14. Pingback: T=Machine » Entity System 1: Java/Android

  15. Pingback: Yummy… « Chris On Everything

  16. Pingback: Mark’s Testblog » Blog Archive » Data oriented design links - …for these are testing times, indeed.

  17. the31

    Thank you so much for writing these articles. After reading them I can see that using entity systems solves the main problem I’ve been running into with my MMO projects–the interface with the database.

    I was linked to this post a while ago and I’m slowly implementing my own entity system. Running into plenty of problems (first time trying to use templates, C++) with the way I’m trying to do it, and I’m just about to begin my first re-read of the posts to help myself rethink my attack.

    Can’t wait for the next article.

  18. Pingback: T=Machine » Entity Systems are the future of MMOG development – part 3

  19. Pingback: State of the Art Game Objects | GBGames - Thoughts on Indie Game Development

  20. Pingback: Galactic Gaming » Blog Archive » Entity Systems

  21. Pingback: Flash component oriented programming | JADBOX: Web Application Musings

  22. ortuman

    Up to now, as I understand.

    Entity Systems are nothing more than relational “databases” where:

    Entities = Primary keys
    Rows = Components
    Tables = Systems

    Besides, systems have their own functions and operate over their data knowing exactly how to deal with it.

    Is this a too simplistic conclusion?

    Maybe I’ll try to make something based on this new and attractive idea.

    Great article!!!

  23. lukasPfeffer

    Hi adam!

    thanks for all the inside! I’m now trying to use your approach in a little flashgame project and started with your objective C code from the wiki( i tried to comment there but couldn’t figure out how to do it).

    As i tried to implement the first components i stumbled over little problem and i think i’m missing something,.

    I tried to implement the following components:

    scrollable ( as i want to do a sidescrolling game, most sprites should move/scroll)
    Moveable ( move sprites via playerinput)
    Renderable ( actually show sprites on screen)

    all these components need access to the sprites (x,y)-position, but according to your posts/wikicode the position( the data) has to be in the components and not in the entity.

    my question now is, how do all components gain access to the same position variable?

    thanks

  24. adam Post author

    No component can ever “need access to” anything – there’s no code in a component. This has come up a few times before, have a read throughthe comments for the various posts here.

  25. Pingback: Game Architecture

  26. sen

    Hi Adam, great posts, thank you!

    Not sure, but maybe what lukasPfeffer meant was that many systems seem to depend on multiple components, at least for me it was puzzling what to do with my transform component that is needed in many systems(sprite, 3d object, ai logic, etc. systems).

    Now i just have components storing a pointer to transform component they need and access it from systems in need of that data. This seems like partially falling back to “gameobject architecture” but didn’t figure any better solution. Is this the way others have gone or have I also missed something essential?

    Thanks

  27. adam Post author

    Maybe I’ve caused too much confusion with the “1 component per system” statements in the past?

    Core to the idea of ES is that your transform component is NEVER referenced from other components – why would you do that? What’s the point? If it’s needed, you add it to the entity, and only the *system* will ever care. If it needs transform data to so something usefule with the AI logic, it will merely request “the transform component for (that entity)”.

  28. lukasPfeffer

    Components needing access to something is obviously not right, sorry. but the systems do, right?

    Yeah, i have problems with the one “1 component to 1 system”-relation too and somehow i still dont get.

    i have a rendercomponent( a sprite) and a position component.

    user input, handled by the positionSystem need the postion comp. everythings fine here.
    but my renderingsystem needs the sprite and the position comp to draw anything on screen.

    shall i put the position info in both components, or should i but everything in one component and let both systems work on it? i’m a bit lost…

    @the wiki: how am i able to comment there?

  29. Pingback: Entity Systems

  30. adam Post author

    Components are pure data, nothing else. So there’s no point in duplicating them. This is a big part of WHY they are pure data: so that you never have to worry about the issue you have.

    Decide the components you need on a system-by-system basis, maybe several for some systems. You expect to see a few dupes (e.g. Position is needed in lots of places). But since you’ll only be making each component once, it doesn’t matter. Everything is “shared” by definition already, so don’t worry about it.

    The wiki si just a normal wiki. http://entity-systems.wikidot.com

  31. Pingback: Incomplete Game Prototype: Gravity | Hanno Braun

  32. Pingback: Incomplete Game Prototype: Gravity « Hanno Braun

  33. Pingback: Entity Frameworks, part 2 » Gray Lantern

  34. Pingback: Component based Game Entity Systems « Dysnomia Games

  35. Pingback: The Making of: SuperChicken Battles the Penguin Horde | Piron Games Insider

  36. Pingback: Day 10: Shooting For The Goal | Pike's Game Dev Diary

  37. Pingback: Entity System | andreashedin

  38. Pingback: Entity Systems « Alexandre Kohen

  39. Pingback: Rectangle Packing | BLACKGOAT

  40. Pingback: Entity-Component game programming using JRuby and libGDX – part 1 « I have a black belt in geek.

  41. Pingback: Game Engine Framework: Overview » 6by9

  42. Pingback: A hybrid Entity-System-Component architecture | Game Dev Rubber Duck

  43. Pingback: Artemis Entity Framework Ported to C# | Ploobs

  44. Pingback: 3D Free-running Shooter: Networking | Apps, Websites and Game Development

  45. Pingback: ES Architecture. | Seed Of Awesome

  46. Pingback: Component based game engine design | Everyday I'm coding

  47. Pingback: Using an Entity System Part 1 | Exploring Lines

  48. Pingback: C# Artemis Entity System Tutorial | Ploobs

  49. Pingback: » The Entity System Brent Owens' Blog

Leave a Reply

Your email address will not be published. Required fields are marked *