Entity Systems are the Future of MMOs Part 5

(Start by reading Entity Systems are the Future of MMOs Part 1)

It’s been a long time since my last post on this topic. Last year, I stopped working for a big MMO publisher, and since then I’ve been having fun doing MMO Consultancy (helping other teams write their games), and iPhone development (learning how to design and write great iPhone apps).

Previously, I posed some questions and said I’d answer them later:

  • how do you define the archetypes for your entities?
  • how do you instantiate multiple new entities from a single archetype?
  • how do you STORE in-memory entities so that they can be re-instantiated later on?

Let’s answer those first.

A quick warning…

I’m going to write this post using Relational terminology. This is deliberate, for several reasons:

  • It’s the most-correct practical way of describing runtime Entities
  • It’s fairly trivial to see how to implement this using static and dynamic arrays – the reverse is not so obvious
  • If you’re working on MMO’s, you should be using SQL for your persistence / back-end – which means you should already be thinking in Relations.
…and a quick introduction to Relational

If you know literally nothing about Relational data, RDBMS’s, and/or SQL (which is true of most game programmers, sadly), then here’s the idiot’s guide:

  1. Everything is stored either in arrays, or in 2-dimensional arrays (“arrays-of-arrays”)
  2. The index into the array is explicitly given a name, some text ending in “_id”; but it’s still just an array-index: an increasing list of integers starting with 0, 1, 2, 3 … etc
  3. Since you can’t have Dictionaries / HashMaps, you have to use 3 arrays-of-arrays to simulate one Dictionary. This is very very typical, and so obvious you should be able to understand it easily when you see it below. I only do it twice in this whole blog post.
  4. Where I say “table, with N columns”, I mean “a variable-length array, with each element containing another array: a fixed-size array of N items”
  5. Where I say “row”, I mean “one of the fixed-size arrays of N items”
  6. Rather than index the fixed-size arrays by integer from 0…N, we give a unique name (“column name”) to each index. It makes writing code much much clearer. Since the arrays are fixed-size, and we know all these column names before we write the program, this is no problem.

Beyond that … well, go google “SQL Tutorial” – most of them are just 1 page long, and take no more than 5 minutes to read through.

How do you store all your data? Part 2: Runtime Entities + Components (“Objects”)

We’re doing part 2 first, because it’s the bit most of us think of first. When I go onto part 1 later, you’ll see why it’s “theoretically” the first part (and I called it “1”), even though when you write your game, you’ll probably write it second.

Table 3: all components

(yes, I’m starting at 3. You’ll see why later ;))

Table 3: components
component_id official name human-readable description table-name

There are N additional tables, one for each row in the Components table. Each row has a unique value of “table-name”, telling you which table to look at for this component. This is optional: you could instead use an algorithmic name based on some criteria like the official_name, or the component_id – but if you ever change the name of a component, or delete one and re-use the id, you’ll get problems.

Table 4: all entities / entity names
Table 4 : entities
entity_id human-readable label FOR DEBUGGING ONLY

(really,you should only have 1 column in this table – but the second column is really useful when debuggin your own ES implementation itself!)

…which combines with:…

Table 5: entity/component mapping
Table 5 : entity_components
entity_id component_id component_data_id

…to tell you which components are in which entity.

Technically, you could decide not to bother with Table 4; just look up the “unique values of entity_id from table 5” whenever you want to deal with Table 4. But there are performance advantages for it – and you get to avoid some multi-threading issues (e.g. when creating a new entity, just create a blank entity in the entity table first, and that fast atomic action “reserves” an entity_id; without Table 4, you have to create ALL the components inside a Synchronized block of code, which is not good practice for MT code).

Tables 6,7,8…N+5: data for each component for each Entity
Table N+5 : component_data_table_N
component_data_id [1..M columns, one column for each piece of data in your component]

These N tables store all the live, runtime, data for all the entity/component pairs.

How do you store all your data? Part 1: Assemblages (“Classes”)

So … you want to instantiate 10 new tanks into your game.

How?

Well, you could write code that says:


int newTank()
{
int new_id = createNewEntity();

// Attach components to the entity; they will have DEFAULT values

createComponentAndAddTo( TRACKED_COMPONENT, new_id );
createComponentAndAddTo( RENDERABLE_COMPONENT, new_id );
createComponentAndAddTo( PHYSICS_COMPONENT, new_id );
createComponentAndAddTo( GUN_COMPONENT, new_id );

// Setup code that EDITS the data in each component, e.g:
float[] gunData = getComponentDataForEntity( GUN_COMPONENT, new_id );
gunData[ GUN_SIZE ] = 500;
gunData[ GUN_DAMAGE ] = 10000;
gunData[ GUN_FIRE_RATE ] = 0.001;
setComponentDataForEntity( GUN_COMPONENT, new_id, gunData );

return new_id;
}

…and this is absolutely fine, so long as you remember ONE important thing: the above code is NOT inside a method “because you wanted it in an OOP class”. It’s inside a method “because you didn’t want to type it out every time you have a place in your code where you instantiate tanks”.

i.e. IT IS NOT OOP CODE! (the use of “methods” or “functions” is an idea that predates OOP by decades – it is coincidence that OOP *also* uses methods).

Or, in other words, if you do the above:

NEVER put the above code into a Class on its own; especially NEVER NEVER split the above code into multiple methods, and use OOP inheritance to nest the calls to “createComponet” etc.

But … it means that when you decide to split one Component into 2 Components, you’ll have to go through the source code for EVERY kind of game-object in your game, and change the source, then re-compile.

A neater way to handle this is to extend the ES to not only define “the components in each entity” but also “templates for creating new entities of a given human-readable type”. I previously referred to these templates as “assemblages” to avoid using the confusing term “template” which means many things already in OOP programming…

An assemblage needs:

Table 1: all Assemblages
Table 1 : assemblages
assemblage_id default human-readable label (if you’re using that label in Table 1 above) official name human-readable description
Table 2: assemblage/component mapping
Table 2 : assemblage_components
assemblage_id component_id

This table is cut-down version of Table 5 (entity/component mapping). This table provides the “template” for instantiating a new Entity: you pick an assemblage_id, find out all the component_id’s that exist for it, and then create a new Entity and instantiate one of each of those components and add it to the entity.

Table 3: all components
Table 3: components
component_id official name human-readable description table-name

This is the same table from earlier (hence the silly numbering, just to make sure you noticed ;)) – it MUST be the same data, for obvious reasons.

Things to note

DataForEntity( (entity-id) ) – fast lookup

If you know the entity-id, you may only need one table lookup to get the data for an entire component (Table5 is highly cacheable – it’s small, doesn’t change, and has fixed-size rows).

Splitting Table 5 for performance or parallelization

When your SQL DB is too slow and you want to split to multiple DB servers, OR you’re not using SQL (doing it all in RAM) and want to fit inside your CPU cache, then you’ll split table 5 usually into N sub-tables, where N = number of unique component_id’s.

Why?

Because you run one System at a time, and each System needs all the components with the same component_id – but none of the components without that id.

Isolation

The entire data for any given system is fully isolated into its own table. It’s easy to print to screen (for debugging), serialize (for saving / bug reports), parallelize (different components on different physical DB servers)

Metadata for editing your Assemblages and Entities

a.k.a. “Programmer/Designers: take note…”

It can be tempting to add extra columns to the Entity and Assemblage tables. Really, you shouldn’t be doing this. If you feel tempted to do that, add the extra data as more COMPONENTS – even if the data is NOTHING to do with your game (e.g. “name_of_designer_who_wrote_this_assemblage”).

Here’s a great feature of Entity Systems: it is (literally) trivial for the game to “remove” un-needed information at startup. If, for instance, you have vast amounts of metadata on each entity (e.g. “name of author”, “time of creation”, “what I had for lunch on the day when I wrote this entity”) – then it can all be included and AUTOMATICALLY be stripped-out at runtime. It can even be included in the application, but “not loaded” at startup – so you get the benefits of keeping all the debug data hanging around, with no performance overhead.

You can’t do that with OOP: you can get some *similar* benefits by doing C-Header-File Voodoo, and writing lots of proprietary code … but … so much is dependent upon your header files that unless you really know what you’re doing you probably shouldn’t go there.

Another great example is Subversion / Git / CVS / etc metadata: you can attach to each Entity the full Subversion metadata for that Entity, by creating a “SubversionInformation” System / Component. Then at runtime, if something crashes, load up the SubversionInformation system, and include it in the crash log. Of course, the Components for the SubversionInformation system aren’t actually loader yet – because the system wasn’t used inside the main game. No problem – now you’ve started the system (in your crash-handler code), it’ll pull in its own data from disk, attach it to whatever entities are in-memory, and all works beautifully.

Wrapping up…

I wanted to cover other things – like transmitting all this stuff over the network (and maybe cover how to do so both fast and efficiently) – but I realise now that this post is going to be long enough as it is.

Next time, I hope to talk about that (binary serialization / loading), and editors (how do you make it easy to edit / design your own game?).

Did this post help you?

Support me on Patreon, writing about Entity Systems and sharing tech demos and code examples

154 Replies to “Entity Systems are the Future of MMOs Part 5”

  1. I know I’m late to the part, but there’s a few bits that I’m still struggling to wrap my head around:

    In the scope of an RPG, how would things like items be handled? For example, if we have a potion that restores health, is this an entity, or an assemblage that is instantiated when a character picks it up? I think it’s the latter, but I’m still not sure.

    I imagine when the game loads its content, it would load a list of definitions for various items, and then create them as required. So a character would have an InventoryComponent object that would just list the ID of any object entities.

    Basically, I think I’m having difficulty visualing the line between entities that do something (monsters, treasure chests, event trigger zones) and entities that describe something (like potions, swords etc).

    I think I answered my own question, but any advice would be very welcome.

  2. Obviously, this depends on the rest of your game – it depends on HOW MUCH DETAIL you choose to put into your different systems.

    Off the top of my head, I’d start with:
    – entity, made up of:
    – position component
    – one-use-special-action component, with an enum type with value of “HEAL_POTION”
    – inventory component (as in: when it’s in your inventory, it has a component that gives a bitmap you can display, and says how much inventory space it takes up, is it stackable, etc)

    Although, in some RPG’s, the potions design is quite complex, so you might well choose to split-out the “potion” bit into mulitple components, e.g. a single “item-use” component, speciying that it’s “one-use” and has “instant-effect” PLUS a set of “item-effect-component” instances:
    – alter-stats component, with values “HEALTH” and “50”
    – alter-stats componeont, with values “STAMINA-PERCENT” and “100”
    – alter-stats-temporary componeont, with values “MAX-HEALTH” and “10” and a variable “Fade-time” of value “60 seconds”
    – …etc

    (As much as is necessary to make the implementation of YOUR game’s particular potions-logic easier to manage…)

    In an RPG, that split of components also starts to make it REALLY easy to create – say – a new magic sword that has a side effect of increaing your max-health; it’s actually re-using the componone from a heal potion.

  3. @Nash – your question in your second-to-last paragraph suggests:

    “I’m thinking too much”

    Don’t. There is no such difference. Entities are just data. Nothing more. There is no “do something” part to an entity. You are thinking of OOP. Stop thinking of OOP. ES’s want your life to be simpler … don’t complicate matters, just let them be as simple as they possibly can be :).

  4. Thanks for taking the time to reply, I really appreciate it.

    These systems always look so elegant on paper, but I’ve always struggled with the implementation. I’ll take the simple approach and see what I come up with :)

  5. it ‘s same that you just talk about the store and initialize the entities,but how those entities interact with each other? does you use the system based on message or something else?

  6. @Neo

    Think of an entity as a dictionary of arrays (each array being a component). In most programming languages, implemented efficiently, that’s all it is.

    A dictionary doesn’t “interact” with anything, it just sits there. You write other code that gets given refecnes to multiple dictionaries, and operates upon them. Your “other code” is the System(s). The entities just sit there, doing nothing.

  7. thanks for replay, you said just “other code” do the interactivity.but
    1.how did the system work?
    2.is every component has a specific system for it or there is a public system for all components to handle the interactivity.
    3.how does the system collaborate with each other?

    and i have other more questions:
    1.how to reuse the assemblage?
    2.how to reuse the components as we want to make one new component with the old one?

  8. “If you know the entity-id, you may only need one table lookup to get the data for an entire component”

    I have a work in progress ES based off of what I learned from your article. I am trying to make a function that loads an entity from the database including all its component data. My question is, can you give an example of how to do this with the least number of queries?

    I have a table for each component’s data, with the table name being in the “component” table. Right now I have it so it does one query to get a list of the component IDs and their table names, and then it does a query for each component table to get all the data.

  9. You’re into the realms of SQL optimization with that question…

    My flippant comment was assuming that you chose a deterministic naming scheme for your components, so that e.g. for component “Position” you knew (without having to look it up) that the table name would be “CPosition” (or maybe “Component_Position” or whatever).

    Component-class names are going to be unique anyway (whether you “name” them with strings, or with integers, or some other scheme) – so you should in theory be able to invent a DB schema that avoids looking up the table-name for each component.

  10. Hi Adam,

    first of all, fantastic post. My profession is in developing web applications with database backends, which I’ve been doing for a while now (too long perhaps?) About a year ago I started my first serious effort at game development. I’m working in C# and XNA, and I intend to release the game under Xbox Indie Games (but just a single player experience, not an mmo).

    Anyway, I’ve been attempting to research as much as I can regarding entity/component systems. In addition to your article, other articles I consider ‘top articles’ are: Mick West: Cowboy Programing – Evolve Your Hierarchy and Marcin Chady: Theory and Practice of Game Object Component Architecture (GCD Canada 2009).

    Chady’s design uses the terminology “Behavior” for the logic part of the Entity, and “Attribute” for the data part of the Entity. I’ve adopted this design approach in my game. In my model, an Entity is represented as an EntityKey with a unique value. Each Behavior has an associated list of EntityKeys. For an Entity to adopt a certain Behavior, it registers its EntityKey with the given Behavior, adding it to the associated Behavior-EntityKey list. When processing its EntityKey list, the Behavior’s logic can reference one or more Attributes.

    Taking inspiration from your article, I’m treating Attributes as records in a list (emulating something like a database table). The Attribute lists are keyed with an EntityKey. An When the Entity Factory creates an Entity, it looks at the Behaviors the Entity needs and then creates ‘records’ in the appropriate Attribute lists.

    Also taking inspiration from your article, I looked at using LINQ queries on my Attribute lists. The version of C# used in XNA supports LINQ-to-Object queries. Apparently LINQ queries are too expensive to use in the game loop, but I have experimented with using LINQ queries for initializing level data behind a ‘loading’ screen.

    Coming from the domain-driven-design philosophy we employ at work, I have run into a quandary in the above entity/component design. I must first mention that I am by no means an expert in software engineering. And I find domain-driven-design is not a panacea, and it may very well be ill-fitted for designing game architecture. Martin Fowler’s ‘anemic domain model’ makes an argument that, in migrating all the logic out of the entity, we are loosing (all) benefits of object oriented design. Though I do see myself continuing with the above entity/component design, the trade-off in separating logic from data makes me wonder about the nature of steering away from object oriented design in the implementation of game entities.

    I guess there was no real question in my post, just wanted to say keep up the great work, and I found your article very enlightening.

  11. Hi Adam,

    I really enjoyed this series of articles and I did go through them a few times. It’s interesting to note that the comments and your answers really add a lot of value to the posts!

    I think a lot of us would love to read the next article :)

    I’m starting to write a text based mmorpg on my free time (a bit different from my .net consulting day job but it’s fun!) and I’m really excited about implementing an entity system that will be used by the server C#. The backend storage is MSSQL.

    1) One of the thing that I cannot yet get my head around is how to respond to individual network requests efficiently using the ES. I think I understand how to system process entities in batches inside the server game loop but this seems a bit different.

    What I mean is that when I get a request from a client saying Entity X is attacking Entity Y, I might have to do security checks, procces the fight and then send back a lot of info like the outcome, XP gained, eventual loot, remaining health, level up info, …

    I may have multiple of these requests per seconds and I’m afraid that since most of this data might be scattered in a lot of different components, it might be slow. Do you see a single System in charge handling these requests, performing the action and returning the results + the additional data. I’m afraid that’s a lot of responsibility for just one system.

    2) If I query/update mssql in realtime, I don’t have to worry too much about multithreading but if decide to have in-memory representation of the data, more challenges appear: synchronize access to the data (game loop systems vs user requests), consistency and also probably persistence.

    3) I’m not very clear on how I can best use the ES to implement of containers (inventory has bags, bags have items, chest contains loot, …), do you have any suggestion that could get me started?

    4) Any objections about having some systems running stored procedures to update some data directly in the DB? This could possibly prevent fetching + storing + saving.

  12. Hi Adam,
    The articles are great. I am looking forward for the next article – hopefully you will give us some more information about AAA performance and use cases.
    Meanwhile, I’m wondering about the implementation of a collision detection system.
    We will need a:
    collisionDetectionSystem
    collisionDetectionComponent (x,y, deltaX, deltaY, velocity, radius etc)
    Entity
    Let’s say that we have 1000 entities, all of them are bind to collisionDetectionComponents.
    Assuming that we need to perform a simple sphere test, how would you suggest of checking if there was any collision between the objects?
    IMO, since the entities are stored in a list, we will need to check them against each other (1000*1000) – which is bad.
    Another suggestion is that the collisionDetectionSystem will have a 2-3D array/hash table for the X.Y,Z and on-top have linked list to hold more then 1 entity on the same X,Y,Z (array[x][y][z]->entitiy_1->entitiy_2)
    Now, we can check for collision only on specific areas and reduce the amount of checks.
    Any idea?
    Thanks,
    Eldad

  13. @Eldad

    Just use any standard CD system, anything you find on google. The ES doesn’t change the algorithms you use, it just changes how the data is stored and fetched

  14. @Adam
    I asked that question in order to get a better understanding on how components interact with each other and how to deal with systems that need to consolidate the component data for better performance.

    Problem:
    Let’s image that we have 4 components that need to interact with each other:
    renderComponent – render enmities.
    invisableComponent – if the entity is on the move, its visible, else invisible.
    collisionDetectionComponent – check collision between entities
    movementComponent – used to me the entity

    In other words, the interaction between the components is:
    movementComponent –> invisibleComponent -> renderComponent
    -> collisionDetectionComponent

    In addition, the collisionDetectionComponent will have better performance if it will hold the position of his entities rather then comparing each entity to each other.

    Solution I:
    http://cmpmedia.vo.llnwd.net/o1/vault/gdccanada09/slides/marcinchadyGDCCanada.ppt
    (Slides #17 and #21) A solution can be messaging between the entities; we send a message to entity and let it decide how to deal with it.
    This will get dispatched to all systems that are interested in attach messages.
    void HealthBehaviour::OnMessage(Message* m)
    {
    switch (m.type)
    {
    case APPLY_DAMAGE:
    Attribute* healthAttr = GetAttribute(HEALTH_KEY);
    healthAttr->value -= m.damage;
    if (healthAttr->value SetLogicState(DEAD);
    break;

    case ATTR_UPDATED:
    if (m.key == HEALTH_KEY)
    {
    Attribute* healthAttr = GetAttribute(HEALTH_KEY);
    if (healthAttr->value SetLogicState(DEAD);
    }
    break;
    }
    }

    Solution II:
    (Slide #20) Another solution is a onUpdate( pass, delta ) function on each system, every time that a component get changed, the previews and current will be sent to the different systems (that have component attachment between the entity and system):

    GameObject::OnUpdate(pass, delta)
    for b in behaviours
    b.OnUpdate(pass, delta)

    I can see some problems by using those techniques:
    1. Performance issue
    2. The systems behaviors should be built with consideration with other system – something that we shouldn’t do in ES.

    Any idea what is the best approach for the problem described above?

  15. @Adam, please ignore the previous post, it got submitted by accident.

    I asked that question in order to get a better understanding on how components interact with each other and how to deal with systems that need to consolidate the component data for better performance.

    Problem:
    Let’s imagine that we have 4 components that need to interact with each other:
    renderComponent – render entities.
    invisableComponent – if the entity is on the move, its visible, else invisible.
    collisionDetectionComponent – check collision between entities
    movementComponent – used to move entitles.

    In other words, the interactions between the components are:
    movementComponent –> invisibleComponent -> renderComponent
    -> collisionDetectionComponent

    In addition, the collisionDetectionSystem will have better performance if it will hold the position of his entities and compare only those that close enough – rather then comparing each entity positionComponent to the other entities(n^n).

    Solution I:
    http://cmpmedia.vo.llnwd.net/o1/vault/gdccanada09/slides/marcinchadyGDCCanada.ppt
    (Slides #17 and #21) A solution can be messaging between the entities; we send a message to entity and let it decide how to deal with it.
    This will get dispatched to all systems that are interested in attach messages.
    void HealthBehaviour::OnMessage(Message* m)
    {
    switch (m.type)
    {
    case APPLY_DAMAGE:
    Attribute* healthAttr = GetAttribute(HEALTH_KEY);
    healthAttr->value -= m.damage;
    if (healthAttr->value SetLogicState(DEAD);
    break;

    case ATTR_UPDATED:
    if (m.key == HEALTH_KEY)
    {
    Attribute* healthAttr = GetAttribute(HEALTH_KEY);
    if (healthAttr->value SetLogicState(DEAD);
    }
    break;
    }
    }

    Solution II:
    (Slide #20) Another solution is a onUpdate( pass, delta ) function on each system, every time that a component get changed, the previews and current will be sent to the different systems (that have component attachment between the entity and system):

    GameObject::OnUpdate(pass, delta)
    for b in behaviours
    b.OnUpdate(pass, delta)

    I can see some problems by using those techniques:
    1. Performance issue
    2. The different systems should be able to interact with other system – something that we shouldn’t do in ES, each system should be independent.
    3. Can cause to infinity recursion.

    Any idea what is the best approach for the problem described above?

  16. Neither – that’s an incompatible design, that loses most of the benefits of the es I’ve described. Entities are data only.

  17. Why exactly do you feel that ES and Event-Driven messaging systems are incompatible? It seems to me that by using messages to handle the communications between multiple Systems/Components (in case your ES implementation allows Components to have methods, like described below), you abstract from the identity of the other Components of an Entity, and just send messages to anyone interested in them, and receive messages of the types you registered for. This results in truly individual Components which can easily be swapped out for others with the same functionality but with, for example, different attributes.

    In the slides linked above by Eldad, Components have data AND methods, namely onUpdate() and onMessage(). You would use the onUpdate() to do tasks that are required in each frame, something like Systems would do by applying operations on all their Components. You would use the onMessage() function to handle messages you registered for, you don’t care where they come from and are thus independent of the sender Component/System.

    Or what about defining multiple Interfaces for Components, and querying Entities for a Component that implements a certain Interface?

    One other question: like you mentioned, a good editor with support for adding Components to Entities is crucial to the succes of using an ES in a project. Do you know of any such editors? I am looking for one for a 2D game, and I would like to be able to visually build up my levels out of Entities and give them certain Components and place them in the game world.

  18. @adam
    I’ll appreciate if you could explain how to build a “collision detection” system and a “movement” system in the same application in the ES way.
    I feel like its the only missing link for me.

    Thanks,
    Eldad.

  19. @Jacob

    If components can received messages, then the ES is an over-complicated way of doing OOP – and you should just stop, and do normal OOP instead. There’s nothing wrong with that – but you’d lose many of the benefits I’ve previously described.

    A lot of the benefits REQUIRE that components are pure data; the moment you let them “receive messages”, those benefits are literally impossible.

  20. @Eldad

    Start again from the beginning. Components are just data – nothing more, nothing less.

    Re-think how you would approach the algorithms. You need to internalise this stuff, or you will never be able to understand your own ES. It’s not easy if you’re used to OOP – but when you get used to it, it’s very easy.

  21. @Jacob:

    The commercial engine Gamebryo LightSpeed has/had the kind of design you’re advocating for its entity system. It works, more or less, but as Adam states, it doesn’t have several of the benefits a “pure” entity system is supposed to bring. Unity3D also has an entity system that reminds me of the approach you’re suggesting, although I think there are some subtle differences.

    I think what did the most to sway me to this line of thinking, beyond lurking on t-machine, was reading Tony Albrecht, particularly “Pitfalls of Object Oriented Programming” – which is a bit PS3 specific. Or Noel Llopis, whose columns in Game Developer are repeated & expanded on his blog at gamesfromwithin.com. I’m hoping that my next job will give me a chance to work with (or, more likely, write) a purer entity system.

  22. @adam:

    Exactly WHAT benefits do you lose with the approach I described?

    Would an alternative be sending the messages to the Systems with a certain ComponentID as parameter? This should still have the benefits of being able to create any kind of objects you want by composing them of Components, without getting a huge hierarchy or bloated classes. And what do you think of the idea about components implementing specific interfaces like I mentioned?

  23. Sorry I don’t have time right now to go and dig out examples for you. There’s lots that have been hilighted and discussed in these many pages of blog posts and comments.

  24. @Adam,
    I never said that my component won’t be pure data.
    I was talking about system additional functionality (entitysystem).
    To be more specific, I was talking about systems that need to be sync in some way (“collision detection” and “movable”).
    In order to have a functional CD system – it should be affected by the movable system.
    How can we achieve that?

    I hope that my question is clear.

  25. @Eldad

    Here’s what made me think you were doing code + data:

    “invisableComponent – if the entity is on the move, its visible, else invisible.”

    Well, that’s not a data component. That’s a piece of code.

    “collisionDetectionComponent – check collision between entities”

    Ditto.

    “movementComponent – used to move entitles.”

    Possibly ditto – the “used to move entities” suggests it has some code?

  26. @eldad

    Say you have:

    Position: { x, y, z, }
    Movement: {dx, dy, dz}
    CollisionInfo: { collisionClass, collisionVolume }

    You might implement a collisionDetectionSystem that each frame iterates over every entity which possesess those three components, and calculates:

    SweptVolume = sweep( collisionVolume, x,y,z, x+dx, y+dy, z+dz )

    …then runs a CD across all the swept volumes, using the collisionClass to e.g. make somem objects not collide with other specific types (maybe a player avatar doesn’t collide with player’s own gunshots, for instance)

  27. @Eldad

    …but optimising this is the same as optimizing any CD in any game.

    e.g. you might do an AABB pre-filter. You could, if you choose, supplement your CollisionInfo component with an AABB var.

    Or add a new component:

    CollisionAABB: { x,y,z, width, height, length }

    (I’d probably do that – so if you change to a CD that isn’t AABB-based, you wouldn’t need to change your main ColliosnINfo compononet at all)

    if you defined your AABB boxes as relative to the object origin in Position component, your CD system could cache the AABB tree, and each frame use the values in Movement to modify the cached copy instead of rebuilding the entire tree from scratch

    …but agian: this is all bog-standard CD. ES hasn’t changed the algorithm. And there are better CD approaches out there, but they shoudl be easy to port across.

  28. OK, I got my answer.
    Basically what your saying is that systems can share data (components).
    in addition we can have systems that uses 1 component while others can use more – Right?

    Do you think that systems should/can have pub/sub or observer implementation?
    For example, if you have “Movement” system, it can publish messages onMove and onStopMove etc.

    Eldad

  29. Yep.

    Re: pub/sub – I love that paradigm, but it doesn’t work well on modern computers. It requires edge-driven CPUs, not clocked CPUs.

    You can fake it in clocked fashion, by havnig a set of components that just contain deltas, that only exist when they’re non-zero. e.g.

    1. During one tick: each system creates + add a bunch of deltas
    2. During next tick: some systems read the delta components, then delete them at end of tick

    In general, it’s often easier to re-write your logic to be non-edge-triggered. e.g. instead of “onStop”, do:

    – “iterate over all movement components for entities that have the EnemyShip component”
    – “for any of those that have a value of 0, look at their associated delta component, and see if it’s non-zero (i.e. it moved in the last frame”
    – “…do the processing…”

    this approach has no internal dependencies, so it’s a lot easier to debug IME

  30. Adam,
    I can think of another approach for implementing that, I want to know what you think of that.
    Instead of having an even-odd ticks we can have an hierarchy of of systems, each level will have its “own” tick.
    The first level will have only independent systems, that mean that the system can make calculation only by their own components.
    The next levels will require components from the previews ticks and not just their own.
    When the last level will end – all the deltas will be removed.

    for example, we have the following components/systems:
    1st level:
    Position: { x, y, z, }
    Clothing: {shirt:x, pants:y }
    2nd level:
    Movement: {dx, dy, dz, velocity }
    3rd level:
    CollisionDetection { collisionClass, collisionVolume }
    last level:
    Render

    so as you can see, if a unit need to move, the Movement system will set a new dPosition, then it will be checked by the CollisionDetection (on error, remove the dPosition) and finally it will be render to the screen.

    What do think of that?

    BTW, I didn’t understand how you will implement the Collision Detection system. its seems like you will need to to compare all the entities position using your algorithm N^N.
    I think that it will be better to store an associate array/vector/hash map for location->entity list, that way when you check for collision you can do it VS units that are close enough.
    Any suggestion on that one?

  31. @adam

    When using an ES in an *offline* game (in my case for iOS devices), are there still any advantages to using a database to store the entities, components and systems? I understand they are the most natural fit, and I also understand the advantages they bring for an MMO game, but as you mentioned in one of your (excellent) posts, hundreds/thousands of queries are probably way too slow for home PC’s, and mobile devices even more so?

  32. @Jacob I wrote an Objective-C version you can download from github – c.f. http://entity-systems.wikidot.com/es-approaches, which doesn’t use a DB, but might help if you’ve not started yet.

    There’s also a blog-post I did on ES with Android – works fine for real-time action games. The limiting factor was the JVM’s poor implementation of HashMap – which I’d *expect* an in-memory database to avoid (using a much more clever / cunning data structure, probably with ByteBuffers to do it in contiguous memory areas, etc).

    I would certainly experiment with an in-mem SQLDB if I were you – but please report back with your results :)

  33. Hey Adam, interesting article series, can’t wait for part 6 (nudge).

    Let’s say I’ve drunk the Kool-Aid and have an ES system that uses only relational queries to fetch data in an MMO. Spiffy.

    Simplest example, I want to add 5 to X coordinate of every entity that has a Position component, and I want to do this every simulated frame. I guess we’re playing “Conveyer Belt Online (TM)” or something. Additionally, let’s subscribe to the design that an entity may have only a single instance of a component associated with it.

    In practice, we should just write an SQL statement to update every row by adding 5 to the X position. Okay, fine, but when we move to non-trivial examples, this isn’t appropriate, so let’s assume you do your logic in C++ and let the DB do the fetching.

    What would the expected approach be here? Fetch all rows in CPosition table, modify the value, update the table? That would take 1 query to fetch all of them (sounds good), a fast C++ loop, then N ‘update … set ‘ queries. That seems really slow – is this really how it should be done?


    A different question: You said that writing back to the DB that you initially queried would be much too slow, and sure, I understand why that might be true. Your suggestion was that another database should be used to write to, and that it could even be a different vendor’s DB, etc.
    Now what the heck do you do when you want to run the same query as last time? It seems like it will just fetch the old values. How does one properly handle this “write to a different DB” situation.

  34. WordPress authors are stupid bastards sometimes. Their comment form is “ooh, how clever we are” AJAX – but they hacked the web browser to do it so badly that if you mis click while typing, WordPress deletes your entire comment. Stupid, stupid, stupid.

    So, briefly, SECODN TIME: in practice, don’t expect to use SQL, just understand that’s the theoretically perfect way of doing it. SQL drivers may optimize writes in clever ways – make no assumptions. There’s always stored procedures too; could be a bizarre and stupid approach, could be genius.

  35. One of the advantages of such an entity system would be, as you mention in the articles, that iterating over all components of a system can be done really fast if they are laid out consecutively in memory. However, this won’t be the case because different systems need to reference multiple components!

    Example: both the rendering and audio system need postion information, along with a bunch of other systems. Where do you store this position information? There are 3 possibilities really:
    1) Store in both the renderable and audible components. This causes duplication of data and issues with keeping them in sync.
    2) Store in one of the components. Then the other components that also need this data must either have a reference to it or query for this information through some kind of messaging system, which complicates things and is probably too slow.
    3) Store in a separate Positionable component. This would be the cleanest solution I think, however now both the Renderable and Audible components need references to this Positionable component, or another way to access it.

    The point is, no matter which solution you choose, components will always need to refer to other components (or their respective systems do), and this makes it impossible to lay out components consecutively in memory for every system that needs to iterate over them. Also, because this kind of communication will always be nessecary, utilizing parallelization by putting each system in a separate thread is not as easy as you make it sound.

    Any thoughts on this?

  36. Well, for starters … where components are being accessed read-only, there’s no reason why there has to be only *one* copy of them in-memory…

    But the wider point is simply: you have enormous flexibility here to implement any of many optimization schemes. The data is now independent, homogeneous, and indexed. That’s about as good as it gets.

  37. Thanks for the quick reply.

    My point is that it’s not independent, as I tried to explain. Also, position data is not being accessed read-only, for example a Moveable component will need to modify it, as well as a physics component. Could you name a few of those many optimization schemes you are talking about?

  38. A given system will often access a given component read-only – something you know at design-time.

    The data in one component is independent of the data in other components from the same entity – instead of all being in a single OOP object.

    Re: optimizations … this is core Computer Science, so any book on algorithm optimization, memory optimization, CPU optimization, etc. Compiler design, maybe? CPU design (especially multicore? Operating Systems (especially kernel-design and thread scheduling)? I’m sure there are good websites too, but I don’t know them myself.

  39. Ok, another example:
    A bullet that travels through the air has a moveable and positionable component. The PhysicsSystem then takes the deltas in the moveable component and adds them to the absolute values in the positionable component. This results in accessing different memory locations for those 2 components. Putting these 2 components after eachother in memory is possible, but what if there are 2 other systems A and B, which both need their own component together with these 2? Do you put moveable, positionable and A after eachother in memory, or moveable, positionable and B? No matter which you choose, either system A or B won’t have the components it needs available consecutively in memory.

    This is only a very simple example with perhaps negligible overhead, but I hope my point is clear.

  40. Good day. Your articles is a fresh air to my dimmed by OOP mind. I have a question about ES concept. How entities can be relate one to another? For example, I have an (adventurer) entities with inventory component, and a (sword) entity with item component. Can inventory component store GUID of entity with item component? Or there can be another way to implement this relation.

  41. @Axis
    Think of it from a different approach. Here’s a OOP (and physical reality) setup:
    Adventurers have inventories which contain more entities

    But using this system you need to use components to describe everything. And if you’re adding multiple copies of the same component to an entity, you might want to approach it from the other end.

    Here is a simplistic setup:
    Sword has a PickUpableComponent with two values:
    Weight, ContainerId

    Adventurer has a HasInventoryComponent:
    MaxInventoryWeight, MaxInventorySize

    then we have a PickUpItem System which:
    When Adventurer makes contact with something with a PickUpableComponent:
    1. checks to see if the Adventurer’s MaxInventoryWeight can fit the Sword’s weight
    2. checks MaxInventorySize…
    3. then adds the item to the inventory by adding the component: InInventoryComponent which contains:
    InventoryOffsetX, InventoryOffsetY, InBagNumber, etc…

    You could either remove the PickupableComponent (and store Weight in the InInventoryComponent) or keep both

    You find the list of items in your inventory by simply retrieving all entities with the InInventoryComponent

    You’ll see quickly that components multiply fast, but that’s fine.

    adam, hope i did it credit.

  42. Clarification on previous comment:
    If you have only one thing that can have an inventory (normal 1-player game), you don’t need ContainerId in InInventoryComponent.

    If you have multiple inventories, you could put either the guid of the entity or a unique identifier of the container (stored in HasInventoryComponent)

    The idea being that the adventurer doesn’t know he has something (that requires one to many). It’s that the sword knows that it’s had by something (many to one). Which is my understanding of how it would optimally work.

  43. @PsionMage
    Thanks a lot for the answer! Now I see how it works.
    I trying to apply entity system’s approach to a non game project with tricky and complex data structures. This structures changes all time and ES may dramatically simplify development process.

  44. A question regarding the “only data” part of components:

    I have a component called “Location”, which represents the current position of an entity. I now have two ways of representing the position:

    Either, I add three int fields (x, y, z) to the component. This way the component is pure data. Or, can I use the data type “Point” (struct) to represents the position? This is a complex data type, which somewhat violates the “pure data” view, because it does contain (static) methods.

    Now, do I use x,y,z and “upgrade” those to Point whenever needed, or would you think it OK to use a more complex data type?

    From my point of view, using Point doesn’t add any code to the component, however the data type itself does contain methods (in regards to that same data type).

    Speaking of .NET, the Int32 data type is also a struct and +, -, … etc. are static methods… so technically, pure data isn’t even possible.

  45. @Wildfire

    Put Point in there if you like. Just be aware that it might make netwrokgin code harder to write later on, and that you might be losing some performance improvements (I don’t know what your Point object is, under the hood – a struct is (usually) by definition “only data”)

    It’s the spirit of the thing you want to stick with – if your language/platform doesn’t support low-level efficient storage, then that’s fine. Just make sure *you* don’t code any methods/logic/etc into the things you put in your Components.

    Although … if your language *does* support this stuff … use it. Store as x/y/z and pull-out a Point object when you need it. This is one of the cases where I like to use a sprinkling of OOP (not recommended unless you’re *absolutely* confident you’re not abusing it): not inside the core, but as icing that makes the code shorter to write.

    e.g. make an Entity class, give it an entity-ID, and stick a method “locationPoint” on there. The class is never used at runtime except as a convenient place to store convenience code, i.e. you’d end up:

    MyEntity = new MyEntity( id ) // this object is thrown away immediately at end of this loop
    MyPoint = MyEntity.locationPoint(); // impl: “return new Point( EntityManager.getComponent( Location, self.id ).x, EntityManager.getComponent( Location, self.id ).y, EntityManager.getComponent( Location, self.id ).z )”

  46. @adam

    Thanks for the answer. Good point about networking, I was thinking something similar in regards to storage. In the worst case each complex type needs custom code for serialization, wheres built-in types are supported by the language itself.

    The more I think about it, the better it seems to go with x/y/z. Structs like Point and Vector add a lot of convenience, however not every system might need that. And conversion, if needed, is trivial.

  47. I really like your idea of “pure” entity systems, and while my game is very small I would like to use it.

    Now implementing the entity manager is the easy part, but I got stuck at two things:
    1) Getting a component to update another component.
    2) Getting a custom script to run on a specific component in order to update it.

    I am trying to make a user input component that states that the entity listens to events such as keyboard and mouse presses.
    Now, I want to give my player this component, with a script that will tell it what to do when an event does occur (say “set velocity to something if you pressed WASD”).
    What I do right now is to give a callback to the user input component when I create it. Then when the system runs, it calls the callback for each existing user input component.
    This seems to be very hack-ish and not a good structure.

    Now getting components to talk to each other – I do not care about efficiency at the moment, so I want to have the messaging method you described previously – sending a message to all the components owned by an entity, and if they are not listening to that message they will just silently ignore it.

    I implemented this by adding a method to the entity manager called getEntityFromComponent (I think the signature describes what it does good enough :) ).
    Every time I want to send a message from a component, I get its owner from the entity manager, then I get all the components owned by the entity, and finally I loop over them and call their “onmessage” method, which is the callback I assigned them at construction time I was talking about before.

    This again doesn’t look to me like a very good design. It’s a bit odd that a component needs to go through the entity manager to get messages to its brothers.

    To summarize, I want to be able to listen to user input, and update appropriate components (position, which will in turn update rendering) in a better structured way, preferably with custom scripts for components rather then hard-coded responses to messages.

    Do you have perhaps a better way to implement these two things?

    Thanks.

  48. You’ve broken two of the cardinal rules/guidelines:

    1. Adding methods to your components

    2. Message-passing between components

    Remove that, re-think “how else could I achieve the same outcome, without breaking those rules?”, and try again. I think you’ll find it becomes much easier/cleaner.

    (lots of discussion of this in the comments already – you might want to read the complete comments to all 5 entity systems posts)

Leave a Reply

Your email address will not be published. Required fields are marked *