Entity System: RDBMS-Beta (a new example with source)

I’ve just added a new, improved Entity System design to the wiki. I’ve also created a github project for it where I’m about to check-in working source.

The only source I’ve provided is Java – but it’s an evolution upon the old “RDBMS with Code in Systems” type, that already has full source for Java and Objective-C – so you could easily upgrade the obj-c version to this new type (hint, hint).

What’s new?

Three major changes:

  1. Included a bunch of sensible java-specific improvements, based on the forks other people had done of the previous Java implementation
  2. Added a bunch of FANTASTICALLY USEFUL methods to the EntityManager – things you definitely need when writing a game
  3. Added a new concept: the Meta Entity

The first one is obvious, the second is obvious when you look at what’s added (e.g. a “removeComponent” method; why wasn’t it there originally? Solely because it wasn’t necessary to show “the simplest possible implementation”).

The interesting one is number three: the Meta Entity

What’s a MetaEntity?

A while back, in the first Java/Android ES I wrote about, I mentioned that I was using a fake Entity class.

I made a big fuss about how this was NOT being used to store any game data – but was instead an OOP trick to make it easier to write code.

Thinking about this later, I realised that there was really no need for that class to be named “Entity” – and calling it that ran the risk of tempting people into using OOP for their core data / components (Remember: Never do that!). Instead, looking back at a couple of example ES’s I’d written, I saw that every method – and all data – in this class was a “meta method”.

For the methods, you need to read the source AND the javadocs to see this.

For the data, it’s a lot more obvious: the only data that a MetaEntity has is:

  • The Entity itself (remember: that’s just an integer – or a UUID object in Java, which is a cleaner Java-specific way of handling it)
  • The EntityManager object which created this Entity, and which contains the Components for that Entity, and … basically provides all access to the data etc

i.e. if you pass around MetaEntity objects, you can feel safe that you know where they came from, where their components are, etc.

Because when you pass around raw Entity’s, they’re just an integer – which makes it easy to create an Entity in one EntityManager, then accidentally start using it in another EntityManager. Technically that’s illegal – but from a compiler perspective, it’s 100% legal … so you’ll get bizarre runtime bugs. Ouch.

Equally, the MetaEntity can contain a lot of single-argument and zero-argument versions of methods that exist in the EntityManager as 2-argument, or 1-argument methods. This greatly reduces typing, increases readability, and reduces opportunities for bugs. It may sound like a very small change (2 arguments dropping to 1), but I promise you: in practice, it makes a big difference.

Why not use MetaEntity’s all the time?

They’re very inefficient in how they use memory, and they throw away many (but not all) of the performance advantages of an ES.

For instance, because you’re moving from “integer + array of structs” to “linked-list of objects”, you’re making your game go from “cache friendly” to “cache poor”, and making your MINIMUM mem usage go from “tiny” to “small”.

In practice … if those differences matter to you, you’re probably writing a hardcore custom ES anyway.

More importantly: even in a “hardcore” ES, you don’t actually *need* that performance all the time. If you’re just pulling out a handful of Entities and their Components – e.g. a player Entity (of which there’s only 1 – or a few for multiplayer) – then the above performance differences are NON EXISTENT (vanishingly small).

…but the code is more readable, easier to maintain, and more robust.

So. I recommend using the MetaEntity for *all* your ES coding … until you reach a point where performance is low. Then look at re-coding *only* the affected loops / methods (remember: you can do this on a System-by-System basis; your code is nicely separated already!). That way, you get the best of both worlds.

However … the reason none of this was included in the first Java ES I posted onto the wiki – the ultra-simple “RDBMS-inspired with Code in Systems” – is that really this is all just gloss on top of an ES. You don’t need it. I believe it makes your ES easier to work with – but it distracts from the core theory.

I’d recommend you start with the simpler ES, and understand it, before moving onto something like this for your practical work.

For more info, check out the wikidot link at the top of this post – and try the github project linked from it (currently empty, until I do a check-in)

Did this post help you?

Support me on Patreon, writing about Entity Systems and sharing tech demos and code examples

79 thoughts on “Entity System: RDBMS-Beta (a new example with source)

  1. Mike Leahy

    @Rafael

    Indeed. How things are accomplished with my efforts is that there is a runtime layer and one of the runtime systems is an input converter. The input converter will wire up to whatever the native platform provides and efficiently converts events to a platform neutral event. When writing apps with my efforts you attach to the runtime input converter and receive neutral events. For J2SE I’m mostly using JInput, but will provide an AWTEvent converter too. One has to keep in mind though that this works great for full screen / window apps, but a little less so if you are say doing an extensive Swing or even writing custom UI Android components via View, etc. I will be providing an essential 2D scene graph for GUIs via OpenGL ES and handling input and such.

    With Android the input converter is an Application level runtime system that initializes on startup and persists between the Activity lifecycle, so when a new Activity starts one can immediately connect to the input converter and receive events. Likewise with sensors, etc.

    For graphics OpenGL/ES is compatible across J2SE and Android and is the best path to pursue for cross-platform efforts. For sound in a similar fashion as input in my efforts there is a general runtime system that will delegate to the native sound APIs as necessary, but an app interfaces with the middleware layer. Likewise with image handling there is a runtime system in my efforts that is platform neutral for loading images and such.

    So as things go my effort provides a runtime layer providing many of the services one would normally go to the native API for access. The benefit of providing system interfaces for this is that the underlying implementation can handle the platform specific details.

  2. .rafael

    *** This comment is now using the Adam’s ES:

    Ok, let’s one fast example…
    Keep in mind that I’m the Button and ToggleButton as example for UI. But you can put in a game if you want as a emergency button or a swith that is on/off, if you want

    I have two scenes:

    P.S.: I’m represting the entity-id with “quotes” followed by { Components }

    SCENE_A:
    “play_button” { CButton.action = “play” }
    “sound_toggle” { CToggleable.action = “sound” }
    “music_toggle” { CToggleable.action = “music” }
    “exit_button” { CButton.action = “exit” }

    SCENE_B:
    “another_button” { CButton.action = “anotherb” }
    “another_toggle” { CToggleable.action = “anothert” }

    These are my Componentes data:
    CToggleable
    + toggled : boolean
    + action : String

    CButton
    + action : String

    —-

    Then I have:

    SceneAController ( CToggleable.toggled = !CToggleable.toggled
    // –> Create an ActionEvent with ActionEvent.action = CToggleable.action
    // –> Send the ActionEvent to current controller parent (or SceneAController or SceneBController)
    // else do nothing.
    }

    ButtonSubsystem
    {
    // when the bounds of entity (by CPosition) is collided with touch (x, y)), then:
    // –> Create an ActionEvent with ActionEvent.action = CButton.action
    // –> Send the ActionEvent to current controller parent (or SceneAController or SceneBController)
    // else do nothing.
    }

    If I have a SCENE_C with:

    P.S.: “entity-id” { Components … }

    SCENE_C (game scene)
    “alarm_button” { CButton.action = “start the alarm” }
    “switch_toggle” { CToggleable.action = “toggle the elevator up-down” }

    I can still use the ButtonSubsystem and ToggleButtonSubsystem.
    They are going to be created (together with entities and components) in the SceneCController.
    So the parent of ButtonSubsystem and ToggleButtonSubsystem is now the SceneCController that has the ActionEventListener and will know what do do in the action.
    And I will not to have everytime make the CTogglable.toggle != CToggable.toggle (in every susbsystem or controller)

    I’m sending messages to the parent controller, but has any other way to maintain the Toggle and Button “generic” like this without using messages?

    What do you guys think? :)

  3. .rafael

    Keep in mind that I’m the Button and ToggleButton as example for UI = Keep in mind that I’m t__using__ he Button and ToggleButton as example for UI

  4. Mike Leahy

    @Rafael
    >What do you guys think? :)

    Erg… :) Hard to say per se. Keep in mind that Adam’s ES framework is more of a lightweight conceptual model than a full blown multi-purpose implementation. Also your question is best served offline rather than through comments on this blog post. Unfortunately I don’t have the bandwidth to help individually at the moment and I’m guessing this may be Adam’s response too.

    I’d generally recommend not to start with a UI example, but a game example. Get that working then consider implementing a UI. I provided essential pseudo-code in a comment above for how to handle hit testing and action handling. Don’t get hung up on Adam’s vs my efforts as for the most part they are functionally fairly similar.

    Essentially you’ll need to iterate over a list of entities (your UI elements) and check for hits and respond. Consider though you really want to avoid event passing by implementation with an is-a relationship which is the standard manner of passing around events in an OOP framework. I’ve provided my essential thoughts on that above already. You mention controller systems for event distribution, but these “controller systems” really should be the mechanism that is iterating over entities and not actually a system itself per se.

    In Adam’s ES that simply is iterating over the list of entities in EntityManager.

    I’d say you just need to go out and create your app and explore the possibilities. Adam even mentioned that above. Start with a game before the UI. What you are mentioning above is a custom way of handing things that I and likely Adam can’t comment much on as things go. It’s time to get coding and exploring on your own! :)

  5. .rafael

    @Mike

    Hi Mike, no problem. I don’t want to bother you or Adam, just talk about ideas and ES related stuff.
    I already did demos and two games. I also had come with my “own” ES by now.
    I put the ToggleButton and Button as example, because the game, like I said, has the switch for elevator and a button to be pressed. I can do the UI using the same components because the behavior is the same, but I put pseudo-code to be fast and not write so much.
    My question was more focused about the event system for the buttons, if this is ok or if I’m kind of “destroying” the ES concept using something like this :)

    Thanks for all tips and comments above, by the way. I will look forward your ES when it’s ready :)

    I’m already exploring by my own since a long time, even reached this blog because it =)

  6. faris

    I’m wondering about the best performance way to implement getEntitiesWithComponents(). I know that, while there will often be a 1:1 ratio between components and their systems, there will also be plenty of cases where a system relies on a set of components (movement w/ position + velocity components, or rendering with position + mesh components, etc). Others have touched on this but I have yet to see a solid implementation.

    Also, unless I’m missing something obvious… doesn’t the use of a hashmap as your component store for entity/component mapping, doesn’t that restrict you to single component types eg, one missile launcher component on a figher jet vs two. Maybe my understanding of it in java is poor, but would something like an stl multimap be more appropriate for this?

  7. adam

    @faris

    Yes, although: that quickly starts to depend on the hardware/architecture you’re working on. How big are the caches? How much are you parallelising? etc. So, like with the “semantic implementations for different languages”, we really need people to start writing “optimized implementations for particular hardware + game”.

  8. adam

    @faris

    If a fighter jet has two missile launchers … won’t they need each a unique position? And a unique render state (if one is firing, and the other is reloading, they’ll have different animation states)?

    …at which point: they should be separate entities, right?

    One of the things with an ES is it gets you to split your stuff into smaller pieces, arranged on “functionality” rather than on “the real-world object it’s related to”. The whole OOP GameObject idea encourages many people to use one OOP Class to represent “the player’s fighter jet”, when that’s obviously wrong from game-dev terms. It would be object-hell if you did that with OOP, but with an ES … it’s comparitively effortless.

  9. Mike Leahy

    This got stupid wordy; hopefully there is some clarity in there somewhere… ;) I do think Adam added some interesting directions to pursue and faris brought up reasonable comments too.

    @adam & faris

    faris mentions:
    I’m wondering about the best performance way to implement getEntitiesWithComponents().

    Adam mentions:
    * Yes, although: that quickly starts to depend on the hardware/architecture you’re working on. … we really need people to start writing “optimized implementations for particular hardware + game”.

    In some respect this is a misdirection regarding what hardware, cache sizes, etc. etc, but the last part about this particular use case needing further work is certainly the reality. As things go Adam’s reference implementation does not take into account searching for a set of components and returning entities by one or more sets. In general the idea is neat and certainly relevant from an interface / contract set oriented lookup it is not practical particularly with Java and lets say on an Android device. If we take an Android device as the lower bounds for the discussion iterating through a list of entities checking against a set of components is abysmally slow. There are many different constraints whether external / internal that can lead to particular customized implementations, however it still is not a frame by frame operation one would want to do.

    This is particularly why I support a type ID separate of a UUID whether an int though I’m moving toward making this an extensible enum for legibility for reasons similar to the deck of cards discussion above. If you have basic information on how a particular type of entity is composed you can quickly search by a type ID rather than a raw set of components / systems. If you don’t know which type IDs match a given set of components / systems quite likely as is the case with my efforts there is external data such as XML or even an RDBMS that stores the composition for given entity types; this may even be stored in memory and can be quickly parsed. Search through this composer meta-data for matching a set of components / systems thus finding the type IDs that are relevant then finding entities by type ID is relatively a lot quicker and suitable for frame by frame usage. Just make sure any algorithm applied gracefully short circuit if there is a missing component / system. I think it’s important to be able to do this kind of search by composition of interfaces / IE a contract set, but it’s definitely more of an offline operation for the most part unless there aren’t a lot of entities or components / systems, but that’s a particular implementation constraint. There are many trade offs in that regard as Adam began to point out.

    ———–

    faris mentions:
    *Also, unless I’m missing something obvious… doesn’t the use of a hashmap as your component store for entity/component mapping, doesn’t that restrict you to single component types eg, one missile launcher component on a figher jet vs two.

    Adam mentions:
    * If a fighter jet has two missile launchers … won’t they need each a unique position? And a unique render state (if one is firing, and the other is reloading, they’ll have different animation states)?
    …at which point: they should be separate entities, right?

    I agree in potentially separating responsibilities along reasonable entity boundaries, but this is a slight dodge as well given that this is a soft spot for you reference implementation. The 1:1 component store with your EntityManager implementation isn’t ideal per se, so @faris does bring up a valid point. There is no way to represent a “compound entity” in your reference implementation unless one create a component to hold a UUID or MetaEntity. Particularly with a fighter jet example the missile launchers are relatively positioned from the jet and the orientation is the same as the jet.

    I’m talking about my ES for the most part below unless otherwise mentioned, but you bring up some good points @faris so, I’d be glad to discuss how I try to solve the issues you brought up in my efforts.

    On your second point.. Indeed this is a current limitation of Adam’s reference ES implementation such that you can’t store more than one component type. In my efforts you can store a single component or system by type or a collection by type. I make a hard distinction between component and system. There is also a more “dangerous” way to store in my efforts one or more same types by an extended, but non-related child interface of the actual component / system being stored. It’s kind of dangerous I suppose or could be viewed as “non-clean”. An example..

    Say you have IMissileLauncher and IMissileLauncher2 interfaces. IMissileLauncher2 essentially being a marker interface extending from IMissileLauncher without any additional method definitions. You can then create a concrete system MissileLauncher that implements IMissileLauncher. In my ES you can do this through an expanded generic method:

    entity.setSystem(IMissileLauncher.class, new MissileLauncher())

    entity.setSystemAs(IMissileLauncher2.class, IMissileLauncher.class, new MissileLauncher())

    setSystemAs enforces that the left hand type “IMissileLauncher2” extends from the right hand type “IMissileLauncher”, but only enforces “IMissileLauncher” on the given system to store.

    You can’t do:
    entity.setSystem(IMissileLauncher2.class, new MissileLauncher())

    You can retrieve things as such:
    IMissileLauncher launcher1 = entity.getAsSystem(IMissileLauncher.class);
    IMissileLauncher launcher2 = entity.getAsSystem(IMissileLauncher2.class);

    The only catch is that when retrieving the system via IMissileLauncher2 well it’s really an IMissileLauncher, so don’t assign it to IMissileLauncher2.

    That is the fast way to retrieve more than one components / systems directly in my ES. You can of course also store one or more systems by type into a collection, but then you have to reference the collection accordingly which can be a lot more code or slower depending on the use case.

    Now to get to the compound entity case… Entities in my ES are component managers and systems and can be nested in other entities. The dynamic inject/extract mechanism can inject the parent manager / entity into a child entities components / systems. Say in the missile launcher case consider that “new MissileLoader()” is actually an entity though creation of entities works differently in my efforts. Let’s just say it is an entity though. When it is added to the jet entity the jet entity is potentially injected into all of components / systems of the child launcher entity. There is a particular system “PhysicsUpdateSystem” in my efforts that the relevant positional systems of the missile launcher may try to retrieve from the parent jet entity and thereby adding itself as an interested party creating an adhoc relationship such that when the jet moves position the missile loaders receive position / rotational updates of the parent and can then respond by updating relative position. When a missile loader is removed it unregisters itself from the parent entities PhysicsUpdateSystem.

    A sort of adhoc scene graph / potential same sort of composition could be used to implement skeletal animation. Have I done this yet… No.. Just getting there and stabilizing my efforts in general. I certainly do plan to work over examples like this before full public launch though.

    So the entity example… Consider IMissileLauncher and IMissileLauncher2 are marker interfaces extended from IEntity.

    IEntity launcher1 = jetEntity.getAsSystem(IMissileLauncher.class);
    IEntity launcher2 = jetEntity.getAsSystem(IMissileLauncher2.class);

    The nice thing is that the input control that marshals input to an entity just needs to target the jet entity essentially.

    Anyway I mention the above as that is the only really “dangerous” use case in my API and I suppose one can consider it “non-clean” from an API / type purity standpoint. It’s up to the dev not to mess up. There is also an explicit getter though that enforces types in a similar manner as the setAs method:

    IEntity launcher2 = jetEntity.getAsSystem(IMissileLauncher2.class, IEntity.class);

    However, in practice storing components / systems by semi-related child interface / type works out rather well and problems are immediately apparent and easy to fix while testing or the more verbose getter can be used. I’d be glad to hear any reactions though on the “non-clean” aspect. I’m not too worried though as the practical use cases seems to trump absolute type safety / purity in this case. I may add before public launch a mechanism to receive a set of parent types that a stored component / system matches by a given type. IE query for IMissileLoader2 and if it is contained / found it returns a set containing IEntity.

  10. Tom H.

    Ugh… I’d like to think I’d never ship IMissileLauncher2, just on aesthetic grounds.

    But this is an area where the entity system I used to work with hit a snag. The original design for the system was exactly what Adam calls “obviously wrong from game-design terms”: each entity was one or more graphical models, but there wasn’t clean support for composing multiple entities into a single graphical or physics model.

    The way they tried to resolve the issue was to make the graphical model layer handle the multiple morphs of the entity’s appearance, use entities to represent the gameplay, and just rely on special-purpose scripting to keep the two in sync. With a modicum of design, you wouldn’t have to have an individual heavyweight entity for each of the missile launchers attached to your jet.

    This all started to fall apart when people were implementing characters with changeable weapons. If I recall correctly there ended up being an extension to support physical/graphical attachment between entities, but it always had some issues because the entity system was layered over a pre-existing OOP graphics engine with a fairly traditional update() structure, and we weren’t allowed to alter the graphics engine to handle the needs of the entity system.

    I think part of the root problem came from the architect’s distributed simulation background; if your concerns are massive network scalability and state replication, you want to have as few entities as necessary. If your concerns are making the toolset easy for the designers to use very flexibly, you probably want to have finer-grained entities like your individually mountable missile launchers. The right design for an MMO ES is not necessarily the right design for a single-player console ES.

    So, here’s a good litmus question: you have characters with inventory (or vehicles with cargo – whether this is EVE or Starcraft). Are the “things in the inventory” represented with entities, or some other data structure? How do you handle their position – can they really even be said to have one?

  11. Mike Leahy

    @Tom H.
    >Ugh… I’d like to think I’d never ship IMissileLauncher2, just on aesthetic grounds

    Name it whatever you want as the 2nd marker interface is a “throwaway type”. ;) I consider it kind of hackish, but it serves a purpose as otherwise you’d just be dealing with a collection of missile launchers indexed by a single type. Sometimes it’s nice to directly reference an entity or duplicate of a component / system in more of a 1:1 relationship.

    Tom H.
    >… but there wasn’t clean support for composing multiple entities into a single graphical or physics model.

    >If your concerns are making the toolset easy for the designers to use very flexibly, you probably want to have finer-grained entities like your individually mountable missile launchers. The right design for an MMO ES is not necessarily the right design for a single-player console ES.

    This is a good observation. It would be fantastic to come up with a generic yet still speedy architecture that can serve both or even more paths and then the problem really becomes a question of composition for known larger design and platform constraints. I’d like to think I’m arriving towards the right side of this so far from the generic architecture side of things. Dunno though until things get into wider practice for many apps / games.

    The missile launcher as system vs separate entity really depends on design constraints and how much dynamic state one may want to push up the chain. If we know every jet of a particular type will always have 2 missile launchers there can be a hard coded relationship to state of the jet and missile launchers as systems and any associated renderer functionality can take this into account to draw the larger / singular entity.

    I’m really interested in allowing dynamic state to be pushed up the chain of a composite entity. I have not perfected the fast extensible enum set implementation yet, but each entity has one in my ES beside a type ID and the ExtEnumSet that is really the two things that make an entity different than just a plain old “system manager”; a system that is also a component manager / store. I suppose a couple more convenience methods to directly access the parent entity aggregator / current trackers is only extra additions over a stock system. Anyway with the ExtEnumSet I’ve yet to try this, but have this in mind to be able to push an extensible enum that say controls the missile launcher state up to the parent jet entities ExtEnumSet when the missile launcher is attached. This way user input control just needs to deal with the jet entity, but new state options appear in one location. In my ES there is only one more level of state tracking which is what I call actions which are one or more extensible enum states set or not. IE the god mode action may be flying, invincible, and eating cupcake states and they are not necessarily composed from the same extensible enum. Basically when the missile launcher entities are removed from the jet entity I want the jet entity ExtEnumSet to remove the extensible enums and higher level actions associated with the missile launchers. The user controls just the jet entity and doesn’t directly interact and there is no behind the scene wiring up to input control of any child entities. This sort of upstream dynamic state & action injection / extraction is potentially enabled with a little extra work by the generic dynamic injection / extraction functionality of the component architecture itself.

    @Tom H:
    >you have characters with inventory (or vehicles with cargo – whether this is EVE or Starcraft). Are the “things in the inventory” represented with entities, or some other data structure? How do you handle their position – can they really even be said to have one?

    Entities. In the case with vehicle cargo and the example is a 3D game and the cargo should be visible one could certainly represent position as a component and render them separately. This is more of a loose attachment where you want to attach to fundamental physics systems of the parent vehicle and perhaps set some threshold before say a box could fall off the back of the truck or be affected upon by any physics system as an independent entity.

    In my ES for more inventory like aspects I suppose the easiest thing would be creating a marker interface IBag extended from IEntity in my system then storing entities like parentEntity.addSystem(IBag.class, childEntity); “removeSystem” also being available. This could easily be ITruckBed collection instead of IBag. truck.addSystem(ITruckBed.class, cargo);

    An aside and small example of the AbstractComponentManagerControl (ACMC) system addition described in above comments is that one could create a “BagManager” ACMC system and attach it to the entity with the IBag collection so when items are added / removed from the IBag collection a separate component tracking the weight carried can be automatically updated, so the dev doesn’t have to explicitly track weight through additions / removal or iterate through the bag every time to find the weight carried.

    For an item in a more static inventory or with say set attachment points I’d just create an extensible enum component like the deck of cards and you can assign that to each entity and put it in a collection in the parent entity. Of course there is the marker interface & setSystem approach to specifically place or associate a child entity with a parent entity. You’d create marker interfaces for all the attachment points.

    A one to many and many to one relationship or at least the ability to associate in that manner is handy. There are a couple of utility methods in my component architecture that given a concrete system instance one can retrieve a collection of system types it is stored under (single system types, collection types, or both). Being Java one can also retrieve type keys or actual instances of systems or components by associated annotation. IE mark all flyweight systems as @FlyweightSystem and one can query an entity and retrieve all systems or type keys with the @FlyweightSystem annotation.

    So indeed I think the core aspect of what is handy in a component oriented ES is generic dynamic injection / extraction functionality where nested entities may be able to self wire up to parent entities or even inject up the hierarchy child state / action options to form one composite entity that an end user may control from one access point.

    This conversation continues to help out as there are a couple of things above I have yet to implement (some of the offline reverse lookups), but are definitely being tracked on my TODO list before public launch. I’d like to think that I’ll get to a point before initial public launch that I have a generic enough ES architecture that supports numerous ES applications and design / platform approaches such that any barriers and trade offs in approach is compositionally imposed rather than ingrained in the general architecture itself. I would love for anyone to mention any “have you thought of” scenarios or “but it sounds like your ES can’t do X”.

    We’ll see how things hold up when held to the flamethrower of wider use though. Now back to work here.. If I write long messages here it’s because I’m procrastinating too.. ;)

  12. .rafael

    Hi Adam!

    Question for you about ES and OOP.
    Do you think the Exception (specially creating new exception classes/messages) is something that must be avoid when working with ES?
    I know that exception is more a s/w arch. But it’s success was about OOP.

    Also, usually you have a list of subsystems in your game that you put in a ordered list to processOneGameTick(). Do you think it’s a good idea to have a subsystem cached as a attribute of the game class if I need? Or this is a bad idea? I know that in “Escape from the Pit” game you used the Game class reference as a attribute for some subsystem classes (like SubsystemTriggers). But what if there are one or two subsystem that I need to call some method in the Game class? Like one conversion of a component value. There’s something wrong with it?

    Thanks,
    rafael.

  13. Dens

    “The Entity itself (remember: that’s just an integer – or a UUID object in Java, which is a cleaner Java-specific way of handling it).”

    In practice it is also useful (for debugging) to store a string indicating the original user type of entity: e.g. “Player” or “MagicSword”.

    Of course in a complex game sets of components might change over time making the actual entity type different. BTW versioning of serialized data in ES is an interesting topic for discussion :).

  14. Mike Leahy

    I’ll pipe in some opinions about the above two posts:

    @rafael:
    >I know that exception is more a s/w arch. But it’s success was about OOP.

    As far as Java is concerned regardless if one takes a component oriented approach the built in mechanisms such as exception / try / catch blocks are still valid control flow. Once component implementation code is made generic though and extended beyond just ES usage one could easily make a RuntimeException container; IE COPRuntimeException extends RuntimeException and also implements all the component manager / container functionality to store components / systems. Now this may not exactly be practical or I haven’t seen the need for it yet, but that is how I’d make exceptions in Java more component oriented. You’d create a COPRuntimeException and put whatever components / systems in it that make sense (likely just components / data). It might be noted this approach also depends on implementation as like I mentioned in the ES discussion above in my efforts components / systems are stored directly in the container and not in a global component store.

    @Dens:
    >In practice it is also useful (for debugging) to store a string indicating the original user type of entity: e.g. “Player” or “MagicSword”.

    While in response to some of Adam’s outline of entity ID I’m still a proponent of using an extensible enum for the entity type though this is separate of a UUID. I outlined practical use cases for this above (somewhere / many comments now). You get a textual representation of the entity, a known, static type w/ clear implementation and practical / actionable enum useful in lookups and control flow. I suppose you could say a bare String could also provide some of this to some extent, but I think an extensible enum is more practical in a larger context.

    @Dens:
    >Of course in a complex game sets of components might change over time making the actual entity type different.

    I think one has to consider the entity type ID as malleable / changeable and a temporary constant; depends on the game though. It’s there for short term use / easier sorting / lookups and conceivably could change based on composition of components.

    @Dens:
    >BTW versioning of serialized data in ES is an interesting topic for discussion :)

    Versioning… Yes… One of a couple of Achilles heals out there. While I’ve been holding off on OSGi integration as long as possible I’m thinking that the basic versioning aspects inherent with OSGi may be useful. I don’t know yet how this may be exposed at the component / system level in a generic manner in my efforts at least such that any OSGi identifier / versioning info could be replaced with another mechanism. I’d at least want to make it as generic as possible to allow other implementations.

  15. Jakob

    Hi adam,

    I’ve read your articles and like the ideas you present. There is one question that is still nagging me: how to model game world object relationships?

    I would really appreciate if you expand your solution to the fighter jet/missile launcher problem. Adding the missile launchers as separate entities is fine, but how to make sure their position/orientation are updated at the same time as the jet?

    Another related problem is how to model objects that in one way or another trigger another. How to model a flip switch that when activated makes a door open? How to model a trap that when stepped upon makes a gun fire?

    Surely the switch and the door should be separate entities, but how should the connection between them be modeled?

  16. Mike Leahy

    Yeah!!!! More ES discussion. I’ve been missing a good continuing discussion on the topic.. ;)

    @Jakob your first question essentially is the “compound entity dilemma”. The latter is more of an architecture question that is tricky nonetheless as there are different answers for the complexity of the game / app. No doubt though one should move away from the Observable / listener pattern. I’ll let Adam take a stab at these two questions then err chime in too.. :)

    BTW Adam you decide on GDC this year? I’m looking for some ES crazed drinking buddies for good beer and ES discussion next week.. heh..

  17. Julkson

    Hello, I’m a new reader, and I got a few questions regarding this topic and the source code in the github repository (I’m assuming it’s the last version).

    Why would I need to have several entity managers?

    Is an entity a type of game object (Asteroid, SpaceShip, etc) or a particular instance of a type (Asteroid1, Asteroid2, etc)? I’m assuming it’s the latter, but in that case, would I need a factory method that creates Asteroid and SpaceShip entities, with default data, so I must only modify the fields I need?

    Wouldn’t there be a need for some kind of component tree? For example, it wouldn’t make sense to have an entity with the “Speed” or “Collision” component if it doesn’t have the “Position” component too, so instead of remembering to add all the components needed by another one, I’d just add, for example, “Speed” and it would ensure me that it will also add the rest of the branch up to the root.

    In the source code, the method MetaEntity.loadFromEntityManager(UUID e) only creates a new MetaEntity, but doesn’t fetch any data from the manager, so if I create a MetaEntity with a bunch of components with a constructor, store the UUID, and then call this load method with that UUID, I’ll get a MetaEntity with no components and no internal name. Also, I think the empty constructor and the constructor with UUID as parameter should be private.

    Is the internalName of the MetaEntity the same as the entityHumanReadableName from the entityManager? Should they be?

    I think I had more questions, but I forgot them already. I hope you can read this comment and it isn’t lost forever in cybervoid.

  18. adam Post author

    Reqd the articles on this blog on entity systems (try the entity systems tag, or google for “entity future of mmo”). Evrythin lg has been explained multiple tikes in the coments

  19. Rougg

    As Jacob asked, how do you handle triggers ?

    I’ve read your articles & comments many times, i’m not the only person to ask that but either i’m running circle because you are saying that it has already been answered either i find some infos but everything is kinda vague to me (and that’s probably my fault).

    Here is what i found from your answers :

    “one unanswered question: how to handle interaction between entities. Common example, rocket hitting a tank. Should I work it out … Or maybe queue all the events and let iterate over them by the systems I have running?”

    Not unanswered, answered multiple times. You’ve answered it yourself :).

    In practical terms, you literally *cannot* send messages to entities. The nearest you can do is send messages to one of the “systems” that is processing entities each loop, and have it batch up a whole chunk of incoming messages, then apply them to its own entities on the next cycle.

    Although you’re probably better off creating temporary components that will be picked up by other systems when they scan the complete entity/component store looking for things to operate on

    I’m sure you know this, but for anyone new to this: you can convert from events > clock by the age-old trick of generating a list of “events that happened since the last clock-tick”, and making a system/sub-system that analyses that list each clock-tick, and makes the appropriate “reactive” function calls.

    What i’m doing right now :

    networkInSystem.processOneGameTick()
    mySystem1.processOneGameTick()
    mySystem2.processOneGameTick()
    mySystem3.processOneGameTick()
    networkOutSystem.processOneGameTick()

    networkInSystem handles what’s coming from the interweb and add components to appropriate entities and sometimes event components. So if CActionEvent component is added, mySystem1 might do something with it, mySystem2 as well and mySystem3 will remove the event because it doesn’t satisfy him, preventing the networkOutSystem to dispatch the info back to the client.

    I can see the whole propagation of the event really easily that’s neat but also way too fragile, if i need to reorder my systems execution one system might have stopped the event too soon… making a hell to reorder it. I guess just thinking about threading that thing would be enough to stop doing this.

    What i understand from your quotes is that i should add events to a queue and process them the next tick.

    Seems kinda cool & easy (except adding 16ms of lag to my players :3) but i guess i’ll have to change my mind a bit. I need to think a bit more about how independent systems actually are to the rest of the world.

    My real world case is that i have a “skill action” information coming from the client, it’s catched by the server via NetworkInSystem, i want to keep that system a dispatcher and not starting to add too much logic in it that’s why i create an ActionSystem which catch CActionEvent components, identifies the source of the action, process some stuff and add new events CPlayerActionEvent or CTowerActionEvent both events being handled by PlayerActionSystem and TowerActionSystem.

    * I guess that wanting to handle events in “real-time” is a bad practice in that case (at least) !?
    * Maybe I’m thinking in sequence of events and intercommunication a bit too much !?
    * Queueing events and processing them the next tick would make the last event of a 3chained events happens after 3 ticks ?
    * Am i being a victim of the rubber duck debugging technique writing this ?
    * Or do i misunderstand something ? everything ?

    PS : You would be a duck.

  20. Luke

    Hi again..

    firstly, if this question has already been asked/answered in one of the thousands of other comments across the 9 or so articles pertaining to this topic, I do apologise. I did my best to scan through the comments to find something sounding like my question below but nothing jumped out…

    secondly, thanks again everyone for answering my queries on the other forum posts, the information was very helpful. I have been continuing my efforts in developing a simple ES in between work, wedding prep and the other random small projects I was somehow drawn into.. I am still using asteroids as my target (just want to make a simple clone of the classic game).

    My question today comes out of my thoughts/concerns about how to handle changing game states, as well as changes in key meaning for each state..

    for example, I want my Asteroids game to have 3 states:
    – Start Screen (with asteroids flying around behind text), can start a game by pressing ‘Enter/Return’
    – Game Proper/In Game (self explanatory, w,a,d movement, space shoot, shift to ‘hyperjump’ esc to pause game)
    – Pause Screen (stop simulation updates, render the game proper in background, darken the screen, esc to return to game, q to quit).

    Once I knew what I wanted I found I was at a loss for how to implement it.. (Full disclosure, this is my first ever attempt at making a game). My first reaction was too:
    – Have one EntityManager
    – Have a GameStateManager which would manage a stack of game states
    – Have a GameState class, each game state would hold a ref to the EntityManager for the game, and have a collection of Systems to execute when that game state is active..
    – Have different types of InputSystem (GameProperInputSystem, GamePausedInputSystem) and add them to the correct GameState, this way each Input System would interpret the state of the keyboard the right way for the given state… When one of the input systems detects the user wished to change the state (such as Pause Game), return a special code back up to the game state manager so it can push/pop to the right game state..

    Great I thought, lets make it happen, but then I started to wonder, should I have split the ‘GameState’ concept out into classes like I did, or should I:
    – Add the ability to turn on and off systems so that the state the game is in be represented by a combination of the data in the components and which Systems are active at a given time??
    – Stick to having one system that handles input for the game but have the keyboard input run through a filter/context for the current state of the game and turn keystates into intentions for interpretation…

    Im sorry if that didn’t make much sense, this issue has got my wondering about the best way to handle game states / changes in input meaning within an ES.. I found myself questioning the purpose of the ES, was It just one part of the game code, existing alongside other OO constructs or am I attempting to implement the entire game (objects, UI, network etc) within the ES paradigm.

    Peoples thoughts/ideas would be very much appreciated.

    Thanks in advance

    _L

  21. Pingback: Entity System 1: Java/Android | T=Machine

  22. Pingback: Why use an entity system framework for game development? | Richard Lord

  23. adam Post author

    @Rougg sorry late to reply, I was out of the country when you posted, didn’t see the comment until now.

    Designing your event system is a separate issue entirely. The questions you’re asking aren’t about entity systems, they’re about event-handling – and any resource on event handling algorithms should give you a good understanding of what’s possible and what’s desirable.

    For commercial AAA games, it’s quite common to have a 1 or 2 frame delay for whole classes of input (especially network input, given it’s slow already). Not ideal, not super-correct, but … easy to deal with if you plan it. Generally the designs I’ve seen / worked with had a hard delay of 1 / 2 frames – no variation.

    Designing your system-ordering is a separate issue entirely. The questions you’re asking are about functional / time-based control flow, and/or “unclocked logic”. Again, there are plenty of resources on this and you need to find them – there are many solved problems in this area, best to borrow from others :).

    Given time, I’d happily write up my own personal preferred implementations of both the above systems – but it’s very dependent on the game you’re making, and the other design decisions you’ve taken.

  24. adam Post author

    @Luke sorry late to reply, I was out of the country when you posted, didn’t see the comment until now.

    For the trivial example you picked, I’ve usually done the following:

    1. In-game uses an Entity system, everything else does not
    2. When you leave the game, you stop running your SubSystems. That literally freezes time as far as game is concerned
    3. All the other menus are implemented in traditional OOP style, or MVC (depending on the platform you’re using, and how powerful it’s windowing system / toolkit is). They don’t need to know about entities, and don’t care
    4. When a different menu needs info about an entity – e.g. how many lives did the player have left when game ended? – it can query the (frozen) entitymanager, find the appropriate entity, and read whatever data it wants.

    For more complex examples – e.g. games that leave the game running while you’re in menus – there are many variations on game-design that each give rise to different implementations. Some games float the menu above the visible game. Others pause the game but leave selective animatins running on a loop (e.g. keep the particle-system running, so sparks continue to fix, smoke continues to burn forever, etc). Some render the menus in-line with the game (GL Immediate mode), others pre-batch it (as the gameengine itself does).

    If you’re just getting started, I’d say: none of this matters to you. Just pick the simplest way you can think of to start with, do it quickly, and learn-by-doing what you like and don’t like about it. On your next hobby-game, pick a different way, and likewise learn-by-doing. After a few small games, you’ll have a good feel for what approaches you like, and what to use for your next game.

  25. rpaz

    As a novice developer who mostly does back-end web development, I find it fascinating that entity-component systems have so much in common with relational databases. I’m currently in the brainstorming phase of an idea I had for an ECS that relied on immutable data structures, written in Clojure. My original idea for how to handle the “data” part of an ECS was brain-dead simple – just use an in-memory SQLite DB. The advantage of it is that it’s easy to query for the data you want and sticks closely to the concept of an ECS being similar to a relational database. The problem is that databases are by definition mutable, so if I represent my game state using one, I lose out on the immutability aspect. I could dump the database to a basic Clojure map (a.k.a. hash, dictionary) before transforming the game state, and then write the new state to the database afterwards, but then I lose out on the ability to run queries on my components in my game state, cause now it’s not a database.

    I want the power (i.e. query-ability) of a relational database alongside the immutability of a Clojure map, but there doesn’t seem to be an obvious solution of that nature that isn’t basically mountains of filtering and iteration. The only thing I can really think of is a NoSQL approach, or some other sort of non-relational data storage system.

  26. adam Post author

    Would you be happy with simply blocking calls to “UPDATE” in SQL? Should be easy to disable in SQLite source, I imagine?

    Or implement a DB backend for SQLite (or one of the others) that is itself implemented to be efficient and immutable. Most of this stuff is open source, so in theory that’s possible…?

  27. rpaz

    Unfortunately, t’s not open source, but after doing some further research I’m wondering if Datomic would be a good choice for a back-end with regards to ECS storage. Datomic treats databases as values, making functional programming a lot more feasible as well as making save games easily serializable; retains the entire history of a database relative to time by recording deltas (changes to the database over time), which drastically reduces the amount of data it takes up and also enables things like stepping through an entire save game (replays, anyone?); has a multi-threading supportive, peer-to-peer based data model, which would be beneficial to both performance and netplay; can operate entirely in memory if desired, eliminating the performance hit from disk read-writes; and best of all, extends the queries you can make against typical SQL-based RDBMSes to be applicable to plain old data structures in the application itself.

    As I understand it, the reason that relational databases aren’t used much in games dev because disk read-write is slow, large amounts of in-app memory is needed, and because their stateful nature makes them hard to reason about over time. Novel databases like Datomic seem to avoid many of these pitfalls. It feels like it’d be both powerful and efficient in games dev, especially since ECS already resembles relational databases in concept (if I understand correctly). Does this make sense?

    Regardless, I might just try to use it as a back-end in a game anyway, just because I can.

  28. adam Post author

    Sounds great! See if you can get a license off them in return for testing it out and recommending to everyone else? :)

Leave a Reply

Your email address will not be published. Required fields are marked *