Entity System: RDBMS-Beta (a new example with source)

*** This comment is now using the Adam’s ES:

Ok, let’s one fast example…
Keep in mind that I’m the Button and ToggleButton as example for UI. But you can put in a game if you want as a emergency button or a swith that is on/off, if you want

I have two scenes:

P.S.: I’m represting the entity-id with “quotes” followed by { Components }

SCENE_A:
“play_button” { CButton.action = “play” }
“sound_toggle” { CToggleable.action = “sound” }
“music_toggle” { CToggleable.action = “music” }
“exit_button” { CButton.action = “exit” }

SCENE_B:
“another_button” { CButton.action = “anotherb” }
“another_toggle” { CToggleable.action = “anothert” }

These are my Componentes data:
CToggleable
+ toggled : boolean
+ action : String

CButton
+ action : String

—-

Then I have:

SceneAController ( CToggleable.toggled = !CToggleable.toggled
// –> Create an ActionEvent with ActionEvent.action = CToggleable.action
// –> Send the ActionEvent to current controller parent (or SceneAController or SceneBController)
// else do nothing.
}

ButtonSubsystem
{
// when the bounds of entity (by CPosition) is collided with touch (x, y)), then:
// –> Create an ActionEvent with ActionEvent.action = CButton.action
// –> Send the ActionEvent to current controller parent (or SceneAController or SceneBController)
// else do nothing.
}

—

If I have a SCENE_C with:

P.S.: “entity-id” { Components … }

SCENE_C (game scene)
“alarm_button” { CButton.action = “start the alarm” }
“switch_toggle” { CToggleable.action = “toggle the elevator up-down” }

I can still use the ButtonSubsystem and ToggleButtonSubsystem.
They are going to be created (together with entities and components) in the SceneCController.
So the parent of ButtonSubsystem and ToggleButtonSubsystem is now the SceneCController that has the ActionEventListener and will know what do do in the action.
And I will not to have everytime make the CTogglable.toggle != CToggable.toggle (in every susbsystem or controller)

I’m sending messages to the parent controller, but has any other way to maintain the Toggle and Button “generic” like this without using messages?

What do you guys think? :)

Sorry guys, the example was cutter
here is the link:
http://pastebin.com/YDqU4myP

@Mike

Hi Mike, no problem. I don’t want to bother you or Adam, just talk about ideas and ES related stuff.
I already did demos and two games. I also had come with my “own” ES by now.
I put the ToggleButton and Button as example, because the game, like I said, has the switch for elevator and a button to be pressed. I can do the UI using the same components because the behavior is the same, but I put pseudo-code to be fast and not write so much.
My question was more focused about the event system for the buttons, if this is ok or if I’m kind of “destroying” the ES concept using something like this :)

Thanks for all tips and comments above, by the way. I will look forward your ES when it’s ready :)

I’m already exploring by my own since a long time, even reached this blog because it =)

@faris

Yes, although: that quickly starts to depend on the hardware/architecture you’re working on. How big are the caches? How much are you parallelising? etc. So, like with the “semantic implementations for different languages”, we really need people to start writing “optimized implementations for particular hardware + game”.

This got stupid wordy; hopefully there is some clarity in there somewhere… ;) I do think Adam added some interesting directions to pursue and faris brought up reasonable comments too.

@adam & faris

faris mentions:
I’m wondering about the best performance way to implement getEntitiesWithComponents().

Adam mentions:
* Yes, although: that quickly starts to depend on the hardware/architecture you’re working on. … we really need people to start writing “optimized implementations for particular hardware + game”.

—

In some respect this is a misdirection regarding what hardware, cache sizes, etc. etc, but the last part about this particular use case needing further work is certainly the reality. As things go Adam’s reference implementation does not take into account searching for a set of components and returning entities by one or more sets. In general the idea is neat and certainly relevant from an interface / contract set oriented lookup it is not practical particularly with Java and lets say on an Android device. If we take an Android device as the lower bounds for the discussion iterating through a list of entities checking against a set of components is abysmally slow. There are many different constraints whether external / internal that can lead to particular customized implementations, however it still is not a frame by frame operation one would want to do.

This is particularly why I support a type ID separate of a UUID whether an int though I’m moving toward making this an extensible enum for legibility for reasons similar to the deck of cards discussion above. If you have basic information on how a particular type of entity is composed you can quickly search by a type ID rather than a raw set of components / systems. If you don’t know which type IDs match a given set of components / systems quite likely as is the case with my efforts there is external data such as XML or even an RDBMS that stores the composition for given entity types; this may even be stored in memory and can be quickly parsed. Search through this composer meta-data for matching a set of components / systems thus finding the type IDs that are relevant then finding entities by type ID is relatively a lot quicker and suitable for frame by frame usage. Just make sure any algorithm applied gracefully short circuit if there is a missing component / system. I think it’s important to be able to do this kind of search by composition of interfaces / IE a contract set, but it’s definitely more of an offline operation for the most part unless there aren’t a lot of entities or components / systems, but that’s a particular implementation constraint. There are many trade offs in that regard as Adam began to point out.

———–

faris mentions:
*Also, unless I’m missing something obvious… doesn’t the use of a hashmap as your component store for entity/component mapping, doesn’t that restrict you to single component types eg, one missile launcher component on a figher jet vs two.

Adam mentions:
* If a fighter jet has two missile launchers … won’t they need each a unique position? And a unique render state (if one is firing, and the other is reloading, they’ll have different animation states)?
…at which point: they should be separate entities, right?

I agree in potentially separating responsibilities along reasonable entity boundaries, but this is a slight dodge as well given that this is a soft spot for you reference implementation. The 1:1 component store with your EntityManager implementation isn’t ideal per se, so @faris does bring up a valid point. There is no way to represent a “compound entity” in your reference implementation unless one create a component to hold a UUID or MetaEntity. Particularly with a fighter jet example the missile launchers are relatively positioned from the jet and the orientation is the same as the jet.

I’m talking about my ES for the most part below unless otherwise mentioned, but you bring up some good points @faris so, I’d be glad to discuss how I try to solve the issues you brought up in my efforts.

On your second point.. Indeed this is a current limitation of Adam’s reference ES implementation such that you can’t store more than one component type. In my efforts you can store a single component or system by type or a collection by type. I make a hard distinction between component and system. There is also a more “dangerous” way to store in my efforts one or more same types by an extended, but non-related child interface of the actual component / system being stored. It’s kind of dangerous I suppose or could be viewed as “non-clean”. An example..

Say you have IMissileLauncher and IMissileLauncher2 interfaces. IMissileLauncher2 essentially being a marker interface extending from IMissileLauncher without any additional method definitions. You can then create a concrete system MissileLauncher that implements IMissileLauncher. In my ES you can do this through an expanded generic method:

entity.setSystem(IMissileLauncher.class, new MissileLauncher())

entity.setSystemAs(IMissileLauncher2.class, IMissileLauncher.class, new MissileLauncher())

setSystemAs enforces that the left hand type “IMissileLauncher2” extends from the right hand type “IMissileLauncher”, but only enforces “IMissileLauncher” on the given system to store.

You can’t do:
entity.setSystem(IMissileLauncher2.class, new MissileLauncher())

You can retrieve things as such:
IMissileLauncher launcher1 = entity.getAsSystem(IMissileLauncher.class);
IMissileLauncher launcher2 = entity.getAsSystem(IMissileLauncher2.class);

The only catch is that when retrieving the system via IMissileLauncher2 well it’s really an IMissileLauncher, so don’t assign it to IMissileLauncher2.

That is the fast way to retrieve more than one components / systems directly in my ES. You can of course also store one or more systems by type into a collection, but then you have to reference the collection accordingly which can be a lot more code or slower depending on the use case.

Now to get to the compound entity case… Entities in my ES are component managers and systems and can be nested in other entities. The dynamic inject/extract mechanism can inject the parent manager / entity into a child entities components / systems. Say in the missile launcher case consider that “new MissileLoader()” is actually an entity though creation of entities works differently in my efforts. Let’s just say it is an entity though. When it is added to the jet entity the jet entity is potentially injected into all of components / systems of the child launcher entity. There is a particular system “PhysicsUpdateSystem” in my efforts that the relevant positional systems of the missile launcher may try to retrieve from the parent jet entity and thereby adding itself as an interested party creating an adhoc relationship such that when the jet moves position the missile loaders receive position / rotational updates of the parent and can then respond by updating relative position. When a missile loader is removed it unregisters itself from the parent entities PhysicsUpdateSystem.

A sort of adhoc scene graph / potential same sort of composition could be used to implement skeletal animation. Have I done this yet… No.. Just getting there and stabilizing my efforts in general. I certainly do plan to work over examples like this before full public launch though.

So the entity example… Consider IMissileLauncher and IMissileLauncher2 are marker interfaces extended from IEntity.

IEntity launcher1 = jetEntity.getAsSystem(IMissileLauncher.class);
IEntity launcher2 = jetEntity.getAsSystem(IMissileLauncher2.class);

The nice thing is that the input control that marshals input to an entity just needs to target the jet entity essentially.

Anyway I mention the above as that is the only really “dangerous” use case in my API and I suppose one can consider it “non-clean” from an API / type purity standpoint. It’s up to the dev not to mess up. There is also an explicit getter though that enforces types in a similar manner as the setAs method:

IEntity launcher2 = jetEntity.getAsSystem(IMissileLauncher2.class, IEntity.class);

However, in practice storing components / systems by semi-related child interface / type works out rather well and problems are immediately apparent and easy to fix while testing or the more verbose getter can be used. I’d be glad to hear any reactions though on the “non-clean” aspect. I’m not too worried though as the practical use cases seems to trump absolute type safety / purity in this case. I may add before public launch a mechanism to receive a set of parent types that a stored component / system matches by a given type. IE query for IMissileLoader2 and if it is contained / found it returns a set containing IEntity.

@Tom H.
>Ugh… I’d like to think I’d never ship IMissileLauncher2, just on aesthetic grounds

Name it whatever you want as the 2nd marker interface is a “throwaway type”. ;) I consider it kind of hackish, but it serves a purpose as otherwise you’d just be dealing with a collection of missile launchers indexed by a single type. Sometimes it’s nice to directly reference an entity or duplicate of a component / system in more of a 1:1 relationship.

Tom H.
>… but there wasn’t clean support for composing multiple entities into a single graphical or physics model.

>If your concerns are making the toolset easy for the designers to use very flexibly, you probably want to have finer-grained entities like your individually mountable missile launchers. The right design for an MMO ES is not necessarily the right design for a single-player console ES.

This is a good observation. It would be fantastic to come up with a generic yet still speedy architecture that can serve both or even more paths and then the problem really becomes a question of composition for known larger design and platform constraints. I’d like to think I’m arriving towards the right side of this so far from the generic architecture side of things. Dunno though until things get into wider practice for many apps / games.

The missile launcher as system vs separate entity really depends on design constraints and how much dynamic state one may want to push up the chain. If we know every jet of a particular type will always have 2 missile launchers there can be a hard coded relationship to state of the jet and missile launchers as systems and any associated renderer functionality can take this into account to draw the larger / singular entity.

I’m really interested in allowing dynamic state to be pushed up the chain of a composite entity. I have not perfected the fast extensible enum set implementation yet, but each entity has one in my ES beside a type ID and the ExtEnumSet that is really the two things that make an entity different than just a plain old “system manager”; a system that is also a component manager / store. I suppose a couple more convenience methods to directly access the parent entity aggregator / current trackers is only extra additions over a stock system. Anyway with the ExtEnumSet I’ve yet to try this, but have this in mind to be able to push an extensible enum that say controls the missile launcher state up to the parent jet entities ExtEnumSet when the missile launcher is attached. This way user input control just needs to deal with the jet entity, but new state options appear in one location. In my ES there is only one more level of state tracking which is what I call actions which are one or more extensible enum states set or not. IE the god mode action may be flying, invincible, and eating cupcake states and they are not necessarily composed from the same extensible enum. Basically when the missile launcher entities are removed from the jet entity I want the jet entity ExtEnumSet to remove the extensible enums and higher level actions associated with the missile launchers. The user controls just the jet entity and doesn’t directly interact and there is no behind the scene wiring up to input control of any child entities. This sort of upstream dynamic state & action injection / extraction is potentially enabled with a little extra work by the generic dynamic injection / extraction functionality of the component architecture itself.

@Tom H:
>you have characters with inventory (or vehicles with cargo – whether this is EVE or Starcraft). Are the “things in the inventory” represented with entities, or some other data structure? How do you handle their position – can they really even be said to have one?

Entities. In the case with vehicle cargo and the example is a 3D game and the cargo should be visible one could certainly represent position as a component and render them separately. This is more of a loose attachment where you want to attach to fundamental physics systems of the parent vehicle and perhaps set some threshold before say a box could fall off the back of the truck or be affected upon by any physics system as an independent entity.

In my ES for more inventory like aspects I suppose the easiest thing would be creating a marker interface IBag extended from IEntity in my system then storing entities like parentEntity.addSystem(IBag.class, childEntity); “removeSystem” also being available. This could easily be ITruckBed collection instead of IBag. truck.addSystem(ITruckBed.class, cargo);

—

An aside and small example of the AbstractComponentManagerControl (ACMC) system addition described in above comments is that one could create a “BagManager” ACMC system and attach it to the entity with the IBag collection so when items are added / removed from the IBag collection a separate component tracking the weight carried can be automatically updated, so the dev doesn’t have to explicitly track weight through additions / removal or iterate through the bag every time to find the weight carried.

—

For an item in a more static inventory or with say set attachment points I’d just create an extensible enum component like the deck of cards and you can assign that to each entity and put it in a collection in the parent entity. Of course there is the marker interface & setSystem approach to specifically place or associate a child entity with a parent entity. You’d create marker interfaces for all the attachment points.

A one to many and many to one relationship or at least the ability to associate in that manner is handy. There are a couple of utility methods in my component architecture that given a concrete system instance one can retrieve a collection of system types it is stored under (single system types, collection types, or both). Being Java one can also retrieve type keys or actual instances of systems or components by associated annotation. IE mark all flyweight systems as @FlyweightSystem and one can query an entity and retrieve all systems or type keys with the @FlyweightSystem annotation.

So indeed I think the core aspect of what is handy in a component oriented ES is generic dynamic injection / extraction functionality where nested entities may be able to self wire up to parent entities or even inject up the hierarchy child state / action options to form one composite entity that an end user may control from one access point.

This conversation continues to help out as there are a couple of things above I have yet to implement (some of the offline reverse lookups), but are definitely being tracked on my TODO list before public launch. I’d like to think that I’ll get to a point before initial public launch that I have a generic enough ES architecture that supports numerous ES applications and design / platform approaches such that any barriers and trade offs in approach is compositionally imposed rather than ingrained in the general architecture itself. I would love for anyone to mention any “have you thought of” scenarios or “but it sounds like your ES can’t do X”.

We’ll see how things hold up when held to the flamethrower of wider use though. Now back to work here.. If I write long messages here it’s because I’m procrastinating too.. ;)

“The Entity itself (remember: that’s just an integer – or a UUID object in Java, which is a cleaner Java-specific way of handling it).”

In practice it is also useful (for debugging) to store a string indicating the original user type of entity: e.g. “Player” or “MagicSword”.

Of course in a complex game sets of components might change over time making the actual entity type different. BTW versioning of serialized data in ES is an interesting topic for discussion :).

Hi adam,

I’ve read your articles and like the ideas you present. There is one question that is still nagging me: how to model game world object relationships?

I would really appreciate if you expand your solution to the fighter jet/missile launcher problem. Adding the missile launchers as separate entities is fine, but how to make sure their position/orientation are updated at the same time as the jet?

Another related problem is how to model objects that in one way or another trigger another. How to model a flip switch that when activated makes a door open? How to model a trap that when stepped upon makes a gun fire?

Surely the switch and the door should be separate entities, but how should the connection between them be modeled?

Hello, I’m a new reader, and I got a few questions regarding this topic and the source code in the github repository (I’m assuming it’s the last version).

Why would I need to have several entity managers?

Is an entity a type of game object (Asteroid, SpaceShip, etc) or a particular instance of a type (Asteroid1, Asteroid2, etc)? I’m assuming it’s the latter, but in that case, would I need a factory method that creates Asteroid and SpaceShip entities, with default data, so I must only modify the fields I need?

Wouldn’t there be a need for some kind of component tree? For example, it wouldn’t make sense to have an entity with the “Speed” or “Collision” component if it doesn’t have the “Position” component too, so instead of remembering to add all the components needed by another one, I’d just add, for example, “Speed” and it would ensure me that it will also add the rest of the branch up to the root.

In the source code, the method MetaEntity.loadFromEntityManager(UUID e) only creates a new MetaEntity, but doesn’t fetch any data from the manager, so if I create a MetaEntity with a bunch of components with a constructor, store the UUID, and then call this load method with that UUID, I’ll get a MetaEntity with no components and no internal name. Also, I think the empty constructor and the constructor with UUID as parameter should be private.

Is the internalName of the MetaEntity the same as the entityHumanReadableName from the entityManager? Should they be?

I think I had more questions, but I forgot them already. I hope you can read this comment and it isn’t lost forever in cybervoid.

As Jacob asked, how do you handle triggers ?

I’ve read your articles & comments many times, i’m not the only person to ask that but either i’m running circle because you are saying that it has already been answered either i find some infos but everything is kinda vague to me (and that’s probably my fault).

Here is what i found from your answers :

“one unanswered question: how to handle interaction between entities. Common example, rocket hitting a tank. Should I work it out … Or maybe queue all the events and let iterate over them by the systems I have running?”

Not unanswered, answered multiple times. You’ve answered it yourself :).

In practical terms, you literally *cannot* send messages to entities. The nearest you can do is send messages to one of the “systems” that is processing entities each loop, and have it batch up a whole chunk of incoming messages, then apply them to its own entities on the next cycle.

Although you’re probably better off creating temporary components that will be picked up by other systems when they scan the complete entity/component store looking for things to operate on

I’m sure you know this, but for anyone new to this: you can convert from events > clock by the age-old trick of generating a list of “events that happened since the last clock-tick”, and making a system/sub-system that analyses that list each clock-tick, and makes the appropriate “reactive” function calls.

What i’m doing right now :

networkInSystem.processOneGameTick()
mySystem1.processOneGameTick()
mySystem2.processOneGameTick()
mySystem3.processOneGameTick()
networkOutSystem.processOneGameTick()

networkInSystem handles what’s coming from the interweb and add components to appropriate entities and sometimes event components. So if CActionEvent component is added, mySystem1 might do something with it, mySystem2 as well and mySystem3 will remove the event because it doesn’t satisfy him, preventing the networkOutSystem to dispatch the info back to the client.

I can see the whole propagation of the event really easily that’s neat but also way too fragile, if i need to reorder my systems execution one system might have stopped the event too soon… making a hell to reorder it. I guess just thinking about threading that thing would be enough to stop doing this.

What i understand from your quotes is that i should add events to a queue and process them the next tick.

Seems kinda cool & easy (except adding 16ms of lag to my players :3) but i guess i’ll have to change my mind a bit. I need to think a bit more about how independent systems actually are to the rest of the world.

My real world case is that i have a “skill action” information coming from the client, it’s catched by the server via NetworkInSystem, i want to keep that system a dispatcher and not starting to add too much logic in it that’s why i create an ActionSystem which catch CActionEvent components, identifies the source of the action, process some stuff and add new events CPlayerActionEvent or CTowerActionEvent both events being handled by PlayerActionSystem and TowerActionSystem.

* I guess that wanting to handle events in “real-time” is a bad practice in that case (at least) !?
* Maybe I’m thinking in sequence of events and intercommunication a bit too much !?
* Queueing events and processing them the next tick would make the last event of a 3chained events happens after 3 ticks ?
* Am i being a victim of the rubber duck debugging technique writing this ?
* Or do i misunderstand something ? everything ?

PS : You would be a duck.

As a novice developer who mostly does back-end web development, I find it fascinating that entity-component systems have so much in common with relational databases. I’m currently in the brainstorming phase of an idea I had for an ECS that relied on immutable data structures, written in Clojure. My original idea for how to handle the “data” part of an ECS was brain-dead simple – just use an in-memory SQLite DB. The advantage of it is that it’s easy to query for the data you want and sticks closely to the concept of an ECS being similar to a relational database. The problem is that databases are by definition mutable, so if I represent my game state using one, I lose out on the immutability aspect. I could dump the database to a basic Clojure map (a.k.a. hash, dictionary) before transforming the game state, and then write the new state to the database afterwards, but then I lose out on the ability to run queries on my components in my game state, cause now it’s not a database.

I want the power (i.e. query-ability) of a relational database alongside the immutability of a Clojure map, but there doesn’t seem to be an obvious solution of that nature that isn’t basically mountains of filtering and iteration. The only thing I can really think of is a NoSQL approach, or some other sort of non-relational data storage system.

Unfortunately, t’s not open source, but after doing some further research I’m wondering if Datomic would be a good choice for a back-end with regards to ECS storage. Datomic treats databases as values, making functional programming a lot more feasible as well as making save games easily serializable; retains the entire history of a database relative to time by recording deltas (changes to the database over time), which drastically reduces the amount of data it takes up and also enables things like stepping through an entire save game (replays, anyone?); has a multi-threading supportive, peer-to-peer based data model, which would be beneficial to both performance and netplay; can operate entirely in memory if desired, eliminating the performance hit from disk read-writes; and best of all, extends the queries you can make against typical SQL-based RDBMSes to be applicable to plain old data structures in the application itself.

As I understand it, the reason that relational databases aren’t used much in games dev because disk read-write is slow, large amounts of in-app memory is needed, and because their stateful nature makes them hard to reason about over time. Novel databases like Datomic seem to avoid many of these pitfalls. It feels like it’d be both powerful and efficient in games dev, especially since ECS already resembles relational databases in concept (if I understand correctly). Does this make sense?

Regardless, I might just try to use it as a back-end in a game anyway, just because I can.