Entity Systems are the Future of MMOs Part 5

Sorry for not explaining properly – the systems themselves handle passing messages through the entity manager when they iterate over the components they control.

About methods – I just can’t find any other way to do stuff with messages in a custom way per-component without sticking callbacks in the components.

I read most of the comments over a very long time, there are just so many.
This same question seems to be the one most asked, and I don’t remember reading a direct answer (but I didn’t actually use entity systems then, so I might have not noticed).

Hi!,

I have just read all your articles about entity systems and they look fantastic. I like the idea a lot but I have some questions I would be very happy if you could answer.

Main question is, in this ES you have explained how do you do things as simple as “playsound()”? I mean, You have components with data and systems processing this components. Thats ok, but what about direct actions as the playsound()?

This question is related to the previous one too. I have game objects that are added to layers in the scene. How would I do a change layer request? For example think that the player has a controllable component and when user hits a key the player avatar must be moved to a closer layer, should I access the render system and tell it to move graphic component for entity id XXXXX to layer X?

Sorry if my explanation is not clear, but it is a bit hard to get the grasp of all this.

Thanks in advance.

Hello,

First of all, thank you Adam for sharing your experience with ES.

And I have some thoughts about ES.

From OOP design (oops it sounds so wrong!) we know that we should:

1. Design to interfaces.
2. Favor aggregation over inheritance.
3. Find what varies and encapsulate it
(encapsulation means hiding not only data, but implementation, types, everything).

Lets see ES in this context.

From what I understand:

1. Only system can write data into their component (and only into their component).

So system will have two types of methods:
a) methods that work with one component and receive it as parameter
b) methods that work with many components (two or more of same type and other parameters)

In a) case methods of system will look like this:

some_return_type method_name(MyComponent* my_component, … other parameters);

and if they can only WRITE data to my_component , what kind of method is this?
This is instance method of MyComponent class.
Because instance methods that change only this class’ instance data are instance methods and
in C++ they look like this:

some_return_type method_name(MyComponent * this, … other parameters);
where *this – is hidden parameter.

So we can easily move these kind of methods from system to component class.
Because this is the same.

What about b) case? It is better for those type of methods to stay in separate “System” class.

So components now not only have “GET” and “transform before GET” methods but also methods that can
change component data.

2. Entity is just a name(id). But why name has separate class in programming language? Actually I
don’t see reason why you need entities at all. Because instances of components are entities
(or entities are just instances of components that can contain other components).
For example I have components: leg, arm, body, head. Using these components I can form “entity”
– human:
1 left leg – instance of leg component
1 right leg – instance of leg component
1 left arm – instance of arm component
1 right arm – instance of arm component
1 body – instance of body component
1 head – instance of head component

But if enemy throw granate to this human, and it will tear her apart,
left leg – can become separate “entity”.

Also that human “entity” actually can be component instance of Army “entity”
(component instance of army component). But I also can remove this human component instance from
army component instance and add it to civil people component instance or dead people
component instance.

In DB you don’t need table 3 (from tables you proposed).
And table 5 will look like this(renamed it to component_instances – to be clear):

Table 5 : component_instances
id parent_id component_id component_data_id

where id – autoincrement field, id of instance of component
parent_id – self link to parent instance component
component_id – link to “type” of instance of component
component_data_id – link to “real” data of this component

If component don’t have parent, then parent_id is NULL.

Why it useful? Because everything are hierarchies.
Instance of human leg is usually a part of human instance. Human instance component is
usually part of some community intance component, etc.

But not only this. You solve problem when you need system that uses data of few components
and maybe even “indirectly” update data of few components. How? Simple.
You create component Collision Detection that “contains” Position component and Collision Data
component and then create system Collision Detection System that works with this component.
Collision Detection System can query all Collision Detection component instances
and for each of them get access to Position component instance and Collision component instance.
But remember, only Position system and Position component can change data in Position component,
so Collision Detection System to change data should use other systems and components itself.

3) Now lets examine ES from OOP design.
Lets start from end, last rule:
“Find what varies and encapsulate it
(encapsualtion means hiding not only data, but implementation, types, everything).”

What is changing in big games? Parts of different fucntionality, parts of objects, etc.
And it can even happen in dynamic way – like leg instance is not part of human instance
anymore. So ES encapsulate these changes.

Next rule:
“Favor aggregation over inheritance.”

Sure, we do this. And if classic aggreagtion looks like this:

class Human {
Leg left_leg;
Leg right_leg;
Arm left_arm;
Arm right_arm;
Body body;
Head head;
};

we do aggregation like this
(here is very not optimized way, because actually list is not contained here,
but idea is the same):

class Component {
list components;
};

class Human : Component {

};

In previous case I cannot move leg out of Human and in ES you can. Also you can add third leg
(mutation???!!).

So yeah, we have aggregation over inheritance here, and not just simple aggregation – but
dynamic aggregation.

And now first rule:
“Design to interfaces.”
We see it here too, as components have common interfaces.
Your component don’t worry about types of other components it contains.

4) So what is difference between OOP and ES? None, ES is part of OOP but it applies some useful OOP rules:

a) Only component or it’s system can change data of component
(pure OOP rule: methods of object can change data only of it’s own object).

b) It gives us very useful OOP pattern to create games: I would call it ES pattern(or CS pattern?).
This pattern propose dynamic aggregation to hide variations of object parts. It does this not only
in programming language level but also in RDB level.
ES pattern (or CS pattern) is similar to Bridge pattern but ES pattern has dynamic aggregation
(of operations or components) and additional encapsulation that hides types of operations (components).

c) You work with instances of components of same type not with instances of
components as containers of other instances of components.

5) Can you use inheritance between components?
Yes you can, but you should avoid it as much as possible because you should favor aggregation
(in this case dynamic aggregation) over inheritance.

Consider you have 2 components – Rectangle and Circle (not sure in real world you
will use such components). Rectangle component looks like this:
class Rectangle : Component {
int x1, y1, x2, y2;
};

Circle component looks like this:
class Circle : Component {
int x1, y1, radius;
};

If we favor aggregation, then we can create Point component
class Point : Component {
int x, y;
};

And now Rectangle component will consist of 2 Point components instances.
Cicrcle component will consist of 1 Point component instance and radius.

But we see now that Circle and Rectangle share 1 Point component instance, can we make Shape
base class for them that contains that 1 Point component instance and then “inherit” Rectangle
and Circle components from it? Why not? You can. But you don’t need to. As creating of instance
of component is matter of writing creating function. For example:

Component* createPoint(string debug_name) {
return new PointComponent(debug_name);
}

createRectangle(debug_name) {
// rectangle doesn’t have it’s own data, so we don’t need class for it in C++ language
c = createComponent(debug_name);
c.add(createPoint(“point1”));
c.add(createPoint(“point2”));
return c;
};

createCircle(debug_name) {
// Circle has it’s own data – radius, so we need to have class for it in C++ language
c = new CircleComponent(debug_name); // class that contains radius only
c.add(createPoint(“point”));
return c;
};

So we don’t need Rectangle class (only Rectangle system) and we only need Circle class because
of radius field (yes, technically we can move radius to some component if that makes sense):

class Circle : Component {
int radius;
};

But in real world you will have all components preconfigured into assemblages somewhere in db
(tables 1 and 2).

So component creation will look like this:

// nothing changes for this component
Component* createPointComponent(string debug_name) {
return new PointComponent(debug_name);
}

createRectangle(debug_name) {
c = createFromAssemblage(“rectangle”, debug_name);
return c;
};

createCircle(debug_name) {
c = createFromAssemblage(“circle”, debug_name);
return c;
};

So inheritance is not needed.

What are default values for data in components? There are two ways, or you use default values
for columns set in DB or you create your own default data initialization layer (usually using some
config files).

6) Another interesting question is,
can same component instance be part of other two componont instances?

Why not? Same human can be part of two different organizations.

On RDB level, ES pattern allows creation of dynamic links between tables. And same component can
be used by 100 other components.

Best wishes,

Thanks for quick answer,

1) Private methods? So systems methods are private? Ok. But they have methods that works only with related components , like PositionSystems works with PositionComponent. For example method PositionSystem.reset(PositionComponent c). You can easily move this method to component and it will became PositionComponent.reset() method.

I mean PositionSystem.reset(PositionComponent c) == PositionComponent.reset(). In C++, I don’t see any performance overhead. Because compiler transforms PositionComponent.reset() to static PositionComponent.reset(PositionComponent* this) – same method as if it is in system. As PositionComponent is class anyway and has base class – Component.
You have to hold your data in some struct or class anyway.

2. Agree, building deep hierarchies will be bad. Especially for MMOs where you have a lot of players and a lot of hierarchies. Because it is too many operations for one table. There should be way to optimize it. I like idea that you can combine few components to create new and have new system for combined components.

My question: so I have Human entity, it consist of few components(leg, arms etc), where I put code that do read data from few components and do something?

From my point of view – I could consider Human as new component that consist of other components and write system for it. Or should I not use ES for that problem?

4. I came from ruby on rails background where we have FAT models and we don’t have any problems with idea that we are using relational db. ES uses Systems with static methods – but in fact many of those methods equals to instance methods (as I described in 1.)

1. So you just query data from memory or db and process it with system? You even don’t need to create instance of your component class for each record from db?

2. I am trying to create AI that uses data from few components (like position, health, weapon possessed, etc.). Should I create it outside ES and it’s rules? I don’t understand this.

4. RoR is comparably slow because it uses Ruby – and Ruby is not compiler but interpretator. Also Ruby is very dynamic language (like you can create new classes , add methods and properties to it and create instances of it during execution). If I will understand question 2., maybe I will understand why ES is not OOP.

thank you for answers Adam

From part 2 of article:

“Each System runs continuously (as though each System had it’s own private thread) and performs global actions on every Entity that possesses a Component of the same aspect as that System.”

So System can read and change data of entire Entity “that possesses a Component of the same aspect as that System”?

I mean, can System read data and write data to other(not of System’s aspect) components of Entity? Or only read? Suppose read mode on components of other aspects should be enough.

Suppose I have AISystem and AIComponent. Now I assign AIComponent to every Entity that should be processed by AISystem. Then I will query all Entities that have AIComponent, read data that I need for AISystem from other components of each Entity, process some ai algorithm and save some results (like decision what to do) in AIComponent of that Entity. Suppose AISystem decided to do action “move” and saved that decision into AIComponent for Entity.

Now I will process MovementSystem, this system queries all Entities that have MovementComponent (not all entities can move). Do for each entity cycle. If Entity has AIComponent and decision made by AISystem is to move, then MovementSystem processes movement for that component.

Sounds good.

But what if AISystem need information not only from Entity it processes but also from other entities that don’t have AIComponent component. For example, to decide if I should move to left I need to check if there is not some dangerous object (like flame Entity – which has not AI) ?

And second question, suppose I have Human Entity and that Entity has two legs. Each leg is component of same aspect. What to do in this situation? Should I support few components of same aspect for Entity?

Thank you, Adam

Very interesting article(s) and comments. As the discussion keeps going this topic remains hot.

I have some questions though I plan building an Javabased ES for a mmo browser-game engine. As rendering happens on clients while non-presentational game logic resides on the server (multiple systems :D) I have a split of the systems in two layers located on different machines. Rendering though is actually not a problem as I focus on the server-side concept of the engine. How would you build a system not based on a game loop but on request/reply concept?

Think of a situaion like this.
Player A calls an action to attack Player B with an Army C in his base located at D.
-> we have 2 player entities, one army entity (which somehow needs to be a component of player a?!), and a base at a given location.
Player A fires an action with the given parameters to the server which has a method for handling this case (possibly the mehtod path is stored as component_data aswell?!). That method belongs to a system in the sense of an ES. As it’s an system belonging to an component it get’s all necessary component_data, calculates the results and both answers the client and store the possibly altered data. Would you think this is a good way to solve that?

How do you think about using NoSQL databases like Couchbase (formerly known as membase) for storage?

Client/Server-Communication is not a problem as I would use jWebSocket for communication between those layers and there for go for a request/reply concept with async-RPC to the relevant systems (e.g. CombatSystem, ShopSystem, …).

Francois had some questions as well, which you didn’t answer. I guess you just didn’t see it or forgot about it. Those questions answered would help me aswell.

thanks, roka.

The game loops main purpose is to update the view depending on the changed game data. However game data may change without some one actually sees it. Think of an attack of an army while both users are logged off, as it doesn’t happen instant because the army has to get to the other players base. In this case the event still is going to happen but no “client” actually sees that change to the game data.

That rendering stuff actually isn’t interesting to me in the first point as the “rendering part” of the engine will have to work on the clients machine and my goal is to really decouple client and server. That way you may build an specific mobile app using the same data (player account) and server while using a different client.

It’s really going to be more async RPC call like or even RRPC call like as jWebSocket for instance supports it.

…and none of that should affect your ES. Your ES should be updated *by* those algorithms, and should carry on running locally as if none of this existed.

As it happens, the datastructures used in an ES generally make it very easy to implement the more advanced / tricky algorithms both for client and server. This is because they are nearly always data-driven algorithms, so the ES obsession with data-centric design fits neatly.

…but you wouldn’t expect an ES to know *anything* about networking.

I posted a comment on a different post but adam suggested posting here would receive more replies. I am re-engineering a platformer game from a few years back to use the Artemis Entity System (primarily as a way to become familiar with ES development). I’m trying to figure out how best to implement a few features –
A player can only jump when they are standing on the ground. Player input is handled in a PlayerControlSystem, collision is handled in a CollisionSystem. What is the best way to deal with only allowing the player to jump when they are standing on the ground? Should the CollisionSystem set a property on a Component that the PlayerControlSystem looks at? Likewise, if I want an AI to react by turning around when they hit a wall, how should that be handled? I can keep adding flags for reacting to every specific situation but that would lead to Component bloat. I could handle every possible collision situation within the CollisionSystem (ie: having an AI turn around, damaging a player, checking if the player is trying to jump and reacting accordingly, etc) but that would lead to an ENORMOUS event handler within that system.

What is the ‘right’ way to do this?

re: collisions.

Create a “current collision” component with an enum of current case + any other data you need. This is essentially how all physics engines work already! (although usually they have more than one possible situation at once, per object)

when the player is in mid-air, remove/delete the collision component until they hit something.

Then other systems can look for and react to the presence and data of that component.

@adam

– You hinted that if it wasn’t for the performance gains, ES might not be the best solution. Wouldn’t the data-driven aspect still be enough, or do you see a better way to build a data driven game with reusable components that is better then ES (performance not being a factor) ?

– Design of systems… Seems like there has to be a systematic way to manage system dependencies and order of operations within systems. In RPG like games everything ends up touching everything else. Without some declarative way to manage this, the system becomes very hard to follow. I’ve thought of using state machines, or some type of declarative syntax so it’s obvious by looking at the system how the logic flows.

Yep, I’d agree that you really want to use something like Adam’s “meta-declaration system”. A former employer built an entity system around assigning a static priority number to each system, and ran into many problems because that was hard to maintain in practice, and very conducive to performance problems.

@chris – please write a blog post about that, with the concrete examples you’re using to test the design – eg your purchase example.

From the brief description it sounds like you MIGHT be overcomplicati g things – but without more detail its impossible for the rest of us to fully appreciatewhat youre running into.

Also, it would be a very useful “worked example” for a lot of people, i think, to help them understand.

Fundamentally … Standard databases are appallingly bad at storing tree structures, so id expect an ES to fail horribly (in performance terms). I dont know what the state of the art is in tree databases, but last time i looked (5+ years ago), there was none, outside of extremely expensive proprietary systems.

Most people seem to have gone even further away from tree structures – eg the recent excitement aboit mongodb and key/value stores?

Adam, thanks for everything, it’s really something :-) And not just for game development i think.

One question i’m struggling with: do you think it’s ok for Systems to have a sort of (simple) messaging/event System to pass (for instance) environment events (window size changed, mouse click, network disconnection, etc). Sometimes i feel that i need a little glue between the Operating System and the game, and sometimes even between Systems. I thought that maybe a simple event system would still make Systems not too dependant, and each System can decide for itself if it should handle an event or not.

Thanks in advance!

Alright, that is very understandable and clear! It’s perhaps very useful to have like 2 or 3 Systems in a separate folder, which serve as OS or network wrappers, to keep good overview for porting to other environments.

One other small question, then i’ll stop and resume programming: Is it possible to add two same component types to one entity? Like a tank having two guns. Or is there a better advice to solve this (ie make two component types)?

Thanks for your quick answer. Very excited about this, even for web development.

Cheers,

Ronald

Hey,
I read all your post and that seems all great.
I’m trying to make an implementation of this system in Javascript (using SQLike, a SQL clone for JS – http://www.thomasfrank.se/sqlike.html ).
Even though I read all your numerous warning about not using classes and not mixing OOP and ES, I still can’t get to understand why.

I mean, I know you said components are not classes and entities are not objects, which is true, but what would keep us to implement them in such forms ? Why shouldn’t I create an entity class that I could instantiate, that had an addComponent methods, and whose components would also be classes instances, giving me the benefits of adding methods, setters and getters ?

I’m writing my ES system, and working with SQL call each time I want to modify data is a pain. And I have the feeling that I’m still accessing an object’s property, but just instead of creating a real object, I created an SQL table, complicating the process of accessing it. Then, next to it, I’ve got a bunch of big functions that I call systems, which will act on the components. That gives me the impression of just giving a bunch of singletons (say, RenderingSystem using all the Renderers), which ends up not really different from actual OOP.
The only benefit I see is how easy it will be to use an SQL database with my server. But on this topic I’m wondering, what about noSQL ? I like the json, it is a great structure, and your ES system could be implemented this way too, would that be bad ?

So do you have some examples of the bad consequences this would bring ? I’m not a very experienced programmer (only a few years, still student) so maybe I lack some of the “coding maturity” required, but I just don’t get it, and it is sort of killing my motivation to implement it.

For example when I look at the Unity’s system, it is the “bad” way of doing ES you mentionned : Unity’s components are classes with methods and properties which are just put inside a gameobject, and that is all running pretty well.

Sorry for so much questions but I think I’m missing something here.

Well I’ve had a lot of problems with managing lots of data for online games (or any sort of content based games actually) with OOP, mostly for objects initialization and real-time data save/loading to/from server, so I kinda see why another system is needed. What I don’t exactly see is how using these entities won’t make me front the same kind of problems, just presented differently…

Anyway I will try to make a basic implementation of this to see how it ends up, maybe I’ll understand its benefits by then.

Adam,

Regarding your last message, do you actually suggest writing the client using an ES? Most client tasks, e.g loading assets into memory, sending network packets, rendering, physics prediction etc are all distinct layers that don’t really fit into the style of an ES. With a single scene graph intended for physics/rendering there actually isn’t much data to process on the client.

The power of the ES, as you have stated many times, comes from the ability to iterate over large sets of data quickly and parallelize your code, which I only see being able to take true advantage of on the server.

Aside from keeping a common ES codebase between the client/server, is there a compelling reasonthat I am missing?

Hi,

I’ve been building my Entity System on node.js and I have a few things I am wondering:

My database is MySQL. I have made a few performance tests to see how it runs, and considering a 10,000 components DB I get select/update times of more than 10 ms. Considering a game loop runs in 16 MS (or 32 for 30 FPS), and that many different requests may be needed by each system, it seems extremely unrealistic to me that I could run a complete game loop in such a short time.
These tests were run on my macbook, not on a dedicated server.

Here are the results of my perf test:

===== PERFORMANCE =====
Time to create 1000 entities: 7
Time to select from a 10,000 entries table: 34
=====Entity creation and modification cycle====
․Time for creating one entity : 1
․Time for creating a component : 1
․Time for getting a component : 11
․Time for setting a component : 15
.Cycle time : 32

Just the get/set requests alone take an entire game frame. So I know you said in your posts that this is meant to be run on servers, but what type of servers do you think would be able to run these sort of requests in a reasonable amount of time?

I just want to do this as a personal project so I started developing my ES system, assuming I would be able to run this correctly in a standard affordable server, but seeing the results on my computer I am worried.
I don’t know much about MySQL, so I have no idea if my performance are usual of if there is a way to optimize all this.

I am also worried about the iteration process. If my computer isn’t powerful enough to run the server, it means that if I am working on the server I have to deploy to my distant, powerful enough server each time I want to test. Isn’t that a big loss of time?

Also, it seems to me that a huge amount of requests will be made in a game. Each system has to get data to process, sometimes it may have to query the database multiple times (for example a physics component getting only a certain category of the data, then another, etc). Is it really possible to have low enough SQL access time to support these constant requests?

That’s what I figured out when I ran the server on my computer that had an old hard drive and I heard it make so much noise. I initially thought that MySQL kept in memory the last used tables and stored in the hard drive automatically in background, but it seems not.

Thanks for your answer, I will rewrite this with a on-memory SQL server to see how it runs

Hi Adam, thanks so much for putting this all together.

I’m having a difficult time interpreting the values stored in Table 3: All Components. Please help clarify it for me. From what I can infer:

– ‘component_id’ represents the “type” of the component (Renderable, Animated, Position, etc.)
– ‘official_name’ is a human readable label to identify the component type (“Renderable”, “Animated”, etc.)
– ‘description’ is a human readable description of what data the component stores, and which System(s) use it.
– ‘table-name’ is a unique look-up value to retrieve the component table.

If these are correct, what is the difference between ‘component_id’ and ‘table-name’, and could ‘table-name’ just be an integer, rather than a (presumed) string?

Also, if ‘component_id’ is actually the unique id of a given component, then does ‘component_data_id’ in Table 5: Entity Components represent the component type to look up? If so, would ‘table-name’ and and ‘component_data_id’ be storing the same variety of values (each representing component types)?

I guess to help clarify for the slow people out there (like me), it’d be helpful to briefly describe each category in every table.

Thank you for your time!