T=Machine

Entity System 1: Java/Android

May 9th, 2010 by adam

Please note: the design described in this blog post has been much improved and updated and put up – with full source – on http://entity-systems.wikidot.com/.

On the wiki linked above, there is now a wholly new Entity System design, based on this one, but much improved. There’s a brief post about it here, recommended instead of this ES design: http://t-machine.org/index.php/2011/08/22/entity-system-rdbms-beta-a-new-example-with-source/.

I’ve been writing about Entity Systems sporadically for the last few years. Recently, I finally had the time and the excuse to build one of my own (i.e. not owned by an employer). If you haven’t read the main series of ES posts, you should do that first. There are many things in the world masquerading under the Entity/Component banner – and even more that just coincidentally share the name, but describe something else completely. It’s worth understanding which variant I’m talking about before you read about what I’ve done :).

Why build an Entity System?

At a generic level, this is covered in the other posts. But it’s taken years for me to have the time/inclination to write a new one from scratch outside of my day-job. What happened?

I left my iPhone in America, and it took 2 months to get it back
Google gave me a free Nexus One, in the hope I’d write something for it (ha! Their cunning plan worked…)
The Android marketplace is such a miserable morasss of third-rate crap that eventually I was compelled to write my own Android game … just so that I would have something to play (there are very few games on the Android store that are even worth the time it takes to download them)

I’ve been making games for a long time. I know how much effort will go into it, how much time, and how much slog there is before it becomes worth it. Writing a game on your own often means putting in 90% of the effort to get 10% of the reward.

Enter … the Entity System. If I were to pick a game-design that mostly used data-driven game features, I could implement it around an ES, and massively reduce the amount of planning needed to get the game running. I could maybe have a working game after a mere 20% of the effort. Hmm…

Building the ES for Android

Android runs something that’s *almost* Java (although more on that later – Android’s version of Java is very slow at some of the core libraries, and it really shouldn’t be). Technically, Android supports all the core data structures from Java (Collections), and the templating system (Generics).

If I were writing an ES in C++, I’d do it using templates without pausing to think; I wondered how well the same might work with Generics, given that Generics is *not* a complete templating system, although it provides quite a lot.

Getting started: early ES decisions

How to design/implement this thing? Well, we know one thing for sure:

Entities have a single name/label/global-ID. Entities MUST NOT contain ANY DATA: these are NOT objects, this is NOT OOP!

There you go, the Entity class wrote itself:

public class Entity
{
   public int id;
}

This immediately raised some concerns for me, being the seasoned coder I am (ha!). How the heck was I going to write any code that dealt with these things if I didn’t have references to them? Obviously, sometimes you do have references, but other times you expect to follow refs from within the objects you have, to get to the objects you need. That wouldn’t be happening here, since there are no inter-object refs.


public class BaseEntitySystem implements EntitySystem
{
	/** I'm too lazy to write a "safe" method to get a globally-unique ID; for now,
		I just return 1 the first time I'm called, 2 the second time, etc... */
	protected int getNextAvailableID();

	/** Whenever you create an entity, you'd better invoke this method too! */
	public void registerEntity( Entity e );

	/** The method to solve my fears above */
	public Entity getEntity( int id )

	/**
	 * Merely removes the entity from the store. It becomes a GC candidate
	 * almost immediately (since all other refs are transient)
	 */
	public void killEntity( Entity e )
}

…but, again, being a Veteran coder, the survivor of many painful battles on the field of programming … I didn’t trust myself in the slightest to “always remember” to invoke registerEntity. Quick trick: give the Entity class a static reference to a default EntitySystem, and have each EntitySystem check if that reference is null when starting; if so, set itself as the “default”.


public class Entity
{
	...
	public static EntitySystem defaultEntitySystem;
	...
	public Entity( int i )
	{
		id = i;

		if( defaultEntitySystem == null )
			throw new IllegalArgumentException( "There is no global
 EntitySystem; create a new EntitySystem before creating Entity's" );

		defaultEntitySystem.registerEntity( this );
	}
	...
}

public class BaseEntitySystem implements EntitySystem
{
	...
	public BaseEntitySystem()
	{
		if( Entity.defaultEntitySystem == null )
		{
			slog( "Setting myself as default entity system (Entity.default... is currently null);
 self = " + this );
			Entity.defaultEntitySystem = this;
		}
	}
	...
}

W00t! I can create Entity’s, and I can find them later on. Awesome. What about those Components, then?

Getting started: Components in Java

I’ve done ES in C++ before, with real templates, so I wasn’t really thinking at this point … I just ran with what seemed natural based on prior experience. The thought process (had there been one) would have been something like this:

This is java, I use Eclipse: I absolutely *must* have the IDE know what data/fields exist in each component so that Content-Assist/Autocomplete works 100%. Otherwise I will gouge my own eyes out having to remember, and doubly so each time the app compiles but dies at runtime because of a typo in a field-name.
- Requirement: each unique Component must be defined as a java Class, with each of the fields being a public member of that class
- Requirement: to access a Component of a given Entity, you must invoke a method which returns something that is typed (as in language typing) to the correct Class

I made a Component class, and had all Components extend it; there is a particular reason for this, but it doesn’t matter right now – essentially, it lets you define shared behaviour for all Component subclasses, and just saves you time on typing.

My first real Component:

(NB: I defined this *inside* another class, because I couldn’t be bothered having N source files for the (large number of) N Components I was bound to create. Hence the “static”):


public class MyEntitySystemExperiment
{
	...
	static class Position extends Component
	{
		float x, y;
		int width, height;
		float rotationDegrees;

		@Override public String toString()
		{
			return "("+super.toString()+" @ ("+x+","+y+") * rot."+rotationDegrees+")";
		}
	}
	...
}

Great. I have a component. Now comes the largest single piece of work in the entire implementation of the ES: writing the methods to:

Add a component to an Entity
Fetch a component from an Entity
Remove a component from an Entity

Fetching a Component from an Entity

This is the win/lose point: if this works well, our ES will be nice and easy to use. The other two methods (add and remove) are simply twiddling bits of data. This one is the challenge: can you make it *easy* to write code that uses the ES, and for that code to be clearly *understandable*?


public class EntitySystemSimple extends BaseEntitySystem
{
	HashMap<Class, HashMap<Entity, ? extends Component>> componentStores;

	public <T> T getComponent( Entity e, Class<T> exampleClass )
	{
	   HashMap<Entity, ? extends Component> store = componentStores.get( exampleClass );

	   T result = (T) store.get( e );
	   if( result == null )
	      throw new IllegalArgumentException( "GET FAIL: "+e+" does not
possess Component of class\n   missing: "+exampleClass );

	   return result;
	}
	...
}

Boom! It works.

Let’s just stop briefly and I’ll explain why. Reading Java generics code from cold (just like reading C++ templates) often takes a lot of hard thinking.

Looking at the “result” of this method, we want it to be (enforced by the compiler):

“an instance of a class that extends Component”
“an instance of the particular class/Component that we requested – not just any old subclass”


public <T> T getComponent( Entity e, Class<T> exampleClass )

Here, I failed. I wanted the signature to be:


public <T extends Component> T getComponent( Entity e,
	 Class<T extends Component> exampleClass )

…but I couldn’t quite get it to work. I’m too rusty with Java Generics (hopefully someone else can point out my stupid mistake(s)).

But for the most part, it works. Because it causes you to write application code that looks something like this:


public void doSomethingWithAnEntity( int globalId )
{
	// remember, we NEVER hold refs to Entity objects for long
	Entity e = entitySystem.get( globalId );

	Position position = entitySystem.getComponent( e, Position.class );
	position.x = 5;
}

…and what’s far more important is that the “type” of the “Position position = …” line is already hard-typed to “Position”. So, the content-assist will *auto-complete* anything put after a dot on the end of that line, e.g.:


	entitySystem.getComponent( e, Position.class ).AUTO_COMPLETE

…so you can instead write your method much quicker, and yet very clearly, as:


public void doSomethingWithAnEntity( int globalId )
{
	// remember, we NEVER hold refs to Entity objects for long
	Entity e = entitySystem.get( globalId ); 

	entitySystem.getComponent( e, Position.class ).x = 5;
	entitySystem.getComponent( e, Damage.class ).hitpoints = 145;
	entitySystem.getComponent( e, Renderable.class ).foregroundColour = Color.red;
}

Time-out: HashMap

HashMap is the “correct” class to use in Java for this setup: it’s the exact equivalent of Hashtable / Dictionary / etc in other languages. We need to map (somewhere, somehow) from one thing (an entity) to another thing (a component).

NB: this does not mean that you have to use HashMap as your data-store for the ES; I positively encourage you to consider other options. I used it here as the most obvious, simplest possible structure that would do the job. If you think back to my posts on Entity Systems for MMO development, I’ve often suggested that the data store could *and should* be any of many different things. In particular, SQL databases make for an excellent data-store (and remember you can get in-memory SQL implementations that do away with all the expensive write-to-disk stuff).

Unfortunately … Android seems to only partially support HashMap. You can use the class, but it runs an order of magnitude slower than you expect for a normal JVM (compared to the speed with which it runs other methods). It seems to have problems with the hashcode methods, but also even with basic iteration over the Map contents. Odd. Later on, I had to do some tricks to speed up the ES, just because of this problem.

Fetching a Component from an Entity: Redux

The examples I gave above for accessing components were lean and clear on the right hand side (thanks to autocomplete and strong typing), but terrible on the left-hand-side. By the magic of OOP, I’m going to clean up the LHS. BUT (and this is a big “but”) … make sure you fully understand what I’m doing here. With what I’m about to do, it would be very easy to fall into one of the traps of ES development: slipping back into OOP techniques.

Looking at the example:


	entitySystem.getComponent( e, Position.class ).x = 5;
	entitySystem.getComponent( e, Damage.class ).hitpoints = 145;
	entitySystem.getComponent( e, Renderable.class ).foregroundColour = Color.red;

… applying OOP mindset, we see that the first argument is redundant; the Entity already knows about the EntitySystem to which it’s registered.

Also, we know that the Entity class will never have any methods or data other than the ID. If that’s the case, the only thing we’d ever “get” from an Entity is a Component. So, we can add this to Entity:


public class Entity
{
	...
	/** Gets a filtered view of the entity's data, only returning the subset that
	 * corresponds to a particular one of its components */
	public <T> T getAs( Class<T> type )
	{
		return source.getComponent( this, type );
	}
	...
}

…which converts our usage example to:


	e.getAs( Position.class ).x = 5;
	e.getAs( Damage.class ).hitpoints = 145;
	e.getAs( Renderable.class ).foregroundColour = Color.red;

Using the ES with Systems

Recap: right now, we can:

Create entities
Add components to entities
Read/Write the data inside each component, on a per-entity basis
Fetch entities by globally unique ID

One last thing is needed before the ES can work: we need a way to fetch Entities “by component”.

e.g.:


public class MyEntitySystemExperiment
{
	...
	public void runLoop()
	{
		while( true )
		{
			// move all the entities
			positionSystem.move( MOVEABLE_ENTITIES );

			// check for collisions
			collisionDetectionSystem.process( MOVEABLE_ENTITIES );

			// render all the visible entities
			renderingSystem.render( RENDERABLE_ENTITIES );
		}
	}
	...
}

We need a way to provide the arguments that are capitalized above. We know that these should be plain-old lists of entities. We know they have to come from the EntitySystem. Finally, we know that the only defining characteristic of these lists is that everything in the list has *at least* a particular Component.

(respectively, in the example above, the lists contain: “all entities that are moveable”, “all entities that are moveable AND all entities that are barriers to movement (e.g. solid walls)”, and “all entities that should be displayed on-screen”)

So, one more method for the EntitySystem interface:


public interface EntitySystem
{
	...
	public List<Entity> getAllEntitiesPossessing( Class... requiredComponents );
	...
}

“Class…” is just a convenience; in many cases, you’ll be insisting on a single Component. In many other cases, you’ll be insisting on a set of components. Java varargs provide the minor convenience of doing both of those in one method, while retaining type-safety.

The implementation of this method is obvious: it iterates over every entity that’s been registered, and checks it against ALL the required components. If it possesses all of them, it goes into the output list.

Finis

That’s it. So easy! Obviously, there’s more to it – the other methods you need to create should be mostly self-evident – but this should be enough to get you started.

Now, I’m not sure where to go from here. I’ve got a working Java ES. I’ve got some performance improvements and feature improvements. But … in practice, hardly anyone writes games in Java (except Android programmers, and there aren’t many of those), so … is it worth it?

Alternatively, I might just run through some of the practical pros and cons I encountered when actually using the ES in writing the game-logic. There’s some interesting things that came up which most people encounter sooner or later when doing their first ES, and which might be worth looking at in more detail.

One last thought…

Did it work? Did this ES allow me to write a decent Android game?

Yep. I wrote a space-invaders / bullet-hell game with it. It worked fine on Android phones for a hundred-odd enemies and bullets on screen. On Android, thanks to the crappy JVM, it started to chug after that (dropped below 30 FPS), so I had to make some substantial performance improvements, and now it’s happily rendering 300 things all flying around at 20-30 FPS. The game is far from finished, but it’s playable and fun for a minute or so – a definite achievement considering how little of it I’ve written so far.

NB: it’s got some way to go before I’ll be happy releasing it. But, given a few more spare evenings, I hope to get this up on the Android Market as a free download in the near future.

I’m pleasantly surprised that the Android phones can handle something as high-level as an ES, in a pure, unoptimized “simplest possible” implementation.

(7 votes, average: 5.00 out of 5)

Loading ...

Filed under : games design, massively multiplayer, programming

153 Responses to “Entity System 1: Java/Android”

Mark May 10th, 2010 at 12:43 am

Thanks for taking the time to write this. It’s nice to see something concrete!
Mehr Entitysysteme « Zeitalter3 – Browsergames Entwicklerblog May 10th, 2010 at 6:08 am

[...] hat in seinem Blog eine Implementation eines kompletten Entitysystems in Java vorgestellt. Genug [...]
Atridas May 10th, 2010 at 1:58 pm

I started to read your blog when I found it researching for “entity system” in google. I can more or less understand them, but I also have my doubts. Could it be possible to see some day a example? You say that that game was easy and fast to produce, would you release the code of it, so I can see a “complete example” running?

Thank you very much either case (If you don’t publish it, don’t mind, I really understand!)
questor May 10th, 2010 at 4:27 pm

Thanks for your article! I’ve used a implementation from game programming gems and never finished the project because it was a big mess. Now I’ve implemented your java-system in c++ and it looks much more usable. One question about the system: is it allowed to have only one component-type per entity or can the user for example add two position-components to one entity?

a first version is available here: http://www.pastie.org/953838 but beware, it’s really not tested and only for reference :) it would be great to remove the template-parameter in the getAs like in the addComponent, but C++ only allows template-deduction from input-types and not from output ones…
adam May 10th, 2010 at 4:30 pm

@questor

Bingo!

(that was the first “non-essential” feature I added to the ES; it made life simpler when I wanted my player space-ship to have extra guns ;))

However … having done that … it adds a lot of complexity to the ES implementation (read: on Android, reduced performance), and I’m not convinced you ever need to do it. It’s usually a sign that you’ve skipped an entity somewhere.
Andrew Hanson May 10th, 2010 at 4:32 pm

I believe the signature you are looking for is

public T getComponent( Entity e, Class exampleClass )
Andrew Hanson May 10th, 2010 at 4:33 pm

Lets try this again without treating generic tags as html tags

public T getComponent( Entity e, Class exampleClass )
Andrew Hanson May 10th, 2010 at 4:36 pm

Third times the charm?

I believe the signature you are looking for is

public <T extends Component> T getComponent( Entity e, Class<T> exampleClass )
adam May 10th, 2010 at 4:50 pm

Great – thanks, Andrew!
Paul May 10th, 2010 at 4:57 pm

Very interesting articles!

Can you extrapolate on how you associate entities with components? For example, in your space invaders game, how does an individual “alien” entity have its Position component associated with it?
adam May 10th, 2010 at 5:35 pm

@Paul

The flip side of “getComponent” (which I provided in the post) is “addComponent”, which looks something like this:

1. Fetches exactly the same hashmap that getComponent fetched:

HashMap<Entity, ? extends Component> store = componentStores.get( exampleClass );

2. Adds a mapping from this Entity to that Component:

store.put( e, c );

So, to make a new alien, you would do e.g.:

Entity alien1 = new Entity( entitySystem.nextAvailableID() );
Position pos = new Position();
pos.x = 10;
pos.y = 0;
entitySystem.addComponent( alien1, pos );

NB: with this setup, you *must* instantiate a new Position for each Entity; this is a definite flaw (it’s preferable to work with some kind of flyweight object). Usually, though, Java is so very fast at instantiating new objects that this isn’t a practical concern (NB: on Android, I am having some minor GC hiccups, but I’ve been very lazy / profligate with object creation there).

On the other hand, it lets you write code like this:

Entity alien1 = new Entity( entitySystem.nextAvailableID(), new Position(10,0), new Weapon(100,10,3), new Enemy(500) );

…assuming you add an extra constructor to Entity that takes a varags of Component instances, and iterates through them, calling “addComponent( this, c )” for each c.
appel May 10th, 2010 at 9:03 pm

Good article, gives me some new ideas.

One of the problem I’ve encountered in entity/component systems is how you prioritize rendering in 2D. I’m making a RTS game whereas I can select a unit. That unit has some highlighting when it’s selected. The highlighting needs to be rendered atop of everything. This may seem trivial at first glance, but it’s not.

renderingSystem.render(ALL) doesn’t really go into much depth.

I’ve actually solved this problem in my own system, whereas each component type has a primary rendering priority and a entity type group has a secondary rendering priority. E.g. Highlighting component would has a high z-index order while a Units entity group would have a low z-index order. Sum of both determine the rendering z-index.

Another issue I’m worried about is the danger of the system implementations becoming a blob class.

In my system I have the data and logic in the components, and no systems. I have interface for e.g. Health for communication between components, and concrete implementations like RegenerativeHealth or StandardHealth.

If I were to do as you suggest, keep the logic in systems, would I need RegenerativeHealthSystem and StandardHealthSystem?
If it’s only a single HealthSystem, how does it determine what logic code to apply to that component data?

Of course, a article that raises more new questions than it answers, is a good article ;)
adam May 10th, 2010 at 11:16 pm

L@appel

Highlighting is not part.of an entity system. It MAY be part of a custom renderer that touve written, but thats an entirely different set of design.

you need to look at your game and see it as many independent bits, only one of which is the game logic (whwre ES becones most useful). one common source of confusion is that its possible (not required!) to write a renderer that pulls all its data from the ES. but many renderers cannot be done purely this way.

in your example, you *might* have a “boolean highlighted ” field on one of your components, but that’s as detailed as you’d go.

re: blobs … you can write the systems with full-on OOP. since they fit what oop was designed for, they tend not to blob.
adam May 10th, 2010 at 11:30 pm

@questor

looking good… youre right that killEntity also neesa to deal with the components; i left that to subclasses, becauae my base entitysystem ONLY deals with entity maintenance.

this is probably a dumb setup, but i wanted it to be easy to write a crap data structure for store/retriwve componnts, and then easily swap in a better one later. i should probable have written some factory class(es), but i hate them on unthinking principle (so much indirectikn and waffle!).

ill probably change this in future, but in the meantime, its an interesting experiment: how much can you decouple the storage of entities from the storage of components?

alsk, nb: this ia nit acadwmic; in the long run, you want to move your components tp flyweight / striding steicuts
adam May 10th, 2010 at 11:32 pm

Ps: the garbled comments are because I’m.writing.on android phone. Android os has MAJOR bugs in b
appel May 11th, 2010 at 12:21 am

@adam

ok, fair enough.

One final thing. I am a bit confused about the relation between EntitySystem and the positionSystem, collisionDetectionSystem, and renderingSystem in the game loop.

Your code code-bits contain multiple references to EntitySystem as being some sort of a manager, maybe EntityManager is a better name, whilst the other systems implement EntitySystem?
adam May 11th, 2010 at 1:15 am

@appel

Yes, you’re right – I gave that interface a very very stupid name. EntityManager is, exactly What I meant :)
questor May 11th, 2010 at 9:29 am

two notes: the c++ system could be a little bit smarter by using the runtime type information, but I don’t like rtti and emulated my own (in position the “static const FamilyId familyId = 1;” is the typeinformation that is used)

second: you can learn from my mistakes: don’t ever put logic into the components besides output for debugging or serialization! In my first try I’ve put update-logic into the components and with this approach one component needs access to other components (for example the physics component needed access to the position component) and things got very quickly very messy and unhandable :/ instead use systems that work on entities like positionSystem, collisionDetectionSystem and so on.

keep us informed about your progress on the android-implementation, I think I can still learn a lot from your articles :)
appel May 12th, 2010 at 12:25 pm

Regarding the performance issue you mention:

I assume each system needs to look-up all dependencies for a component in each iteration from the map? This causes heavy performance decrease IMO.

It’s probably best to resolve the dependencies only once, and store the appropriate references in the components themselves. Hitting the map heavily can be performance draining.
adam May 12th, 2010 at 12:32 pm

@appel

With a production system, especially a server-based one (e.g. for MMO servers), you’d make the meta system that runs the whole game a bit cleverer.

For instance, rather than just a simple game-loop that blindly iterates, you do careful management and batching of component queries. Ideally, you pass to each sub-system a pre-batched “set” of entities + components that it needs. This obviously requres each subsystem to implement some protocol that lets them tell the meta-system what they will be needing.

My quick-fix on Android is a very basic entity-system query-cache. This was very easy to write, since we’re only dealing with data – and because I’m not doing any flyweighting (yet).

My cache is only caching the “find the entities that match X” right now – I haven’t even tried to cache component fetches.
adam May 12th, 2010 at 12:34 pm

NB: before writing a better cache … I’d want to seriously look at directly plugging-in to SQLite or similar, and tweaking the SQL caching.

I’m afraid (i.e. I haven’t tested it – I’m just being pessimistic here) that SQLite perfomance on Android is likely to be way too low for real-time, but … there’s plenty of java-based in-memory SQL cache systems around, and Android has a lot of RAM. I *expect* that one of those caches would be lean enough to effortlessly improve things here – even with the trudging slowness of a full SQLite behind it.
appel May 18th, 2010 at 12:08 am

Hey adam.

I made a “rough” draft of an entity system. I’d appreciate your input on it. What I’m doing wrong and right. Hopefully I’ll be able to build on that.

The source files are zipped together here (should be easy enough to simply drag it into Eclipse if you wish to run it):

http://gamadu.com/temp/es.zip
Atridas May 23rd, 2010 at 8:32 pm

I still have some questions on the matter.

For example, in collision detection, what do you do when you detect a collision?

I understand that th “collisionDetectionSystem.process( MOVEABLE_ENTITIES );” method picks all entities with the moveable component (and, I assume, the collidable component) and check if they did collide. And when they collide… you call a script that is defined in the collision component? Is that ES-friendly? Or you trigger a event that someone other system picks? (kind of ExpolionSystem.checkTrigger(EXPLOSIVE_ENTITIES)” )

I’m trying to implement such a system and while I understand the basics (entitie just are IDs, Components store data and Systems do the work) I don’t find a way to implement it…
Adam May 24th, 2010 at 2:32 pm

@Atridas

Whatever you want. The purpose of the ES is *not* to implement a CD system for you.

The ES keeps track of *what* is collidable, *where* it is, etc – while attaching that info to the game-logic-info (e..g “this collidable is … a spaceship; that collidable is … an orc chieftain”) – in a way that you can write your CD system to *be completely unaware of* the “extra” info that is attached.
Atridas May 24th, 2010 at 11:38 pm

This brings some interesting crazy ideas:

Let’s suppose I have a system that runs on all collidable objects. Then if one of them collisions with other, will add a “collided” component to both with all relevant information.

Then run a system for “collided missiles” that triggers a great explosion.

Is that a valid way to go or I’m just crazy?
adam May 25th, 2010 at 3:50 pm

Sounds pretty cool :).

I’ve not done so much with this aspect – using the frame-to-frame presence/abscence of a component as a key part of the logic. It works very nicely (I use it for a lot of complex state-changing. Instead of having a complex state machine, just check for presence of components. It’s de-facto still a state machine but ITS MANY MANY TIMES easier to debug!)

Going the other way … instead of add/remove the components, leave them always present, but perhaps with “empty” data … My Android game is going very nicely with a similar two-step process for movement.

I have:
– Movable (containing a delta-x, and delta-y)
– Position (containing an x and y)

Everything that moves objects affects the Movable. Then my core CD system runs once per frame and applies all “movable”s to all “positions” – I did it this way so I could implement a swept-CD more easily.

When I know something wont move for a while, I subtract the movable component, so there’s less processing (although this doesn’t matter hugely).

It goes horribly wrong when you want more detail than Movable. For instance, I quickly switched to using parametric position for enemies (i.e. there is no “velocity / delta”, instead there is “at time T, one exact value for x,y”.

It was quite easy to fudge – I read the “next” value for x,y, then subtrace the “current” value, and set that as the movable.deltax/y.

But I’m sure there’s a better way to handle this. Perhaps a cleverer way to handle the difference between “absolute” and “relative” movement?
appel June 13th, 2010 at 11:30 pm

Hi again

Adam, could you please elaborate on the rendering aspect:
“renderingSystem.render( RENDERABLE_ENTITIES );”

I see in your game you have all sorts of things there, healthbar, player spaceship, enemies, bullets, etc.

How exactly does the renderingSystem go about displaying all these different things?

Also, isn’t the renderingSystem dependent on the environment it is rendering in? For a 2d game there would be some sort of graphics context to write into.

If I understand all your articles correctly, there should be multiple renderingSystems, e.g.: PlayerShipRenderingSystem, HealthBarRenderingSystem, BaddiesRenderingSystem, BulletsRenderingSystem etc.?
adam June 14th, 2010 at 12:36 am

95% of things are rendered using a common system, as with most traditional 2D game programming. The only special case here is the HUD, because I didn’t want to include the HUD itself in the entity-system.

(although, NB: there are some potential benefits to doing so. For instance, rendering your health-bar as an integral part of your player-ship-graphic – like some of the FPS shooters used to do with ammo-readouts on the in-game weapons)

player, baddies, bullets – these are all just “things that render a sprite”, so I have a component that contains the name of the sprite to render, and I scale the sprite according to the objects “size” (which actually comes from the collision-detection component in this case, although it could have come from elsewhere).

(obviously, I have a cache of “sprite name” -> “image data”, so one image is shared across all things rendering that sprite)
appel June 14th, 2010 at 4:36 pm

Clearly, not everything is a sprite image.

You mentioned in some article that components were handled by their corresponding systems, in most cases. I assume this is the case for renderable components also?

In my RTS game that I am developing I have a FogOfWar component. Now, in order to render this special component I would need to have some sort of:
FogOfWarRenderingSystem.render(FOGOFWAR_COMPONENTS)

Unless you have some sort of sub-systems within the RenderingSystem to deal with all rendering-aspects?
Paul June 25th, 2010 at 7:01 pm

I just posted some thoughts about using ES in my undergraduate game programming class. I am interested in any feedback you (Adam) or you (the others who posted here) might want to share.

http://paulgestwicki.blogspot.com/2010/06/entity-system-vs-oo-game-engine.html
appel July 8th, 2010 at 7:48 pm

After doing some prototypes of these entity systems I’ve concluded that heavy usage of maps/dictionaries makes them impractical, I’m experiencing a huge drop in the framerate. I will need to think how each system processes the entities and their components without doing hashmap lookups.
adam July 8th, 2010 at 8:11 pm

Incidentally, the new Android update fixes some of the HashMap performance problems in Google’s JVM – my game framerate under heay load went from about 15 FPS to 40 ish, without changing any code :).

But yes, I wouldn’t recommend Maps except as a “first pass” implementation. A good starting point is to look at the architecture of your game, work out where it’s going to stress the system, and think about suitable approaches.

For instance, how far will caching get you?

How far will batching (especially MAP(function, data)) get you?

etc
appel July 8th, 2010 at 8:46 pm

Here’s how each system is designed in my prototype:

A “System” has two methods, (1) initialize() takes care of retrieving all the entities from the EntityManager which possess certain component(s), (2) and it has update() method which processes those entities.
Now, the system only has a collection of the entities, not their components. So, for each entity it processes it needs to lookup the components it will use from the EntityManager. This is really the bottleneck. Doing this each game loop, for all entities, in all systems, kills the framerate. I’m already seeing 50% fps drop with only 2-3 test systems doing not much but drawing 10 sprites. A few more systems and I’ll be down to unplayable fps.

What I could probably do is to encapsulate the entity+components into a special SystemEntity container class, which has references to the components of that entity (strong references).
I could even put the processing logic into those SystemEntity classes, so the System really just iterates a collection and invokes some sort of “process()” method on those.

Now, but what would then be the point of the Systems if they were just there to execute the SystemEntities? I might as well just have one System which did that, or none?

Why couldn’t I create something like RegenerateHealthEntity class which had the logic to regenerate the health of an Entity using the RegenerativeHealth and Health components? E.g.
SystemEntity se = new RegenerateHealthEntity(entity, regHealthComp, healthComp);

But that’s code duplication, memory overload, and just bad design.

Caching, batching, I’m not sure what you mean. FlyWeight I use though.

But without a better idea how to really implement a good fast well designed architecture around I’m really just going to drop this idea as “impractical”.
adam July 8th, 2010 at 9:51 pm

Stop thinking about entities as having any importance – that’s classic OOP, and has no place here. You fetch components, not entities. Most of the time, your systems don’t even look at the “entity” – they never have any reason to. Also, Systems SHOULD NOT HAVE A COLLECTION OF ANYTHING, they are merely data-processors – you hand them a list of bunched components, they iterate over them, The End.

Why is the “components that match criteria X” function slow? This is one of the parts in an ES that needs to be lean and fast. Which methods / stacktraces are the bulk of your lost CPU time? Standard profiler questions, etc…
appel July 8th, 2010 at 10:20 pm

I’m sorry, but this:
“you hand them a list of bunched components”

contradicts:

positionSystem.move( MOVEABLE_ENTITIES );
and
public List getAllEntitiesPossessing( Class… requiredComponents )

Do systems process entities, and then look up their components, or do they process components? Perhaps as:

regenerateHealthSystem.process(healthComponents); ?
adam July 8th, 2010 at 10:42 pm

Yep. That was shorthand. By “ENTITIES” I meant “whatever custom datastructure you use to hold a bunch of components as a single entity and put those bunches – one per matching entity – into a single container”.

e.g. I’ve made ES’s where the arguments were a single pointer, and an int. Receiving systems then stuck one or more structs over the top, and zoomed through memory, fast, hoovering up long stretches at once, stopping only after (int) bytes.

e.g. Equally, my Android system hands-in a List-of-Lists. Each sub-list is “the components for a particular entity, but only the ones you actually want, Mr System – whichever ones you asked for”. Much less efficient, but easy enough to switch to a faster version later, and conceptually the same thing.

Anyone trying this needs to read the original ES articles carefully first, and be sure within themself that they can “think” in ES terms, before going further. It’s so easy to accidentally slip into OOP design, some people (myself included) have to really push themselves to stop doing OOP when they start ES.
adam July 8th, 2010 at 10:44 pm

(the MMO ES articles go on at length about how you use the ES and why the bits that are there are … the way they are. I believe they explain reasonably well how/why *in practice* you do all your interaction with references to Components, and never concern yourself with which “entity” they are part of, unless you’re debugging your ES library itself, or serializing it or something else where you want to store/retrieve some identifying lable)
lefty July 28th, 2010 at 3:36 am

I thinking I’ll really like this approach once I understand it, however I’m a little confused about the systems, components, and individual entity behaviors. In this example it looks like an entity is just a place holder and components are just bags of data associated with that place holder. And I’m guessing a system is the code the actually manipulates those components. Is that about correct?

As an example lets say I’m writing a game where all the enemies are chess pieces and they must move like chess pieces. I believe I would create components for position on the board, rendering information, and one that describes which chess piece it is (White or Black and Rook, Queen, Pawn, etc). Then would I have a ChessPieceMovementSystem that manipulates the pieces and moves them around?

Why not use an approach where entities have Components and Behaviors? Components hold the data and behaviors contain the code that manipulates those components. The behaviors could be very generic or very specific depending on the need of the game. I could write it using a single ChessPieceMovementBehavior or I could write it with multiple behaviors like QueenMovementBehaviour. In the later case all the piece movement handlers would work against the same position component. Is there some major disadvantage to this approach that I’m missing?
adam July 28th, 2010 at 9:07 am

You can have behaviours if you want – they just aren’t part of the ES. The internal implementation of a system can be *anything* you like – it doesn’t matter, just so long as the entities themselves are pure data, no code.

In your description, why do you think “entities have .. behaviours”? Unnecessary – the entity doesn’t need to have any awareness of this “behaviour” thing – the movement system can handle that mapping internally, perhaps it has a lookup table of “PieceComponent.Knight == BehaviourKnight” etc.
Michael July 29th, 2010 at 3:08 pm

Thanks for the informative articles.

I was curious, since Entity is never anything more than an identifier and you are trying to avoid getting OOP in your ES, why did you choose having an Entity class at all? Would it not be enough to simply pass the id itself around?
adam July 29th, 2010 at 3:21 pm

yes, indeed.

The only reason for a class is for convenience – you can, for instance, instantiate it. And put convenience methods on it.

(you have to write your constructors to do something a bit clever, since the real reason for “instantiating” it isn’t to get an OOP objct, but to cause the ES (somewhere) to allocate a new ID and reserve it for you. e.g. “find the singleton EntityManager, request creation of a new entity, grab that ID, and stick it in my public variables”)
Michal July 29th, 2010 at 4:02 pm

Are you planning on publishing the full source code for the game?
adam July 29th, 2010 at 4:11 pm

No plans right now – just haven’t thought about it. The ES part I’m more likely to, but I wouldn’t bother unless I or someone else was committed to maintaining it as a project. I hate “dead” open-source projects.

The first challenge is to finish it and get it up on the Marketplace – there’ll definitely be a free download, but I MIGHT do a paid version too (mainly just to get more practice using the Google payments system).
Michal July 29th, 2010 at 5:26 pm

@adam – I’m interested in ES part + for example Movement/Render systems. More in form of a complete code to back the article then the game itself.. There are some issues I’m having issues implementing ES that I’m sure you had to solve in your game that are to hard to explain in form of a comment to a blog post.
ingramb July 30th, 2010 at 2:02 am

Just found your blog and I love it. Exactly the kind of stuff I’m interested in.

In one of your earlier articles, you write
“There is a one to one relationship between Systems and AVAILABLE aspects (i.e. types of Component)”

Reading this article, it looks more like a given system will work on entities that contain a given set of one or more components. So the rendering system would operate on all the entities that have both a position component and a renderable component. Maybe you would go to write a new system, and it doesn’t even need a new type of component, just a different permutation of the existing component types.

Is this what you mean, or am I missing something?
lefty July 30th, 2010 at 5:23 am

@adam – That makes sense I guess. A behavior in my model would have basically been a system, just tied to the entity. It really wouldn’t have saved me much code wise but would have sucked up more memory and possibly performance. In my example I could just as easily write a system for each chess piece if I really needed.

The code would probably be interesting to see, however the entity system isn’t really where I’m having a problem. I’ve got a basic entity system based on the above code that I think will work great. (Thanks for the great article!) Where I’m having issues is with the systems. As an example, one of my design decisions was to include the game background as an entity. It would allow me to animate it if I wanted, or apply effects to it, etc. I also have buttons that will be constructed of a sprite and then a label overlay (and occasionally another overlay on top of that. The problem I’m having is figuring out a good way to handle the z-order of my entities during rendering. I may just settle with a TreeMap for now and just ensure that I instantiate objects in the order they need to be rendered.
adam July 30th, 2010 at 8:52 am

@ingramb

You’re right – the 1-to-1 thing was a theoretical ideal, rather than what tends to happen in practice.

My point was that you should *generally* assume a 1:1 relationship. Even in a large project, it will be roughly that – because, frankly, why bother adding a new component type unless it’s because you need a system that requires it?
adam July 30th, 2010 at 8:56 am

@lefty

I’d still expect just one system to handle movement of all pieces – it might use Behaviour objects internally, or it might have a giant “switch” statement internally.

re: Z ordering, why don’t you just assing things to named virtual layers, and use that to sort them? Or, more powerfully, standard practice these days seems to be to create a hierarchy of nested containers (seems like almost every language has used that paradigm since the 1990s)
MIkeD August 3rd, 2010 at 6:35 pm

Using a C ES system as an example, is wiring Components via pointers to an individual data item of other component an acceptable approach, for example:

Physics2DComponent
Vec2* pos;; // set to &pos of Position2DComponent
Vec2 vel;
Vec2 acc;

Position2DComponent
Vec2 pos;

I would imagine with Sql you would store the comp_id and the wiring as some type of required struct meta-data, where the struct is simply a mapping of the component data.

I am also assuming when iterating components it would be acceptable as well as performant to use a switch/jump-table on component->type to update/render/etc ?

Thanks for the fantastic series of articles, I started writing a component based system without even knowing what it was simply for performance and flexibility, and I am definitely looking at sql for clean scalable storage, with raw byte code caching for performance.
adam August 3rd, 2010 at 7:44 pm

@MikeD

Why would you do this? What’s the advantage you’re gunning for?
MIkeD August 4th, 2010 at 5:13 pm

@Adam

In the example a Physics component store velocity, acceleration and a pointer to the position member on a Draw component. The goal is to have component-wiring allowing greater flexibility with fewer components.

More specific / complex components can be coded as needed. I can also see wiring being useful in assembling smaller components for prototyping and implementing simple logic, for example a conditional component for simple behavior.

This was our solution to: How do components communicate with each other?

Thanks…
adam August 4th, 2010 at 5:36 pm

I don’t think you mean “communicate” – components *never* communicate with each other. I say this over and over and over again in the articles – if you haven’t read them, read the series linked at the top of the post.

Components are data. Data cannot “do” anything, there’s no code … so it can’t communicate :).

In your example, I still can’t see the reason why the physics component needs to have any kind of “wiring” to the location component. What would be the point?

What’s the point of “having fewer components”?

I’m not sure, but I think what you’re trying to describe is:

a. Location component: x, y, z
b. Physics component: x-accel, y-accel, z-accel

…and you’re going to write a system that takes – say – an array of entities:

(void) doPhysics( Array a )
{
for( ComponentHash entity in a )
{
entity.get( “location”).x += entity.get( “physics” ).xvelocity;
entity.get( “physics” ).xvelocity += entity.get( “physics” ).xacceleration;
}
}
MIkeD August 4th, 2010 at 6:27 pm

@Adam

I really appreciate the replies, and the discussion, I read all 5 articles twice, plus this one… Let me clarify because I think we are on the same page, after looking at your code.

Our system does what you have above, it just does it a little differently. The physics component is wired to the location component’s position(x,y) this is done when both components are created. We do this because it’s now possible to add a physics component and wire it to a component that controls the rotation, this would be something like a Physics1D component. The same Physics1D component could also be wired to location.x when an object only needs to move left/right. We have a Physics2D and Physics3D component as well.

The goal is to have fewer more flexible components, at the core both implementations are very similar, components dont have any methods, however at some point code has to “act” on the component, example, your doPhysics method is the code that acts on the component.

The only difference is, instead of hard wiring the relationship between the physics and the location, the physics component has a data member called “position” which is a pointer to the location object’s “position” data member. Using C this is trivial to implement.

When doing the physics or update phase as we call it, we walk all “update” components and use a simple switch statement to fire the “doPhysics” or whatever code is acting on the component data, end result is the same, just an implementation detail.

Again, I really appreciate the time you have put into writing these articles and answering the comments.

Thanks
adam August 4th, 2010 at 6:39 pm

You’re burying logic inside these relationships. Bad move. Keep the logic outside your components.

I might have misunderstood, but … It sounds like you’re trying to solve an unrelated problem, something about re-use of physics code, and the way you’re going about it is poisoning your ES.

?
MIkeD August 4th, 2010 at 7:29 pm

Just want to clarify and make sure I am with you…

You are saying that linking two components as a configuration/data is the poison?

Physics2D // component data id = 1
vec2* pos; // database value 2->pos // links the physics component to the location component
vec2 vel; // database value 1,1
vec2 acc; // database value 0.5,0.5

Location: // component data id = 2
vec2 pos; // database value 100,100

Although this is a very simple example, I see flexibility here. You have more options without changing working engine code. Similar to how you talk about expanding or fixing the world by simply updating the component data, and having the ability to do so after shipping the game. You would of course still create specialized components when needed.

BTW I do understand that “vec2* pos; // database value 2->pos” has to be handled when the data is delivered to the engine (lookup the other components data location etc..) several ways to do it without breaking the ES concepts, but I see this as a “configuration” no different that vel, acc and not “logic”

Great discussion, Thanks…
adam August 5th, 2010 at 12:28 pm

This is why you have “entities” in an ES – to link instances of components together.

…otherwise there’s not much point in having the concept of an “entity”. Or even an ES. Instead, just have a sea of random components that are internally connected to each other in opaque manner.

But if you do that, you lose lots and lots of things you could otherwise have done.
thebluedragont August 21st, 2010 at 2:17 pm

questor, a few months late, but your source posted here has a problem.

In the getEntities() function you don’t return a list of entities containing the ID of the component, but rather return the entities which their OWN ID is the one of the component.
This means that if you want a component with ID=3, it will return the 3rd created entity.

I believe you meant to do this:
[php]
template void getEntities (set& result)
{
map m = mComponentStore[T::mId];
map::const_iterator it = m.begin();
map::const_iterator end = m.end();

for( ; it != end; it++)
result.insert(it->first);
}
[/php]
thebluedragont August 21st, 2010 at 2:17 pm

No tags here? :(
Luke F September 4th, 2010 at 2:44 am

ok i have made an attempt at implementing this in c++. here is the src on google code http://code.google.com/p/metalheavensurvivor2/source/browse/#svn/trunk/fallen-engine

some of the src includes the windowing with sdl/opengl and image handling but i have a question. right now i only have a rendering component and would love to have stull like collision system and animation system but how do you allow the components to communicate with eachother like if i have an animiation component and i want to update the rendering component to render the correct image… just a thought should i just inherit from the rendering component for the animation ………. WAIT dose this pull me back into the hierarchy oops well what do you think
adam September 4th, 2010 at 1:27 pm

Components have no code.

Therefore, they cannot “communicate” with each other.

A system that is operating on a component – say, an animation system – can operate on as many components as it needs to. It might take data from the animation components, use them to do some calculations,and write that data back into the rednering components.

(for instance)
Luke F September 4th, 2010 at 5:29 pm

thanks! that makes more sence ill work on it
Unique class ID in C++ « Pixelmania September 24th, 2010 at 10:09 pm

[...] the same time I (once again) wanted to try to implement a component system, something like this one here. For this purpose I needed a unique ID for my classes derived from a generic Component class. These [...]
Nathan September 30th, 2010 at 10:20 am

Earlier you commented that you might release the source code. Have you given that any more thought?
adam September 30th, 2010 at 10:31 am

@Nathan

Right now, I’m too busy – I got a new desktop recently, and the source is on my old desktop (which is now packed away ready for move to a new home).

I’ve got a new project I want to release before Xmas, which will use this ES, so I’ll have to dig it all out then anyway. But it won’t be for weeks / 1-2 months yet.
Leon November 1st, 2010 at 5:16 pm

Hi adam,

I’ve read your posts and this article’s comments and I’ve got one question left:

You said that you pass a bunch of components to a system and the system then does its processing on them. Some systems (I guess many) might require more than one kind of components to do their work. So let’s say I’ve got a pretty simple rendering system. It just draws sprites at postions x,y.

So to do its work this system needs a bunch of Position components and Renderable components. So I pass those components over. But how does the system know which position to use for which sprite.

How do you solve this problem? Do you pass entities to the system so the system can take the components it needs to do its work or do your components have some sort of “belonging to entity”-reference?

Forgive me this stupid question but I’m rather new to the ES thinking and don’t want to bring over my OOP habits – so I rather ask :)
adam November 1st, 2010 at 6:23 pm

This comes down to whats avialable in your language, and what performance bounds youre working within

Lowest performance you can send a list of dictionaries: each dict has an entity id, and a key+value for each component.

By contrast, say if youre aiming for cache-efficient, in C derived languages, you can send an array of structs, wheere the method thatll do the processing knows how many structs to expect per entity, and theres no delimeters: it relies upon knowing that the stream is, eg, “8 bytes: entity id … Then struct1, then struct2, then repeat”

(thats a very network-programmer peerspective on it, for obvious reasons :))
T=Machine » Phone numbers, and making iPhones and Androids switch networks November 8th, 2010 at 3:35 pm

[...] platform until we know the platform well ourselves. So, as well as writing some Android apps (e.g. the shoot-em-up I blogged about earlier this year), I switched from my iPhone to a new Nexus One as my personal phone. The plan was to use it daily [...]
alex November 9th, 2010 at 7:05 pm

How would you handle a parent child relationship between entities?
for instance:
1) I have a ship and torpedos. when the ship dies I want the torpedos to die also.
2) I have one entity attached to another so when the physics run I want the parent to set the child’s properties.
adam November 10th, 2010 at 12:52 am

There may be a better way, but … I usually just add an extra field that contains the numerical entity-ID of the parent (I always use an integer for my entity-ids – it makes data interchange simpler, although in theory you could use any struct/object/whatever).

One advantage is that it makes it transparently clear (when looking at an entire entity) where the dependencies are upon other entities.

NB: because you’ll be adding a field to a component, it also avoids namespace clashes. E.g. you can have a “torpedo.parententity” and a “physics.parententity”, and all is fine. No clashes, even though they’re serving similar purposes. NB: the purposes are similar but not identical – so, keeping them as separate namespaced vars is safer in the long run than trying to share them in e.g. a single reference.
alex November 10th, 2010 at 10:46 pm

Thanks for posting all these articles. I rewrote the bulk of my game using the entity pattern and i have to say once the entity manager was written it all went together ridiculously fast. The entity pattern works particularly well in javascript. I noticed that in some places my code was already headed in this direction but my ideas were nowhere near as coherent as yours ;)

I messed around early with trying to get the getEntities(component) function to return an ANDed array of entities based on a list components. That got kind of messy so I changed it to accept an array of entities to search through. I then realized that in all the cases I’ve fiddled with so far I didn’t need to get an ANDed array of entities. So for instance in the physics function I use the following pseudo code

entities = EM.getEntities(‘acceleration’);
for(entity in entities) {
acc = EM.getComponent(entity, ‘acceleration’);
vcc = EM.getComponent(entity, ‘velocity’);
if( vcc == null )
EM.addComponent(entitiy, ‘velocity’, {x:0, y:0});
}

I’m terrible at explaining myself so I hope that makes sense ;)
State of the Art Game Objects | GBGames - Thoughts on Indie Game Development November 12th, 2010 at 12:53 am

[...] T=Machine’s Entity System 1: Java/Android [...]
Leon November 12th, 2010 at 2:06 pm

alex, on your “not needing ANDed component queries”:

yeah, you could use that but your approach adds IMHO unnecessary logic to the straight forward data processor a system should be. all the filtering logic should have went into the query where you specify a subset of data you want to process.

on the topic of entity systems: I must confess that this design is really awesome. I’ve built a subset of components in the last few days and now I’m able to prototype games really, really fast. With the addition of lua it’s really a walk in the park to try out new game ideas.
Introducing ›Ginkgo‹ » Broken Rules Blog November 16th, 2010 at 5:45 pm

[...] come to the conclusion that we want the engine to be component-based but do not want a full-fledged Entity System. Our main influence was Terrence Cohen’s Paper “A Dynamic Component Architecture for [...]
Erkki Lindpere November 16th, 2010 at 8:50 pm

I’m not a real game programmer so this might be just me being naive, but I still like an entity system that is based on type hierarchy (or rather: mix-ins / cake pattern).

Granted, I’m thinking about this in the context of Scala, which has mix-ins. I implemented the entity system for my last game project as explicitly based on subtyping so every Entity type (player, ball, paddle etc.) had it’s own class and mixed in any components that it needed (position, Box2D body support, state support, rendering etc.)

This system has some drawbacks, namely the update(delta: Int) method which is overriden in many different entity traits… you must not forget to call the super.update method when you override it and it may become hard to understand the call hierarchy when there are several supertypes for an entitiy and if the entity has multiple states.

That aside, I like my system so far, but I’ve read enough about component-oriented systems that I can’t just dismiss them. So my next experiment with this is a hybrid system where both mixin-based (code-driven) and composite (data-driven) entities can happily co-exist.
One would have to be implemented in terms of the other.
Erkki Lindpere November 16th, 2010 at 10:05 pm

To rephrase the last paragraph of my previous comment, for my next game project I’m going to experiment with an entity system, which has both compile-time composition and run-time composition.

e.g. each Component is implemented as
1) a Trait that can be mixed into a compile-time composite entity class
2) a concrete Class extending the Trait, that can be added to a run-time composite entity object

My hope is that the compile-time composition is better for quick prototypes and for small games that don’t have full-featured level editors and are lead mainly by a programmer.

And the run-time composition is better for flexibility, large games, level editors etc.

I’m not yet sure how the EntityManagers/Subsystems will fit into this exactly, there are a few different ways to do it while supporting both the compile-time (mix-in) and run-time (composite) approaches.
jeff November 21st, 2010 at 3:16 am

This is really interesting stuff. I have been following these post with great interest and have implemented an Entity System of my own along these lines.

I am looking at how I can make my implementation more thread safe. For example my rendering, ai and input systems all I want to be on different threads.

One thing I thought is that the subsystem owns the list of components that it just cycles through but went against as I think its not correct that a subsystem will necessarily have a single primary component that it will read/write to unless its a very simple game. I am thinking about double buffering the data in the component in some nice way and allowing only one subsystem write access to the data.
Rauru November 30th, 2010 at 12:44 am

Hi adam. Thanks for your articles.

If i understand the behaviours of a component is managed by a system. It’s that correct? So, for example, if we have a PhysicsComponent then it will contain only data such as: mass, inertia, velocity, etc.
Without methods of any kind? I mean, without an update() or other methods. Thus we would have an PhysicsSystem/manager/whatever who controls the behaviour, i.e via an update() method. I’m correct?

On the other side, about communication… You say that if components are pure data, there is no need to communicate them but, how about this case: if we have an ControllerComponent and a PhysicsComponent and we want to change the velocity in the Controllercomponent (let’s say with UP_key of the keyboard)… If the velocity is in the PhysicsComponent, how we change it? With a ControllerSystem accessing a PhysicsComponentList?

Thanks!
adam November 30th, 2010 at 10:00 am

@rauru

Yep – have your system directly access whichever components it needs to.

This (unfettered access to all components) could be horrendous. In practice, it’s fine – Since you can pre-filter the components that are passed-in to a system on each update() tick, it’s easy to have simple but effective checks that make sure each system is only accessing the things you thought it would access.

(although that level of self-checking isn’t *required*, its usually very welcome in larger codebases, where it makes it much easier to see what’s going on)
adam November 30th, 2010 at 10:12 am

@jeff

Why *those* systems on separate threads? For an MT CPU, those don’t sound like great choices – rendering and physics I would expect, *maybe* ai (but how complex is your AI, really?) … but input? on its own thread?

Short answer: yes, you can architect *your* chosen set of components so that only one system ever writes to one particular component (that’s usually quite easy), although you will likely get some strange artifacts (sometimes these will cause siginificant bugs – often they will just be momentary hiccups / cosmetic) if a different system reads from two components that are in different partially-updated-state.

Alternatively (probably my preferred route these days), thread safety is enforced at the “who is accessing the components of type X right now” level. This isn’t particularly difficult: you record which components (list of type + entity_id) were passed to which system, and then refuse to hand out those components again until that system returns from that method call.

(a system WILL only be accessing some-not-all component-types. In a larger game, it may also be only accessing a subset of the entities that have those components – although this *MIGHT* be a sign that one of your components should really be refactored into two (or more) components. i.e. if you’ve got a component “health” and you have two different ways you use the health, so that a system only reads one set of entities with health OR the other set of entities with health BUT NOT both … that suggests you should split it into two components)

i.e. it’s a batched lock/unlock semantic – and it works pretty well. It has the major advantage that you can automatically re-order (if you *really* need to) the calls to systems to achieve the “most efficient” use of your CPU cores.

…since you have enough information to calculate which calls are dependent upon each other, and which are not (although this does require some extra info from your own game-design to decide whether it’s “required” that e.g. “rendering always happens last, no matter what”. You’ll usually need to add a few manual rules like that…
Evan December 9th, 2010 at 12:12 am

I have a couple questions which I would be grateful if somebody could clarify for me.

1. Regarding the global defaultEntitySystem… the way this is implemented it looks like the first entity system/manager that’s created assumes this role. This could be any system then, the physics system, the rendering system, etc, depending on the order they’re created in, and it doesn’t matter? It seems like it would be more “symmetric” to have a special single instance that assumes this role, and then none of the specialized systems would have to.

2. I’m not very familiar with Java, so I’m not sure I understand what a HashMap<Class, HashMap> is. It’s a nested map (map of maps) which, given an entity id, gives you the map between component ids and component instances for that entity?

3. getNextAvailableID is declared as protected on the BaseEntitySystem, which means only the entity systems can generate new ids. The Entity constructor registers itself with the default manager, and inside registerEntity I assume the manager creates a new id and assigns it to that entity instance? Why then, does the Entity constructor take an id parameter, if nobody but an EntitySystem would be capable of generating one to begin with?

Thanks!
adam December 9th, 2010 at 12:50 am

1. A subsystem would have a different base class; only (re-)implementations of the overarching ES would extend EntitySystem. NB: your question makes it clear that “EnitySysrem” is a poor name for this class! This setup is merely convenience so that you can instantiate an EntitySyatem instance an not even be aware of the global/sngleton references.
adam December 9th, 2010 at 12:53 am

2. Hashmap is the academically correct name for a hashtable.

In this case, you give it an identifier for a “kind” of compoent, and it gives you back a set of all the components – oganized by entity – which exist of that kind. Iirc (I’m writig this reply on iPhone and cannot re-read the article at same time; wordpress has seemingly poor JavaScript which is killing iPhone performance)
adam December 9th, 2010 at 1:00 am

3. This is poor design on my part – the knowledge that i will almost certainly want a feature in future where I can serialized and deserialize an entity, and I would LIKE to deserialize it with the same entity Id it originally had.

This method signature makes it easy to add that feature.

(although IMHO there are better ways of providing that functionality and/or actually I think it’s a stupid feature and I would prefer never to implemnt it)

In reality, I shouldn’t have jumped the gun like that. In my defence, I genuinely expected to add that feature in this iteration, but never got around to it :).
Evan December 9th, 2010 at 1:25 am

I see, so there’s really only one EntitySystem in the game. Specialized sub-systems are implemented completely independently of each other (and of the BaseEntitySystem). All they have in common is that they operate on Entities and Components?

Anyway thanks for the quick reply. These posts are very interesting.
eses December 10th, 2010 at 9:28 am

I think its best to leave out the term component in Entity System. Entity System is 2 items, Entity and System. The entity itself should not contain id but the properties of an object. For instance, PositionEntity possesses x and y coordinates. The system is basically behaviors that acts upon the entity such as manipulating the properties of the entity. So this is more like a MVC structure where M (Model) is the entities, C (Controller) are the ones who manipulate these entities. No behaviors in Model (no getter/setter/methods)
Brad December 10th, 2010 at 11:22 am

This is a great article. I’ve read this along with your five part article on entity systems in MMOs.
I’m curious though, what is the purpose of calling registerEntity() in the EntityManager? Is it to maintain a list of Entity’s somewhere, kept within the EntityManager?
adam December 10th, 2010 at 1:09 pm

@eses

You seem to be missing the point of an ES: an entity/thing/gameobject is composed of many different lumps, e.g. a lump for “position”. The position is not itself a thing/entity/gameobject – it is merely one aspect of it.

It would be much better to say e.g. PositionAspect than to say PositionEntity.

(except “aspect” has specific meanining thanks to AOP)
adam December 10th, 2010 at 1:11 pm

@Brad

Yep – IIRC I mostly used this just for debugging: I could print out a list of all entities on-demand, and maintain a “kill list” of entities that had been deleted (but internally I kept them cached for a while, to help with debugging) etc.
Brad December 10th, 2010 at 4:06 pm

Nice.
Regarding assemblages, what’s a clean way to implement them in Java?

I was thinking having a base Assemblage interface with a create() method that returns an id/entity.
Next I’d implement a separate class for each Assemblage I require.
Lastly I’d probably end up making an AssemblageManager of some sort, with a similar format to your EntityManager.

Does this sound appropriate?
adam December 10th, 2010 at 5:24 pm

I highly recommend NOT using a class for each assemblage – that poisons your system with lots of classes you don’t actually want.

Rather, I’d implement a single Assemblage class, that is data-driven.

i.e. it has a method “createFromFile( File )” or “createFromProperties( Properties )” or “createFromXML( String )” or similar, that reads in a list of components (and maybe default values for those components’ variables) and presents an Assemblage object.

You’ll have many different instances of this class, each one representing a separate instantiable Entity+Components.

But only one class – very little typing / maintenance / etc, and keeps you safely away from being stuck with OOP
ryan December 29th, 2010 at 8:03 pm

I have a couple of implementation questions about your entity system that I haven’t been able to work out myself. I currently work at a game studio who is considering moving to a component system after working in a OOP system with a fairly deep hierarchy for a long time.

I like splitting up the data (component) from the logic (entity system) however I’m having trouble working out how a trigger system would work in this architecture. I need to be able to define a shape (Axis Aligned box), some flags (PlayerIsOnGround), and then be able to hook other entities into this trigger to know when an object enters or exits this trigger and respond accordingly.

From what I’m reading, I need to have an Trigger Component which contains the shape, the flags, and a list of current inhabitants. I can write a Trigger System which takes trigger components, checks to see if any collidable components are inside of the trigger volumes and adds them as well as making sure all current inhabitants are still valid inhabitants.

The problem I run into is building the messaging structure. When some object enters my volume, I want AI to spawn or an explosion to occur. How would you go about hooking this messaging structure up in data and code using your ES?

Thank you!
adam December 29th, 2010 at 9:08 pm

Trigger-based logic is incompatible with ES logic – it’s an event-driven paradigm, where ES uses a clock-driven paradigm.

However … fundamentally, that statement is true of 99% of game development – it’s very rare to see a true edge-triggered game, and yet designers talk about it all the time, and developers often pretend their systems are edge-triggered when they’re not.

ES doesn’t mandate that you do this – you could build an edge-triggered logic into your game, it’ll just lead to a really big opaque blackbox which the ES doesn’t interfere with. But for a lot of cases, it’s much more efficeint to do clocked/batched logic (makes it easier to optimize for performance, caching, streaming, etc)

I’m sure you know this, but for anyone new to this: you can convert from events > clock by the age-old trick of generating a list of “events that happened since the last clock-tick”, and making a system/sub-system that analyses that list each clock-tick, and makes the appropriate “reactive” function calls. However, you have to be careful about the content of those function calls – they shouldn’t be written as event-triggered logic (e.g. they shouldn’t make nested calls to other objects, that react, and re-react, etc – INSTEAD they should be doing just one clock-cycle of work, and generating any new “events” as items that go onto the master list for NEXT clock-tick).

…theory aside… things to note:

– that “list of events since last clock-tick” does NOT need to appear anywhere in the ES – it’s an implemetation detail of one of your Systems. Your systems *can* hold data that does not appear in the ES; the ES is purely for holding game-data – it’s not a replacement for using “variables” in your programming language.

– USUALLY, I end up splitting the CD system into 2 or more systems: one system that purely handles “detecting and classifying collisions”, and other systems that handle “reacting to collisions”. The first part can be written purely accessing position + velocity components – it will generate “fake” collisions for things in your game that can legitimately pass-through each other (or you can make it more advanced, by also accessing other components to filter the collisions better). The latter parts contain more and more game-specific specialization

…but honestly, I have no “best practice” suggestoins for how to do CD in an ES. It needs a lot more people to discuss it, really, and to come to some recommednations/conclusions.
T=Machine » Entity System 1: Objective-C December 30th, 2010 at 3:35 pm

[...] (this is a simplified, but complete, version of the “Entity System 1: Java/Android”) [...]
T=Machine » Entity Systems are the future of MMOG development – Part 1 December 30th, 2010 at 3:42 pm

[...] Entity System on Android phones – a concrete example using Java [...]
barsiwek December 31st, 2010 at 3:25 pm

I would like to ask if it’s possible to take a peek at your implementation of space invaders game?
I’m trying to use your ES approach to write Tetris (as an exercise for myself) but I’m having problems designing the right components and systems corresponding to each other. I could elaborate but i think a peek at the code would be much more beneficial.
Adam December 31st, 2010 at 7:30 pm

Follow the link two comments before yours
barsiwek December 31st, 2010 at 8:50 pm

@adam
I actually saw the ES in Objective-C that you made available, but what I was curios about what components you use in your game and how are they used in systems (like Graphics, CollisionDetection etc.)- thats why I’m actually curious how you use it rather how it’s constructed.
Haro January 5th, 2011 at 1:10 pm

Thanks for your articles Adam. But i have a couple of questions about the entity system.

For example, in your design you need a “System” for every component, because there’s a 1:1 relationship. So, the question is clear: if you have 250 components you need 250 systems? In that case, the “gameloop” is going to be a big one because your are going to call 250 systems!!!

About the other question, what about the “states” (attack, jumping, idle, active, etc..) of an entity. Are components? Or there is a “State” component (so we only need 1 system) ?
adam January 5th, 2011 at 1:16 pm

The 1:1 is a guideline, really aimed at getting you going in the right direction.

Ultimately, on a medium to large project, you’ll have multiple components per system – but you should be generally starting from 1:1 and only increasing that as and when you need to. 250 components is insanely many for most projects.

You would only need a single state component (although in some cases you might want to do it the other way, it would be very unusual)
Alex January 11th, 2011 at 5:58 pm

The articles are great. I am looking forward for the next article.

Can you expand the networking approach? I am very curious about the communication between client and server. I want to do an implementation of the client-server framework.

While trying to get components from the HashMap I get null because the key of the HashMap is an Entity.

I make a new Entity;
Entity e = new Entity(es.getNextAvailableID());

I add the Position component
es.addComponent(e, new Position());

I am making a PositionSystem containing:
public void doSomethingWithAnEntity( int globalId )
{
Entity e = entitySystem.get( globalId ); // on this line the Entity reference changes
…
}
Then I call public T getComponent(Entity e, Class exampleClass)
and the “result” is null because the key of the HashMap is the Entity reference which is changed when I’m trying to doSomethingWithAnEntity( int globalId );

How should I deal with this? what’s the most logical approach?
Should I change to HashMap<Class, HashMap> ?
where Integer would be the globalID

Best regards,
Alex
Alex January 11th, 2011 at 6:01 pm

HashMap<Class, HashMap>
Alex January 11th, 2011 at 6:03 pm

2nd HashMap

Sorry for the double post please edit them. It didn’t let me add multiple “less-than, more-than signs”
adam January 11th, 2011 at 6:21 pm

@Alex

Two quick solutions … one is to change the methods and store the int in the hashmap as key instead (which is a PITA in Java but works fine)…

…the other is java-specific (but holds true for MOST OOP languages): in your Entity class, you override the two methods:
(boolean) equals( Object o );
(int) hashCode();

(I may have the names wrong). The first one is what Java uses to test for “equality” in a collection – if you set that to return true IFF the two objects are both Entity instances *and* have the same integer value inside them, it will work fine.

However, the second one is a performance optimization that you nearly always want to do: specifically for hashmaps, you can control the hash that is generated on a per-object basis. Gross simplification, but: … to make java’s HashMap class work at peak performance, you should have objects that are the same according to equals have the same hashCode too (google for how HashMap’s work, independent of language, if interested to know “why”).

(int) hashCode
{
return this.entityIntegerValue; //whatever the internal int var is called
}

…which should be extremely fast, and yet correct.
Alex January 11th, 2011 at 11:49 pm

Thank you for fast replying. I red about equals and hashCode but I never had the chance to override them so I will try it now :D

To get the whole process straight….there is the initialization process( adding entities, components), then systems Updates all Entities which contain the required Components. This Update method takes every Entity containing the required Components and for each Entity change the data in the corresponding Component. Is this right?

As a suggestion maybe it is good for everybody if you can post the whole Entity System process as a short story/description better than I wrote above.
adam January 12th, 2011 at 2:53 pm

@Alex

Right now, I’m focussing on the community wiki to get source-code for several ES’s in several languages – c.f. http://t-machine.org/index.php/2010/12/30/entity-system-1-objective-c/

and the wiki itself: http://entity-systems.wikidot.com

…so it’s very unlikely I’ll have time to consider anything else, until that wiki has more contributors and there are more open-source code examples
snorre January 12th, 2011 at 2:55 pm

I’m wondering about the implementation of
List getAllEntitiesPossessing( Class… requiredComponents ).

The way I have it setup right now is exactly like you describe: An entity list is returned, which I use to call getComponent to retrieve the actual components. Ideally I would like getAllEntitiesPossessing to return a list of lists (like you commented somewhere above), but I fail at Generics I think.

My only real attempt has been to return List<List>, but this requires casting on the calling end. Is there a smart way to return a typesafe list with mixed Components which DOESNT require casting? I imagine returning the components instead of the entites would be more in line with the ideas described here, and would also improve performance, so perhaps I should just deal with the cast and move forward?
adam January 12th, 2011 at 3:27 pm

@snorre

this is getting into the next-level of “slightly more advanced” ES design, which I’ve talked about in theory, but not gone into much detail. If I can get more people contributing to the Entity Systems wiki, this is where we’ll go next.

(at this point, you’re getting into the kind of thing we were doing on commercial / AAA games – cleverer design around performance, efficiency, etc)

I *do not* have a pithy description for you – I find it takes me a lot of time to come up with derscriptions/blogposts that are clear on this topic (so little is defined, so little shared vocabulary, so many wasys to confuse / get the wrong idea)

The best I can say is:

– in C (and anything C-derived), my preferred route was to work with ( lists-of ( structs-of-structs ) )

i.e:
– each Component is (essentially) a struct
– for a given System, you know you want, perhaps, the Position, Velocity, Destroyable, and Human components
– so … you make a struct of: mySpecialStruct { P, V, D, H }
– …and now you can pass a list of that mySpecialStruct from the ESManager back to the System – with no typecasting – and with no indirection / entity-ID-de-referencing

Problem is: I’m sure I’ve seen good ways of doing that “make a mySpecialStruct” with C++ templates, but I can’t remember exactly what/how :(.

The alternative is to – essentially – create miniature “monolithic GameObject” classes (which, remember, is the nightmare we’re trying to escape), just to avoid the typecasting etc.

NB: I believe that is *not* necessarily a failure, to go down that route: some systems will never need changing (i.e. anything that’s pretty generic already, like a movement system). It’s still a failure for other systems (e.g. AI, that’s very sensitive to the invention of “new” components …. or e.g. anything that runs game-logic (death, damage, etc), for the same reason).

If you can preserve *some* runtime binding of that mySpecialStruct’s contents, you could evade those problems above. Which is what I remember trying to do with C++ templates before. But I’m not a templates expert – if you can find someone who is, they might be able to provide a perfect solution.
Snorre January 12th, 2011 at 4:14 pm

So if I get your drift, using Java, I should spend some time figuring out how to create generic “container”-classes I can pass to the getAllEntities method, to ensure type safety.

I will do some thinking and get back with a post if I can develop something useful.
adam January 12th, 2011 at 5:06 pm

For java, just swallow the cast for now. Until you need / think of smoething better.

OR … make a custom List class, which you use just for passing things in to the Systems. A class which acts like the Facade / Flyweight pattern, e.g.:

MyList
{
public getCurrent( T ); // for the current index, gets the component of type T
public void advance(); // jumps to the next index; all future calls to getCurrent will access the components of the *next* entity
}
Snorre January 12th, 2011 at 5:18 pm

Tell me if I’m on to something here or if I am missing the point completly (this is a as-it.-comes-to-me-with-no-regards-to-maintainability-implementation):

class TwoContainer{
public T one;
public X two;
}

public List<TwoContainer> getAllEntitiesPossessing(Class class1, Class class2) {
TwoContainer container;
List<TwoContainer> returnList = new ArrayList<TwoContainer>();
Component fetchedComponent;

outer:
for(Entity e : entities){
container = new TwoContainer();

if((fetchedComponent = getComponent(e, class1)) != null){
container.one = (T)fetchedComponent;
}
else
continue;

if((fetchedComponent = getComponent(e, class2)) != null){
container.two = (X)fetchedComponent;
}
else
continue;

returnList.add(container);
}

return returnList;
}
Snorre January 12th, 2011 at 5:27 pm

Hmm, all my typesafety got eaten by the replay parser. Let me clarify with (T, X)

public (T extends Component, X extends Component) List getAllEntitiesPossessing(Class(T) class1, Class(X) class2) {
TwoContainer(T,X) container;
List returnList = new ArrayList();
Component fetchedComponent;

outer:
for(Entity e : entities){
container = new TwoContainer(T, X)();

if((fetchedComponent = getComponent(e, class1)) != null){
container.one = (T)fetchedComponent;
}
else
continue;

if((fetchedComponent = getComponent(e, class2)) != null){
container.two = (X)fetchedComponent;
}
else
continue;

returnList.add(container);
}

return returnList;
}

What I’m thinking is this: Have some monster getAllEntitiesPossessing method that takes and have overloaded methods for fewer classes, aka

getAllEntitiesPossessing(Class(T) class){
return getAllEntitiesPossessing(Class(T) class, null…)
}

Not maintainable?
Snorre January 12th, 2011 at 5:29 pm

Damn, the list is oc : List(T,X)
Snorre January 12th, 2011 at 5:33 pm

OR, realize that you posted a sweet solution. Thanks! I’ll use the list approach, that will work well. Feel free to remove the above mess. Talk about going about this the wrong way!
Ted Brown January 18th, 2011 at 7:43 am

Adam, thank you for your detailed blog series, and for being active in the comments. It was tremendously enlightening, and for better or (probably) worse), I’d like to share what I came up with after your indirect help.

Using Processing, I cobbled together a working entity component system that uses the id of each entity to grab a specific object from that system’s vector array. Because my power over Generics is weak, I settled short of the promised land, but this method seems like it will work for small, contained projects.

Thanks again. Naturally, critical feedback is welcome from anyone who cares to dive in.

http://www.openprocessing.org/visuals/?visualID=18023
adam January 18th, 2011 at 3:47 pm

@Ted

Cool. Could you add a page to the http://entity-systems.wikidot.com/ wiki, with a brief writeup of this approach, and with a link to your source project at the end? Also very helpful if you could write a brief overview of the source code in a way that makes it easy for people to port it to other languages.

c.f. http://entity-systems.wikidot.com/rdbms-with-code-in-systems == complete “perfect” example of a writeup page

NB: the three most important bits for a minimal writeup are:
– the first para of the page (the bit that explains verbally *only* whatever is special/different/unique about your approach – in this case, it could be as simple as saying “it’s like the RDBMS-Inspired approach, except that each system holds the components of its kind, instead of the entitymanager holding all components of all kinds” … but better worded than that :))
– the “Implementation > Concepts” section: c.f. the example, it not only lists the words being used, but it gives the *type* of each (e.g. in some implementations, Entity is not an integer. So it helps a lot to state this up-front)
– …and, finally of course: the link to your sourcecode project :)
Spiegel January 25th, 2011 at 4:54 pm

Hi, First of all great articles…

I’ve been using an entity system created by me for some time now, and since I’m doing another (improved) ES I have a couple of questions about the render part of the ES.

Example 1:
A tank is composed of tracks, body and turret, in my ES the render would take care of rendering the entire tank, having no need for child entities.

Now making the new ES, I come to the conclusion that,perhaps, I need a tree structure in entities.

Example 2:
A ship with 3 cannons, its the ship entity that has 3 children entities, one for each cannon, this is a tree concept while doing the render, in the sense that, when the ship moves the cannons should move as well.

How do I implement this? And Do I need this at all?

Should the ship be a complete entity?
Should I have entity trees?
Should I have a tree concept for the render components?

Also regarding the render components, are they data as well? or should they be seen as different since they are the view in the model view controller pattern?

Or am I just making a big mess out of this?

Thanks in advance…
adam January 25th, 2011 at 8:29 pm

ES aren’t well-suited to tree structures, although for small-scale situations, they work fine.

The problems come when you have e.g. a 50-node deep Scene Graph: you can’t iterate over components any more, your fundamental algorithms aren’t iterators – they’re tree-traversers.

In general, my advice would be to run through your data twice: once in a non-ES fashion, which pre-processes data and stores it back in the ES components, then once in an ES fashion, that uses the pre-processed data.

e.g. for rendering a scene graph, have a “pre-render” stage where you traverse your graph, writing-out the absolute coordinates of every node … then you can process the list of nodes in an ES fashion, iterating over each one and processing it without having to refer to its parent, it’s parent’s parent, … etc

…but there are obvious performance issues with running through your data twice, and in the long run you’d want to think carefully about how that interacts with your hardware.

NB: that said, many projects find that this kind of dual-run approach fits well within the performance of the hardware available. I’d perhaps try it this way to start, and see what happens.
Alex February 5th, 2011 at 4:15 pm

Hello adam, I am really interested in networking using ES. How would this work? Can you describe a little how the server will implement ES?
adam February 6th, 2011 at 6:41 pm

@Alex – sorry, I don’t know what you mean. Your question is too non-specific :(
Hans February 7th, 2011 at 5:07 am

In your code, you store components into a hash map with the following specification:

HashMap

However, adding objects to collections defined with bounded wild cards is not allowed in Java. Moreover, since you already know that your collection will only contain classes derived from the Component class, why not make it just:

HashMap

Note that in order to maintain the easy code completion feature in Eclipse, you do still need the getComponent method to have the signature you specified above.
Alex February 8th, 2011 at 12:40 am

In a networking environment, the server will implement ES with clients as Entities?

How many threads will run?

I’m thinking about a thread that will manage the game logic and other threads for listening client messages. The messages from the clients will be some events for the game logic.

Is this right? How it’s done for maximizing potential in MMOs?
adam February 8th, 2011 at 10:47 am

These blog posts are really about using an ES to write a game. What you’re talking about is how to write low-level networking code, which is a different problem. I probably wouldn’t use an ES for that – I’d just use straight imperative code with a little OOP mixed in.

If you definitely wanted to use an ES for network code, I’m not sure how I’d do it.
Tom H. February 8th, 2011 at 12:50 pm

So networking is a question I’m curious about, too, albeit at a much higher level than Alex. The commercial entity system I’m familiar with embraces message-passing, and has messages between systems (and between scripts attached to entities); it also uses the same messaging system to replicate data between instances of entities on separate computers. However, I thought I’d read you arguing against using message passing / event-driven systems with entities (although I can’t find that in your blog at the moment). How would you network an entity system?
adam February 8th, 2011 at 8:29 pm

One core aspect of the ES is that everything is merely simple data – its very easy to write generic code that will serialize, compress, transfer etc your game data around, at the granularity of components.
Alex February 10th, 2011 at 1:37 am

Can you give me some references, books about low level networking coding?
Sorry if this is off topic but I really want to find more information about this subject
and I didn’t found many articles. Thanks in advance!!!
Eldad February 23rd, 2011 at 12:56 am

Hi Adam,
The articles are great. I am looking forward for the next article – hopefully you will give us some more information about AAA performance and use cases.

Meanwhile, I’m wondering about the implementation of a collision detection system.
We will need a:
collisionDetectionSystem
collisionDetectionComponent (x,y, deltaX, deltaY, velocity, radius etc)
Entity

Let’s say that we have 1000 entities, all of them are bind to collisionDetectionComponents.
Assuming that we need to perform a simple sphere test, how would you suggest of checking if there was any collision between the objects?

IMO, since the entities are stored in a list, we will need to check them against each other (1000*1000) – which is bad.

Another suggestion is that the collisionDetectionSystem will have a 2-3D array/hash table for the X.Y,Z and on-top have linked list to hold more then 1 entity on the same X,Y,Z (array[x][y][z]->entitiy_1->entitiy_2)
Now, we can check for collision only on specific areas and reduce the amount of checks.

Any idea?

Thanks,
Eldad
Simon March 11th, 2011 at 2:19 pm

Hey there,

this is a great article. I tried this myself and it didn’t work “out of the blog” for me.
I had to cange to After that it worked for me. I might have done something wrong, because I’m not that fit with Java Genereics as well.
By the way, one can put java primitive types into collections. Just write HashMap . You can then write map.put(42, new Component) since Java 5 the compiler supports Auto-boxing and will create the Integer-Wrapper for you.

What I didn’t quite understand is the big picture. Imagine a network packet comes in, is unmarshalled and then I know, what should happen, e.g. Player moved. Will the network system itself get the Position-Property of the player and change the values or does it forward the incoming message to something like a Position-Service (which then changes the property.)? After that I need to tell other entities about the new Position of the player.
adam March 11th, 2011 at 2:58 pm

@Simon – sorry, wordpress deletes code. It’s crap. Blame WP! (I’ve tried a few plugins, but all so far caused conflicts or bugs elsewhere :( :( )

you may need to apply the corrections listed higher-up in the comments. I can’t remember if I edited and added them all :(.

No, you NEVER “need to tell other entities”. You just wait. They will automatically “discover” this fact the next time they iterate over the entire list of components (usually: once per gametick).

If you need the object to *animate* into position, then *some* additional work will need to be done. e.g. you might have a “interpolatingmovement” component, where you set fields saying what the new position is, how long it should take to get to that position, etc, and maybe do NOT update the position at all.

Instead, you have a system that checks these interpolationmovement comps each tick, moves things a little, and if an interpolation is complete … deletes that component (or zeroes it).
adam March 11th, 2011 at 3:00 pm

PS: if you’ve made this, please add it to http://entitysystems.wikidot.com/ on the page “RDBMS with code in systems”. Ideally, if you can create a (free) github account for it, then include that link on the page.

(someone will ned to change the names of some classes / methods – I came up with better names after writing this blog post. But if you’ve already got a working verison, that’s easy for someone else to do in github)
Simon March 11th, 2011 at 6:38 pm

@adam
Thank you for your fast reply.
I didn’t think of the gametick. But what about the server side of a multiplayer or even massive multiplayer game? I once read that “event based” is the way to go. However, this was written by a former Sun programmer, who worked on *Project Darkstar*, which is now know as *Red Dwarf Server*. It’s entirely event-based and one was encouraged not to programm in the “old gameloop”-fashion.
So the servers have their own gameloop / tick-mechanism? I hope it’s not getting too off-topic, but I’m really interested in the server architecture und implementation. I’m doing some research on locationbased mobile games at my university and, to be honest, a server with an own gameloop might solve some problems i encountered.
adam March 11th, 2011 at 6:45 pm

Event based game logic on any non-trivial scale is generally a stupid idea coming from people with little or no experience of building real systems. Or from people who are used to having insane amounts of spare processors (far too expensive for a game company to afford).

Because modern computers can’t do event-based processing. They can only fake it, and for non-trivial examples, the fakery breaks down, and the performance crashes to the floor.

It’s also the most natural approach for OOP. Which is why you see it come up again and again from people who haven’t been taught better (or haven’t yet learned the hard way). I learned the hard way – I made awesome small games with it, then tried to scale up, got badly burned, and gradually realised quite how stupid I’d been.
adam March 11th, 2011 at 6:48 pm

Just to be clear: I’m talking purely in the context of computer games, and the main game. Things that have to operate with millions of objects being updated with complex CPU processing hundreds of times per second.

There’s plenty of industries where their data / interactivity rate is orders of magnitude slower than games, and they can get away with things like event processing.

Even games industry uses it – e.g the networking layers ride on the back of special hardware in the PC, or code in the OS kernel, that’s there to provide a faked version. But the networking layers typically run very simple code at a much slower rate than the main game – and they have special hardware / OS support to help them.

(But even so, I’m making a gross generalization here. My point is: you should be asusming a game-loop until you’ve got a damn good reason to do otherwise)
Entity frameworks | Gray Lantern April 2nd, 2011 at 4:17 am

[...] http://t-machine.org/index.php/2010/05/09/entity-system-1-javaandroid/ [...]
The Ginkgo and the Unanswered Questions » #AltDevBlogADay April 13th, 2011 at 9:06 am

[...] coder, and me, our main C coder. I wanted a virtual-function-free component system closer to the Entity System architecture proposed by Martin Adams. Peter wanted to stick to the OO features that C++ offers. In the end, we settled for a compromise [...]
glr May 30th, 2011 at 9:17 pm

#1
Would it be a better idea to let Systems call the appropiate components from EntityManager? Leaving System such as render() without parameters, except for systems that need delta time.

Good:
- Multiple component request can be made through one method call.
- No need to make sure the correct components are being inserted through parameter. (or create definition of names for a family of components)

Bad:
- System now know about EntityManager.

—

Instead of this:
mPhysicSystem.processMovable(mEntityManager.getEntitiesOf(ENTITIES_MOVABLE), dt);
You get this
mPhysicSystem.process(dt);
or
mRenderSystem.render();

Any thoughts?
adam May 30th, 2011 at 9:40 pm

I don’t think there’s anything intrinsically wrong with systems knowing about the entity manager – so long as they’re mostly delegating to it the work of retrieving the sets of components they need from memory / wherever.

In my examples I didn’t pay much attention to the call semantics of systems – I kind of waved my hands and said “however you like”, so long as you don’t mess up any of the core rules around e.g. Don’t put logic inside compOnents
thelinuxlich June 17th, 2011 at 3:30 pm

Adam, thank you for such extensive writeup on such a paradigm shift. I’ve ported a ES implementation really close to your thinking to C#, which can be found here(work-in-progress): https://github.com/thelinuxlich/artemis_CSharp

Also, an example of game using this can be found here:
https://github.com/thelinuxlich/starwarrior_CSharp

So far, so good, but now I’ve stumbled upon a tricky topic: how can I bridge the ES with existing engines that take care of rendering, physics etc.

These tasks require complex objects(like a Body to physics) and from what I read here, should be in systems(so, a temporary system state which is generated from the “true” state of primitive component attributes)

I think these complex attributes can be initialized on systems and could be a analogue to temporary tables, a state that won’t be persisted but it is generated based on the “true” state(components). So I thought about a special component that would be populated with these complex objects by the system, to keep the entity reference and easy value lookup(also, we could write systems based on the existence of this component, which means some complex property is already initialized and ready to process). These special components would not be persisted, they are generated in game because of the primitive components…

Thoughts?
adam June 17th, 2011 at 4:11 pm

Great, thanks for doing the C# impl.

Can you add that to http://entity-systems.wikidot.com/ ?

Copy the layout (i.e. the “Source Code” header + section) of: http://entity-systems.wikidot.com/rdbms-with-code-in-systems
…making a new “C#” section to: http://entity-systems.wikidot.com/artemis-entity-system-framework
adam June 17th, 2011 at 4:17 pm

re: physics — I think this deserves a whole new page on the wiki.

I’ve created this:

http://entity-systems.wikidot.com/using-an-es-with-physics

…put your question in there, and see what answers we can come with on that page
thelinuxlich June 17th, 2011 at 6:15 pm

Ok, I’ve added it to the wiki.

My question doesn’t concern specifically physics stuff, that was an example. It is about the “glue” needed between ES and another engine(frequently OOP), hence the complex component objects.
adam June 17th, 2011 at 7:00 pm

These kinds of issues are generally quite small/obvious/simple with other systems; Physics is a special case – and makes a great case-study/example – because it’s so processor intensive and yet so common, with the same data structures/algorithms used in almost every game.

So … I think it’s a good one to discuss and work on.

Briefly, on your example, everthying you say makes sense – there’s nothing new there, AFAICS? I hope it’s obvious that many components don’t usually get persisted (for instance, not least: player-inputs from tick to tick :)).

More extreme example: in network code, lots gets serialized that normally you wouldn’t persist, and lots gets ignored that’s critical, locally.

…but with Physics, there are important questions of: efficiency, integration with 3rd party physics libs, etc. I’ve used a few physics engines, but I’m by no means an expert on them, and I’d really like to throw the question out to people who are :)
thelinuxlich June 17th, 2011 at 7:15 pm

Thank you, Adam. Hope this wil solve my problem :)
thelinuxlich June 21st, 2011 at 3:24 pm

Adam, I need to integrate the ES with a rendering engine that has at its core a messaging system: it acummulating messages and sending them at once. (and it has a number of messages that he can send each frame).
Some messages can be sent lots of times to an object, if the message is a component, it would be strange.
Messages are used to tell something about the engine (triggers for example notifies objects using messages)
If we put components as messages, are we coupling both systems too much? Do you have any advice about integrating ES with external engines that work on a evented way?
adam June 23rd, 2011 at 6:03 pm

In practical terms, you literally *cannot* send messages to entities. The nearest you can do is send messages to one of the “systems” that is processing entities each loop, and have it batch up a whole chunk of incoming messages, then apply them to its own entities on the next cycle.

Although … without knowing the details, my gut feel is that a rendering-engine based on messaging system sounds like a nightmare anyway :)
João Gonçalves September 29th, 2011 at 10:47 pm

I know there’s been so much time since the last update on this thread however this thread made be so frenetic an excited that I just want to spend some time around this model.

I was wondering if, for example, a Command Pattern would fit in here lets say for ease of networking (and not only).. For example, if I want to make a level of abstraction on the way that entities take actions and not to differ from input actions or network actions, would it be correct to create a stack that groups the actions that need to be executed tick after tick and in the specific systems we just create those actions and add them to that stack?

Don’t know if I made myself clear but basically I’m wondering how I can add that level of abstraction so that we can simplify how the systems interact with the “outer” world.

Best regards, João Gonçalves
Sinasquax November 29th, 2011 at 6:48 pm

Great article but i have some problems with this design :

You said that components can’t communicate between them, ok because there are only data but what about systems ?

Imagine you have a physic system that uses Box2D engine to process physic and if you need to disable a contact with the presolve callback of Box2D, how we can do that without using an event or notification system ?

My solution is to have a Box2D physic system that sends events with a contact as parameter of event and others systems which register to this event when needed.

Is that correct ? Thank you
adam November 29th, 2011 at 10:47 pm

Yes you can do that.

Although you’re probably better off creating temporary components that will be picked up by other systems when they scan the complete entity/component store looking for things to operate on
adam November 29th, 2011 at 10:50 pm

@Joao – one of the long term advantages of entities is that they make it a LOT easier to write generic code for memory management, network synchronisation, etc.

This is by no means unique to an ES – it happens with all coding approaches that provide a strongly enforced set of rules on object creation and class design. ES happens to be one of the easier ones for doing this stuff – simPly because the component approach is VERY strongly enforced, de facto.
seth May 13th, 2012 at 7:48 pm

I know this article’s rather old, but it seems to be the most relevant for my question. I’ve read through all your articles on Entity-Component design (as well as any others I could find), and I’ve been trying to implement an ES in C++ for a game idea I’ve had.

You mention that if you were working in C++ you would use templates. And for the life of me I can’t figure out what that would actually look like. I know enough about templates to see the benefit, but I’m just not seeing how an ES would work like that. Do you have any examples of an ES implemented in C++? The Entity Manager is where I’m having the real problem, but anything would be greatly appreciated.

And great job on the articles! I just wish there was more information on Entity-Component design in general (everything I’ve found either redirects here, or has a not-quite-right ES design).
adam May 13th, 2012 at 8:47 pm

@seth – I’ve made a couple of half-hearted attempts last year at doing a pure C++ version based on template metaprogramming, but I’m too out of practice with templates, and I wasn’t happy with any of them.

In theory, it ought to work well. The problem is that I haven’t written any C++ in a long time, and I’m very rusty. In practice, when I struggled and went looking for 2nd opinions, most of the hardcore C++ people I know haven’t adopted ES’s yet and won’t focus on the problem – they instead try to sell me on other, “more C++ style” solutions. Which is usually missing the point, sadly. In practice, C++ coding with the current free IDE’s I have access to makes my head hurt (Eclipse C++ support is (to put it nicely) “weak”, and let’s not even mention Xcode). I actually find it easier to write C++ on paper and then b*tch-slap the IDE into accepting it, than to write it with the IDE’s ‘help’ :(.

So … sorry!

I suggest you try tweeting at some well known tech / games people who tweet about C++ programming – there are interesting problems here, and they might find it fun to have a go with.
seth May 13th, 2012 at 10:28 pm

Well, not exactly what I wanted to hear, but thanks for the reply! I guess for the time being I’ll try asking around like you suggest, and continue hammering together code until it “clicks”… I plan on getting this game finished one way or another, so if I have to rewrite it a few times at least I’ll learn how *not* to write it.

Anyway, thanks again!
Tom H. May 16th, 2012 at 4:02 pm

The C++ skeleton of an Entity System at http://entity-systems.wikidot.com/ really does scale up to a small game (in my hobby playing-around). It needs a couple more convenience functions, like removing components from an entity. It uses some of the antipatterns Adam disparages, like having the Entity hold a map of its components, but I think when you hit performance problems with it it wouldn’t be hard to factor them out.
adam May 16th, 2012 at 5:06 pm

If you can’t see a way to do a wonderfully elegant template metaprogramming monster … please instead follow the “Entity System Beta” (described on this blog ans on http://entity-systems.wikidot.com/rdbms-beta ) – it’s been ported to several langs, but not C++ yet. It would be great to get a clean port of that.

(it’s “good enough” for prototyping – I’ve had it running 100′s of game entities at 30 FPS on old Android phones)

Entity System 1: Java/Android

Why build an Entity System?

Building the ES for Android

Getting started: early ES decisions

Getting started: Components in Java

Fetching a Component from an Entity

Time-out: HashMap

Fetching a Component from an Entity: Redux

Using the ES with Systems

Finis

One last thought…

153 Responses to “Entity System 1: Java/Android”

Leave a Reply

Get updates automatically

Blogroll

network programming

Meta

Categories