T=Machine

Hume just posted his Lessons Learned from the warmup for Ludum Dare 23 (48 hours to write a game from scratch – starts this weekend!) – and his positive experience using an Entity System.

In his epic comment (sparked by a different Adam – not me, honest), is this gem:

“Using the entity system for the first time was unreal to me. It’s like polymorphic code. I did really weird things on the fly. For example:

- In the health processor, if the enemy was just destroyed, set a flag in the lifecycle component.
- In the lifecycle processor, if the fresh kill flag is set, extract its loot component and put that into a new entity with a small randomized velocity component and a gravity component so that the loot drops; then, remove most of the other components from the entity and add an explosion component.

The “enemy” still has the same entity ID — any other components that are looking for that entity will still find it (e.g. missiles homing in on the wreckage, or score processors looking for slain entities) — but by swapping one set of data with another, its implementation has changed from an enemy to some kind of non-interactive effect object.”

(emphasis mine)

Identity. It’s important.

(Quick sidenote: for all the people asking questions like “but … which variables do I put in Component A as opposed to Component B? How do I manage Events in an Entity System? … etc” – Hume’s approach above is a good concrete example of the first-draft, easy-to-write way of doing things. Copy it.)

Identity in games

This is one of those things that newbie game programmers seem to underestimate, frequently.

And when I say “newbie” I include “experienced, skilled programmers with 10+ years of coding experience – but who haven’t yet shipped a game of their *own*”.

(e.g. I’ve seen a couple of studios that started as Digital Agencies, or as Animation Studios, etc – that then transitioned to writing their own games. This is the kind of thing that they often struggle with. Not for lack of skill or general programming experience, but for lack of the domain-specific experience of game coding)

Examples of Identity in games, off the top of my head – all of these are independent, and interact in complex ways with each other :

1. Game-role: e.g. … “enemy”, “powerup”, “start location”
2. Code ‘object’ (in OOP terms): e.g. … “the sprite you are drawing at position (4,5) is part of Object X. X is coloured THIS colour”
3. Gameplay ‘object’: e.g. … “the sprite at (4,5) represents a Tank. If a Tank sprite ever touches a Glass sprite, we need to play the Broken Glass noise”
4. Physics element: e.g. … “5 milliseconds ago, our Physics Engine thought this thing was THERE. Now it’s over HERE. Don’t confuse the Physics Engine! Make sure it ‘knows’ they are the same object – not two separate objects”
5. Network “master/clone”: e.g. … in multiplayer, there are N copies of my avatar: one per computer in the game. One of those N is the original – and changes to the original are constantly used to overwrite the clones; changes to clones are LOCAL ONLY and are discarded. Which is original? What do we do with incoming “changes” – which local Code Object do we apply them to? (your Code Object will be different from my Code Object – but they’ll both be the same identical Network Object, save mine is flagged “clone”)
6. Proper Noun object: e.g. … “The Player’s Tank” is a very specific tank out of all the Tanks in the game. Many lines of game code don’t care about anything except finding and operating on that specific tank.
7. Game-Over representation: e.g. … after the player has killed all the enemies, and they see a Game Over (won/lost/etc) screen, and you want to list all the enemies they killed … how do you do that? The enemies – by definition – no longer exist. They got killed, removed from the screen, removed from memory. You could store just the absolute numbers – but what if you want to draw them, or replay the death animations?
8. …etc

Identity in Entity Systems

ES’s traditionally give you a SINGLE concept of Identity: the Entity (usually implemented as a single Integer). Hmm. That sounds worryingly bad, given what I wrote above. One identity cannot – by definition – encompass multiple, independent, interrelated identities.

But we’re being a bit too literal here. ES’s give you one PRIMARY identity, but they also give you a bunch of SECONDARY identities. So, in practice…

Secondary Identities in an ES

In OOP, the Object is atomic, and the Class is atomic. You cannot “split” an Object, nor a Class, without re-defining it (usually: re-compile).

In ES, the Entity is atomic, and the Component is atomic. But the equivalent of an OOP Object – i.e. “an Entity plus zero or more Components” – is *not* atomic. It can be split.

And from there comes the secondary identities…

A Primary Identity: e.g. “The Player’s Tank” (specific)
A Primary Identity: e.g. “a Gun Component” (generic)

A Secondary Identity: e.g. “The Gun component … of the Player’s Tank Entity” (specific)

Revisiting my ad-hoc list of Game Identities above, I hope it’s clear that you can easily re-write most of those in terms of secondary identity.

And – bonus! – suddenly the relationships between them start to become (a little) clearer and cleaner. Easier for humans to reason about. Easier *for you to debug*. Easier *for you to design new features*.

Global Identity vs. Local Identity

Noticeably, the network-related Identities are still hard to deal with.

On *my* computer, I can’t reference entities on *your* computer. I cannot store: “The Gun component … of YOUR player’s tank”, because your “Player’s Tank” only exists in the memory of your computer – not mine.

There are (trivially) obvious solutions / approaches here, not least: make your Entity integers global. e.g. split the 64bit Integer into 2 32bit Integers: first Integer is the computer that an Entity lives on, the second is the local Entity Integer. Combined, they are a “global Entity ID”.

(I’m grossly over-simplifying there – if you’re interested in this, google for “globally unique identifiers” – the problems and solutions have been around for decades. Don’t re-invent the wheel)

But … at this point, they also offer you the chance to consider your game’s network architecture. Are you peer-to-peer, or client-server?

For instance, P2P architectures practically beg for unique Global entity numbers. But C/S architectures can happily live off non-global. For instance:

• On each client, there are ONLY local Entity numbers
• When the client receives data from the server, it generates new, local, Entities
• …and adds a “ServerGenerated” component to each one, so it’s easy to see that they are “special” in some ways. That component could hold info like “the time in milliseconds that we last received an update on this object” – which is very useful for doing dead-reckoning, to make your remote objects appear to move smoothly on the local screen
• The server *does* partition all entities from all machines. But none of the clients need to know that

Or, to take it further, if your network arch is any good at all for high-paced gaming:

• The server differentiates between:
  1. The entity that the game-rules are operating on
  2. The entity that client 1 *believes* is current
  3. …ditto for client 2, client 3 … etc (each has their own one)
  4. The entity that the game-rules accept (e.g. if a hacked client has injected false info, the game-rules may override / rewrite some data in the local object)
• The server also tags all the entities for a single in-game object as being “perspectives on the same thing”, so that it can keep them in synch with each other
• The server does Clever Stuff, e.g.:
  • Every 2 milliseconds, it looks at the “current entity”, and compares it to the “client’s belief of that entity”. If it finds any differences, it sends a network message to the client, telling it that “you’re wrong, my friend: that entity’s components have changed their data. They are now X, Y and Z”

… or something like that. Again, I’m grossly over-simplifying – if you want to write decent network code, Google is your friend. But the fastest / lowest latency multiplayer code tends to work something like that.

How?

Ah, well.

What do you think?

(hint: you can do wonders using Reflection/Introspection on your entity / components. By their nature, they’re easy to write generic code for.

But you WILL need some extra metadata – to the extent that you may wish to ‘upgrade’ your Entity System into a SuperEntity System – something with a bit more expressive power, to handle the concept of multiple simultaneous *different* versions of the same Entity. Ouch)

Yeah, I’m bailing on you here. Too little time to write much right now – and it’s been a *long* time since I’ve implemented this level of network code for an ES. So, I’m going to have to think hard about it, refresh my memory, re-think what I think I knew. Will take some time…

Filed under : entity systems, games design, network programming, networking, programming

2 Comments »

Thanks to Mike Leahy for spotting this:

http://blog.gemserk.com/2012/02/02/how-we-use-box2d-with-artemis/

…a short blog post (with code) on how a team is integrating Box2D (a very well known open source physics lib) with Artemis (a java implementation of Entity Systems which starts from the same point as my Entity Systems posts, but fleshes it out)

Filed under : MMOG development, entity systems, games design, programming

1 Comment »

For a couple of years now, Firefox has had a nifty feature where if it crashes – for ANY reason – the next time you run it, it gives you a “menu” of the windows you had open, and lets you selectively re-open them. As a bonus, this menu is just a plain window – you can ignore it (if you’re too busy right now), and open some other windows in parallel.

(even if it’s not Firefox that crashed – e.g. a laptop battery ran out, or someone tripped over your power cable – you still get this window. This is a huge help on OS X, which crashes quite a lot if you’re running Apple’s dev tools :))

I’d wondered a few times “what happens if it crashes again, before you’ve selected which things from that menu you want to re-open?”

I just found out the hard way: actually, it gives you a new menu, where one of the items is … the previous menu. It nests, all the way down (I had a crash-of-a-crash-of-a-crash – and I got everything back perfectly. Very useful, too – one of the deeply nested open windows was an important form I’d forgotten I’d not yet sent).

On the downside, if you close Firefox, and re-open it normally, the menu gets blown away – but that’s probably to do with the “five-years-and-counting still unfixed” bugs in Firefox where “remember open windows on startup” doesn’t work properly (there’s some epic threads in their bug tracker, many complaints, but no fix yet).

Still, for the most part, this is a very nice approach to application-crashing. One to remember when designing your own document-based applications…

Filed under : iphone, programming

1 Comment »

In the ongoing, epic comments (300+ and counting!) for my Entity Systems posts, one of the recurring questions is:

What makes a good Component?
How should I split my conceptual model into Entities and Components?
How should I split my algorithms and methods into Systems?

It’s often difficult to answer these questions without concrete examples, which are thin on the ground.

Good news, then…

Paul Gestwicki runs a CS315: Game Programming course, and last year his students used an Entity System to implement their game – Morgan’s Raid. In a recent email conversation, he mentioned he’d been monitoring the actual number – and nature – of the Components and Systems that the teams developed and used on the project.

He’s now posted this list, along with some brief analysis.

All systems and Components

Read Paul’s post – there are some caveats he mentions, and there’s a useful diagram showing roughly how many systems were using each component.

I strongly recommend you play the game too (it’s free, and quick to play) so you can get an idea straight away – just from the names – what data and code some of these contain.

To recap, here’s the list:

“For your reading convenience, here’s a simple tabular view of the systems and components

Systems	Components
BackgroundTileSystem CityNameSystem DestinationSystem FadingSystem GPSToScreenSystem HoverableSystem ImageRenderingSystem IntInterpolationSystem MinimumSleepTimeSystem MorganLocationSystem NightRenderingSystem OnClickMoveHereSystem OnScreenBoundingBoxSystem RaidSoundSystem RailwaySystem ReputationSystem RevealingTextSystem SpeedSystem StepwisePositionInterpolationSystem SunSystem TimePassingSystem TimeTriggeredSystem TownArrivalSystem TownArrowSystem TownUnderSiegeSystem	AnimationRenderable ArrivesAtTownIndex BackgroundTile BeenRaided CentersOnGPS CityData CityImages CityName CityPopulation CityTargets CommandPoint Destination DoesCityHaveMilitia DoesCityLoseGame DoesCityWinGame GPSPosition GPSPositionList HasMorganGPS Hoverable ImageRenderable ImageRenderLayer InGameTime IntInterpolated	MinimumSleepTimeOverride Morgan MorganLocation MovesOnClick OnClickMoveHere OnScreenBoundingBox OnScreenBoundingBoxList PositionInterpolated Raidable Raider Railway Reputation ReputationValue RevealingText Road RoadsToCity RouteTaken Speed Sun Terrain TimeToRaid TimeTriggeredEvent TownAdjacency TownArrow TownUnderSeige

”

Things I noticed straight away:

1. There’s approximately 2:1 ratio of “components” to “systems”
2. In Paul’s post, all the Systems are accessing *something*
3. In Paul’s post, quite a few Components are NOT accessed
4. A couple of components are used by almost every System
5. The names of some Systems suggest they’re very trivial – perhaps only a dozen lines of effective code
6. The names of some Components suggest they’re being designed in an OOP hierarchy

NB: I haven’t had time to look at the source code, but it’s freely downloadable here, so I’d recommend having a look if you have time.

How many Components per System?

I’ve generally started people off with: “aim for 1:1 ratio”. This is mainly to kick them out of the traditional class-based OOP mindset. In practice, there’s really no need to stick to that religiously – once you get the hang of ES design, you should be freely adding and subtracting components all over the place.

In reality, the pressures on “number of systems” and “number of components” are independent. Ideally, you add a new system when you have a major new concept to add to your game – e.g. “previously I was using hand-made jumping, now I want to add a complete physics-driven approach. This will mean changing collision-detection, changing the core game-loop, etc”.

Ideally, you add a new component when you have a new “dimension” to the game objects. For instance, if you’re adding a physics System, you may not need to add any new Components – it might be that all you need is Location (containing x,y,x position and dx,dy,dz velocity) and RenderState (containing screen-pixels x,y) – and that you already have those components.

Zero systems per component

One of the advantages of an ES is that old code can just fall off the radar and disappear. So I’m not surprised at all to see some components that appear to be unused – and it’s MUCH easier to simply delete this code from your project than it would be on a traditional OOP project. Does anything reference that data? If so, it’s a set of particular systems. For each system, you can look at MERELY the system and the component, and make a very quick decision about whether you still need this access – or if you can refactor to move (some of) it somewhere else. The amount of code you need to read to make such decisions safely is typically very small – i.e. easy, quick, and less error-prone.

Many systems per component

This is fine. However, it can also be an early-warning sign of a design or code-architecture bug. Sometimes, there are components that – innately – are just needed all over the place. For instance, in a team-based game, the component saying which “team” a given object/player/item/building belongs to is likely to affect almost every piece of algorithm code across the board. It’ll be referenced by many systems.

On the flip-side, it may be a sign that you’ve put too much data into one component. There are two usual versions of this:

1. You have – say – 8 variables in the struct where you should instead have two structs (components), one with 5 variables, the other with 3.
2. You have – say – 4 variables in the struct, but different systems are using those variables to mean different things. It works OK for now, but it’s very fragile – as soon as the different meanings diverge even a little, your code is going to start breaking

Of course, you get this exact problem in traditional OOP setups, but with an ES it’s trivial to fix. Split – or duplicate – the Component, change a few references in the Systems, and you’re done. If it turns out a week later that the split wasn’t necessary – or worse, was a step backwards (e.g. you find yourself frequently synching the data between those components) – it’s extremely cheap to swap it back.

By contrast, with OOP, this is a nightmare scenario, because you have to worry about every method on the original class. Does that method:

1. Need to exist on both the new classes, or just one?
2. Work correctly for the new class it will be on – or does it currently rely on some of the data (and shoudln’t) and will need to be re-written?
3. Get used by other parts of the codebase in ways that will break if/when you split the class?

Thoughts, Suggestions?

…this is just a lightning quick analysis, but I strongly invite you to do you own digging into the classes – and the codebase – and come up with your own thoughts and feedback. We have here a convenient, real-life, list of components/systems – something to dig our teeth into, and debate the rights and wrongs of each decision. And I’m sure the students involved on the project would be interested in your feedback on their approaches :)

Filed under : entity systems, games design, programming

19 Comments »

Ever seen this hard to notice bug in an iPhone / Mac project? Caused by a flaw (bug?) in Apple’s own API:

NSDictionary * map;
…
NSObject * key;
NSObject * value;
…
// NB: next line will frequently crash at runtime
// …but even if doesn’t crash, it probably won’t do
// what you’d expect it to:
[map setObject:value forKey:key];

if( value != [map objectForKey:key] ) // HA!
{
NSAssert( 1 == 0, @”What the **** just happened??” );
}

NSDictionary is one of the nastier things I’ve seen in a programming language: in an Object-Oriented Language, it’s a class that refuses to take Objects as arguments, *but pretends to*. If you attempt it, it either crashes, or it invalidates your objects, breaking contracts all over the place.

ObjC’s bizarre design

In the days pre-OOP, a “dictionary” was something that mapped:

“a string”
to
“anything” (usually: basic datatypes – e.g. strings, integers, floats, etc).

In the days of OOP, the same thing is usually called a “map” (which is a better term) – although the terms are synonymous – and maps:

“an object”
to
“another object”

What did Apple/NextStep/Crazy-Guys-Behind ObjC do?

NSDictionary: maps:

“STRINGS ONLY” (no objects allowed!)
to
“OBJECTS ONLY” (no core datatypes allowed!)

But … I can use an NSObject as key?

Yep – but Apple’s internal implementation takes a *copy* of the object, and uses that as a key – rather than using the object that you gave it. This is a common problem in OOP languages and implementations of Map – e.g. Java does the same thing.

Unfortunately, this means that you can call “setObject:forKey:”, and then “objectForKey:” will return nil *for the same key*.

In Java and other OOP languages, you are required to implement a custom “isObjectEqualToObject” method. In ObjC too – except that that method is ILLEGAL if you’re using CoreData.

And ObjC will crash too, as a bonus

I’ve never seen this in other OOP languages, but in ObjC *additionally*: if you don’t manually add to the object you pass-in … it crashes at runtime. Yay!

How come? AFAICT, Apple’s header file is wrong:

- (void)setObject:(id)anObject forKey:(id)aKey // You lie!

it seems the implementation of that method is:

- (void)setObject:(id)anObject forKey:(id<NSCopying>)aKey; // the correct signature?

Net result? Code that happily compiles … will crash. ARGH!

Two reasons to beware…

…so what’s the other one?

Ah, just the one we see again and again on live projects, wasting hours and hours of time:

NSDictionary* dictionary = [NSDictionary dictionaryWithObjectsAndKeys:
object1, key1,
object2, key2,
nil];

NSLog( @”dictionary = %@”, dictionary ); // but it only has one key/value pair. ?!?!?

Ah, well … that nil … what happens when object2 is nil? Oh, damn.

What about if key2 is nil? Now we’re really nasty … we’ve given it “half” of key/value pair. Nice!

Filed under : iphone, programming

5 Comments »

It looked so good, finally – a decent Git client for Mac.

But, no. It just – simply – doesn’t work (version 1.2.1). Great for browsing, but sometimes you simply can’t commit changes to a repository! (a few hours earlier, it was committing fine. Nothing had changed in the meantime)

And what do you get as an error, when you try?

“A git error occurred.”

Seems the folks at GitHub know this is unacceptable. Because they also put a message onto the (invisible) system console:

-[GHErrorSheetController presentWithError:] [Line 84]
Presenting with error, but let’s do better guys: Error Domain=GHGitControllerErrorDomain Code=556 UserInfo=0x117f63fa0 “A git error occurred.”

For the record, there was nothing wrong, and no errors – all other clients worked fine. So it’s probably an internal problem with GitHub’s code. They know it’s a problem (they wrote an error for it) but they won’t tell you what that problem is. Classy :)

Filed under : programming

2 Comments »

Two things here: if you run any Rails site, check out the security hole ASAP if you haven’t already. You might be safe – but given that even GitHub wasn’t, I’d double check if I were you. (The Rails community seemingly isn’t patching it – and there’s nothing recent on the Security list. Which leaves me going: WTF? The evidence is right there on GitHub of how bad this is right now, in the wild).

Secondly … what just happened? Apart from doom and gloom and “the end of every unpatched Rails site on the planet”, there’s a fun story behind this one. As someone put it “it’s the whitest of white-hat attacks” (i.e. the “attacker”‘s motives appear extremely innocent – but foolish and naive)

It seems that GitHub got hit by the world’s nastiest security hole, in Rails – trivial to take advantage of, and utterly lethal. The hole appears to allow pretty much anyone, any time, to do anything, anywhere – while PRETENDING to be any other user of the system. So, for instance, in the attack itself, someone inserted arbitrary source code into a project they had no right to.

Hmm. That’s bad. It effectively destroys GitHub’s entire business (it’s already fixed, don’t worry)

But it gets worse … it’s a flaw in the RoR framework, not GitHub itself (although apparently GitHub’s authors were supposed to know about the flaw by reading the Rails docs, as far as I can tell from a quick glimpse at the background). Rails authors have (allegedly) known about it and underestimated how bad it is in the wild, and left Rails completely open with zero security by default.

So, allegedly, the same attack works for most of the web’s large Web 2.0 sites – any of them that run on Rails.

WTFOMGBBQ!

Who was the perpetrator of this attack? Ah, well…

made an impossible issue, a post that GitHub’s database believed was created 1,000 years in the future.

Classy. Dangerous (high risk of someone calling the police and the lawyers), but if people won’t believe you, and *close* your issues, claiming it’s not that important, what more amusing way to prove them wrong?

Whoops, shouldn’t have done that

I can’t state this strongly enough: never attack a live system. Just … don’t.

Any demonstration of a security flaw has to be done very carefully – people have been arrested for demonstrating a flaw allegedly *at the owner’s request*, because under some jurisdiction’s it’s technically a crime even if you’re given permission. In general, security researchers never show a flaw on a real system – they explain how to, and do it on a dummy system, so no-one can arrest them.

(why arrest the researcher? Usually seems to be no reason beyond ass-covering by executives and lawyers, and a petty vindictiveness)

Homakov appears to have been ignorant of this little maxim, hence I’m writing it here, let as many people as possible know: never attack a live system (unless you’re very sure the owners and the police won’t come after you)!

GitHub’s response

On the plus side, they fixed it within hours, on a weekend. And then proceeded to tell every single user what had happened. And did so in a clever way – they put a block on all GitHub accounts that practically forces you to read their “here’s what happened, but we’ve fixed it” message. They could have kept it quiet.

Which is all rather wonderful and reassuring.

On the minus side, IMHO they rather misrepresented what actually happened, portraying it more as a malicious attack, and something they fixed, rather than what it was – the overspill from an argument between developers on some software that GitHub uses.

And they initially reported they’d “suspended” the user’s account. Normally I’d support this action – generally it’s a bad idea to let it be known you’ll accept attacks and not fight back. But in this case it appears that GitHub didn’t read the f***ing manual, and the maintainers apparently (based on reading their tickets on the GitHub DB) refused to accept it was a serious problem – and apparently didn’t care that one of their own high-profile clients was wide open and insecure. The attack wasn’t even against GitHub per se – it was against the Rails team who weren’t acting. IF it had e.g. been a defacement of GitHub’s main site, that would have been different, both in impact and in intent. Instead, the attack appears to be a genuinely dumb act by someone being naive.

Seems that GitHub agreed – although their reporting is a bit weak, it happened days ago, but they never thought to edit any of their material and back-link it.

“Now that we’ve had a chance to review his activity, and have determined that no malicious intent was present, @homakov’s account has been reinstated.

…and it’s pleasing to see that their reaction included a small mea culpa for being unclear in what they expect (although anyone dealing with security ought to be aware of this stuff as “standard practice”, sometimes it’s not security experts who find the holes):

“We haven’t been as clear as we should have been on how to responsibly disclose security problems, and for that I’m sorry. To prevent future confusion about security-related account suspension, and to make explicit our stance on responsible disclosure, we have added a section entitled Responsible Disclosure of Security Vulnerabilities to our Security policy.”

Rails’s response

I’d expect: shame, weeping, and BEGGING the web world to forgive their foolishness. I’m not sure, but it’s going to be interesting to watch. As of right now, the demo’s of the flaw are still live. I particularly like one commenter’s:

drogus closed the issue 5 days ago

kennyj commented

5 days ago

“I’m closing it (again).
@drogus was close it, but it still open.
github bug?”

Closed

kennyj closed the issue 5 days ago

“github bug?” LOL, no – massive security flaw :).

Filed under : amusing, marketing and PR, programming, security, server admin, system architecture, web 2.0

0 Comment »

I’ve only played with this for a few minutes, but so far it seems to have an excellent, simple, clear GUI with the core features I’d expect. Way better than any other Git client I’ve seen for Mac. And it’s free!

GitHub for Mac v1.2.1 (NB: works with any Git server, not just GitHub.com!)

I’ve noticed some cosmetic bugs – e.g. it renders all the user-account avatars completely wrong (puts wrong image next to each commiter) – so I’d advise “use this with caution” until you’re sure it’s got no fatal bugs. Although, since it’s from GitHub.com, I expect it’s pretty robust.

Filed under : agile, programming

0 Comment »

There’s good reasons for adopting Mongo, I’m unconvinced (but open-minded) that performance is one of them. Here’s a ROFLMAO viewpoint on it:

“If your write fails, you’re ****ed”

Obviously, MySQL’s not perfect, but in most cases I’ve seen, it’s been lack of competence on the developer side, and the lack of basic DBA skills – not problems with MySQL itself – that’s broken scalability. In which case, I’m a little suspicious that a company that fails to scale MySQL will equally fail to write their code correctly on Mongo. In many ways, throwing away SQL makes it much easier to prevent scalability…

Filed under : MMOG development, amusing, network programming, programming, system architecture

0 Comment »

Looks like a “normal” KickStarter project for a new Tower Defence game.

Halfway through the demo video, it switches to “here’s how I’ve been using GA to detect game-design flaws, and to test ideas in tweaking game design”.

Something I’ve wanted to do for more than a decade, but could never find a company who’d take it seriously :). I really hope this iPad game does well – would be great to see a poster-child / real-world demonstration of a workable technique here.

Filed under : games design, iphone, programming

0 Comment »

As a free-time project, I’ve been writing a Risk clone (*) for iPad.

One of the bits I like best right now is that you can give it the URL of *any* SVG file on the web, and it automatically turns it into a Risk map.

(e.g. all the maps in Wikipedia articles are SVG files – it’s a common file format with good browser support)

This was one of those “interesting” technical challenges – I had to find an algorithm that would automatically work out which territories a human would “assume” were connected to each other.

I’m using an open-source SVG library which works fine for basic SVG files but has a lot of bugs with the more esoteric ones. I’ve already fixed a few of the major bugs (they’re now merged into the GitHub project) – but I’d like to get more SVG files to test with.

The one thing to bear in mind is that the colour-data gets wiped when it imports. So … SVG files that make heavy use of different colours or gradient-fills/pattern-fills lose detail when imported.

Also, files where none of the elements are close enough to be deemed “connected territories” … work poorly.

Everything else works fine.

So … if you’ve got any, please post a comment here with URL, or email them to me directly (address in the About link at top of this page).

(*) – I say “clone” because it’s the same genre – but the gameplay is “fixed” quite a lot. If you once loved Risk, but grew to hate it, you’ll see why I wanted to change the baic game design :).

Filed under : computer games, games design, iphone, programming

0 Comment »

What happens when you get 2 developers working together, sharing their source? What about 10? … or a 100?

There was a dream, 20 years ago, that the total would be greater than the sum of the parts. That developers could *re-use* each-other’s code.

Sadly, that dream – in 2012 – is poisoned.

What I’m going to describe here happens a lot – although in absolute terms, I hope it’s just a drop in the ocean. Maybe it’s nothing to worry about. Or maybe … well. In the last 15-odd GitHub projects I’ve tried to use, it affected more than a third of them. Such tiny stats are statistically meaningless, of course – but if you look at the causes of this, I think it’s more likely part of a general trend – and that really is worrying.

So. What’s going on?

The curse of Github

I love GitHub, I’m a paying member (and I regularly sell it to clients and colleagues) but … in some ways, it’s IMHO actively preventing collaboration.

Just to be clear: it doesn’t have to be this way – you can run your own projects on GitHub and prevent this happening.

But … GitHub makes this the path of least resistance, and that means – in the world of Open Source – it’s the path that gets most followed

When you fix a bug on GitHub, you have to wait for the original project author to “accept” your fix.

If they don’t accept it, as far as collaboration goes: you’re screwed. There is no “plan B” for collaboration.

Your only option is to tell the world:

“Stop using his project! It sucks! Use my project instead! I promise I’ll be a better merger!”

But then … if *you* stop accepting fixes for a while, one of the developers fixing YOUR bugs will have to do the same thing.

And each of these “Stop! Use mine instead!” calls is one-way: once another developer who’s making use of the source moves to a sub-fork, they can never go back. In theory, the original Author could do a back-dated merge … but in reality, that won’t happen, because of the cost involved:

Back-dated merging is combinatorially expensive

In practice, that’s more expensive than a normal person can afford, in terms of time and effort.

For each SubAuthor they want to back-merge with, they have to check every single change that person has made … against every change that they’ve merged already, from every single source. Otherwise they break the previously-merged code. Usually, each individual SubAuthor makes an incompatible change sooner or later – and so prevents the original Author from ever merging with them.

It’s no surprise – usually by this point the Sub Author has given up on the original Author (can you blame them? the Author has disappeared and ignored merge requests for months or years by this point)

So, in practice, very few GitHub authors (so far: none that I’ve seen) re-merge SubAuthor projects once the SubAuthor has really got going. On the projects I’ve been involved in, when a popular SubAuthor disappears for a while, there’s been a desperate scramble by the SubSubAuthors to find the guy/gal and beg/bribe/bully them into merging – otherwise we know that our combined efforts are about to be blown up.

What? Well …

The actions of the Author can undo the work of the Collaborators

Say you have Author “A”, and 3 people making changes and fixes to the code (“B”, “C”, and “D”).

At first, while A accepts merges quickly, B, C and D are all sharing code together – in practice, they are collaborating. However, they are not truly sharing code – GitHub does not allow this – they are sharing code with a Master (A), who is forwarding their work to all 3 of them.

When A disappears, B C and D can no longer collaborate. If A disappears with merges pending … then B/C/D find they have 3 distinct codebases, and no way within GitHub to do a simple cross-merge.

Now, the situation is not lost – if B, C, and D get in contact (somehow) and negotiate which one of them is going to become “the primary SubAuthor” (somehow), and they issue manual patches to each other’s code (surprisingly tricky to do on GitHub) … then they can resume collaboration. I’ve done this myself – it works. But it’s massively more complex than the process they were using before, which was *one-click-merge*.

In practice, at this point B/C/D will stop collaborating. Sad, but true. This happens over and over again on GitHub projects – when a SubAuthor arises, the other collaborators stop collaborating and become new SubAuthors in their own right.

Often it feels like watching a religion split, with each of the senior priests declaring themself “the new Prophet”, and going forth to spread (their) word…

Net effect: GitHub may be killing open-source projects

In theory, GitHub is wonderful.

But the combination of its bad design around some core use-cases, and its intransigence when it comes to the VERY common case of a single person disappearing … have lead to the point where I believe it’s killing projects. This is a gross generalization – and not every project that loses its Author will get this problem – but I’ve encountered more and more “dead” projects on GitHub over the course of 2011.

Of course … the way GitHub is designed, *those projects do not appear to be dead*. Often they appear to be very much “alive” – there’s tonnes of activity.

But all that activity is going on in radically different and massively incompatible forks. It’s wasted time and energy, it’s programmers fixing the same bugs – multiple times – because they are NOT collaborating any more.

In the case I cited at the start, 100-plus developers have (probably) re-written the same fixes for the same problems.

i.e. the total effect of this project is tending towards ONE HUNDRED TIMES less than the sum of its parts.

Note: LESS … not more!

There’s some value there, still – anyone can come along and start from the original project and make their own fork. But it’s a sad and sorry fraction of what the Open Source world dreamed of when the word Collaboration was fresh and exciting.

This is UnCollaboration. And its becoming depressingly common.

Filed under : advocacy, community, programming

25 Comments »

(I’m prototyping a new game (working title: “ChessQuest”) – original post here)

Major changes:

1. Enemies have health, and can be killed by touching them
2. Performance is another 30% faster (should be running OK on most phones now?)
3. Enemies have a direction indicator (not necessary right now, but it’ll become important in a later version…)

Download link

Chess Quest-0.4.0

Filed under : android, computer games, dev-process, games design, programming

0 Comment »

http://www.java-gaming.org/topics/lwjgl16k/25093/view.html

“the LWJGL16k competition starts right here, right now.” – Cas

The rules

First deadline: 25th November 2011
First task: get a black screen running using LWJGL

“you’ve got 7 days. All I’m looking for at this stage from you is a blank window opening up and maybe a rotating square or whatever. Of course complete games are even more welcome but the idea is to get something shipped.”

Well, what are you waiting for?

If you’ve got Eclipse installed, all you need do is download the LWJGL library, copy/paste the 50-line minimal project from the Wiki, and submit your entry!

(I believe Cas is onto a good thing here … force people to realise how easy it is to make a game if you focus on small-but-visible steps done *quickly* – No more procrastination!)

Filed under : agile, games design, programming

0 Comment »

UPDATE: So … it seems Suffusion has (buried deep inside the config) a way of displaying category pages with a higher-level workaround to the WP “missing feature”. Although so far I can’t find a way to set the settings on individual pages (which is what you generally need). So, I’ll leave this post up – it’s a good starting point for customizing Suffusion’s page-of-pages (which is also, it seems, how you would customize its Category pages.

Just to be clear, this is not a bug in Suffision, although it’s an oversight and I hope it’ll be included in a future version. The core problem is in WordPress: people have been requesting/expecting this feature for the past 5+ years, it’s pretty basic, but for now the WP folks haven’t added it.

Problem: List all the posts in a single category

This is VERY frequently requested by WP users: you want to have a Page on your site which lists the posts that are in a single Category.

WP has a very low-level way of doing this – which is very error-prone, hard to maintain (it’s hard-coded to database ID’s!), and requires you to break every single theme you own. Every time the theme is updated, you have to manually re-implement the fix!

Fortunately, there’s a minimal workaround (which is documented in the WP docs now (scroll to bottom)) which still (!) requires some theme editing.

Fortunately, Suffusion already has most of this workaround implemented, so we can make the fix without screwing around with the theme so much. The source code in Suffusion is almost identical to the sample provided by WP – but unfortunately it’s missing a key part. In Suffusion, you CAN list posts on a Page, but you CANNOT filter those posts by category.

Solution: Combine WP’s source with Suffusion, with a bit of safety improvement

So, edit the file “posts.php”, titled (in Suffusion): “Page of Posts”.

Where it has these lines:

$args = array(
	'orderby' => 'date',
	'order' => 'DESC',
	'paged' => $paged,
);

…instead copy/paste the following:

$category_exclusive = get_post_meta($posts[0]->ID, 'category', true );

if( $category_exclusive )
{
$cat = get_cat_ID($category_exclusive);
$args = array(
	'category__in' => array( $cat ),
	'orderby' => 'date',
	'order' => 'DESC',
	'paged' => $paged,
);
}
else
{
$args = array(
	'orderby' => 'date',
	'order' => 'DESC',
	'paged' => $paged,
);
}

Usage

As per WP’s official suggestion … if you want a page to filter by category:

1. Edit the page
2. Select the “Page of Posts” template (this is Suffusion specific; in other themes, it might not exist)
3. Add a “custom field” (scroll down on the edit screen), called “category”, with a value of the NAME of the category you want to be included

Explanation

One major thing here: We are being SAFE and non-destructive to the Suffusion theme.

Critically important is that we ONLY do a “filter by category” if the Page itself requests it. WP’s sample code does NOT have this protection (which is why the code above is slightly different).

This means that the rest of Suffusion (which doesn’t know about our new feature) is unaffected, and works as normal

Filed under : PHP, fixing your desktop, programming

3 Comments »

Someone just wrote this comment on one of my StackOverflow answers:

Fundamentally, this question/answer pair is saying:

“Apple: one of your non-programming managers made a stupid mistake in one of your core tools, one used every day by hundreds of thousands of people; since you won’t fix it, here’s a (tricky) workaround that anyone can use”

Apple “doesn’t do” anything open, doesn’t do community support, or community development – you’re not even allowed to know if you’re the first person to report a bug, or the millionth.

But if they did, just imagine how much better their tools would be, and how much more productive the iOS developer community would be…

Filed under : advocacy, programming, social networking, startup advice

2 Comments »

(I’m prototyping a new game (working title: “ChessQuest”) – original post here)

A couple more hours work, a few more changes:

Overview of what’s changed

• Obvious from the screenshot: added alternating black/white background tiles. I want it to feel like the traditional chessboard has become the “floors” of the dungeons/towers you’re exploring
• Major fixes for collision detection: player and enemies now do pixel-precise collisions (previously, if e.g. a piece moved 5 pixels per tick, it would move either 0 pixels (if 5 would collide) or 5. Now it moves “as many pixels as it can before the collision happens”)
• Major upgrade to renderering: instead of just “render everything, in random order” it now sorts all sprites once per frame. Each sprint is instantiated with a sort-layer; things in the same layer are rendered in random order, but before all things in the next layer
• Minor tweaks to movement: not happy with this, but I found the “never stops moving” annoying. Now I find the “stops too easily” annoying.
• Various experiments with zoom level – I tried 20px, 32px, 64px and 100px (very zoomed in!). I found 32 px was working nicest on the Nexus 1, so I’ve stuck with that for now. In a future build, I’ll probably bring back the option to switch to a zoom level of your choice.
• Added Flurry: I’ve never done this with a prototype before, but then I’ve never prototyped in public :). I’ve added Flurry just so I can keep track of how many people are running the prototype; as I add to the prototype, I’ll start adding in-app feedback options, so you can effectively “vote” on things you like/dislike, and I’ll use Flurry to graph it all automatically.

Download link for this version

Version 0.2

Filed under : android, devdiary, games design, programming

0 Comment »

Steve Yegge’s Google Platforms Rant is not so much a rant as a sign of someone fighting an inappropriate culture. We saw stuff like this a lot at NCsoft when people were trying to turn around the $100 million clusterf*ck that created hundreds of redundancies all the way to director level.

It’s a great article, but a couple of the key points resonated with my own experience of Google UK’s hiring practices a couple of years ago. There was clearly a lot wrong with the internal culture, but as an outsider I couldn’t quite put my finger on it. Here’s the crux of Steve’s post (but seriously – read the whole thing, it’s rich and meaty):

That one last thing that Google doesn’t do well is Platforms. We don’t understand platforms. We don’t “get” platforms. Some of you do, but you are the minority. This has become painfully clear to me over the past six years. I was kind of hoping that competitive pressure from Microsoft and Amazon and more recently Facebook would make us wake up collectively and start doing universal services. Not in some sort of ad-hoc, half-assed way, but in more or less the same way Amazon did it: all at once, for real, no cheating, and treating it as our top priority from now on.

But no. No, it’s like our tenth or eleventh priority. Or fifteenth, I don’t know. It’s pretty low. There are a few teams who treat the idea very seriously, but most teams either don’t think about it all, ever, or only a small percentage of them think about it in a very small way.

It’s a big stretch even to get most teams to offer a stubby service to get programmatic access to their data and computations. Most of them think they’re building products. And a stubby service is a pretty pathetic service. Go back and look at that partial list of learnings from Amazon, and tell me which ones Stubby gives you out of the box. As far as I’m concerned, it’s none of them. Stubby’s great, but it’s like parts when you need a car.

…and finally, reading that, it clicked for me what I saw that was so wrong:

Google has forgotten what a Product is

“It’s a big stretch even to get most teams to offer a stubby service to get programmatic access to their data and computations. Most of them think they’re building products.”

That pair of sentences, back to back, is the problem: people outside Google would put the word “except” in between. Googlers put the word “because” in between. Google’s cultural definition of Product has got lost and perverted somewhere along the way, and now looks and smells like the real thing but is – to the rest of the world – a fake. Except Google – internally – can’t see this.

Every Googler I talked to worshipped at the altar of Product-as-King; three quarters of them would then – even in the same sentence – admit that they hated Product, didn’t believe in it, and felt it was a waste of time — “get out of my face with your product BS, and let me write beautiful code in my Ivory Towers, and leave me alone”.

They were smart people; they never said this explicitly (although one came very close – and you could see the moment when he had the thought: “oh crap; if anyone else hears I said that…”, then backtracked very hastily) – instead they frequently said mutually conflicting things, and dressed them up in enough abstractions that you could pretend that they weren’t conflicting. They were very good at it – I could tell there was a crack, but I couldn’t work out where the fault-line lay.

Google’s illusions of Product

As Steve puts it later on:

Google+ is a prime example of our complete failure to understand platforms from the very highest levels of executive leadership (hi Larry, Sergey, Eric, Vic, howdy howdy) down to the very lowest leaf workers (hey yo). We all don’t get it. The Golden Rule of platforms is that you Eat Your Own Dogfood. The Google+ platform is a pathetic afterthought. We had no API at all at launch, and last I checked, we had one measly API call. One of the team members marched in and told me about it when they launched, and I asked: “So is it the Stalker API?” She got all glum and said “Yeah.” I mean, I was joking, but no… the only API call we offer is to get someone’s stream. So I guess the joke was on me.

Product. Platform. Since when have those been mutually exclusive? Not in this Millennium, I believe…

And even when Google gets over their hatred of Platform, even with something as simple as the pixels that their apps put on screen, they’ve jumped the shark:

You know how people are always saying Google is arrogant? I’m a Googler, so I get as irritated as you do when people say that. We’re not arrogant, by and large.

But when we take the stance that we know how to design the perfect product for everyone, and believe you me, I hear that a lot, then we’re being fools. You can attribute it to arrogance, or naivete, or whatever — it doesn’t matter in the end, because it’s foolishness. There IS no perfect product for everyone.

And so we wind up with a browser that doesn’t let you set the default font size. Talk about an affront to Accessibility. I mean, as I get older I’m actually going blind. For real. I’ve been nearsighted all my life, and once you hit 40 years old you stop being able to see things up close. So font selection becomes this life-or-death thing: it can lock you out of the product completely. But the Chrome team is flat-out arrogant here: they want to build a zero-configuration product, and they’re quite brazen about it, and Fuck You if you’re blind or deaf or whatever. Hit Ctrl-+ on every single page visit for the rest of your life.

It’s not just them. It’s everyone.

Any genuine Product person would run screaming from that situation – there’s nothing salvageable. It’s like someone coming to you with their design for a chocolate teapot: “Once you’ve had your tea, you can have a tasty chocolate treat too!”, leaving you wondering: where do I even start with trying to explain to this person what they’re missing?

Filed under : android, facebook, programming, project management, social networking, startup advice, web 2.0

1 Comment »

With Scrum, you’re constantly focussing on:

How does the application look / work for the user *right now*?

…to the extent that you care more about “does this feature work for the user?” than “is the code/art/architecture for this feature ideal?”.

“It’s not done” … “but it looks done!”

We regularly get situations where a feature *appears* to work, to a casual observer – but on deeper inspection, it’s clearly broken in one or more significant ways. Sometimes, the “broken” parts are so obscure that you’d need help to even find them. Other times, they’re obvious if you try to to use the feature more than just once or twice.

In Scrum terms, it’s pretty clear what’s gone wrong: the Product Owner didn’t describe the feature clearly enough (they implicitly included functionality they didn’t really care about, … or they described it too vaguely to be implemented well).

Scrum’s in-built check/balance against that is the Team. The developer who adopted the task should have rejected it during the Planning meeting, should have insisted on a clearer User Story (or a more explicit feature description).

But in the real world, this stuff happens. Leaving the issue: What do you do next?

One technique: Divide and Conquer

Here’s an approach I’ve been experimenting with recently.

When it happens, you split the feature description in half, re-defining one half as the part which is done + working, and the other half as the part which isn’t working. Or into 3, 4, etc – if there’s multiple “player visible” ways in which it’s not working.

This seems to work pretty well – it lets us independently prioritise “the bit we’ve already got (hence: zero extra dev cost)” and “the hard stuff that’s not working”. And quite often, we end up redesigning some other part in a way that makes the broken edge-cases no longer exist – so we never need to fix them.

…but I’m still experimenting with it. I’m sure we could do our Planning meetings better – both from the PO side (more detailed descriptions, more PO planning) and/or from the developer side (more questioning, demands for more detail).

Filed under : agile, programming, project management

1 Comment »

Response from ImageMagick folks, when I asked them to either re-instate the working binaries, … or stop building as Lion-only:

“We only host and maintain current versions of ImageMagick on one OS
release level. We have a small development team and do not have the
time to support multiple releases and multiple OS levels. The fix is to
download the MacPorts version of ImageMagick which runs under Leopard.
Another solution would be to donate a Mac with Leopard installed so we
can create binaries. We only have one Mac and it hosts Lion.”

Fine – it’s their software, they can do whatever they want. And I think they’ve done a great thing over the years by sharing this command-line tool with the world.

Except … their alternatives aren’t as reasonable as they sound.

Firstly, MacPorts is incredibly difficult to use (even as a former sysadmin and programmer, I find it painful). Simply put: I know it will take me at least a day to get that working, possibly several.

Secondly, deliberately deleting their own working software, and replacing it with non-working software, is deeply irresponsible. If this is how they approach the overall product, how long before you get “caught out” as a user when they pull some other rug out from under you? “Using ImageMagick today? Well – get it while you can, because tomorrow, they might arbitrarily delete it.” (this is what just happened to me: in the space of a few weeks, the first version I downloaded was deleted and replaced with a knowingly-broken version. My backup copy got corrupted, and I thought I could re-download from the web – nope!)

Asking them about this, they pointed out that the version from a few weeks ago had a bug which was a potential security hole. Fine, so they should discourage people from using it – but that doesn’t excuse *deleting* it, and providing only upgrade paths that are painful or expensive (Lion is not free).

It pains me to say this – as noted above, I think the IM product has been a great thing – but I have to conclude:

Don’t use ImageMagick. Just when you need it, it’s liable to let you down.

As for me, I see no other choice but to give Adobe more money, buying a more expensive copy of Photoshop that I don’t really need. I can’t afford to waste days fiddling around with MacPorts – and not even be guaranteed of success. I just need to do one, tiny, simple operation (an image resize!), but unless I can find a kind person who’s got an archived copy of ImageMagick, it’s not going to happen :(.

Oh, well.

Filed under : programming, server admin

7 Comments »