It’s slightly buried in there, but I spotted this:
Continue reading “MMO scalability is finally irrelevant for Indie MMOs”
It’s slightly buried in there, but I spotted this:
It’s slightly buried in there, but I spotted this:
Continue reading “MMO scalability is finally irrelevant for Indie MMOs”
In his epic comment (sparked by a different Adam – not me, honest), is this gem:
“Using the entity system for the first time was unreal to me. It’s like polymorphic code. I did really weird things on the fly. For example:
– In the health processor, if the enemy was just destroyed, set a flag in the lifecycle component.
– In the lifecycle processor, if the fresh kill flag is set, extract its loot component and put that into a new entity with a small randomized velocity component and a gravity component so that the loot drops; then, remove most of the other components from the entity and add an explosion component.
The “enemy” still has the same entity ID — any other components that are looking for that entity will still find it (e.g. missiles homing in on the wreckage, or score processors looking for slain entities) — but by swapping one set of data with another, its implementation has changed from an enemy to some kind of non-interactive effect object.”
Identity. It’s important.
(Quick sidenote: for all the people asking questions like “but … which variables do I put in Component A as opposed to Component B? How do I manage Events in an Entity System? … etc” – Hume’s approach above is a good concrete example of the first-draft, easy-to-write way of doing things. Copy it.)
This is one of those things that newbie game programmers seem to underestimate, frequently.
And when I say “newbie” I include “experienced, skilled programmers with 10+ years of coding experience – but who haven’t yet shipped a game of their *own*”.
(e.g. I’ve seen a couple of studios that started as Digital Agencies, or as Animation Studios, etc – that then transitioned to writing their own games. This is the kind of thing that they often struggle with. Not for lack of skill or general programming experience, but for lack of the domain-specific experience of game coding)
Examples of Identity in games, off the top of my head – all of these are independent, and interact in complex ways with each other :
ES’s traditionally give you a SINGLE concept of Identity: the Entity (usually implemented as a single Integer). Hmm. That sounds worryingly bad, given what I wrote above. One identity cannot – by definition – encompass multiple, independent, interrelated identities.
But we’re being a bit too literal here. ES’s give you one PRIMARY identity, but they also give you a bunch of SECONDARY identities. So, in practice…
In OOP, the Object is atomic, and the Class is atomic. You cannot “split” an Object, nor a Class, without re-defining it (usually: re-compile).
In ES, the Entity is atomic, and the Component is atomic. But the equivalent of an OOP Object – i.e. “an Entity plus zero or more Components” – is *not* atomic. It can be split.
And from there comes the secondary identities…
A Primary Identity: e.g. “The Player’s Tank” (specific)
A Primary Identity: e.g. “a Gun Component” (generic)
A Secondary Identity: e.g. “The Gun component … of the Player’s Tank Entity” (specific)
Revisiting my ad-hoc list of Game Identities above, I hope it’s clear that you can easily re-write most of those in terms of secondary identity.
And – bonus! – suddenly the relationships between them start to become (a little) clearer and cleaner. Easier for humans to reason about. Easier *for you to debug*. Easier *for you to design new features*.
Noticeably, the network-related Identities are still hard to deal with.
On *my* computer, I can’t reference entities on *your* computer. I cannot store: “The Gun component … of YOUR player’s tank”, because your “Player’s Tank” only exists in the memory of your computer – not mine.
There are (trivially) obvious solutions / approaches here, not least: make your Entity integers global. e.g. split the 64bit Integer into 2 32bit Integers: first Integer is the computer that an Entity lives on, the second is the local Entity Integer. Combined, they are a “global Entity ID”.
(I’m grossly over-simplifying there – if you’re interested in this, google for “globally unique identifiers” – the problems and solutions have been around for decades. Don’t re-invent the wheel)
But … at this point, they also offer you the chance to consider your game’s network architecture. Are you peer-to-peer, or client-server?
For instance, P2P architectures practically beg for unique Global entity numbers. But C/S architectures can happily live off non-global. For instance:
Or, to take it further, if your network arch is any good at all for high-paced gaming:
… or something like that. Again, I’m grossly over-simplifying – if you want to write decent network code, Google is your friend. But the fastest / lowest latency multiplayer code tends to work something like that.
What do you think?
(hint: you can do wonders using Reflection/Introspection on your entity / components. By their nature, they’re easy to write generic code for.
But you WILL need some extra metadata – to the extent that you may wish to ‘upgrade’ your Entity System into a SuperEntity System – something with a bit more expressive power, to handle the concept of multiple simultaneous *different* versions of the same Entity. Ouch)
Yeah, I’m bailing on you here. Too little time to write much right now – and it’s been a *long* time since I’ve implemented this level of network code for an ES. So, I’m going to have to think hard about it, refresh my memory, re-think what I think I knew. Will take some time…
It seems to come down to: Software Developers (programmers who write apps that a company sells) and Ops people (sysadmins who manage servers) don’t talk enough and don’t respect each other; this cause problems when they need to work together. Good start.
But I was feeling a gut feel of “you’ve spotted a problem, but this is a real ugly way to solve it”, and feeling guilty for thinking that, when I got down to this line in Wikipedia’s article:
“Developers apply configuration changes manually to their workstations and do not document each necessary step”
WTF? What kind of amateur morons are you hiring as “developers”? Your problem here is *nothing* to do with “DevOps” – it’s that you have a hiring manager (maybe your CTO / Tech Director?) who’s been promoted way above their competency and is allowing people to do the kind of practices that would get them fired from many of the good programming teams.
Fix the right problem, guys :).
Incidentally – and this will be a long tangent about the nature of a TD / Tech Director – … my “gut feel” negativity about the whole thing came from my experience that any TD working in large-scale “online” games *must be* a qualified SysAdmin. If they’re not, they’re not a TD – they’re a technical developer who hasn’t (yet) enough experience to be elevated to a TD role; they are incapable (through no fault of their own – simply lack of training / experience) of fulfilling the essential needs of a TD. They cannot provide the over-arching technical caretaking, because they don’t understand one enormous chunk of the problem.
I say this from personal experience in MMO dev, where people with no sysadmin experience stuck out like a sore thumb. Many network programmers on game-teams had no sysadmin experience (which in the long term is unforgivable – any network coder should be urgently scrambling to learn + practice sysadmin as fast as they can, since it’s essential to so much of the code they write) – and it showed, every time. In the short term, of course, a network coder may be 4 months away from having practiced enough sysadmin. In the medium term, maybe they’ve done “some” but not enough to be an expert on it – normally they’re fine, but sometimes they make a stupid mistake (e.g. being unaware of just how much memcached can do for you).
And that’s where the TD-who-knows-sysadmin is needed. Just like the TD is supposed to do in all situations – be the shallow expert of many trades, able to hilight problems no-one else has noticed, or use their usually out-dated yet still useful experience to suggest old ways of solving new problems that current methods fail to fix. And at least be able to point people in the right direction.
…but, of course, I was once (long ago) trained in this at IBM, and later spent many years in hardcore sysadmin both paid and unpaid (at the most extreme, tracking and logging bugs against the linux kernel) so I’m biased. But I’ve found it enormously helpful in MMO development that I know exactly how these servers will *actually* run – and the many tricks available to shortcut weeks or months of code that you don’t have to write.
Ubisoft just bought Qazal – one of the last providers of networking tech for games. Congrats to the Qazal folks; although the price isn’t mentioned, I’d imagine a lot of them have picked up a nice windfall from this.
Two problems: one immediate, anti-competitive, affecting games using this tech; the other long-term, and damaging for the *entire* games industry.
Simple. Ubisoft is a large publisher, big enough that whoever is publishing *your* game probably competes directly with them.
Even if you don’t use Qazal yet, if you’re a developer you now have the problem that Ubisoft has an extra stick to beat you into submission when you’re looking for a publisher for your next game:
“Yes, this publishing deal screws you over, and you get 50% less royalty than with our competitors. But you want to use Qazal, don’t you? Well, unless you accept this 1-sided deal, we’ll make sure you either can’t use Qazal, or we’ll quietly – unofficially – make sure you get terrible support, screwed over, etc”
NB: Qazal guys have made it clear they won’t stand for this. Unfortunately, Criterion guys said much the same thing when EA bought them. Turns out that guys with sticks can protest all they want, but the guys with machine guns tend to win the argument.
Remember when the Swine Flu started, govts panicked that this virulent disease – started no-one quite knew where – might become a pandemic? Maybe just one chance interaction between pig and man that would bring down mankind. Maybe …
Over the past few years, there’s been a series of aggressive, destructive, anti-competitive acquisitions of games-middleware companies.
It seems *likely* that the Ubisoft/Qazal purchase will be more of the same – no matter what they say. It may prove to be the straw that breaks the camel’s back.
Already, development teams – or, more accurately, the people that run them – are actively avoiding using middleware. The common refrain is “we’ve been so badly burned by Renderware/Demonware/[insert here] that we now have an exec-level policy that we must not use middleware unless it’s from a company that has practically zero chance of being purchased by our major competitor”.
In the short term, EA and Activision got to laugh at everyone else and dance and sing and claim their brilliance.
But already there are lots of smart industry people who have avoided founding new middleware companies (and I suspect there are plenty that started but died) because of this: too many of their desired customers are refusing to buy, on principle.
It may take 5+ years for the full impact to be felt, but I’m confident it will be felt.
To be clear: we’re not talking about things like Unreal3 here, we’re talking about focussed technology solutions to game-development problems. The dominant players of the industry bemoan the escalating costs of production, but the great innovations in cost – cheap, effective, nimble tech – have traditionally come from small middleware startups. Not from monolithic tech spinouts like U3.
No matter the protestations of innocence, AFAICS there’s a problem of mis-aligned interests.
Ubi will materially suffer *not at all* if Qazal is no longer licensed – the revenue (so far as I can see) is relatively tiny, and it’s non-core to their business. Unless they want to become a “tech selling company”, they will never care much about it. They won’t even care that much about the lost “shared R&D costs” they’ll just be happy to have the smart people in-house.
However, Ubi now has an enormous sword of damocles they can use both to bully and to materially harm their competitors: at any moment, they can turn off the tap.
You can write the cleverest contract in the world, to protect your project, and yet STILL Ubi may screw you over. The team leads may not notice – protected by said contract – but the TDs will (wasted investment of time and research/training with Q).
And the CTO will be livid (because the lead time to build up an internal expertise is measured in years – and they’ll have to start from scratch just to persuade the Finance division to let them have the money, which could take a year or more on its own).
Wifi and internet at all is a priviledge – but Free Wifi is something that in our modern society, and the society we’re set to become, needs to be treated as a right. When I started writing this, I was looking at the benefits we have yet to see (ubiquitous free wifi); in the week I’ve been offline with jetlag, the preceding benefits we already have that would make them possible – flat rate internet – are being ripped away from us, and . Both are understandable, but … yikes.
Casual, assumed, free internet access is now ubiquitous (even if the access itself isn’t as operationally ubiquitous as services assume). I can’t even access half my music collection any more unless I’ve got a wireless high-bandwidth connection available (Spotify). The other half lives on my MP3 player (iPhone) – but is static, unmeasured, unconnected, and unshareable.
This is a problem. Right now, sitting in San Francisco, the city of a thousand broken, crashing, low-bandwidth, pay-per-minute (min charge 24 hours) wifi connections, next door to Silicon Valley, a world center of innovation that only exists because the right infrastructure here and the wrong mistakes elsewhere allowed it to form, it’s particularly on my mind. SF is a great example of what will push the next Silicon Valley to happen elsewhere. A lot of people ought to be worried by that – and doing a little more about it.
In Brighton, my current (temporary) home city, the first repeated free wifi hotspots were set up – as I understand it – effectively as an act of charitable benevolence by “a couple of guys” (looseconnection.com/Josh Russell). They weren’t even rich, or old – just some kids doing something cool, and useful. Anyone could do this. Too few actually do. I’ve heard it suggested again and again (where are the mesh networks that were supposed to be ubiquitous 4 years ago?) by people in the UK – especially in and around Cambridge, in tech the UK’s closest replica of Silicon Valley – but always with excuses about why they aren’t doing it yet, aren’t able to until someone else does something else to make it easier for them. That’s crap. Just do it. Do it this weekend; what better are you doing right now?
Will Apple single-handedly save Wifi? Maybe. It could be the biggest gift of iPhone: that it finally turns the rest of the world on to building bigger, better, and above all FREE, wifi networks. Everywhere. Ironic, considering that’s exactly what will kill the fundamental device that drives the iPhone: the “cell” phone. Does anybody else remember that before we had cell phones we had hotspot phones, back when cells weren’t good enough, and were so expensive to use? So we go full circle, but this time with an ecosystem and a tech interconnection system (API’s, protocols, layers) big enough to support the worldwide rollout of such hotspots (well, and that’s what mesh was supposed to be about, right?)
Skype is a great example. Sadly, it’s also overloaded with additional meaning that clouds the issue – because Skype is an internet app (good) that is mostly about phone calls (bad / confusing the issue).
Skype is now available on iPhone, and it’s a great, highly polished, iPhone App. It *works* (as well as anything can on iPhone – with the current version of iPhone Apple does not allow *anyone* to have their app listen for incoming connections and auto-start, so you can only “receive” Skype calls on your iPhone if you are not using any other app and instead are currently inside the Skype App.
But … the voice part only works over Wifi. This is the concession it took for Skype to be “allowed” on iPhone (NB: Apple allegedly forced the network operators to give away free / flat rate data in return for being “allowed” to sell network-locked iPhones; if Apple had also allowed Skype-on-3G/EDGE/cell network, then they would have caused people to stop paying call charges en masse. Although this is the natural future of cell phones, and everyone knows this, the network operators would probably assassinate Steve Jobs if he tried that today).
So, Skype is – effectively – a “wifi-only” application.
But wait … there’s more. The iPhone platform has an installed userbase of almost 40 million handsets as of first quarter 2009 (yes, that’s only 20% less than the entire global sales PS3 and 360 combined; the iphone is already one of the top games consoles in the world; Sony (Computer Entertainment) is doomed, and Nintendo’s cash days are numbered, even though they’ll make loads of cash for the next 3 years – the DSi was defunct due to iPhone *before it launched*, so after those few years, the cashflow will drop off / vanish).
But … around half of those are not iPhones, but iPod Touch’s. This is very important to understand: the two devices are compile time identical, and *almost* feature identical. They are more similar than almost any pair of cell phones in the world, even ones from the same manufacturer. And by default all iPhone developers are writing code that runs seamlessly on the iPod Touch – it doesn’t (usually) “break” on iPod Touch if it uses an unsupported iPhone-only feature … rather, that part of the app silently is ignored.
So … nearly all those iPhone developers are actually also iPod Touch developers. Many of them deliberately steer clear of using iPhone-only features. Some of them (myself included) write their apps to cleverly detect whether they’re on an iPod Touch, and work around the limitations (it’s not hard – e.g. if I can’t upload scores to the game server because I’m on a Touch that isnt in wifi range, I save it and upload it next time the phone is online. As a bonus, this makes my games work “better” on iPhone when the iPhone has to go offline, e.g. when it goes on an airplane).
Back to the point… There aren’t many Wifi-only Apps out there on iPhone … yet.
But there will be. More and more of them. And this summer, when Apple brings out the 3.0 update for iPhone, making ad-hoc discovery much easier (i.e. my phone will be able to auto-detect / find your iphone when they’re in the same room), wifi-local Apps will blossom.
A simple example: real-time fast-action games.
e.g. a Racing Game, that works like this:
All that is possible, and relatively easy, come summer 2009. You *can* attempt to do it over a 3G network, but it’s hard. But as a wifi-only app it becomes easy. Guess what’s going to happen?
I predicted around 30-40 million iPhone* devices sold by now, and Apple’s 37 million official figure made me look clever (although admittedly it was only a 6 months extrapolation and a 33% error margin I quoted there ;)). I predicted around 75-100 million sold by the same time 2010, and I’ve noticed a lot of other people have come up with the 100 million estimate for 2009 since the official 37 million figure came out.
So, although I think it’s optimistic to expect 100m by the end of the year, I’m confident it’s going to be close. 100m wifi enabled game consoles sitting in cafes, restaurants, bookshops, trains, buses, hotel lobbies, city squares, pubs, etc.
Oh, and don’t forget – that iPod Touch, with no “network contract” to pay for, is a perfect gift for kids. Plenty of people have lined up to tell me that kids can’t afford them; the market research that consistently shows under 18’s as the second largest demographic for iphone* ownership suggest that’s an ill-informed opinion. So there’ll be a lot of those devices sitting in the hands of bored children / used to keep them occupied while parents are doing other things. And we all know how strong a child’s “pestering power” can be.
Monetize local wifi? Screw that; who can be bothered to monetize it when it becomes as essential a driver of custom to your store as having coke/pepsi/coffee on the menu (even though you’re actually, e.g. a bookstore…). Re-think how that affects the “monetization potential” of local wifi (hint: look to the already vast field of *indirectly monetized* Freemium / F2P for inspiration)
So, I’m optimistic. And rather than focus on how “iPhone is going to destroy the cell phone / network operator hegemony, and bring around fair pricing for consumers”, I’m focussing on how it’s going to usher in the long-envisaged era of high-bandwidth, low-latency, high quality console games and apps that focus on the local area. I’m happy with that: I’ve spent almost a decade learning how to make online games for millions of players where the core experience takes place in the local group, so I feel extremely qualified to do well out of this. What about you? What will you be doing with it?
With some WordPress-Fu, I’ve added a page that’s a category and auto-includes links with custom meta-information.
Or, in other words, there’s now a page where I can effortlessly post all my various bookmarked links to do with MMO development – and add my own commentary to each link – which you can’t ordinarily do. Which is why it’s taken me some time to get around to it (previous efforts to do this without customizing WordPress, or using plugins only, failed).
The (practically empty) page in all it’s (non-)glory can be found here:
Over the coming weeks, I’ll be posting much more stuff to it. I hope.
I’ve made a small simple but complete java NIO server (with full source included) that is free for anyone to use for anything. At the moment, it only deals with sending and receiving strings, and isn’t optimized, but if anyone wants to improve it and send me the changes then I’ll post up an improved version here.
Download, documentation, license details, tutorial etc after the jump…
Also known as: Nobody expects the Spanish Inquisition!
(because I’m now deviating from the original schedule I outlined in Part 1; what the heck, it was only a rough agenda anyway…)
First of all, there’s a bunch of good questions that have been raised in response to the first two posts:
Let’s start with that last one first.
Years ago, I found the spritesheets + source code from the author of SFE, who was offering them up if anyone wanted to improve it, make it 4 player multiplayer again (like on RISC OS) etc (or something like that).
Last Sunday afternoon I was very bored, and found just the spritesheets lying around on an old disk, so I wrote the gamecode from scratch. Didn’t quite finish it that day, but I think one more boring Sunday and I’ll have over-the-internet multiplayer and highscores server working, which would rock.
Kevglass asked for a screenshot, so…
Networked games use the internet, and the difficulties of making these games evolve on Internet Time, which means that the articles people wrote as recently as a year ago on how to make a networked or multiplayer game are already out of date. Most of the literature is more than 5 years old, and some as much as 10 years old – hopelessly out of date in the modern world of internet and online gaming.
Anyway, to get you thinking (I’m not providing definite answers here, but just some stuff to make you think about more carefully about how you’re doing your networking), here are some common rules that perhaps no longer apply the way they used to:
A few years ago, I wrote an article for Develop magazine – “10 MMOs you don’t want to do”.
Here’s 8 things you really shouldn’t do but that might seem like a good idea if you’ve never made an MMOG before.
All these are examples of things that have been done on real MMO projects, usually MMORPGs.
I reserve the right to come back and edit this to make it ten once I’ve had more sleep and can remember two more :)…
…was my section in the fourth GPG book from Charles River Media. And, sadly, although I tried to put some resources up on the web, a series of unfortunate events led to that all disappearing.
But now … they’re back! (and I’ll be adding more followup stuff in the coming weeks/months)
Continue reading “Thousands of Clients per Server (Game Programming Gems 4)”
UPDATE: short, complete, 42-slide version now available from the CMP website – https://www.cmpevents.com/sessions/GD/S5762i1.ppt
…but if you want the 144-slide version (!), see below. No extra content.
A few years ago, entity systems (or component systems) were a hot topic. In particular, Scott Bilas gave a great GDC talk (http://scottbilas.com/files/2002/gdc_san_jose/game_objects_slides.pdf – updated link thanks to @junkdogAP) on using them in the development of Dungeon Siege. The main advantages to entity systems were:
An article I wrote for Gamasutra’s Game Career Guide:
…hopefully this will be useful to some of the many people interested in network programming but not sure where to start.
Also, you might want to have a look at my post on Career Progression for Games Programmers