Categories
dev-process iphone programming

iPhone Development FAQ

In my joyous travels developing a fun little iPhone game recently, I kept track of all the many tips and tricks and gotchas I came across. There are a fair few bugs in Apple’s IDE (including at least one critical one that bit me), some stunningly bad flaws in the Objective-C language (it’s *horrible*), and some slightly surprising lack of docs from Apple in key areas (like: how/when do I get paid?)

There was too much to blog it all, so instead I installed a free FAQ software and I’m gradually transferring my notes over to this FAQ (only got a few questions in there at the moment, just fresh installed):

http://iphonefaq.t-machine.org/index.php?action=show

(the one you probably want to start with is: How to make applications for the iPhone?)

There’s an awesome feature to this FAQ too (not that I’ve tested it): anyone can go to the site and click “Ask a Question” and it gets added to a list for admins to answer and post. You can even answer your own question at the same time as asking it.

If you’ve been developing iPhone apps yourself and have some burning questions or neat tips of your own, feel free to go to the FAQ and add them in. Any problems, email me (email address is on the About link at the top of this page)

Categories
games design iphone programming

My first iPhone game – download now!

As part of our super top secret new startup we’ve been making some educational games, and for fun (in my spare time) I tried porting one to my iPhone. It’s a simple maths game which I thought would work well with a touch-interface. If you have an iPhone, you can get it here:

Download from Apple iTunes Store

(this is a bit of an experiment – if enough people like the game, I’ll take the time to plug it in to some serverside stuff we’re doing which will add a bunch of feedback/stats/scores features for anyone who’s got the game)

Categories
dev-process programming recruiting startup advice

Google cutting 20% time

Oh, wait, actually they’re not. But people love to misinterpret statements with the word Google and the number 20 in them.

As I put in the comments:

Frankly, I’ve generally not bothered to correct anyone who didn’t bother to research it themselves – except in the cases where they were in my own organization, and attempting to make decisions about related matters based on misconceptions of the supposed Google rule.

Of course, my dear lovely friends may have been lieing to me. I very much doubt it (and even if I didn’t, what they’ve said over the years has made a lot more sense than most of the hearsay you hear on the web) – but the point here is that I’ve bothered to ask.

Google has so many employees that if you preach on subjects like 20% time – which, by the way, I think is one of the most fundamentally important (and least well-understood) issues in corporations, job choice, and why you get up in the morning, but that’s another post – then you have no excuse at all for not going and asking an employee how it works. Last time I looked, they had bajillions of offices around Europe, not to mention the sprawl all over the US.

Categories
dev-process iphone massively multiplayer mmo signup processes programming

iPhone: why’s my download stuck at 1.4 of 1.6 Gb?

Your possible answers include:

1. Because … Apple engineers have never heard of the concept of a “patch”, and require you to re-download the *entire IDE*, with all libraries, all documentation, all binary code – everything – when they release an update? So the current “SDK” for iPhone (hint for Apple: when most people say “SDK” they don’t mean “plus a copy of a bloody operating system”, they just mean “the few custom bits that are specific to that app”) is a whopping 1.6Gb?

[NB: actually in general I think that’s a good thing – avoids a lot of mis-configuration / version mismatch problems – but as an MMO developer the idea of *not* patching gigabyte-sized packages horrifies me, and avoiding those problems actually isn’t THAT hard (it’s been solved many times by now!) these days. Writing (or buying) a good patcher is one of the first steps you do in MMO dev projects…]

2. Because … Apple didn’t think to split The Behemoth into multiple files, perhaps make them something reasonable, like a few hundred meg each?

3. Because … Apple decided to put this monster behind an authentication check on their website, presumably for legal reasons, and there is no other “official” mirror (all the ones you find on google are technically-illegal torrents or else, ultimately, redirect you back to the apple.com link), and their authenticated sessions TIMEOUT after 1 hour of “not fetching any new pages from the site” (completely ignoring whether you have any transfers in progress!), and refuse to send you data once your authenticated session runs out?

4. All the above?

NB: I wasn’t brave enough to try resuming the downoad without first re-authenticating and loading at least one web page from the apple developer site to prove I was logged in. I suspect (*suspect*) that the web browser would receive an HTTP 300 redirect to the login page, at which point most browsers are going to delete the partial download. Ha. Haha. HAHAHAHAHAHAHAHAHAHAHAAARRRRRRGHHH!.

Expect to see some comments/tutorials/advice on iPhone game development here at some point in the near future. If I can ever get the download to complete…

Categories
NIO Server programming

Project moved: java NIO servers

Because WordPress is so poor at handling file uploads (AND because the new WP versions introduced incompatibilities somewhere that make some of the original blog pages 404 even though they are in the WordPress folder), I’ve created a SourceForge project for this (http://javanioserver.sourceforge.net/), and uploaded all the files there.

For those of you who wanted the source recently, there’s links to the downoad page from the SF page – the three files marked “version 1.0.0” are *precisely* the same ones I originally uploaded to the blog post.

If I get time, I’ll check the source in to the SF SVN too, so you can check it out directly (and upload any modifications you make).

Categories
maintenance programming

Is there any good way to upload files to WordPress blogs?

As far as I can tell: no (?). I’m hoping it’s just that my Google-Fu is too weak, and there’s a solution out there that someone has already put together…

Categories
facebook network programming programming

Installing and using PHP Eclipse IDE on OS X

I wanted to knock something up for Facebook, and I thought I’d try out PHP development on my shiny macbook air laptop (I usually develop on a much more powerful windows/linux PC).

This is tortuous, painful, and mostly undocumented on the official site. Sigh. After some experimentation, here’s how to make it work (IMHO you shouldn’t need to do this – although the Eclipse project allows plugin developers to package their plugins in really stupid ways, and doesn’t make it easy for anyone (users, developers, etc) – it’s still the fault of the plugin developer if they ALSO do not make it easy for the users to install).

First problem: get Eclipse

IME, most programmers who would use Eclipse already have it. The PHP plugin website won’t really help you, mostly taking the attitude of “install an extra copy of eclipse, just for doing PHP development; if you already have eclipse … work it out yourself”. I kind of understand why, but still feel that the first duty of every developer is to make their stuff easy to install!

If you don’t have Eclipse, I highly recommend you do NOT follow the PHP instructions (by downloading their pre-made “PHP + Eclipse all in one”) because then you are doomed to having multiple parallel installs if you ever need to use any other programming language; learn how to do it properly instead.

Download eclipse here (you want the latest stable version, currently called “Ganymede” (no, I don’t know why they stopped using Version numbers either – yes, it does make life more difficult for all normal users who haven’t memorized the funny names. Sigh)). You can get “Eclipse with Java” or “Eclipse with C/C++” or whatever you want – they are all identical, some just have extra plugins pre-installed.

NB: the people that run the Eclipse website have some pretty icon-artists, but a cruel sense of humour (or just suck at website design, maybe? I shouldn’t complain – it used to be a LOT worse than this) and don’t provide bookmarkable links to the OS X version (that I could find, at least); you’ll just have to go to that page and scroll till you find it. Right now, the current “standard” Eclipse version is at the VERY BOTTOM OF THE SCREEN (c.f. my previous comment re: sense of humour), and at the right hand edge of that box is a tiny link saying “Mac OS X”)

If you’re new to Eclipse, you will probably find that downloading eclipse is one of the most confusing downloads you’ve ever done; if so, this is part of that same problem mentioned above where the Eclipse project makes plugin installs ridiculously difficult: all those many confusing different Eclipse versions that you cannot tell the difference between are actually the same, but differ only in which plugins are pre-installed.

Yes, this is stupid. Yes, it’s badly documented. Sorry. You’ll learn to live with it – they’ve been doing this for almost ten years now so don’t expect it to be fixed any time soon.

I suggest you run Eclipse once, now, before going on to the next step – if Eclipse doesn’t start at this point, don’t waste time confusing yourself with the PHP plugins until you can get Eclipse working on its own!

Second problem: get PHP IDE (now renamed to “PDT”)

You can try to do the automated-install; Eclipse is bad at handling automated installs, has very poor error-handling if anything goes wrong (it just crashes and doesn’t explain), and plugin developers usually screw-up the auto-install API in ways that can actully render your copy of Eclipse unusable (this happens *a lot*). I would advise never using it if you can avoid it.

This one’s pretty messed-up. According to the website and the list of requirements, if you want to use the latest version (vesion 2.0, not yet “packaged”), then you have to download approx 6 massive files.

I found that if you download the “Eclipse for Java” pacakge then it has some of those built-in already, and even if you don’t, several of those have EACH OTHER built-in (WTF?). I suggest you don’t take any risks, and that you do this the long way (download EVERYTHING).

First, go to this download page.

Decide which version you want; right now you want “2.0.0 Stable Builds”, but soon that will be what you get from “Latest Releases”, so check there too.

Then download ALL the zip files listed under “PDT” AND everything listed under “Build Dependencies”. Right now, there are 3 files for “PDT” (SDK, Runtime, and Tests), and 5 files under “Build Dependencies” (Eclipse gtk, EMF, DTP, GEF, and WTP).

Third problem: OS X can’t unzip the files

If, like most OS X users, you’re using Stuffit Expander to unzip the files, by default it won’t do it, because they all overwrite the same directory name (and StuffIt is designed to “protect” you from that, which is nice).

That’s only slightly annoying to get around, but you are still screwed, because OS X itself is (apparently – I couldn’t find a way around this using Finder) hard-coded to prevent you from copying the contents of the directories into the Eclipse directory. When you try to it says “delete the target directory first, or cancel?” (unlike windows, which says “only overwrite files which are the same, otherwise copy all the missing files … or cancel?” which is 99% of the time what you wanted. I have no idea why Apple uses a “destructive” copy – and gives you no alternative!)

Here’s how to get around both problems: The solution – Manual install via Terminal

Fortunately, if you switch to using the Terminal, and run “unzip” by typing it in manually, by default it’s setup as a unix variant that acts in the same way that Windows works by default.

First, make sure you are in the directory where you saved the ZIP files, e.g. by typing:

cd ~/Downloads

(assuming you saved them to your personal Downloads folder)

Then you just type:

unzip [name of zipfile1]
unzip [name of zipfile2]

unzip [name of zipfileN]

…which resolves the incompatibilities in the distro files, and then to install the plugin you type:

cp -Ri eclipse [location of your eclipse folder – usually: /Applications/eclipse]

Note that the “-R” is *required*, and that there is NO trailing slash after “eclipse”. The “i” after the “-R” is optional, it might be good to know if you have problems, but it allows you to get confirmation before overwriting any files. Thankfully, you can just hold down the enter key and it will do the defautl (do not overwrite) as it goes through each file; there are many hundreds of files to copy, and you may already have hundreds of them, so this is handy.

Installation complete

Now start Eclipse – this took 5 minutes for me, where normally it takes 30 seconds, don’t ask me why) – but eventually it worked.

Test it works – go “File -> New Project” and scroll down to the PHP folder, and select “PHP Project”. If there is no PHP folder in the list, then the install has failed. Start again. Good luck.

Otherwise, it should let you create a project, in which case: You now have PDT / PHP-IDE for Eclipse installed and working.

Writing a PHP file

I’m assuming you know how to do this, or can find a tutorial (any PDT tutorials should be fine – it works the same way as the mainstream language plugins for Eclipse, so *any* tutorial on creating a source file and building it ought to work).

Fourth problem: Running your PHP

Oh crap. If you go ahead and create a project, give it a name, and hit OK, you’ll find that the PHP IDE seems designed to not allow you to develop or test PHP on a server; it only supports developing and testing on a local (inside Eclipse) private PHP interpreter. If you’re new to programming, this is fine to get started and learn some basic PHP.

If you’re an experienced programmer, you’ll probably hat that: unless you enjoy tracking down unreproducable bugs and tearing your hair out, you need to develop on the same software + version that runs your production server (in most cases, this will be an Apache2 server running the PHP5 module). Since OS X comes with Apache2 *and* PHP5 built-in, you *already* have a server on your machine that is probably 98% the same as the live server you would use (so far Apache2 + PHP5 on OS X seems to act almost identically to the same versions on Linux, FreeBSD, etc – as you would expect).

(98% is annoyingly short of 100%, but it’s a lot closer than using the bulit-in interpreter)

I can’t find any options in the Run Dialogs to control how it invokes the running of the code from a remote server (or even a local one!) – if you go digging through all the config options, they’re just missing.

NB: there IS something that looks like it might do the trick, where it has a list of “Server” and lets you choose a “PHP Server” – but THIS IS A LIE (there is no cake), do not believe it, this is for something else entirely; it’s just that someone made a poor choice of name for those labels).

Instead, what you have to do is be a lot more careful when creating new projects. Do this:

  • File -> New… -> PHP Project
  • Fill in project name as per normal with Eclipse
  • UNcheck the “default” option for “Project Contents”
  • Click the Browse button under “Project Contents” and navigate to wherever you keep your source DIRECTORIES for all your projects (see more on this below – and you may end up crying when you see what has to be done)
  • Click Finish
  • …NB: BE CAREFUL: it asks you a sneaky extra question, and your answer depends on how you manage source control (see below)

Special Note: where do you keep your PHP?

If:

  • you are just developing locally for now on OS X
  • AND you want to use the Sites directory to save your PHP files in
  • AND you want to use your Apache2/PHP server instead of the “fake” one that comes with eclipse

Then:

  • select your personal Sites folder as the Project Contents above
  • AND answer “Create project in /Users/[your name]/Sites/[project name]” when you press Finish above

This will:

  • automatically create a new sub-directory in Sites with the same name as your project
  • mean that the address of your project is “http://localhost/[projectname]/”
  • mean that if you delete the project in Eclipse, and select “delete from disk as well” when you do, Eclipse will delete ONLY the “[projectname]” subdirectory from your Sites folder, and leave everything else intact

…ALTERNATIVELY…

If:

  • you use proper source control which has a unique root directory for each source project

Then:
EITHER:

  • select the folder for that project as the Project Contents above
  • AND answer “Create project in [source root folder]. (Deleting the project will delete the entire [source root folder])

OR:

  • select the PARENT folder that contains the project-specific source folder as the Project Contents above
  • AND answer “Create project in [PARENT folder of source root folder]” when you press Finish above

I actually strongly recommend the first option, since this will ensure that Eclipse doesn’t mess with your source control’s PARENT folder (which in most source control systems will either screwup the system (happens with crappy source control like CVS) or just be ignored because you won’t have write-access (happens with the more high quality source control) – but this can upset Eclipse if you do some other things wrong in the future.

FINALLY!

After all that, I finally had a working PHP IDE on OS X. Yes!

I haven’t tried debugging yet, but I found the following links that look pretty sound for setting it up. Bear in mind that the first one tells you to setup your Project Contents differently – just adapt what it tells you depending upon what you did above – the author doesn’t seem to fully understand Eclipse’s arcane approach to projects (given the name he uses for his Project!), which is fine, but IMHO not recommended.

How To Setup a Free PHP Debugger using Eclipse PDT + XDebug

Categories
community computer games databases design dev-process games design games industry massively multiplayer network programming programming recruiting

MMO Blogger Round-up

On this site I have a rather subtly-hidden Blog Roll. When I started blogging, the site had less on it, and the roll was easy to find – and short. Now it’s not. And it’s long. And each link on there has been carefully considered. There’s some gems in there (although a lot of them are updated so infrequently few people track them).

So it’s time to call-out some of the interesting things to be found in the blogging world of MMO people.

By the way … you can tell who’s working on uber-secret or personally exciting projects these days because they’ve suspiciously stopped blogging for months at a time. Lazy slackers, the lot of them. The more you do, the more you should blog! :P

There are some that should be on the blogroll but aren’t (yet), and some other bloggers I should mention (but I’m sticking to the blogroll only for this post – I’ll go through others next time). Feel free to add your own recommended reading in the comments.

Blogs to read:
Brinking (Nabeel Hyatt)
* Who? serial entrepreneur, raised funding and sold companies
* What? currently running a funk-tastic social / music / games company with a bunch of Harmonix guys
* Why? big commentator on the games/apps/making money/predictions parts of All Things Facebook

Broken Toys (Scott Jennings / LTM)
* Who? became infamous in the early days of MMOs as a player of Ultima Online who ranted publically, amusingly, and sometimes even insightfully
* What? ex-NCsoft, now doing intriguing web games at John Galt Games
* Why? In his heart Scott’s still a player, and more than anyone else I’ve seen he interprets the world of MMO design, development, and playing through the players’ eyes. Interesting point: he’s mostly concerned with life-after-launch. Funny that. Players kind of find that bit the most interesting. Also keeps a close eye on community-management screw-ups, and WoW generally

Bruce Everiss
* Who? ex-head of marketing for Codemasters
* What? um, I’m not sure what he’s doing these days, apart from becoming a “professional blogger”
* Why? he aims to comment on every single interesting piece of news in the mainstream games industry. That’s a lot of commentary. Always something to read! IMHO he is often completely wrong about anything online-games, and a lot of business and “future of industry” stuff – Bruce is from an older age of the industry. But … he says a lot of interesting things and sparks a lot of interesting debates in the process. Worth reading. Just remember he is extremely (deliberately, I’m sure) provocative, and don’t take it too seriously.

Coke and Code (Kevin Glass)
* Who? A programmer working in mainstream IT
* What? An insanely prolific author of casual games “in his free time, as a hobby”
* Why? Because he’s better at making games than many professionals I’ve met, and he is very very prolific, making new libraries, toolsets, editors, games, game engines – and commenting on it all as he goes, and throwing up new games for you to play all the time

Erik Bethke
* Who? ex-Producer for Interplay
* What? CEO of GoPets, an online casual virtual world that’s especially big in Asia (and based in South Korea)
* Why? A hardcore WoW player who analyses the game-design as he goes, and relates very honestly a stream of both emotional experiences and seminal events in the game that should give you lots of things to be thinking about, especially if you’re a designer, business person, or product manager.

Extenuating Circumstances (Dan Hon)
* Who? ex-MindCandy, current CEO of SixToStart
* What? one of the first Bloggers (on the whole of the internet!) in the UK, and an awe-inspiring assimilator of “everything happening on the internet, with technology, with media, with entertainment and the future of the world” for all of the ten years I’ve known him.
* Why? He’s still an excellent tracker of all those things, and finds memes very quickly. Nowadays he just auto-posts links (lots of them, every day) with a few words of commentary scattered here and there (del.icio.us descriptions) – making his blog a ready-made news filter for you :)

Fishpool (Osma Ahvenlampi)
* Who? CTO of Sulake (makers of Habbo Hotel)
* What? a very technical commentator, often in great detail, on the issues of running a 100-million user virtual world, with observations about Habbo’s community, business, and culture thrown in
* Why? He posts very rarely, but when he does, they’re usually full of yummy detail

Futuristic Play (Andrew Chen)
* Who? ex-VC (Mohr-Davidow Ventures)
* What? entrepreneur with a web-background who’s come into the games industry and bringing lots of useful stuff with him
* Why? blogs a LOT on advertising (and how to make money out of it in games and web and casual), and on metrics, and how you can use them to run you games or web business better. Also has a long fascination with what are the best parts of the games industry, and the best of the web industry, and how we can each put those best bits together to be even better

Off the Record – Scott Hartsman
* Who? ex-Everquest, ex-Simutronics
* What? Senior Producer for MMOs – but previously an MMO lead developer, and once (apparently) a Game Designer.
* Why? he’s funny, he knows his stuff, and he’s worked on some of the most important MMO projects outside Asia, so he’s got an interesting perspective going there.

Orbus Gameworks (Darius Kazemi)
* Who? ex-Turbine, now CEO of Orbus (a games-metrics middleware company)
* What? Likes the colour orange *a lot*, infamous for networking his ass off at games conferences (*everyone* knows Darius), very friendly, generous – and mildly obssessed with the use of metrics and stats to improve the creativity and success of game design (in a good way)
* Why? If you liked the Halo heatmaps when they came out, you’ll love some of the stuff they post on the Orbus company blog. A year ago they were posting heatmaps-on-steroids. If you thought “metrics” equalled “spreadsheets of data” then prepare to have your view changed pretty thoroughly.

Prospect Magazine/First Drafts (Tom Chatfield)
* Who? section-editor of the highly respected socio-political print magaine Prospect
* What? a highly-accomplished English Literature post-grad (bear with me here) … who also happens to have been a lifelong hardcore game player, I think the only person I know who got a hardcore character to level 99 on Diablo2, and now plays WoW a lot.
* Why? although Prospect only very rarely (like, only a few times ever) covers games, it’s very interesting to see what the rest of the world – especially the highly educated and highly intelligent but non-technical, older generations – thinks of us. And a bit of culture in your blog reading is probably good for you, too.

Psychochild (Brian Green)
* Who? ex-3DO/M-59, now the owner and designer of the revamped, relaunched, more modern Meridian-59
* What? an MMO game designer who disingenuously describes himself as an indie MMO designer but like most of the others has probably spent too long doing this and knows too much (compared to many of the modern “mainstream” MMO designers) for that to be true any more
* Why? lots and lots of great design ideas and commentary here for anyone wanting to do MMO design

Scott Bilas
* Who? programmer on Duneon Siege
* What? …in particular, responsible for the Entity System (one of my main areas of interest)
* Why? Scott’s phased in and out of blogging, but when he does blog he tends to do good meaty programming posts that contain lots of source code and some useful lesson or algorithm.

Sulka’s Game (Sulka Haro)
* Who? lead designer for Sulake (Habbo Hotel)
* What? more of a Creative Director than game designer, more of a web background than games, but above all a community/product/creative person who knows his stuff. Also a big player of MMORPGs
* Why? are you cloning Club Penguin or Habbo Hotel and want some pointers about revenue models, community management, and how to be successful with virtual-item sales? You might want to read his posts ;)

The Creation Engine No.2 (Jim Purbrick)
* Who? ex-Codemasters, ex-Climax (both times working on MMO projects)
* What? originally a network / MMO academic researcher, then a network coder, and now the person who runs Linden Lab (Second Life) in the UK. Very big proponent of all things open-source, always doing interesting and innovative things with technology
* Why? Keep an eye on the more innovative technology things that are done with Second Life (stuff you don’t tend to read about in the news but – to a tech or games person – is a heck of a lot more interesting by a long long way), and get some insight into the life of serious open-source programmers who succeed in living and breathing this stuff inside commercial environments

The Forge (Matt Mihaly)
* Who? developer of one of the earliest commercially successful text MUDs, now CEO of Sparkplay Media
* What? spent many years running Achaea, a text-only MUD that made a healthy profit from pioneering the use of itemsales (virtual goods) – and the things weren’t even graphical – and has now finally (finally!) moved into graphical games with the MMO he’s developing
* Why? one of the few MMO professionals who talks a lot about his experiences playing on consoles (especially Xbox), which makes for a refreshing alternate view – especially from the perspective of an MMO person talking about social and community issues in those games. Just like Brian Green, claims to be an indie MMO designer, but probably knows far far too much for that to be even vaguely justifiable

Vex Appeal (Guy Parsons)
* Who? ex-MindCandy
* What? Guy is an extremely creative … guy … who had a small job title but a big part in inventing and rolling out a lot of the viral marketing stuff we did for Perplex City (online game / ARG from a couple of years ago)
* Why? Awesome place to go for ideas and info on the cutting edge of doing games stuff with social networks. Usually. Also … just makes for a fun blog to read

We Can Fix That with Data (Sara Jensen Schubert)
* Who? ex-Spacetime, currently SOE
* What? MMO designer, but like Lum / Scott Jennings, comes from a long background as player and commentator, and shorter background as actually in the industry. Like Darius Kazemi, spent a lot of time in doing metrics / data-mining for MMOs
* Why? Take Darius’s insight into metrics for MMOs, and Scott’s knowledge of what players like, don’t like, and ARE like, and you get a whole bunch of interesting posts wandering around the world of metrics-supported-game-design-and-community-management. Good stuff.

Zen of Design (Damion Schubert)
* Who? ex-EA (Ultima Online), currently at Bioware (MMO)
* What? MMO designer who’s been around for a long time (c.f. UO)
* Why? Damion writes long detailed posts about MMO design, what works, what doesn’t, practicalities of geting MMO development teams to work together, how the playerbase will react to things, etc. He also rather likes raiding in MMORPGs – which is fascinating to see (given his heavy background as a pro MMO *designer*)

[NC] Anson (Matthew Wiegel)
* Who? ex-NCsoft
* What? Dungeon Runners team
* Why? was doing lots of interesting and exciting things with data-mining/metrics in the free-to-play low-budget NCsoft casual MMO. Watch this space…

People with nothing to do with games, but you might want to watch just because they’re interesting:
Bard’s World (Joshua Slack)
* ex-NCsoft
* Josh is one of the key people behind Java’s free, hardware-accelearted, game engine (JME)
Janus Anderson
* Who? ex-NCsoft
* What? um, he’s been taking a lot of photos recently
* Why? watch this space
Mark Grant
* Who? non-Games industry
* What? an entrepreneur, web-developer, and Cambridge Engineer
* Why? very smart guy, and interesting posts on web development (no games tie-in)

Categories
computer games programming

Tilt-sensitive game using Macbook / Macbook Air

This is a tiny game we made more than 2 years ago as part of Perplex City. Many people are unaware that the tilt-sensors inside the iPhone are also available (or something very similar) inside every apple laptop (all Macbooks/most Powerbooks/some iBooks); we made a game that made use of this.

Unfortunately, this feature of the laptops is undocumented, and so Apple has changed the specs a few times, so the original files no longer work. With a small bit of mangling, I’ve fixed the very simple input script, and it all works fine on my Macbook Air.

Links:

Instructions:

  1. Download the file “maze-inertial.zip” from the second link above, and unzip it
  2. Download the AMStracker dmg from the third link above, and open it, and put all the contents in the Maze subdirectory of wherever you just unzipped to; it will want to overwrite a file – say “yes”
  3. Make a new file in that directory using a text editor, call it e.g. “start.pl”, and copy/paste the lines below (bottom of this post)
  4. Run the file start.pl; you might be able to double click on it, you might have to go to the directory in Terminal and type “./start.pl” to make it run
  5. Wait for the java app to load, you should see a ball on a chequered background
  6. Pick up your laptop and start tilting it to make the ball roll around
  7. NB: the lack of graphics was deliberate! the idea was to close your laptop lid and play just using the sound (this requires you to disable OS X’s auto-suspend feature though)
  8. You will get cryptic messages as you complete levels; there are hundreds of levels, and no save feature (!). Read the website links above to find out why…

PS: if you read the source and think “ugh, what a brutal and inelegant way to make this work” then congratulations – this whole game was done start to finish in something like 2 weeks (part-time; if we’d all been working full-time on it it was probably more like 4 days). We had to cut a lot of corners…

Source of file start.pl – copy/paste everything below this line

#!/usr/bin/perl -w

use IO::Socket::INET;
use Getopt::Long;

$| = 1;

my $flipx = 0;my $flipy = 0;my $port = 5399;my $ams = “./amstracker”;my $maze = “./maze.jar”;my $scale = 5;

GetOptions(“flipx” => \$flipx, # –flipx flip x axis
“flipy” => \$flipy, # –flipy flip y axis
“port=i” => \$port, # –port 5399 port to talk to
“ams=s” => \$ams, # –ams ./amstracker location of amstracker
“maze=s” => \$maze, # –maze ./tiltmaze.jar location of tiltmaze.jar
“scale=f” => \$scale); # –scale 3.0 tilt scaling factor

# Needs AMStracker v0.34 or above, currently at:
# http://www.osxbook.com/software/sms/amstracker/

print STDERR “Starting amstracker ($ams)…\n”;
open AMS, “$ams -s -u 0.1 |” or die “Can’t start amstracker: $!”;
my $line = ;
print “$line”;

if ($line =~ /software/) {print STDERR “No tilt sensor found.\n”;exit 1;}

print STDERR “Starting maze ($maze)…\n”;
system “open $maze”;

print STDERR “Connecting to game…”;
my $sock;my $count = 0;

while (1){
$sock = IO::Socket::INET->new(PeerAddr => ‘localhost’,PeerPort => $port,Proto => ‘tcp’);
last if defined $sock; # success!
$count++;
if ($count > 20){die “Can’t connect to game: $!”; }
print “.”;sleep 2;}
print “\n”;

while () {
my ($x,$y,$z) = split;
next unless $x =~ /\d+/;
$x = -$x unless $flipx; $z = -$z if $flipy; $x = int($x * $scale); $z = int($z * $scale);
last unless $sock->print(“[$x,$z]”); }

Categories
computer games massively multiplayer mmog links networking programming system architecture

New page – MMOG Development Links

With some WordPress-Fu, I’ve added a page that’s a category and auto-includes links with custom meta-information.

Or, in other words, there’s now a page where I can effortlessly post all my various bookmarked links to do with MMO development – and add my own commentary to each link – which you can’t ordinarily do. Which is why it’s taken me some time to get around to it (previous efforts to do this without customizing WordPress, or using plugins only, failed).

The (practically empty) page in all it’s (non-)glory can be found here:

http://t-machine.org/index.php/category/mmog-dev/

Over the coming weeks, I’ll be posting much more stuff to it. I hope.

Categories
agile devdiary programming

Writing games in Korean – part 3

Computer grammars, Unicode in Eclipse, and problems with Java’s XML parsers…

Categories
programming

Why did people start calling ‘rewriting things better’ refactoring?

(re-posting from private forum thread)

My response to this question was a bit of a gross generalization, and made several assumptions about the questioner; I hope I didn’t misinterpret what they were getting at. I *think* I got it right:

Because it’s not “rewriting things better”.

It’s “changing the implementation of things without changing the observable effects of the overall system, even if the system is BROKEN at the moment”.

By cutting out a much smaller problem than “rewriting things better”, you get a more powerful suite of techniques: there are complete processes which just don’t work if you allow yourself to spend time improving the code at the points where, instead, you are meant to merely “refactor”.

Damn useful technique – although I found it hard at first to get over the idea of leaving broken features broken: I felt guilty doing it. But it works, and eventually the guilt goes away, as you see that it really is the right thing to do (…in the appropriate situations, of course, and as part of the right process(es)…)

In the forum thread, I didn’t go into the details – you can google a lot of uses for refactoring.

There’s one great example I ALMOST wrote about last year, but didn’t get around to (I didn’t realise until now that I’d never posted it…Doh!): using refactoring with unit testing to convert an old code base to a new code base, but doing it with many many passes.

Brief summary of the parts that would be relevant here:

  • Traditionally, with a horrendous mess of a codebase that needs converting, you’d either suck it down and spend many months (maybe many man-years) “converting” it – or else give up and rewrite it from scratch, one part at a time
  • With Refactoring, you can rewrite it from scratch INSIDE the existing codebase, LESS than one part at a time, so that you get to use all the new code you’re creating immediately (instead of waiting for the complete system to be rebuilt-from-scratch)
  • This works because Refactoring does NOT “fix” the code, it merely moves it around a bit, perhaps changes some minor subparts of it, often parts that have no effect on the overall system
  • “no effect” means NO effect: no performance improvement, no change to the end-users
  • …which allows you to use a particular Refactoring task in your schedule to merely “make the code slightly more amenable to be improved in the future; make it a bit less messy”

Yes, this shows no overall improvement in the system after each Refactoring pass – but you make those passes so EASY and so QUICK to do that one individual programmer can do literally HUNDREDS of them in a few weeks.

Traditionally, even doing just one “rewrite this small part of the system better” task would take months to even get started, because each time you think you’ve “fixed” it, you uncover another broken part, somewhere else in the system, somewhere that SHOULD be unrelated but sadly isn’t, that prevents your fix from working.

With refactoring you just shrug, smile, ignore it, and get on with making small incremental improvements every day, and go home happy in the knowledge you’ve at least achieved SOMETHING today, even if no-one else will see the result of it until you’ve achieved many more days of little somethings. But then they’ll be amazed, because in the limit – as the complexity of the codebase / number of refactorings tends towards large numbers – the sum total of small steps of refactoring quickly becomes net improvements to the code base that were literally IMPOSSIBLE with traditional software engineering: they would have taken so many man years, you’d never have finished them in the liftetime of the product you’re working on.

So. Just one example there, of how refactoring is a useful – and unique – tool, quite separate from “rewriting things better”.

Categories
computer games games design java tutorials programming

Writing games in Korean – part 2

Using a custom font renderer that lets me position text exactly where I want it, I put together a simple game that teaches you to count from 1 to 10 in Korean characters.

Screenshot of numbers test

First, you HAVE to manually download the Korean font I’m using (because I don’t think I can legally include it in the executable file :( )

http://www.i18nl10n.com/fonts/UnBatang.ttf

NOTE: you MUST (windows is rubbish) do “Save to” and select C:\Windows\Fonts (this causes windows to automagically “install” the font) – I can’t remember how to install new fonts on Linux or OS-X, but it should work fine with both of those OS’s too.

Then just stick the two attached files in a directory somewhere, and double-click on the one called korean-numbercounter-all.jar

Essential library (download this)
Main file (download this to same directory as the library file, and run this file)

NB: this is mainly just a proof of concept to test my rendering code, I’ve got a couple of other simple “games” like this I’ve knocked up as tests.

Categories
computer games dev-process java tutorials programming

Writing games in Korean – part 1

First, check whether I can actually render Korean characters. I know that Unicode supports nearly all of the glyphs (or, more correctly it seems, “ideographs”) needed for Chinese, Japanese, Korean (typically abbreviated as “CJK”) – but I also know that MS Windows fonts are infamous for missing some/most/all of the unicode characters (there are tens of thousands of them, so this is not all that surprising – except that Microsoft has been shipping OS’s localized to those countries for many years now, so I’d hoped maybe they would include at least ONE “CJK + Latin” font on all machines by now, no? No; they don’t).

Quick test program using Java (NB: java one of the easier languages for this because it was invented late enough that Unicode had settled down, and so it has extremely good Unicode support built-in to the core libraries, unlike C++ et al. By default, Java uses unicode almost everywhere, so I don’t need to debug unicode support, yay!):

NB: …but then when I tried it I quickly discovered a problem. I quickly gave up on the obvious route, so might have missed a workaround, but… I could have done this using Java’s built-in GUI toolkit (Swing), which supports using any font as a text label. However, it seems there’s a design flaw in Swing that’s been around since 1997 (yes, folks, 11 years and counting): the only component that can have the font changed – JLabel – which is a “general-purpose” label for any GUI component (good idea!) … is incompatible with the only component capable of being a button – JButton – which is a “general-purpose” button. Crap. If I’m not missing something here, with one small piece of poor API design, Sun have made their *almost* international-friendly GUI system almost completely useless for multi-language GUIs. Just to be clear: it’s not specifically internationalization they’ve broken here: its a general issue – any forms of customized rendering cannot use the JButton / JLabel combo. Why? Why, why, why would you do such a stupid thing, after going to the effort of making these generic widgets? Oh, well. Custom rendering it is…

/**
 * Simple test program that renders Unicode characters from a particular font of
 * your choice, a couple of thousand at a time, automatically wrapping them based on
 * render width on screen.
 * 
 * You need to manually change the font name and the start/end co-ordinates of where
 * in the unicode constellation to render.
 */

import java.awt.*;
import javax.swing.*;

public class PaintSomeUnicodeChars extends JFrame
{
	void setup()
	{
		getContentPane().add( new charPane() );
	}
	
	public static void main( String[] args )
	{
		PaintSomeUnicodeChars window = new PaintSomeUnicodeChars();
		window.setDefaultCloseOperation( JFrame.EXIT_ON_CLOSE );
		window.setup();
		window.pack();
		window.setSize( 600, 500 );
		
		window.setVisible( true );
	}
}

class charPane extends JPanel
{
	public void paint( Graphics g )
	{
		int si = 44032;
		int ei = 46000;
		g.setColor( Color.red );
		g.setFont( new Font( "Verdana", 10, 40 ) );
		g.drawString( "Chars from " + si + " to " + ei, 30, 30 );
		
		g.setColor( Color.black );
		g.setFont( new Font( "Lucida Sans Unicode", 10, 20 ) );
		
		int accumulator = 0;
		int height = 20;
		int xwidth = getWidth();
		for( int i = si; i < ei; i++ )
		{
			char[] chars = Character.toChars( i );
			String drawString = String.copyValueOf( chars );
			int advance = g.getFontMetrics().charsWidth( chars, 0, chars.length );
			
			if( accumulator + advance > xwidth )
			{
				accumulator = 0;
				height += g.getFontMetrics().getHeight();
			}
			
			g.drawString( drawString, accumulator, height );
			
			accumulator += advance;
		}
	}
}

Why choose Lucida Sans Unicode?

Apart from the fact that it’s present on all modern Windows machines automatically, I cheated a bit and used Character Map (Start > Programs > Accessories > System Tools > Character Map) to quickly inspect each of the fonts already installed and find out which ones had lots of unicode in them. Character Map has a neat feature where it completely ignores missing characters, automatically skipping over them, so you can very quickly see if a font has lots of unicode, some, or very little.

Stepping through in the java app, I fairly quickly found lots of Unicode characters. success – I can correctly render arbitrary Unicode characters in java.

NB: important because it took a lot more lines of source than I expected; java’s built-in “char” datatype is incapable of being used to render Unicode, because it’s too small a range of values. That also means you can’t use ANY methods that take a char as argument (this is documented in the Java API docs for Character). I’d never really used the methods that take int’s as argument before…

Fonts…

Not really happy with the font, though, and it’s clearly missing a whole bunch of characters. Some googling for free CJK fonts found me that allegedly Arial Unicode MS would work – which comes free with Microsoft Office (which I have on one of my machines).

NB: the way font licensing works, it’s probably illegal to copy a font you have on one machine to another machine you own. Unless you can obtain the font from the original source (e.g. in this case by buying a second copy of MS Office).

Arial Unicode MS has lots of characters, but … they’re wrong. At least, one third of the ranges of Korean characters are completely useless, as far as I can tell, because whoever made this font (whoever Microsoft licensed it from?) didn’t read the spec carefully and stuck the wrong characters in from U+1100-U+11FF. More on this later…

Giving up on that font, I went looking for others. The first four or five I tried that didn’t look ugly were all from download sites in Japan or Korea and kept crashing on the download. I found a couple of places hosting UnBatang (“UnBatangOdal.ttf”) and claiming it was free, but the first I found where the download didn’t crash was on http://www.i18nl10n.com/fonts

UnBatang (the name you have to use in Windows / Java from inside an application in order to load it) seems to work very well. It has nicely painted ideographs for the main range of Hangul characters/syllables, and it has *correct* glyphs for the other two ranges (although they’re a bit ugly).

Unicode Hangul

Specifications for official standards tend to be big. Really big.

Specifications for anything to do with internationalization tend to be huge.

Add in localization and data-transfer between different cultures, and you can expect something gargantuan.

So, I was really really happy to find a downloadable copy of only the CJK Chapter of the Unicode 5.0 specification (it’s still a big document!). This contains a full explanation of what Hangul is in Unicode, where to find it (yay!), and why there are not one but three separate sets of Hangul glyphs.

Here’s the preface to the Chapter 12 PDF I downloaded, in case you want to find the spec / electronic version yourself:

Electronic Edition

This file is part of the electronic edition of The Unicode Standard, Version 5.0, provided for online
access, content searching, and accessibility. It may not be printed. Bookmarks linking to specific
chapters or sections of the whole Unicode Standard are available at

http://www.unicode.org/versions/Unicode5.0.0/bookmarks.html

Purchasing the Book

For convenient access to the full text of the standard as a useful reference book, we recommend purchasing
the printed version. The book is available from the Unicode Consortium, the publisher, and
booksellers. Purchase of the standard in book format contributes to the ongoing work of the Unicode
Consortium. Details about the book publication and ordering information may be found at

http://www.unicode.org/book/aboutbook.html

Unicode 5, Hangul, and Arial Unicode MS

So, armed with the offical spec, I read up on Hangul. What did I find?

The Unicode Standard contains both the complete set of precomposed modern Hangul syllable
blocks and the set of conjoining Hangul jamo. This set of conjoining Hangul jamo can
be used to encode all modern and ancient syllable blocks.

(this is the glyphs at U+1100–U+11FF)

“conjoining Hangul jamo” means “these glyphs have been positioned inside their spaces in the font so that if you need to make one ideograph out of, say, four Korean letters, you just pick the top-left version of the first letter, the top-right version of the second, etc, and OVERLAY all the glyphs, and what comes out will autamatically be correctly spaced out etc”.

What did the author of Arial Unicode MS do?

Made all those glyphs take up the full available space, and centre them horizontally and vertically.

Why? Seriously, why? Because any application that tries to render those characters is going to render them on top of each other (according to the specification, this is the ONLY point of having those characters), and you won’t be able to read at all what the letters say.

If you want to render just individual letters from the Korean alphabet, there’s a different range of Unicode where you can find them all centred etc (which Arial Unicode MS also has … so it seems to be just copy/pasting internally).

I guess the font author just didn’t read the spec. Or I’m completely misunderstanding the spec. But the fact that UnBatang spaces the conjoining jamos out in such a way that this works as I expected it to suggests to me that it’s the Arial Unicode that’s broken…

The joy of Conjoining

I couldn’t get hold of the Unicode spec section on how to conjoin, because of some mimetype problems between their server and my PDA’s web browser. I figured I could probably work it out by trial and error fairly quickly.

I can’t remember how to spell my name in Korean yet – I know the letters, I’m just a bit flaky on what the placement of them is. So, a quick experiment, to see how easy it is to position characters using the Conjoining Jamo from UnBatang:

	... constants used later - the Unicode values for the letters of my name: ...
	
	int a = 0x1161; // jungseong vowel
	int d = 0x1103; // choseong consonant
	int d2 = 0x11ae; // jongseong consonant
	int m = 0x1106; // choseong consonant
	int m2 = 0x11b7; // jongseong consonant
	int blank = 0x110b; // silent char you place at front if first letter is a vowel
	
	... the body of the paint method: ...
	
	g.setColor( Color.black );
	g.setFont( new Font( "UnBatang", 10, 30 ) );
		
	int w = 0;
	int lead = 0;
	w += renderIdeograph( g, blank, lead, 40  );
	w += renderIdeograph( g, a, lead+w, 40  );
	w += renderIdeograph( g, d2, lead+w, 40  );

	lead+=30; w = 0;
	w += renderIdeograph( g, blank, lead+w, 40  );
	w += renderIdeograph( g, a, lead+w, 40  );
	w += renderIdeograph( g, m2, lead+w, 40  );
		
	lead=0; w = 0;
	w += renderIdeograph( g, blank, lead+w, 80  );
	w += renderIdeograph( g, a, lead+w, 80  );
		
	lead+=30; w = 0;
	w += renderIdeograph( g, d, lead+w, 80  );
	w += renderIdeograph( g, a, lead+w, 80  );
	w += renderIdeograph( g, m2, lead+w, 80  );
	
	... and then this method to do the paint of each part of an ideograph: ...
	
	/** Doesn't really render an ideograph, renders a single glyph from the Font */
	public int renderIdeograph( Graphics g, int i, int x, int y )
	{
		char[] chars = Character.toChars( i );
		String drawString = String.copyValueOf( chars );
		int advance = g.getFontMetrics().charsWidth( chars, 0, chars.length );
		
		g.drawString( drawString, x, y );
		
		return advance;
	}

Which renders exactly like this:

hangul-adam-unbatang.PNG

Which makes me want to point out the nice thing about properly specified fonts: in the source code, I simply did “the natural thing”, as if I were outputting characters in an arbitrary conjoined language:

  1. Render the first part of the first ideograph at (x,y)
  2. Ask the font to tell you how many pixels wide (w) it just rendered that part
  3. Render the next part of the first ideograph at (x+w,y)
  4. …repeat until first ideograph is complete, increasing w more and more each time…
  5. Choose a value for how much you want the ideographs separated from start to start (ideograph_width)
  6. Render the first part of the second ideograph at (x + ideograph_width, y)
  7. …repeat as for first ideograph

And it worked. First time. I didn’t expect it to – I expected to have to do something strange like manually “reset” the (w) value each time I went from the first line of the ideograph to the second (Korean orders letters top-left, top-right, bottom-left, bottom-right, (repeat) … as opposed to Latin which is just left, right, more right, even more right, etc).

For this to have worked, it means the font is deliberately rendering the lower letters (e.g. d2 and m2 in my constants) a long way to the left of the origin that you tell it to render them at. This would be very, very confusing if you just tried to render these letters individually (well, duh).

Of course, if you try to run that code using Microsoft’s Arial Unicode MS font, you get a complete mess instead, because that font is FUBAR, as mentioned before. You get this:

hangul-adam-arial-unicode.PNG

…which is completely incomprehensible.

Categories
computer games databases entity systems games design massively multiplayer programming system architecture

Entity Systems are the Future of MMOs Part 4

Massively Multiplayer Entity Systems: Introduction

So, what’s the connection between ES and MMO, that I’ve so tantalisingly been dangling in the title of the last three posts? (start here if you haven’t read them yet).

The short answer is: it’s all about data. And data is a lot harder than most people think, whenever you have to deal with an entire system (like an MMO) instead of just one consumer of a system (like the game-client, which is the only part of an MMO that traditional games have).

Obviously, we need to look at it in a lot more detail than that. First, some background…

Categories
programming

Data and Digital Computers don’t exist?

There’s no such thing as a digital computer. Except in extremely controlled or constrained environments. I suspect those exist … somewhere … but I don’t work in hardware development, so I’m pretty ignorant of that area.

This is something I didn’t realise until I was studying Computer Science at university, and it was something of an epiphany for me: not that “computers aren’t infallible” but that – at least in the way I spent my life imagining them – they don’t exist AT ALL. All you’ve got is an analogue computer that pretends a billion times a second to be digital, and some number of times a second fails. Fortunately for you who rely on digitalness, a bunch of other analogue circuits inside your analogue computer exist specifically to detect when the simulation diverges from a what a theoretical digital computer would do, and overwrite parts of the analogue data with new data to make it conform to the digital ideal it is simulating. It’s only ever a simulation, and – for the most part – a poor one, with lots and lots of stuff going on to try and keep the simulation accurate.

This isn’t an analogy, or metaphor – this is how modern computers actually work.

Is this useful? Probably not :),

I think indirectly it helped me, over time, to let go of organization in software development and embrace the unknown, which is pretty useful in any explorative or uncontrollable project management situation (just to be clear: I don’t mean only games development here, I’ve seen plenty of similar situations in mainstream IT). The last time I had a grasp of current physics theory, it was fairly widely accepted that there’s nothing in the real world that’s ever “known”, there are only estimates, but digital computers quietly squat there in literature and culture apparently sticking two fingers up at the whole concept. Only, they don’t, it’s really just us being lazy about terminology, and being self-delusional about the concept of “digitalness”.

It also emphasizes some of the fundamental differences between two aspects of systems: substance and data. A very poor explanation, I’m too tired, but: the substance of the system that is your computer is in no way digital; the data that conceptually “exists” (but doesn’t, in fact, exist) within that system is truly digital. The substance and the data are two extremely different things. So, part of each system (e.g. your Operating System, or your application, or your networking library, etc) actually exists, and has real-world behaviours. Other parts do not exist, and are more conceptual.

I find this in turn helps me appreciate the fundamental incompatibilities of data and systems, except for systems designed to store particular kinds of data, and vice versa, and above all to appreciate data as the lingua franca for all digital electonics: it’s the only thing in that universe that actually is digital. All the rest is an approximation…

Categories
databases dev-process programming

Improving your dev process: Automating collection of game-crash info

(As spotted in last month’s Game Developer magazine’s Inner Product column. Good enough that I wanted to draw some attention to it, and hopefully add a bunch more of these good ideas as I come across them / remember them)

Most games, when they crash hard enough, memory dump. The customer (or internal tester during development) can then be requested by the dev team to find a specific file in a specific directory (the mem dump) and email it to them for analysis.

Some games actually popup a dialogue to the user, to tell them about this file, and where to find it, and who to email it to.

Some even include some contextual information automatically (“the user was on Level 3, and has the following DirectX hardware info”). Most just ask the user to fill in a field “telling us what you were doing when it crashed”.

Most games, during development, automatically either popup – or display in a custom textarea – a message every time somethign unexpected happens, something that MIGHT be a symptom of a bug. There are even conference talks about how to avoid making your playtesters so sick of error messages that they stop bothering to read them, and hence the messages become mostly a waste of time.

How many games do a simple call to the SQL client library to send to the local SQL server a simple detailed report of what happened, when, why, and how bad it was? This information is ALL already available, but how hard is it to copy/past the following into your source code and execute it as an SQL command?

INSERT INTO reports.errors (time, problem, severity, stacktrace, user, IP, clientversion) values (TIMENOW(), “+assertion.getMessage()+”, “+assertion.getLevel()+”, “+assertion.getStacktrace()+”, “+assertion.getUsername()+”, “+system.getLocalIPAddress()+”, “+client.getVersion()” );

Nice idea – free, detailed, metrics on exactly why, when, and where your game exe is crashing, any unexpected problems, anyone who’s running an out-of-date version, etc.

And with the dev team no longer having to rely on users *actually bothering to tell them* when they see big flashing warnings and errors, we might avoid some of the programmer/artist antipathy (“[artist] hey, can you help me with something?” “[programmer] its got an error! no wonder it doesnt’ work!”, “oh, ignore that, it always does that”, “um, how long has it been ‘always’ doing that?”, “ever since around the time the game started running really slowly, 2 months ago. It sucks – now everything takes me twice as long to test. But that’s just life, I guess” [cue: programmer grinds teeth, smashes forehead on desk, or similar]).

NB: I have nothing against artists. They are just an easy scapegoat for a not-very-funny-but-actually-quite-painful-if-you’ve-been-there joke. No artists were harmed in the making of this post.

Categories
games industry programming system architecture

Central Tech: Essential strategy, or complete waste of time?

I work at a company where managing and directing a subset of the global core-tech is part of my day-to-day job, and I’ve been trying to find good, positive ways forwards. After a year in this role, I’m looking back and wondering whether it’s working for us, and where it’s been good and where it’s been bad. I don’t know what the perfect solutions are, so I’m just going to document the issues I’ve been wrestling with, and my current thoughts on what causes them, and my current ideas on what can be done to fix or avoid them. Personally, although I’m confident there is at least some role for some form of central technology in many games companies, I’m not sure yet what that role is.

Definition

Interestingly, there are no clear definitions on Google/Wikipedia for either “central technology” or “core technology” (the two main names I’ve heard used throughout the games industry). I see that as a warning that this is pretty under-explored territory I’m heading in. Cool.

The vague definition is: “sharing technology between multiple game-project teams and taking any new technology created during a game project and re-using it afterwards for other projects”.

There appear to be two major concepts wrapped up in the central-tech teams that I’ve seen. Some teams have only one of these, some have both. Usually, there are substantial examples of both, mixed within the same team. Often, the team has one focus in particular, but people outside the team in the rest of the company pick one or the other to *believe* the team has as their focus, depending upon what they personally most want out of the team. When this is divergent from the team’s own internal focus, things are off to bad start already.

Centralised technology (control, direction, authority, mandating):
– control: deciding what technology to create for “general use” by game-project teams
– direction: researching and reporting to the teams what tech is needed over the coming years by all game teams (yes, including telling teams themselves what they need)
– authority: deciding all general and long-term technology choices for ALL game teams, and making decisions on what tech teams are allowed to create and use
– mandating: forcing game-teams to use the centralised technology

Shared technology (re-use of common code, standing on shoulders of others, increased stability, decreased startup costs)
– re-use: finding code that gets written the same way on successive projects, and turning it into a library then never re-writing it again
– standing on shoulders: getting a domain expert to write difficult code once, turning it into a library, so that people who don’t understand enough of the tech to write it themselves can leverage it into future games without having to learn it themselves
– stability: using code in multiple successive projects so that eventually, after a few projects, it is extremely heavily tested and bug-fixed, much more so than any code written and used for the first time in a given project
– decreased startup: reduce the amount of time after a project goes into production that it takes to get the “first playable” version of the game running, because a lot of “typically useful” code already exists; unlike “re-use”, this is NOT necessarily code you will keep in the long-term – it’s just a leg-up to get you started quickly

Problems (real or perceived)

In some cases, Central Technology / Core Technology teams have been growing on game development companies like mould on a piece of bread in recent years – gradually expanding their remits (and team sizes) until they end up affecting (usually negatively, despite the best of intentions!) every part of game development. I’ve had lots of experiences with centralised tech depts, although all of them in the games industry – interestingly, I never saw any software company even consider doing this when I worked in mainstream IT (across various industries), only games companies. And it seems a relatively recent phenomenon, too – at least at the scale that we see nowadays.

Typical problems cited by game-teams and third-parties affected by coretech (NB: these may not be fair!):
– a bunch of non-game programmers inventing arbitrary (undirected by user) solutions for potential (not verified because no concrete use-case) problems for not-yet-existent (haven’t started production yet) projects
– forced upon game-teams who have to twist and force their game into the inflexible (already built, without knowing the game-teams requirements because the game-team wasn’t around at the time) mould of the now-built core-tech (ask any EA employee about centralised technology initiatives…)
– generates pointless tools that are always in search of a problem to solve

Ultimately, the more bitter and cynical end of the spectrum of those who’ve experienced coretech come up with something that veers towards one of:
– it’s an excuse to spend infinite amounts of money forever without clear benefit and no oversight or restraints on feature set or spending
– it’s an excuse to take freedom and power away from game-teams and centralise it under an authority who has no reason to care if the game ships on time or with most appropriate feature-set

I suspect those two ultimate, generic criticisms map to the two typical definitions – the former to “shared tech” and the latter to “centralised tech”, although I’ve not looked at that in detail.

Most (not all!) of the experiences I’ve personally had in games companies have been failures, some narrowly (managing to achieve good things, but failing at what they were originally setup for), some spectacularly (failing to provide anything good at all, and even dragging down one or more game-projects with them). Some flailed about and killed projects in their wailing death-throes, others gorged on the corpses of the projects they’d killed and rose again from the bones of what was left, like some undead creature (which is a worrying precedent: “how do you kill that which has no life?”). It’s fair to say my experiences have been mostly bad ones :) … BUT: until it was called “central tech” or “core tech” the *same kind of initiatives* (re-using code, centralising decision-making, making powerful proprietary libraries) seemed to work well, generally, in teams that I was in or working with. The failure cases seem (at least in my small amount of experience) to be a relatively modern problem.

What’s the point?

Speaking to people throughout the company where I currently work, and working with some of our pre-existing core-tech teams, I’ve found myself more and more asking a simple question: “why do we have Core Tech?” The answers have been illuminating…

“to write everything-including-the-kitchen sink, then on each game project only use the bits you need”
“recycle code from an existing project”
“I don’t know”
“write code that can be used and re-used in multiple projects”
“share the really cool and difficult stuff that one team did with other teams so they can benefit from its brilliance”

This has made me realise / remember a couple of things:
– non-programmers, and many inexperienced programmers, grossly underestimate the ratio in development time/cost between “making code that solves one problem” and “making a generic solution to a whole class of problems of which the current problem is just one example”. The first explanation described above is usually prohibitively expensive by *a large factor, maybe even an order of magnitude*, but many people seem unaware of this
– CT is sufficiently ill-defined and *poorly marketed internally* at most games companies that many internal dev staff genuinely don’t know why it exists or what purpose it’s supposed to be solving, let alone have the opportunity to rate the success or failure at that purpose
– many people notice that game development often involves re-solving or re-performing similar tasks, and wish for a perfect world in which you only ever have to do one “kind” of solution once, and then you get to re-use it forever after. That is largely true in mainstream IT, but sadly this seems to be far from the truth with games development. Personally, I suspect that the day the technology becomes derivative is the same day you realise your game itself is derivative, and has lost most of it’s fun.

I also wonder: if we don’t have CT, are you saying we wouldn’t re-use code? As a programmer, do you avoid finding code from old teams and finding ways to use it in your current project? Isn’t this a *basic part* of your job, part of what we pay you to do?

Is CT something that gives game-programmers an excuse to be less diligent than they would otherwise willingly be, thereby setting the scene for them to later complain that it’s “not as good as if I’d written it myself” (which is one of the cruelest things I’ve seen game-teams do to their CT teams, even if it’s true).

Other side of the coin: why is central tech hard?

Project teams steal the best hires
– Project teams, especially Producers and Leads, have a hugely difficult problem (ship a game on time, and make it “fun”, so it sells well) and a huge amount of personal buy-in (if it fails, they have no second chance: their sole responsibility is that one project), and a huge amount of measurability of success (how many sales does the game make?). They have a vested interest in finding the best of the best people, and they have – ultimately – large budgets to recruit them with. If someone really good at developing code for games is in central tech, they usually get stolen for a game-team sooner or later. Central tech departments have too little visibility (how do you measure the brilliance or crapness of your coretech? there’s no sales figures!), no concrete deadlines (they randomly invent them, or borrow a game-project’s deadlines, but they really don’t sink-or-swim on that project-deadline like the game-team itself does!), and usually restricted budgets (if not, well, that’s when the unlimited-spending-without-reason problem kicks in…). They also are rarely as cool as working on a “real game”: most people joined the industry to Make Games, not to Write Random Code For Someone-else’s Game. CT finds it hard to retain the best people…

Writing code a second time is generally a lot cheaper than modularising the original version
– …I’m hoping this is self-obvious. But sadly it means that it is ALWAYS easier for any individual game-team to re-write *from scratch* the code they wrote for their last game than it is for CT to make the generic version that they’re usually asked for. Game-teams then get pissed that the CT team seems incompetent compared to them (they notice that the time they have to wait for CT to do something is MORE than it would take them to do it themselves).

CT is always setup as an “ongoing” task: there is no defined beginning or end to what they do. Games companies (developers and publishers) are very specifically geared towards “project-based” tasks.
– From the funding, to the project management, to the staff rewards schemes, everything about the games-industry is setup for project-by-project operation. We know (sort-of) how to make project-based development work (reasonably) well. We don’t really have ANY experience in non-project work – certainly anyone who’s only ever worked in the games industry (who are strongly self-selected by current standard recruitment procedures) will have no experience of non-project development. Is it any surprise we often screw it up? Moreover, is it any surprise that we have such difficulty making the non-project CT team be respected and understood by the purely project-based game-teams?

Ideas

I’ve been kicking around some ideas on new directions to pursue. These may not be that great, but at the moment I don’t have to make any decisions, I just have to provide options, so I’m interested in any suggestions or alternatives…

Think of CT as a human resource, not a code repository.

This is one thing I’ve been trying: what *I* needed most urgently out of CT was the *expertise* component, and actually I didn’t really care about the code re-use etc. I knew I would, but I had a more pressing problem of two projects that lacked the necessary expertise in Network Programming to make an MMO. So … I created a tiny team dedicated to doing the network code for any game, and explicitly NOT to writing reusable network technology (NB: many years ago I ran a failed network middleware company, so I had a healthy apprecaition of how difficult and time-consuming it is to write generic network systems; I knew we simply did not have the time to write that stuff when we had projects already in full production that needed network code NOW).

Because of that, we can’t share the technology they create. Not yet, anyway. But we *can* share the expertise of the people in the team. They have to do a fair amount of international travel (bummer). So far, it seems to be working OK, but … it’s too small a sample and it’s been running too short a time (do you have any idea how hard it is to recruit really good network coders? :( ) to make any firm measurements of the success or failure of the approach.

Architect all your CT initiatives on a project-basis, not a service basis

I’ve seen a couple of examples of this, usually where senior people were frustrated by the progress of CT but needed some new example of the kind of stuff it was providing – so they went and started their own new pet-project to provide that, but outside the remit of the CT group. I’ve seen examples of this that were positive, and examples that were negative. Politically, it’s likely to cause a shitstorm as far as I can see, no matter how you sell it (unless this is blessed by CT itself, and none of the examples I’ve seen were their idea, although IIRC one of them did get CT’s agreement – they just knew they didn’t have BOTH the expertise AND the budget to provide it, so were happy to not have to take responsibility for it).

Certainly, it’s been the fastest way to get real significant progress on anything major: by doing it as a project, and going through the “standard” greenlight – pre-production – greenlight2 – production – ship cycle that all games companies are accustomed to for their games, large amounts of money have been secured quickly, and the projects have got going quickly.

Embed CT teams into the game-teams

The idea with this one is to overcome both the antipathy between game-teams and CT (by making them part of the same team), and also to ensure “automagically” that the CT team-members are working stuff that is guaranteed directly applicable to a real game – because they’re reporting to the Producer of a given game-team all the time.

If it works out well, this could solve all the problems. Although I’m not yet sure I understand how – in the long term – this is any different from having no CT at all. As noted earlier, this is closest to how I remember things working well before the idea of CT gained widespread popularity. OTOH, I’m not sure if that same situation will even work nowadays – the industry has changed a lot since then.

History of CT

I’m not sure about this, but my vague memory is that CT grew in popularity and – more importantly – commonness (i.e. how many companies were doing it) in parallel with the axing of internal game-engine teams and the move towards licensing external game-engines.

I have suspicions that a lot of the confusion of “what do we have central technology for?” comes from this, two-fold:
– a) those companies that didn’t entirely get rid of their engine teams had people left a bit stranded, who ended up being folded into CT, and then tended to carry on working much as they had inside the engine team, despite the different overall task of the dept (and this happened partly because of the lack of clear direction from management about what the new team was for anyway)
– b) many of the companies that dispensed with their substantial proprietary engines they’d been using for years then jumped to Unreal et al found that actually the licensable engines weren’t as good as they’d been lead to (or had lead themselves to) believe: they couldn’t go back to having an engine team, but they needed a way to reduce the huge budgets they were spending on just basic modification and fixing of the external tech, that weren’t easily justifiable on the game-budget (because they weren’t forseen, and were adding NOTHING to the game’s end feature list), so they ended up creating a CT team to hide / provide the budget to do that kind of work

I could be completely and utterly wrong here – this is just some ideas that have been kicking around in my head recently.

EDIT: I’d also like to point to this good post about game-engines, and why you shouldn’t build them (not necessarily a “never do it” argument, but an explanation of why you might want not to).

Categories
java tutorials programming

Java considered harmful as first programming language…

Today, I was pointed at an article suggesting that java is responsible for the decline in the quality of Computer Science graduates.

Specifically if you want to be a Computer Scientist, I’d say that the initial claim that “java is bad as a first language” is true, just as “C++ is bad as a first language” is true. What disappoints me is that they go on to suggest that C++ is good as a first language (disclaimer: my uni course taught ML as first language, ARM assembler as second).

Categories
java tutorials networking programming

Java NIO Server Example

I’ve made a small simple but complete java NIO server (with full source included) that is free for anyone to use for anything. At the moment, it only deals with sending and receiving strings, and isn’t optimized, but if anyone wants to improve it and send me the changes then I’ll post up an improved version here.

Download, documentation, license details, tutorial etc after the jump…