How much should you use a Scripting language when writing a Game?

Ask 10 game developers if you should use more script code or less, and you will get 11 different answers. They are all correct: it’s very situation-specific. Use of scripting languages is highly dependent on the humans and the practical / project-management issues.

Why?
Continue reading

Makerbot Replicator2 Upgrade: new fan duct

I reviewed a whole bunch of these on Thingiverse, and the best combination of simplicity and effectiveness appeared to be this one:

…but you want to print these at 0.1mm layer height, or less, and that’s difficult with the model above. So I tweaked it a bit:

NOTE: The WordPress authors can’t be bothered to implement safe/secure uploads; instead, they block ALL uploads (which is very stupid and doesn’t solve ANY security problems). This file has to be renamed when you download it. Stupid, stupid, stupid programmers.

WordPress Is CRAP.stl

Changes

The original model has two small circular screw-holes. Makerbot printers can’t print these very well – Both the official slicer, and the main commercial / open-source slicers get it wrong and print the holes in a bad way (untethered; it’s an obvious heuristic to fix, but no-one has implemented this yet). Net effect: the holes slide about during layer 2, and usually detach, and ruin the print.

To fix this, I cut a slice out of each hole, making them into hooks. This follows the design of Makerbot’s original fan duct, and makes the print easy to do even at high speed and low layer-height.

3DPrinting tips: cooling, fans, and PLA temperatures

This is hugely important to 3D printing, and generally: no-one talks about it online (at least, not with any authority: more “blind leading the blind”). Which is silly: the engineering behind this is obvious (and it’s often obvious from looking at a 3D printer that it’s been under-engineered in this critical area).

NB: I don’t know which parts of “correct, obvious” cooling are blocked by stupid patents that should never have been granted. That might excuse the poor design of most printers today. Might.

My experience so far is that “more cooling of printed object is better”. But that’s an over-simplification…
Continue reading

Towards a Unity3D ECS: what GUI? What features?

This year, one of my goals was to build a sane, powerful, viable Entity System / Entity-Component System within Unity3D. Something I could build current and future games on, and save a lot of dev-time (both in ease-of-use and in easily adding missing features that I wish Unity had).

So, progress?
Continue reading

Site disappeared? Apache ARGH!

Transferring server to new home.

Allowed it to install latest versions of core software.

Turns out … the lovely people at Apache have made major site-breaking changes to their config system, so that upgrading WILL break existing sites.

Took an hour to discover that :(.

Epic saga of Unity3D deserialization of arbitrary Objects/Classes

(a summary writeup of the things I tried – based on this Unity blog post: blogs.unity3d.com/2014/06/24/serialization-in-unity/ – to get Unity 4.5 to correctly serialize/deserialize simple object graphs)

UPDATE: I belatedly discovered that EditorUtility.setDirty() is fundamental to Unity; if you have even one single call of this missing, Unity will corrupt data when you save the project. In some of the cases below, where everything was fine EXCEPT on fresh startup, I suspect that was the real problem: a missing setDirty(). I thought this method was cosmetic, but tragically: no.

This is not easy to read, it’s bullet points summarising approximately 2 weeks of testing and deciphering (plus a month or so spent learning-the-hard-way the things that are in the Unity blog post above. Note that the blog post only came out very recently! Until then, you had to discover this stuff by trial and error)

Serializable-thing:

– example on blog won’t work if e.g. your Class is an object graph.
– e.g. class Room { List doors; }. class Door { Door otherDoor; }
– …can’t be serialized. Can’t be stored as a struct, unless you have a GUID per “Door” instance. That’s no surprise, didn’t bother me: This is standard in most serialization systems.
– So we go on a quest to create a GUID…

GUID1:

– class Door : ISerializeCallback { Door otherDoor; public string guid = Guid.NewGuid().ToString(); /* use OnBefore / OnAfter callbacks to store the guid of the other door / find the door from guid */ }

Unity inspector lists:

– if you add an item to the list, 2 things corrupt your data:
1. all existing objects are NOT MOVED, but are CLONED into new list. This breaks our GUID.
2. the new item is a carbon-copy clone of the old – so it ends up with the SAME GUID (ouch)

GUID2:

– class Door { [SerializeField]private string _guid = Guid.NewGuid().ToString(); }
– now the inspector no longer clones the guid … BUT … instead it sets it to null.
– it seems to run the constructor, then use serialization to delete all values it finds

GUID3:

– class Door { private string _guid = Guid.NewGuid().ToString(); public string serializedGuid; /** use serialize callbacks to maintain serializedGuid */ }
– Disable the Editor’s list-renderer: it has at least two major bugs that corrupt data. Use the UnityEditorInternal.ReorderableList.
– NB: ReorderableList – with no customization – does not have either of the two bugs I saw with built-in Inspector’s lists.
– NB: as a bonus, with RL, we can avoid the “run constructor, delete all data, deserialize, reserialize” cycle, and directly create a new, blank instance with a safely generated – FRESH! – Guid.

Ctrl-D in Editor:

– breaks this, because we have no idea if a deserialize event comes from “ctrl-d” or from “read back from disk” (or from a prefab instantiating into the scene).

GUID4:

– class Door { public Room serializedRoom; private string _guid; /** use callbacks to peek into the serialized room, and use the guid as a “local ID within this room” */ }
– Because we’re now leaning on Room (which is a GameObject) to provide part of our GUID, in theory we don’t care about prefab/ctrl-d/etc — the code inside MonoBehaviour will automatically separate-out those cases for us.

Work with everything except prefabs, I think

– eventually gave up, removed the GUID completely, and changed game design to only allow ONE door to connect any pair of rooms. Yes, I was desperate!

THIS WORKS! Until you restart Unity:

– restarting unity causes this to deserialize on a different thread, and now you’re NOT ALLOWED to peek into the serialized room. You cannot use it as a substitute ID. You get crashes at load-time).
– …apart from that, this works fine. We have a solution that worked for ctrl-D, for prefabs.

GUID5:

– class Door { public Room serializedRoom; private bool _isUnityStillOnBackgroundThread; /** use the bool to say “deserialize happened but the data is NOT VALID, DONT RUN ANY METHODS” */ }
– …also use an AutoRun script with [InitializeOnLoad] and “EditorApplication.update += RunOnce;”
– …the autorun script waits for Unity to startup, then goes and Finds all objects, and finds all Rooms, all Doors — and does the fixup.

Fail, but can’t be debugged:

– I have no idea why this failed. It appeared perfect. Simply moving code that worked inside the deserialize (but crashed because it was run on “wrong” thread) out into the post-launc RunOnce method … silently fails. No errors, but no data.
– NB: this is impossible to debug, because we can’t even call ToString() on the deserialize method :(.

Conclusion

Give up. Make 10,000’s of Component instances instead. Waste memory, kill performance (especially at runtime: imagine how bad the AI’s going to be iterating over this!) but hey – IT WILL WORK. Because it leans on the built-in hacks that MonoBehaviour has had for years (which we’ve all been using millions of times a day without realising / appreciating ;)).

Debugging A* Pathfinding in Unity

I needed fast, efficient, … but powerful, adaptable … pathfinding for my hobby project. My minions have to fly, walk, crawl, teleport, tunnel their way across a procedural landscape. Getting to that point has been tricky, but the journey’s been interesting.

We want something that intelligently routes creatures around obstacles, and picks “shallow” routes instead of scaling cliffs (or vice versa for Giant Spiders etc). Something like this:

Screen Shot 2014-08-10 at 20.09.36

Continue reading

OpenGL ES 2 – Video on iPhone in 3D, and Texture-Map sources

This is a blog post I wrote 6 months ago, and my life got too busy to finish it off, finish testing the code, improving the text. So I’m publishing this now, warts and all. There are probably bugs. There are bits that I could explain better – but I don’t have time. Note that the code is ripped from apps where I ACTUALLY USED IT – so I know it does work, and works well!

In brief:

  1. Using different “sources” of textures in OpenGL, not just images!
  2. Using CALayer’s as a source — allows for video in 3D!
  3. Using “Video straight from the video decoding chip” as a source – faster video!

Continue reading

Review: good SMS apps for Android

Why do we need this?

In a bizarre move, Google deleted the perfectly good SMS app from Android. Yeah. Hmm. Well, read this TechRadar piece for some thoughts on that.

Sadly, Google’s corporate policy is that the consumer is always wrong: you’re not allowed to simply “use the app I already had, that worked great”. Time for a trip to the Android App Store…

I tried:

Quick review of each

TextSecure

  1. does what it’s supposed to: show all conversations, send texts
  2. uses the “standard” view of SMS that we’re all accustomed to

…Deal breaker

  1. there’s no widget. Without a widget, you can’t get your phone to show “at a glance” new and recent SMS texts on the home screen / dashboard

Evolve SMS

  1. has a widget that works

…Dealbreakers

  1. enormously unpleasant “forced swipe” control system. I’m not sure why. Maybe: “Facebook did it, and it sucked, so Facebook stopped. But we want to be Facebook, so let’s copy their failures!”
  2. the back button is actually broken. This is a core feature of Android, one of the biggest improvements compared to iPhone. And Evolve broke it. (Sad face).
  3. the widget can’t be resized. Fixed size widgets are a real pain – you can’t place them where you want, cant put the things you want on same screen, can’t control the info displayed

HelloSMS

  1. basically works

…Dealbreakers

  1. stupid forced-swipe control (see above)
  2. On first start, requires you to swipe in wrong direction
  3. no widget

Go SMS Pro

  1. has a widget!
  2. basically works, but ugly

…Dealbreakers

  1. The widget WILL NOT DISPLAY incoming/recent texts. Instead it displays ONE text, and you have to hit buttons to “scroll” the widget to next / previous text. Wat?
  2. The app hangs if it’s not your “default SMS app”: you type something, it hangs. No other app had this problem, they all did a popup explaining, including a single-tap “fix this” button.

chompSMS

  1. has a widget
  2. looks really nice

…Dealbreaker

  1. The widget … is simply “an ugly resized version of the app icon” … and it does nothing. Seriously, a developer chose to add this feature, and implemented … nothing. Wat?

Summary / Winners

TextSecure is a perfect recreation of ” an SMS app that just works, the way its supposed to” … Apart from the lack of widget (essential!). It even has Apple iPhone style “free text messages to other users of the app”. If they’d fix that missing widget, it would be worth paying for…

chompSMS is very much like TextSecure, perhaps slightly prettier (personal preference). But the broken widget is just as bad as TextSecure’s, maybe worse. And it lacks the “free SMS” mode and “secure SMS” modes of TextSecure.

Evolve has the only working widget (essential!). But … you’ll frequently exit the app unintentionally when you hit the back button. And it’s unusable unless you like “swipe all the time to control UI instead of using the back button” behaviour.

Final thoughts

The rest … are not winners. I’m using TextSecure for now. When people say “people on Android don’t buy stuff”, partly it’s because the apps are incomplete: most of these apps haven’t been fully developed (how can you have a broken widget? Why did you launch without a widget? etc).

NB: as a developer, I can attest that “adding a widget” is very easy. It’s stressful the first time, and then you read the docs and examples, and realise “wow! this is going to be super quick to implement!”.

World-exploration game, done differently

Inspired by Markus / @notch, and how he got MineCraft going in the early days, I’m live-prototyping a game idea I’ve been thinking about for ages. It’s about exploring a landscape … where cities affect dungeons, and dungeons affect cities.

(this is stress-relief for me, an “evenings and weekends” project. By day, I’m helping Schools teach children to program, but that’s HARD (no investors, lots of costs – at the moment, I get paid nothing))

Play in web browser here (very early prototype!)

Screen Shot 2014-07-25 at 12.31.23

July 2014

Current build focusses on the “City building” interface. Initially, you won’t be building cities, you’ll be building small camps / trading forts, etc. So this lets me play with the interface, the scale, the rendering, etc. Make sure it’s fun.

Create something – but take screenshots! Because there’s no “save” yet. Super-early prototype.

Long term gameplay

  1. Explore a huge landscape in FPS
  2. Any location, “create a camp”, and it switches to the isometric view you see in current demo. Build yourself a home/fort/etc.
  3. Fast-travel between locations, but this triggers “random encounter” events … or run between locations manually.
  4. Turn-based / DungeonMaster style dungeons are randomly generated near to each camp. Explore the dungeons for resources and to level-up.

Unlike e.g. Skyrim … when you find an interesting location, you can’t fast-travel to it. You have to build a camp next to it, and fast-travel to that. But your camp can be overrun/destroyed/stolen while you’re away.

So … you’ll be building (and maintaining) a lot of bases around the world. I want to be sure that base-building is easy and fun, even after you’ve done it lots of times already, before progressing with the rest!

(I’m trying to make it “repeatably fun” by having the landscape HUGELY affect your base design)

Webmail on your Debian server: exim4 + dovecot + roundcube

95% of linux configuration on Debian servers is simple, well-documented, well-designed, easy to do, with only a tiny bit of reading of docs.

Sadly, “making email work” is most of the 5% that’s: nearly impossible, very badly designed, badly packaged/documented. This OUGHT to take an hour or two, in practice it takes ONE WEEK to setup. WTF? In 2014? Unacceptable!

So I took several incomplete/broken guides, dozens of pages of help and advice, and synthesized this complete, step-by-step guide. This should get you the webmail you actually want (!) in an hour or less.

Continue reading

Your Pricing Models dictate your business success

Many businesses underestimate the power of a clever Pricing Model. I’m selling a new product at the moment (we’re helping schools teach children to program), so pricing models occupy a lot of my head right now.

Startups are so unsure of what model to use that they often say “anything, so long as we get sales”. They usually focus on simplicity (with a “new” product, a complex pricing model can tip people over the edge and make them “give up” on buying).

Which is fine, but a cunning pricing model can work wonders.
Continue reading

How to tidy your Unity code by putting it in a DLL

The importance of Re-use

Here’s a story that may sound familiar:

You downloaded Unity, watched some tutorials on YouTube, and had a physics “game” running the same day. Excited, you started writing the game you’ve always wanted to make. Possibly – bitten before when trying this with GameMaker etc – you thought to organize your source code: folders, subfolders, logically-named scripts, etc.

Fast-forward a few months, and you have 50 scripts in 40 sub-folders, some of it re-usable, some of it hardcoded for your game. Hoooo-Kayyyy… Well, it’s not too bad. And then a new opportunity comes along – a paid contract, a game-jam, a collaboration with a friend – and you want to re-use some code from your main project. You look at the folder structure, you look at how intertwined it’s all become … and you weep.

Without re-use, what seemed like an “easy” project becomes just a little too much work, and you never finish it. Back to square one :(.

Code re-use in Unity

There are five major aspects to code re-use when making Unity apps. Most of these are shared with general programming, but these are the ones I’ve found *especially* important when doing Unity development.

Good names

The best way to get good at naming things in a program is to focus on “what makes a bad name”, and “Don’t do that”. e.g.

  • If you have to open a Script to remember what it does … it has the wrong name.
  • If you keep typo’ing the script when referencing it in other scripts … it’s the wrong name
  • If your script is more than a thousand lines long … it’s the wrong name AND you’ve put too much junk in there; split it!
  • If you can’t find a script, because you can’t remember which folder it’s in … the folder name is wrong, AND the script name is wrong (should have been a name that forced you to put it in the right place first time!)
  • …etc

Renaming scripts in Unity is a minor pain: you rename it, then you have to close the file in MonoDevelop, re-open it, and (assuming it was a C# class) rename the class in source-code too. (I keep hoping latest Unity will do this automatically, but I’m a version behind at the moment).

We need Sub-folders. Lots of Sub-Folders

… you can never have too many. Give them good names, though!

De-couple your code

Ah, now it gets tricky. The art of decoupling takes much practice to master.

Decoupling is why C# (and Java) has the “interface” keyword. The concept is much easier to understand after you cocked it up, written code you can’t re-use, because everything’s too inter-dependent.

“I can’t just copy/paste that script to my new project, because it references 3 other scripts. Each of which … references another 12. ARGH!”

Unity itself doesn’t use Interfaces (though it really ought to!) – and you have to use a little ingenuity to use them yourself (Google it, it’s only a few small caveats). But C#-interfaces simply give you a compiler-compatible way of decoupling: you have to design your classes as decoupled to start with. Again, google this and ask around – it’s too big a topic for me to do justice to here!

Package your code, so you can re-use it

Libraries. This is why The God Of Programming invented Libraries. So, how do you do a Library in Unity?

Libraries in Unity: the easy DLL

There are two kinds of DLL’s:

  1. Real DLL’s, as used by Real Men, who write all their code in C++
  2. Fake DLL’s, as used by the rest of us, who just want an easy life so we can focus on making our game

A DLL is a great way of packaging code – it’s a very widely-used standard. So, Unity uses DLL’s for this (good move) – but if you google “Unity make DLL” you’ll get distracted by the huge complexity and depth of C++/Unity integration, and you don’t need any of it. Instead, you’ll be doing “DLL light”, which is easy.

But, as with most Unity features, it’s almost entirely undocumented. And, out of the box, it will fail. For bonus points, Unity will give you completely the wrong error message when it goes wrong. Yay!

Step 1: write a Unity script in Unity

Do something simple. I wrote a class that makes random numbers, following XKCD’s advice:

using UnityEngine;

/** In Unity 3, you cannot have namespaces. So comment out this next line. Unity 4 is fine */
namespace MyTestDLL
{
	public class DLLClass
	{
		public static int GetRandom()
		{
			return 4;
		}
	}
/** In Unity 3, you cannot have namespaces. So comment out this next line. Unity 4 is fine */
}

Make a second script that uses that first one, e.g.

using UnityEngine;

public class TestScript : MonoBehaviour
{
	void Start()
	{
		print( DLLClass.GetRandom() );
	}
}

…attach it to a GameObject, check it works.

Step 2: follow Unity’s docs on creating and using a DLL

You’d think this would be enough, but it isn’t. However, the docs are simple, direct, and I found them easy to follow.

So http://docs.unity3d.com/Documentation/Manual/UsingDLL.html = read this and do what it says

NB: they write a slightly more complex script to use than mine, with but whatever. The idea is the same.

Step 3: Use the namespace, Luke. And Interfaces. And …

Now that your DLL is compiling/building in MonoDevelop … you are no longer stuck with Unity’s arbitrary rules and restrictions! The world is yours!

(so, even in Unity 3, you can now use the namespace. Which makes it MUCH easier to keep your code well-organized ;))

Step 4: What about the [square brackets]?

These Just Work ™ exactly as they did in Unity scripts. e.g.

using UnityEngine;

namespace MyTestDLL
{

/** Hey! Look! Unity editor-features ... coded outside of Unity */
[ExecuteInEditMode]
[System.Serializable]
	public class DLLClass
	{
...

Step 5: Editor extensions, editor scripts, editor GUIs, etc

This is where it breaks. If you had any Editor scripts (which, in Unity, MUST be stored in the “Editor” root folder, or one of its subfolders), you can include them in your DLL – but they won’t work.

Worse, when you trigger them (e.g. by selecting a GameObject that did custom editor rendering), you’ll get:

Error: multi-object editing not supported !

…at least, you do in Unity 3.x. Fixed in 4.x, I suspect? But I haven’t gone back to try it since I fixed the bug :).

There is some mumbo-jumbo on the internet (and Unity forums) about complicated workarounds, notably courtesy of AngryAnt (with a Jan 2014 post here, with some extra info worth reading but NOT required!). But these are long-since outdated and unnecessary. The fix is very simple: we must create TWO DLL’s!

  1. DLL #1: the one we already had
  2. DLL #2: will contain ONLY the editor-scripts, and we’ll drag/drop it into Unity’s “Editor” folder.

This is extremely logical, simple, and easy to remember. And it works beautifully! But .. how?

Step 5a: Upgrade MonoDevelop project to output not one, but two, DLL’s

First, right-click on the top-most item in MonoDevelop’s “Solution” panel, and select “Add New Project…”:

Screen Shot 2014-05-24 at 20.54.53

As you did earlier, choose “Library (C#)”. I recommend naming this “SomethingSomethingEditorScripts” (where “SomethingSomething” is your library name, which you used for the first DLL).

Second, you need to edit the References (as you did with first DLL), and this time repeat the steps and add ALL of:

  • UnityEngine.dll
  • UnityEditor.dll
  • AND: instead of the “.Net Assembly” tab, use the “Projects” tab and select your main DLL/library. It should be the only option

That gives you something like this:

Screen Shot 2014-05-24 at 20.58.22

…see how I have two “projects” in MonoDevelop, one which just uses normal Unity stuff, the other which additionally uses Unity Editor stuff? And the second one “references” the first, so it can see/use/create the classes and methods from the first one.

Step 5b: Put the DLL’s in different places

Build, and find the output files. The first project/DLL will still only be making one DLL, but the folder for the SECOND project/DLL will conveniently build BOTH projects and contain BOTH DLL’s.

Make sure you put one DLL “anywhere except Editor” and the other one “in Editor, or a subfolder of Editor”.

Finally, create some GameObject’s, open up the DLL’s in Unity’s Project view (expand the triangle) and drag/drop the Scripts onto your objects as desired. Click on them in the Scene view, and all your OnGUI, OnGizmos etc methods should run exactly as normal.

Step 6: Rejoice!

Now you can share your code with other Unity projects simply by drag/dropping the 2 x DLL’s into a new Unity project. BOOM!

Beautiful! Easy, simple, impossible-to-screw-up ;). That’s how I like my code re-use…

Should freelancers in gamedev industry sign NDA’s?

I’m sure I’ve written about this before, but it’s come up again on Reddit, so here’s my thoughts…

IANAL, but … I have been offered more than 100 NDA’s, both as an individual and working in large companies (including a games publisher, and including a funding team).

I have signed fewer than 30 – and most of those were before I realised how “wrong” it is to sign an NDA.

These days, I only sign 1 in every 10 NDA’s sent my way – and yet I still end up working for, or doing business with, at least 50% of the people who initially “required” an NDA.

Often, people send me a bad, broken, poorly worded, typo-riddled NDA and I say “I’m not signing that. Fix it, get it down to 1.5 pages max, and I’ll reconsider” — and they usually don’t bother, they just tell me their “secrets” anyway. They didn’t really need or want the NDA!

Also: In most jurisdictions, an NDA can’t realistically be more than 2 pages, maybe 3 if it’s badly worded … if it’s more than 2 pages, you’re either in a niche industry (not games), or … it’s not an NDA, it’s a contract that’s pretending to be an NDA, perhaps for evil reasons. When someone sends me a 5-10 page NDA I say “send me a 1-2 page NDA. Don’t send me a secret employment contract”. AGain, they often simply carry on talking to me without NDA.

One large games company (100 staff) sent me (IIRC) a TWENTY TWO page NDA. Buried on page 17 was something like: “we own all converations with you, and at YOUR cost we can confiscate all hard-disks and storage belonging to you or your company if we have any suspicion you might be developing ideas similar to our own”.

(It specifically took ownership of any ideas similar – even if we’d never discussed them. It was disgusting.)

There are some exceptions to be aware of – but the company can easily explain + justify this to you. For instance, if the company is based on a patent, then in Europe they may be required to prove everyone is under NDA or risk invalidating the patent (not the same as USA, where you can talk about patents publically, IIRC).

TL;DR – many “NDA’s” are a scam. Refuse them without hesitation. Or question them, and discover the company didn’t really need it anyway. Most “decent” companies won’t care that you’re not under NDA; the obsession with NDA’s usually comes from the kind of dumb-ass employer that is a PITA to work for.

(to reiterate: IANAL. I’m happy giving advice, but legally you shouldn’t trust random strangers on the internet, and I take no responsibility for your actions ! :))

Great use of UX: too-close signs on cars

5 years as an iOS developer has pushed me to understand and improve my product/physical/functional design skills. Especially “industrial” design – problems and solutions.

Here’s a great one I saw the other day: combining a Snellen Chart with Stopping Distances, i.e.:

Screen Shot 2014-04-26 at 15.24.14 + dg_070645

Existing (bad) solutions

The status quo on driving “too close for safety” is provocative, antagonistic. Have a look at a search for eBay: “too close bumper sticker”:

Screen Shot 2014-04-26 at 15.57.35

There’s a couple of major problems with this:

  1. It’s wrong. If you’re stationary, 6 feet behind the vehicle, you can read it – but you’re definitely NOT too close.
  2. It’s combative and offensive. It appeals to the anger and frustration of the driver in front – when we’re trying to modify the behaviour of the driver behind. Offending people who you’re trying to convince to change … is generally unsuccessful.

But combining the charts above, we get…

Net result

A plate you attach to your vehicle that informs the viewer how much too close/far they are right now, using a dynamic scale:

Screen Shot 2014-04-26 at 15.26.36

I couldn’t get a close enough photo to show it clearly, so here’s my redrawing of what the plate looked like:

too-close-am-vector

Why you shouldn’t use webfonts instead of images

Warning: principled rant against sloppy design and bad coding about to start; kids these days! Get off my lawn!

There’s a terrible disease affecting modern web developers – Twitter.com has just fallen ill with it, and it could be a long time before they cure themselves.

Deleting all images, and replacing them with the dreaded “web font”.

It’s the wrong solution for the problem, it doesn’t do what you think it does, and it pisses all over some of the Web’s core principles. If you’re a web developer, and you respect your craft, you shouldn’t even consider this, not for one moment. Here, for instance, is how Twitter currently looks for me – ugly, and very hard to use!

Screen Shot 2014-04-15 at 10.18.17

GitHub was another recent victim of this disease. They’re still in recovery, but they at least made their site “slightly usable” by adding tooltips:

Screen Shot 2014-04-15 at 10.30.18

What’s going on?

The problem

Core Web principle:

Information is everything; presentation is optional. It’s acceptable to forego presentation, so long as everyone can access the information.

Effect: When you load a webpage, your browser requests every image separately. This is a long way short of the “most optimal” code implementation.

Is this a big problem? For most of us … Not really. The system works, it’s flexible, it’s powerful – it’s a little inefficient, but for the corporations that care there are plenty of hacks and optimizations they can deploy.

The other problem

Most artists should be creating Vector images most of the time, but the software vendors who made the editing tools for Vectors … all died out around 15 years ago. Back then, the advantages of Vectors were small, because most screens were low res.

We still haven’t recovered. There are many standards for bitmap image files, and two very popular ones – PNG, JPG. There are many standards for vector image files, but no popular ones.

Effect: web developers end up looking to Web Fonts as a de facto “vector image” standard.

SVG, or not to SVG?

There is an official Web/HTML approved vector standard – SVG – in wide use, with strong support in all current browsers.

Does it work? Let’s see (http://caniuse.com/svg) …

Screen Shot 2014-04-15 at 10.45.46
Screen Shot 2014-04-15 at 10.45.33

… but many software companies ignore it. For instance, Apple allows programmers to use both PNG and JPG (why?) in core iOS, but not SVG (despite having a full SVG parser built-in to their web browser). Many programmers I speak to believe that SVG isn’t supported at all – FUD wins again.

The other, other problem

A core principle of the Web is that information is accurately described (the M in HTML). A recent trend in web development is “progressive enhancement”.

def. “Progressive Enhancement HTML”: a webpage written the way you were supposed to write it, instead of being hacked-together by unskilled monkeys

HTML has always been “progressive” – this was a core principle 20 years ago. But HTML was so easy to use and abuse that many of us (most of us? nearly all of us?) have been writing poor HTML most of that time. Shame on us (shame on me, certainly – been there, done that :( ).

But … the key point here is: Progressive Enhancement isn’t an “optional extra”, it is the Web. If you fight PE, you’re fighting the entire web infrastructure – and we know how that war will end.

…whatever. What about Web fonts?

So, when you remove an “image” and put a “web font letter” there instead, and change that letter so that the font contains an image you wanted …

…your HTML is now a lie.

Maybe you’re the kind of web developer who scorns blind people (and partially blind), who ignores the Internationalization features of software. You laugh in the face of Accessibility Standards, so you can reduce development time.

But HTML doesn’t make these things optional. They are so core to HTML that they are “always on”, even if you personally never use (need) them. One of the beautiful features of HTML is that if you do nothing, most of the Accessibility is automatically done for you.

With HTML, you have to go out of your way to prevent Accessibility. For instance: replacing images with magic-letters from a magic font.

You’re not blind; why does the Web Font fail?

The thing about custom Web Fonts is … the user can disable them.

Again, this is fundamental to the web. Partly for the Accessibility issue (who are you to decide which users require Accessibility? No. It’s for the user to decide).

But also to support the web principles of openness, and user-control (not corporation-control). My machine, my browser, my choice.

Just as you cannot prevent users from hitting the “View -> Zoom” menu option and making your web page take more or fewer pixels on their screen (I’ve worked with web designers – mostly ex-print designers – who HATED this, and felt it was a feature that should be banned) … you cannot force a crappy font on the user.

Information is everything; presentation is optional. It’s acceptable to forego presentation, so long as everyone can access the information.

In my case: I have a MacBook Air. While wonderful in many ways, they have tiny (11″), non-retina screens, and they’re laptops – so the screen is often further away than I’d like. When WebFonts came to CSS, a lot of the websites I use (art sites, design agencies) started using “beautiful but TINY” fonts that were unreadable. Game Studios still do this today, sadly – lots of hard-to-read but “edgy!” fonts and bad colour choices.

Sometimes, the only way I can do my day to day work is to disable the crappy 3rd party fonts.

Another solution?

SVG says “Hi!”. Think about it.

Debugging odd problems when writing game servers

Cas (@PuppyGames) (of PuppyGames fame – Titan Attacks, Revenge of the Titans, etc) is working on an awesome new MMO RTS … thing.

He had a weird networking problem, and was looking for suggestions on possible causes. I used to do a lot of MMO dev, so we ran through the “typical” problems. This isn’t exhaustive, but as a quick-n-dirty checklist, I figured it could help some other indie gamedevs doing multiplayer/network code.

Scenario

  1. Java, Windows 7
  2. DataInputStream.readFully() from a socket… a local socket at that… taking 4.5 seconds to read the first bytes
  3. and I’ve already read a few bits and bobs out of it

Initial thoughts

First thought: in networking, everything has a realm of durations. The speed of light means that a network connection from New York to London takes a few milliseconds (unavoidably) – everything is measured in ms, unless you’re on a local machine, in which case: the OS internally works in microseconds (almost too small to measure). Since Cas’s problem is measured in SECONDS – but is on localhost – it’s probably something external to the OS, external to the networking itself.

Gut feel would be something like Nagle’s algorithm, although I’m sure everyone’s already configured that appropriately ;).

That, or a routing table that’s changing dynamically – e.g. first packet is triggering a “let me start the dialup connection for you”, causing everything to pause (note that “4 seconds” is in the realm of time it takes for modems to connect; it’s outside the realm of ethernet delays, as already noted)

My general advice for “bizarre server-networking bugs”: start doing the things you know are “wrong” from a performance view, and prove that each makes it worse. If one does not, that de facto that one is somehow not configured as intended.

Common causes / things to check

1. Contention: waht’s the CPU + C libraries doing in background? If system has any significant (but small) load, you could be contended on I/O, CPU mgiht not be interrupting to shunt data from hardware up to software (kernel), up to software (OS user), up to JVM

Classic example: MySQL DB running on a system can block I/O even with small CPU load

2. file-level locking in OS FS. Check using lsof which files your JVM is accessing, and what other processes in the machine may have same files open.

(NB: there’s a bunch of tools like “lsof” which let you see which files are in use by which processes in a unix/linux system. As a network programmer, you should learn them all – they can save a lot of time in development. As a network programmer, you need to be a competent SysAdmin!)

Classic problem: something unexpected has an outstanding read (or a pre-emptive write lock) which is causing silliness. I remember several times having a text editor open, or etc, that was inadvertently slowing access to local files (doh).

3. can you snoop the traffic? (i.e. its socket, not pipe)

Classic problem: some unrelated service is bombarding the loopback with crap. eg. Windows networking (samba on linux) going nuts

Try running Wireshark, filtered to show only 127. and see what’s going through at same time

Also: this is a good way to check where the delay is happening. Is it at point of send, or point of receive? … both?

There’s “net traffic” and there’s “net traffic”. Wireshark often shows traffic that OS normally filters out from monitoring apps…

4. Check your routing table? AND: Check it’s not changing before / aftter the attempted read?

5. Try enabling Nagle, see if it has any effect?

My point is: use this as a check: it ought to make things worse. If not … perhaps the disabling wasn’t working?

6. Have you done ANY traffic shaping (or firewalling) on this machine at any time in the past?

Linux: in particular, check the iptables output. Might be an old iptables rule still stuck in there – or even a firewall rule.

Linux + Windows: disable all firewalls, completely.

7. Similarly: do you have any Anti-Virus software?

Disconnect your test machines from the internet, and remove all AV.

AV software can even corrupt your files when they mistakenly think that the binary files used by the compiler/linker are “suspicious” (IIRC, that happened to us with early PlayStation3 devkits).

8. On a related note: security / IPS tools installed? They will often insert artificial delays silently.

CHECK YOUR SYSTEM LOG FILES! …whatever is causing the delay is quite possibly reporting at least a “warning” of some kind.

9. (in this case, Cas’s socket is using SSL): Perhaps something is looking up a certificate remotely over the web?

…checked your certificate chain? (if there’s an unknown / suspicious cert in the chain, your OS might be trying to check / resolve it before it allows the connection)

10. (in this case, Cas is using custom SSL code in Java to “hack” it): Get a real SSL cert from somewhere, see if it behaves any different

(I think it would be very reasonable for your OS to detect any odd SSL stuff and delay it, as part of anti-malware / virus protection!)

11. “Is it cos I is custom?” – Setup an off-the-shelf webserver on the same machine and check if YOUR READING CODE can read from that SSL localhost fast?

(it often helps to get an answer to “is it the writer, the reader, both … is it the hacked SSL, or all SSL?” etc)

12. (getting specific now, as the general ideas didn’t help): if you bind to the local IP address instead of 127 ?

13. Howabout reverse-DNS? (again, 4 seconds is in the realm of a failed DNS lookup)

Might be a reverse DNS issue … e.g. something in OS (kernel or virus protection), or in JVM SSL library, that’s trying to reverse-lookup the 127 address in order to resolve some aspect of SSL.

I’ve had to put fake entries into hosts file in the past to speed-up this stuff, some libs just need a quick answer

Result

Turns out we were able to stop here…

Cas: “Well, whaddya know. It was a hosts lookup after all – because I’m using 127.0.0.2 and 127.0.0.3 it was doing a hostname lookup. I added those to hosts and all is solved

Me: “(dance)” (yeah. It was skype. The dance smiley comes in handy)

Unconference on Entity Systems – Pre-Signup

Entity Systems are widely used in gamedev and starting to appear in mainstream IT / software development. But we want MORE developers and designers to benefit from this…

So, Richard Lord and I are going to do a mini conference on ES ideas/designs/uses/implementation/etc.

I’ll arrange venue, agenda (e.g. a Keynote) – but this will be an Unconference, so most/all sessions will be interactive, freeform, with no fixed Speakers.

To book a venue, I need an idea of numbers. So, if you’d like to come, please fill out this google form. Note: I’m planning to do this in Brighton, (UK) – less than 30 minutes from an International Airport (Gatwick).

https://docs.google.com/forms/d/156Sb9Qhx4RIX1d0BZg_22MVlUSoUb-gRqwrOq0zeT-w/viewform

Key info:

  • Date: Summer/Autumn 2014
  • Location: Brighton, UK
  • Cost: minimal (depends on demand + venue)