Day: March 28, 2015

Categories
programming Unity3D

#Unity3d’s missing core: No runtime objects

Here’s a game:

  • Player
    • Inventory
      • Gun 1
        • AmmoHolder
      • Gun 2
        • AmmoHolder

Some of those are 3D visual objects: Player, Gun 1, Gun 2.

Some of those are abstract concepts, or data: Inventory, AmmoHolder.

(in a Diablo clone, the Inventory might be a visual object, but only when you bring up the popup-window to display it. By contrast, in a simple FPS, your inventory may not be displayed at all – no need for it!)

Unity cannot handle this situation.

What you need

Classes:

  • PlayerClass
  • InventoryClass
  • GunClass
  • AmmoHolderClass

In Unity, you have to make “everything” extend MonoBehaviour. That’s fine for PlayerClass and GunClass: you make a dummy GameObject for each, place it into the scene, and attach the class as a Component.

But that’s impossible for InventoryClass and AmmoHolderClass: if you make them into MonoBehaviours, Unity prevents you from using them as standard classes any more:

  1. Cannot instantiate them
  2. Cannot use Garbage Collection

…because they have no GameObject to be attached to – they don’t exist! They are abstractions only.

This is core to programming: we use abstractions all the time. Without them, coding would be horrendously inefficient both for the programmer and (to a lesser extent) for the computer.

What Unity offers

You have three alternatives here:

  1. Create fake GameObjects anyway. Put them “somewhere”. Work really hard to make sure they never accidentally appear in front of the camera (!), or interact with the scene due to physics (!), etc
  2. Use plain C# classes. This is an extremely bad idea: Unity has never supported C# classes for anything; Unity has a custom memory-management solution (called, slightly confusingly, “Serialization”) which is incompatible with plain C#. It requires every object to extend Unity.Object. You can workaround this; it requires major work and is very hard to get right.
  3. Use ScriptableObject, which appears to have been invented for this. According to many Unity developers: “this is what ScriptableObject is for”. However, that’s wrong. Very wrong.

ScriptableObject to the rescue?

So … ScriptableObject is something that extends UnityObject (fixing the problem of “plain C# classes aren’t compatible with Unity”), and you can use in a Scene, and can be referenced by MonoBehaviours, and just work, right?

Wrong.

ScriptableObject is all those things, except: it doesn’t exist in the Scene. It’s an Asset.

This is where Unity docs really screw the pooch. We know for a fact that an SO is an Asset; when we try to use it like a MonoBehaviour, several things break and stop working.

…but it also “kind of” works in-Scene. The docs have no explanation for this. What are the limits? Well, basically: no-one knows, except through trial and error. This is so core to your code that it sometimes instead means:

trial and oh-crap-that-didn’t-work-time-to-rewrite-every-damn-class

Why is Asset-hood so bad?

You already have all that – it’s called the Class – and to have an Asset too simply creates problems:

  • An Asset is a special thing in Unity. It interacts with Unity’s core systems in very different ways to a C# object. Building, runtime Loading, editing in the Editor – all are “different”.
  • You can’t create them in the Editor. (you can write your own hack/script to fix this. It’s only 10 lines of code. But … yeah. It’s a hint that this isn’t what Assets are intended for)

But .. what’s ScriptableObject for?

Unity’s official docs are rather … incoherent … on the subject. My guess is that they were written by someone who didn’t really understand ScriptableObject, and was looking at the source code thinking:

WTF is this? Um. Err. Well … it kind-of seems to sort-of do … this? Maybe?

…and worked it out from observation.

What the docs are TRYING to say is:

(gross over-simplification here to make things easy to understand): Unity’s memory-management COPIES every instance many times, and doesn’t support pointers/references.

This is a HUGE problem in game-development.

ScriptableObject is a workaround that uses diskspace to make templates out of objects, and then simulates pointers by using one shared-file on disk to hold the data for thousands of in-game objects.

This works very well, because the way we built Unity’s core architecture (3D Meshes, Textures etc) already supports that workflow: File-on-disk generates optimized in-memory-object, which is shared/batched wherever possible.

Going off on a tangent, it waffles about “ShopStore” and “Multiple Scenes” and stuff that the author clearly didn’t really understand. The obvious way of implementing what’s described in the docs is many times simpler, and works much better – you would never do it using ScriptableObject.

What the docs perhaps should have instead said was:

You can also abuse this system to make hardcoded fake in-scene objects that don’t have to be attached to a GameObject.

This will fail if you have any Procedural Content – but Unity was specifically designed never to be used in games that have procedural generation, and no-one should be doing procedural work in Unity, so that’s fine.

Wait … what? What do you mean “we all do procedural generation”? Wh … why? Why would you do such a thing? do you have any idea how slow that will be in Unity? Most of our code assumes everything is done at compile time! We can’t handle dynamic code!!!

(NB: Unity used to be really, really bad at procedural work. Way too slow. There’s really no reason not to be doing tonnes of procedural code in Unity games today – except that Unity itself still has a load of political blocks on it. It doesn’t support procedural properly in places where it could)

ScriptableObject is great, but most game-development works with procedural content. I’m not talking about complex cool stuff like Sir, you are being hunted. I’m talking about stuff like:

The set of objects in your inventory changes from moment to moment while you play the game. Because otherwise it would be a pretty boring game

When you try to use ScriptableOject here, you are abusing it. It’s not intended for that, it doesn’t work correctly with that, and it’ll make your life hard. IT’s not ScriptableObject’s fault; it’s your fault.

What does Unity need?

At a conceptual level, Unity has:

  • “Physical Runtime objects”: things that have 3D physical presence in the editro (not Physics as in “Physics Engine”, but rather: everything has a Transform)
  • “Runtime OOP Objects, dependent upon GameObject”: generated from OOP Classes (but MUST be attached to a Physical Runtime object)
  • “Compile time OOP Objects”: these are Unity Assets, and can be used for some performance and coding tricks. They are they poorly-documented ScriptableObject, and should probably have been named ScriptableAsset in the first place since they are absolutely not Objects!

What’s missing?

  • “Runtime OOP Objects, independent of GameObject: the classic “Object” from OOP.

I think … not sure, this seems obvious, but if so: why is not already the case? … I think: Unity should be saying “make your classes extend ‘Unity.Object’, and then use them. All will be fine. We’ve made it work under-the-hood; trust us, guys, it’s cool. Everything’s cool.”.

My guess is that’s some internal bug with Unity Serialization that makes this untractable. Maybe something like:

GUESS: Unity Seriailization is internally hardcoded with a fixed number of recognized classes, somewhere, that can only be added to by hardcoding new ones. Because we don’t have access to Unity’s source code, we cannot re-generate those, and the Unity Editor (for same reason) can’t do it for us.

So: our legitimate C# classes can never be fully treated as a Unity Class.

Instead, Unity has to support a fixed number of hardcoded classes (O hai! ScriptableObject, MonoBehaviour), and those have to use reflection tricks at runtime to provide (limited) support for the infinitely many user-authored clases from C#.

OK, so this is slightly more than a “guess”. They have form here: When trying to figure out some obscure Unity Serialization bugs, I found some places where Unity would load extra, hidden, information for particular Unity classes. There was nothing in memory on this, suggesting that they had hardcoded a lookup for their set of “known” Unity classes, and were getting the data that way.

…but that’s the kind of thing that’s going to be a genuine mind-f*ck to try and unravel. This makes me inclined towards it: even with the money and resource Unity has, it would be non-trivial to fix. And I’m pretty sure no-one likes the status quo.

Conclusions

We’ll just keep writing source code like it’s 1999:

  • Use C# as if it’s C
  • F*ck Garbage Collection (we can’t have it :( Sob.)
  • Manually memory-manage the scene and objects; make everything a GameObject, and manually Destroy() everything each time you replace it or it goes out of scope
  • …
  • As a bonus: the Unity Editor has much better support for Developing and Debugging games which use GameObjects than it does for any alternatives (C# Classes; ScriptableObjects)

I’m crossing my fingers that Unity version 6 will dump Serialization, and this problem will go away. Like magic, Unity will fully support C# (Dictionary’s! Objects! Garbage Collection! YAY!)

…of course, by then, I might have finally switched to Unreal. Because frankly I’ve got better things to do with my life than work in low-level programming languages!