#Unity3d’s missing core: No runtime objects

Here’s a game:

  • Player
    • Inventory
      • Gun 1
        • AmmoHolder
      • Gun 2
        • AmmoHolder

Some of those are 3D visual objects: Player, Gun 1, Gun 2.

Some of those are abstract concepts, or data: Inventory, AmmoHolder.

(in a Diablo clone, the Inventory might be a visual object, but only when you bring up the popup-window to display it. By contrast, in a simple FPS, your inventory may not be displayed at all – no need for it!)

Unity cannot handle this situation.

What you need

Classes:

  • PlayerClass
  • InventoryClass
  • GunClass
  • AmmoHolderClass

In Unity, you have to make “everything” extend MonoBehaviour. That’s fine for PlayerClass and GunClass: you make a dummy GameObject for each, place it into the scene, and attach the class as a Component.

But that’s impossible for InventoryClass and AmmoHolderClass: if you make them into MonoBehaviours, Unity prevents you from using them as standard classes any more:

  1. Cannot instantiate them
  2. Cannot use Garbage Collection

…because they have no GameObject to be attached to – they don’t exist! They are abstractions only.

This is core to programming: we use abstractions all the time. Without them, coding would be horrendously inefficient both for the programmer and (to a lesser extent) for the computer.

What Unity offers

You have three alternatives here:

  1. Create fake GameObjects anyway. Put them “somewhere”. Work really hard to make sure they never accidentally appear in front of the camera (!), or interact with the scene due to physics (!), etc
  2. Use plain C# classes. This is an extremely bad idea: Unity has never supported C# classes for anything; Unity has a custom memory-management solution (called, slightly confusingly, “Serialization”) which is incompatible with plain C#. It requires every object to extend Unity.Object. You can workaround this; it requires major work and is very hard to get right.
  3. Use ScriptableObject, which appears to have been invented for this. According to many Unity developers: “this is what ScriptableObject is for”. However, that’s wrong. Very wrong.

ScriptableObject to the rescue?

So … ScriptableObject is something that extends UnityObject (fixing the problem of “plain C# classes aren’t compatible with Unity”), and you can use in a Scene, and can be referenced by MonoBehaviours, and just work, right?

Wrong.

ScriptableObject is all those things, except: it doesn’t exist in the Scene. It’s an Asset.

This is where Unity docs really screw the pooch. We know for a fact that an SO is an Asset; when we try to use it like a MonoBehaviour, several things break and stop working.

…but it also “kind of” works in-Scene. The docs have no explanation for this. What are the limits? Well, basically: no-one knows, except through trial and error. This is so core to your code that it sometimes instead means:

trial and oh-crap-that-didn’t-work-time-to-rewrite-every-damn-class

Why is Asset-hood so bad?

You already have all that – it’s called the Class – and to have an Asset too simply creates problems:

  • An Asset is a special thing in Unity. It interacts with Unity’s core systems in very different ways to a C# object. Building, runtime Loading, editing in the Editor – all are “different”.
  • You can’t create them in the Editor. (you can write your own hack/script to fix this. It’s only 10 lines of code. But … yeah. It’s a hint that this isn’t what Assets are intended for)

But .. what’s ScriptableObject for?

Unity’s official docs are rather … incoherent … on the subject. My guess is that they were written by someone who didn’t really understand ScriptableObject, and was looking at the source code thinking:

WTF is this? Um. Err. Well … it kind-of seems to sort-of do … this? Maybe?

…and worked it out from observation.

What the docs are TRYING to say is:

(gross over-simplification here to make things easy to understand): Unity’s memory-management COPIES every instance many times, and doesn’t support pointers/references.

This is a HUGE problem in game-development.

ScriptableObject is a workaround that uses diskspace to make templates out of objects, and then simulates pointers by using one shared-file on disk to hold the data for thousands of in-game objects.

This works very well, because the way we built Unity’s core architecture (3D Meshes, Textures etc) already supports that workflow: File-on-disk generates optimized in-memory-object, which is shared/batched wherever possible.

Going off on a tangent, it waffles about “ShopStore” and “Multiple Scenes” and stuff that the author clearly didn’t really understand. The obvious way of implementing what’s described in the docs is many times simpler, and works much better – you would never do it using ScriptableObject.

What the docs perhaps should have instead said was:

You can also abuse this system to make hardcoded fake in-scene objects that don’t have to be attached to a GameObject.

This will fail if you have any Procedural Content – but Unity was specifically designed never to be used in games that have procedural generation, and no-one should be doing procedural work in Unity, so that’s fine.

Wait … what? What do you mean “we all do procedural generation”? Wh … why? Why would you do such a thing? do you have any idea how slow that will be in Unity? Most of our code assumes everything is done at compile time! We can’t handle dynamic code!!!

(NB: Unity used to be really, really bad at procedural work. Way too slow. There’s really no reason not to be doing tonnes of procedural code in Unity games today – except that Unity itself still has a load of political blocks on it. It doesn’t support procedural properly in places where it could)

ScriptableObject is great, but most game-development works with procedural content. I’m not talking about complex cool stuff like Sir, you are being hunted. I’m talking about stuff like:

The set of objects in your inventory changes from moment to moment while you play the game. Because otherwise it would be a pretty boring game

When you try to use ScriptableOject here, you are abusing it. It’s not intended for that, it doesn’t work correctly with that, and it’ll make your life hard. IT’s not ScriptableObject’s fault; it’s your fault.

What does Unity need?

At a conceptual level, Unity has:

  • “Physical Runtime objects”: things that have 3D physical presence in the editro (not Physics as in “Physics Engine”, but rather: everything has a Transform)
  • “Runtime OOP Objects, dependent upon GameObject”: generated from OOP Classes (but MUST be attached to a Physical Runtime object)
  • “Compile time OOP Objects”: these are Unity Assets, and can be used for some performance and coding tricks. They are they poorly-documented ScriptableObject, and should probably have been named ScriptableAsset in the first place since they are absolutely not Objects!

What’s missing?

  • “Runtime OOP Objects, independent of GameObject: the classic “Object” from OOP.

I think … not sure, this seems obvious, but if so: why is not already the case? … I think: Unity should be saying “make your classes extend ‘Unity.Object’, and then use them. All will be fine. We’ve made it work under-the-hood; trust us, guys, it’s cool. Everything’s cool.”.

My guess is that’s some internal bug with Unity Serialization that makes this untractable. Maybe something like:

GUESS: Unity Seriailization is internally hardcoded with a fixed number of recognized classes, somewhere, that can only be added to by hardcoding new ones. Because we don’t have access to Unity’s source code, we cannot re-generate those, and the Unity Editor (for same reason) can’t do it for us.

So: our legitimate C# classes can never be fully treated as a Unity Class.

Instead, Unity has to support a fixed number of hardcoded classes (O hai! ScriptableObject, MonoBehaviour), and those have to use reflection tricks at runtime to provide (limited) support for the infinitely many user-authored clases from C#.

OK, so this is slightly more than a “guess”. They have form here: When trying to figure out some obscure Unity Serialization bugs, I found some places where Unity would load extra, hidden, information for particular Unity classes. There was nothing in memory on this, suggesting that they had hardcoded a lookup for their set of “known” Unity classes, and were getting the data that way.

…but that’s the kind of thing that’s going to be a genuine mind-f*ck to try and unravel. This makes me inclined towards it: even with the money and resource Unity has, it would be non-trivial to fix. And I’m pretty sure no-one likes the status quo.

Conclusions

We’ll just keep writing source code like it’s 1999:

  • Use C# as if it’s C
  • F*ck Garbage Collection (we can’t have it :( Sob.)
  • Manually memory-manage the scene and objects; make everything a GameObject, and manually Destroy() everything each time you replace it or it goes out of scope
  • As a bonus: the Unity Editor has much better support for Developing and Debugging games which use GameObjects than it does for any alternatives (C# Classes; ScriptableObjects)

I’m crossing my fingers that Unity version 6 will dump Serialization, and this problem will go away. Like magic, Unity will fully support C# (Dictionary’s! Objects! Garbage Collection! YAY!)

…of course, by then, I might have finally switched to Unreal. Because frankly I’ve got better things to do with my life than work in low-level programming languages!

13 thoughts on “#Unity3d’s missing core: No runtime objects

  1. Jesse Flanagan

    I use plain-old C# classes all the time in Unity and have been since version 3. You say it’s an extremely bad idea but gloss over why.

  2. adam Post author

    By definition, they corrupt your project, e.g.

    – Whenever you hit Play.

    – Whenever you Save the scene.

    – Whenever you click in certain places in the Editor.

    …and a load of other times/ways. Unity’s core absolutely cannot work correctly or reliably if you have plain classes floating around the system.

  3. Jesse Flanagan

    I guess if by “corrupt” you mean “not saved”, then sure. But for things that I create at runtime to encapsulate gameplay logic they work just fine in all the games I’ve shipped.

  4. adam Post author

    You can’t edit them (Inspector will corrupt the data, randomly).

    You can’t save any kind of in-game progress that ever – anywhere! – accidentally references any of them.

    etc.

    These aren’t minor issues :)

  5. Sampsa Lehtonen

    I’m regularly using plain C# classes with Unity (with [System.Serializable] for persistent and without for runtime stuff) and I haven’t had any problems with Unity randomly corrupting data. Persistent here being something that gets saved into a scene or a ScriptableObject, not savegame kind of persistence.

    If you have the Serializable attribute, you will also get to edit the data in inspector – you already know this. And because the serialization works so that each component is serialized as whole (except for references to other Unity.Objects), if you happen to reference the same class from other component’s public field, you get duplication which is seldom something you want. Is this the corruption you’re talking about?

    For the savegame persistence Unity doesn’t have anything built-in, I agree. I haven’t had problems with that so far — saving a whole scene would not usually be practical and I haven’t needed that. And whipping up a simple JSON/whatever serializer has been the solution for me. I’m not sure how Unity could solve this one without being either too limited or have some odd restrictions.

    Regarding ScriptableObjects… I’m not sure if you’ve understood them correctly. SO’s are something you _can_ save to an asset file if you want to, but SO’s are happy to reside in memory/scenes too, and if they’re not based on an asset file, they actually get saved in the scene file as if they were part of the scene. Accessing the SO’s in the scene is not trivial, at all (there’s no list view for them), but they do offer you this way to have classes that are not bound to a physical GameObject. If you want to know how to do this, ask. I’ve tried it out :) It might be just what you need.

    To be honest, I haven’t needed these scene-only ScriptableObjects at all in the past – I’ve only used them for storing data in the asset database that can be shared between scenes etc. Like GameDatabase, which might contain all the prefab names for all of my enemy types or list of scene files etc.

    BTW, saying that Unity’s serialization is same as memory management is a misnomer. It’s used for persistence, only certain fields get serialized, anything else doesn’t. And because of this, GC will then happily clean whatever objects were left dangling after the serialization. Serialization doesn’t destroy anything, it just tells if something should be saved for the particular object being serialized.

  6. Adam

    “Accessing the SO’s in the scene is not trivial, at all (there’s no list view for them”

    – yes, I’d love to know more about your experiences here!

    (as I think would a lot of people :))

  7. Adam

    “Unity’s serialization is same as memory management is a misnomer. It’s used for persistence”

    AIUI, unity uses serialization as its primary form of memory management: all game-related data is managed by serialization, which in turn sits on top of C#, but provides a platform-independent and Unity-controllable, high-level, gateway to memory.

    So, for instance, Unity can implement Undo (even though it doesn’t work in Unity, in theory: it lets them impleemnt it perfectly)

  8. Pingback: Using ScriptableObject for data objects in Unity scenes | Grove Code Blog

  9. adam Post author

    Followup:

    Nope, that doesn’t work. ScriptableObjects do NOT save in the scene – Unity has code to save them in the scene, but someone working there decided to add extra code that then deletes them. Even after you’ve saved the scene!

    They put up a little info message (not even a warning, natch) to say “Yeah, we just deleted your data. Basically, you’re too stupid to be allowed to write code, so … we decided to delete it for you.”

    (My guess: this bizarre “feature” was invented as a clever way to hide some of the terrible code that newbies were writing in Unity. Not fix it, not fix the Unity bug that lead to them doing it – but just hide it)

  10. max

    Excuse me, adam, but you sound like a 6-year old, script kiddie.
    Have you ever worked with a custom game engine ?
    a) No explicit data presentation, No Monobehavior, No scriptableObjects.
    b) No gui or editor to handhold things for you. All have to be done manually by hand.

    Your post presents an unusual degration of quality. I remember old posts backed by data that following a scientific approach. I had higher standarts for you. It seems in your bitterness over failing to master unity, you start creating false arguments just to back yourself, to justify why you failed to use unity. Maybe you rushed to implement your code too fast before following the tutorials. Or you forgot how hard coding can be if you take a long period of break from it. Coding isnt for everyone, become patient instead wanting out-of-the-box easy solutions.

    Next time: Do research before you starting blaming unity for your own faults.
    No one forced you to Ab-use these classes, like you did. Learn their strengths and use them to save time in development.

  11. adam Post author

    Yes, I have written game engines from scratch, both for indie projects and (working with teams of up to 50 people) on AAA projects.

    Since publishing that post, I’ve done a lot more work with ScriptableObjects, using Unity’s latest round of fixed. Guess what? They’re still broken: Unity won’t/can’t fix them.

    All the other problems hilighted in the original post also still stand: I recently tried to do some physics, to make wheeled vehicles. Ever wondered why Unitys’ WheelCollider appears to be fake (doesn’t render like normal colliders; has a strange tiny sphere at its foot?) and buggy? … apparently it’s partly because of the problems I touched on in this post: even Unity’s own staff can’t get stuff working correctly, has to resort to hacks. (in this case: the hierarchy system breaks unity’s custom hacks in their core integration of PhysX, in ways that when it comes to wheels-on-cars are really important and problematic :( )

    …but most people blame themselves, or fail to ask “Hang on; why – exactly – does WheelCollider break all the rules of Unity colliders???”. I ask the questions, and the answers with Unity are usually ugly.

Leave a Reply

Your email address will not be published. Required fields are marked *