SFM "Out of Key value string space"

So I completed a SFM video. All I need to do now is put in the SFX for the video. Whether its playing SFX previews, scrubbing the timeline with the music in the background, or just waiting for the sound folder to scan, I keep getting “Out of Key value string space” and my project crashes. Nothing of value is lost thanks to my constant saving. Some sources tell me its because I have too many models or too many models in my usermod and workshop folders. How do I fix this? I never had this problem when I was nearing completion or letting SFM scan my folders for models.

Right, so, I’ll give you the solution to this, and then explain the problem.

The solution: Restart SFM, and then only load up your sounds or your models or your textures (for the particle editor).

The problem:

So this is a technical problem, I’ll try to be brief here.

Basically, SFM uses caching or indexing for all of its assets that load from file - that is to say, models, sounds, and textures. The first time you bring up the model / sound / texture viewer, it scans through all of the files and builds a cache of the files.

That’s why you see it do this scan the first time you open these viewers per SFM session, but after that, there is no more file loading - everything is instantly there for you.

To accomplish this, SFM slams the relevant filepaths into a big-ass array, or list, of filepaths. Once it says “Okay, I scanned all of the folders”, it actually stops trying to scan the folders at all, and instead directly accesses this array for you.

This array is indexed with a two-byte integer, known as a unsigned short. The mathematical limit of an unsigned short is 2^16-1, or 65535.

This means that this list has a maximum size of 65536 entries (because computers index at 0, not 1). Once the list is at 65536 entries, and you tried to add a 65537th element, you get an overflow. How systems handle overflows varies, but for SFM, it handles it by crashing and throwing a “Out of Key Value string space” error.

So, why is this occurring?

The answer is because, in Valve’s infinite wisdom, they decided to make all of the filepaths use the same array.

All of your models, sounds, and textures are all loaded into the same array. Meaning that you can only have a total of 65536 elements across all three spaces cached at any time. If you attempt to go over that, SFM crashes.

This is why the solution posted above works: by never loading the models, sounds, or textures in a session, SFM never caches them, and so they never take up that space in the array, and so the array doesn’t overflow, and so SFM doesn’t crash.

Fun fact: I actually have too many materials in my SFM installation to load them all in the particle editor by itself. I literally cannot make a particle using the last material in my SFM installation without typing it in by hand - the browser crashes every time before it gets to it, because I have more than 65536 total textures.

My models folder is getting close as well, with 61486 total models in my SFM installation.

So its because I have too many models, materials, and sounds in my usermod and workshop folders in combination, or that I have too much in one session ie my saved video project? So to fix that I would have to delete all of the models, materials, and sounds that I’m currently no using? Sounds like a lot of effort to get my one minute-long to work without a hiccup. This seems to happen only when I’m looking in the “All Mods” sound directory, but I haven’t tested it with models.

No, the issue is you’re loading too many of them into cache.

To fix it, just don’t load them all at once.

I do my SFM sessions in 3 minimum passes:

Pass 1: Load the models, spawn all the models I’ll use. Close out of SFM.

Pass 2: Load the sounds, load all the sounds I’ll use. Close out of SFM.

Pass 3: Load the particle editor, create all the particles I’ll use. Close out of SFM.

Obviously in practice you don’t do them all at once like that, but that is what it breaks down to.

Now, if you’re overflowing the cache and you are only loading one space, then you’re going to have to cull some out of your install.

I wonder if it would be feasible for the array to be larger than 65536 entries.
Valve oughta patch it to be larger but haha SFM updates now? Not likely

Wasn’t the last update 2 years ago? It sucks that Valve didn’t think about that.

[editline]2nd January 2017[/editline]

Ah, so only get to work once you have everything you need in the session? I could try that. In fact, I think I’m doing it with the sound now since everything else (including the animation) is done.

In theory, yes. That’s the most optimized solution.

In practice it’s more.

Load a few models. Restart SFM. Load a few sounds. Restart SFM. Load more models. Restart SFM. Load more sounds. Restart SFM. Create particles. Restart SFM. Load a few more sounds. Restart SFM. Load a few models. Restart SFM. Create more particles. Restart SFM.

But if I were smart and fully planned everything out before I first touch SFM, then that is the way I would do it. :v:

Odd. I’ve never had that problem before. Good to know for the future.

Curious, if I wanted to remove the problem entirely, would lightening the usermod and Workshop folders of addons work? Would going from 24GB to 15GB solve that problem?

I can’t tell you from precise numbers, but lightening it will of course work.

If you can trim your SFM installation down to the sum of Models + Sounds + Materials < 65536, then you’ll be able to cache them all without crashing.