Why is workshop download so slow?

Hey,
I really don’t get it why the god damn download of addons in the loading screen is so slow and unreliable. It takes like 10 minutes to download one 60 MiB addon.
And apparently because 60 MiB is so “big” it fails like half of the time. Especially on the srcds side it’s completely impossible to download larger files.
How can this be? I mean it’s not that fucking hard to make it download stuff. All it takes is one SteamAPI lookup and then you can download the bloddy file from the link provided.
And then when people join our server they are just annoyed at how they can’t see shit because half of the workshop addons didn’t download properly. I mean it’s not like
this is some really useless feature that no one uses anyways. This is one of the main features of the game, it has to work properly.
It’s literally so easy to implement this properly. A 14 year old kid could implement a better (working) version.
There look, I even did it myself, it’s written in Java and I used some libraries, which admittedly makes it much easier, but it’s not like Facepunch Studios is lacking the skills to implement it properly in c++.
Source Binary (probably terrible and CLI only because lazy, still better than Garry’s Mod’s implementation)
This simple Java program let’s you download pretty much as fast as your internet connection and the steam servers allow. It even gives you a
percentage while downloading while the Garry’s Mod implementation just starts over and over again 300 times.
Garry’s Mod: 10 minutes for 60 MiB | Java Tool: 10 minutes to download all weapons and tools (3000 addons) from the workshop (~22 GiB compressed) at 200 Mbit/s (tested and confirmed)

_Kilburn, please fix this shit.
Thanks

It’s _Kilburn, not Killburn.

Maybe linkfilter? It shits up everything else.

No, the SteamAPI returns a link to the steam servers, no link filter is required. The only thing necessary is to make sure that the workshop id is a number.

Fixed it just for you :wink:

If anyone wants to test how fast the tool is, do this:
Download the list of all workshop weapons and tools
Save it as input.txt into the same folder as the jar file.
Run java -jar WorkshopDownloader.jar < input.txt from CMD/Terminal.
You will be surprised how fast the download can be.

I was used to think that’s a source engine’s problem

I don’t think so. I could write a c++ addon for Garry’s Mod that downloads it just as fast as the Java tool.

I have noticed other games like Left 4 Dead download workshop content like maps at least 3x faster but I never thought about it being a problem with Garry’s Mod I just assumed it was how the Steam workshop handled downloads. I mean if you have a server with 30+ people all downloading the same map at same time you would expect it to slow wouldn’t you even with Steams strong infrastructure it’s going to throttle the connection. But if it’s a problem with Garry’s mod then by all means a fix would be nice.

Just curious, how did you generate that file?

You could use a recursive loop and save the workshop IDs to a data file in lua.

I can only agree on the srcds side.

Weirdly, on Windows everything is fine, on Linux it can’t download big files (like m9k weapons).

What is the 64mb limit on servers for?

Made a crawler that inserts information about all addons of a category (tools+weapons) into a database

It’s much better on windows but it still fails sometimes and it’s quite slow. On linux it just fails every single time, it’s really sad.

I have no clue, I guess it’s some limit in the code.

Call me stupid, but where does the file save to?

Apparently I was doing it wrong. So what I have to do is is put an input.txt with the jar file in the download. Then I execute this line in cmd:

java -jar (LOCATION TO WorkshopDownloader.jar) < (LOCATION TO input.txt)

The location to = C:/ and etc.

Can you tell me what I am doing wrong?

Put both jar and txt in a folder (for example C:\workshop)
then start up cmd and type:

My problem isn’t the Workshop download speed, since that appears to be fine for me, but how hard the game freezes up once that download is finished

I hope that’s just my PC being shit and not how it is for everyone else

That’s them being mounted after downloading if I recall correctly.

That has nothing to do with workshop download.
When the download is done the actual loading and precaching (map, models, sound) begins.
Precaching is unfortunately implemented as a blocking operation and thus freezes the game until it’s done.

Hmm this happens to me too. If the addons are being downloaded in a queue it only happens at the very last addon. It doesn’t bother me anymore though.