Add Files In A Thread Pool To Speed Up Join Times?

Quick question, it looks like all the downloads stared by resource.AddFile() are going sequentially with most of the time (in my case) being spent on starting the download, my cpu isn’t being maxed out and I am nowhere near my max download speed. It looks like join times could be greatly improved by having each download started from a new thread.

In theory if I had 100 files to download wouldn’t it be much faster if I had a thread pool and just had individual threads call resource.AddFile()? Even if I only had 10 threads in my pool it should go much faster because I would be able to perform more simultaneous downloads as opposed to just doing them all sequentially?

Each time a file is downloaded with that system it kills the connection after each file then reestablishes for the next. It could be done quicker, but right now the only way is to use Workshop collections. They’re really quick!

The files I am having my users download are not on the workshop mostly due to them being somewhat custom and I don’t feel like adding them to the workshop.

I understand that most of the overhead with resource.AddFile comes from the creation of a new connection, I don’t plan to fix that. My thought is since all that is done sequentially if I were to make a new thread that called resource.AddFile() itself then wouldn’t new people load much faster since the downloads would be happening at the same time, instead of sequentially?

Unfortunately, it doesn’t work like that. You call resource.AddFile or resource.AddSingleFile and/or resource.AddWorkshop once per file / addon. You do not keep calling it when players join inside of a coroutine.

First, Coroutines aren’t true threads; still only 1 think happening at a time.

Second, resource.AddXXX adds the file to a list; re-adding files to the same list over and over does nothing except eventually overflow it. It doesn’t change how fast they download.

You could set up a fastdl to get faster speeds, it is still slower than a compressed Workshop file. Additionally, I did some tests; when I extracted all of my GMA addons from Workshop, it increased loading time exponentially, also auto-refresh took 30 to 60 seconds instead of the typical 3-5 seconds. Keeping the files as gma and not extracting them increased loading time massively, on top of reducing auto-refresh back down to 3-5 seconds ( Over 3mb of code, 70,000+ lines of code excluding comments and blank lines ). A custom auto-refresh loads it in half a second.

You could set up a DLL which the client would need to run ( not many would agree to it ), which would download the server files as fast as the speed, but you might create other issues if you go this route and don’t properly secure it.

Oh wow I never realized that having things packed as a workshop addon was such an improvement, I’ll move my addons over to that format.