So I thought I'd start learning C#. Let's just say it was a very enjoyable experience.
I created WorkshopDLKiller for use on my upcoming server. At the moment, hundreds of shitty DarkRP servers still use the capped WorkshopDL method for distributing server content to their players.
This tool was designed for those who have a Workshop collection they want converted to FastDL. It provides easy to use mass & bulk extracting and also names the folders containing the addons appropriately.
[t]http://i.imgur.com/VAuzqJ5.png[/t]
Simply select the folder where your .gma files are located (a dialog will show up) and it will start extracting to that folder with appropriate addon names.
[t]http://i.imgur.com/QM9M4GV.png[/t]
You can also choose where you want your addons to be extracted to.
Along with all of this, I have added support for simply adding the -folder and -output arguments when you run the tool. This means you can automate the tool or make it even simpler to use.
An idea I had for this was to create a TCAdmin module, so you can just click a button for all your Workshop addons to be extracted into FastDL.
Therefore, you can simply make a batch file like this:
[code]@echo off
WorkshopDLKiller -folder C:/ServerFiles/GMAFiles -output C:/ServerFiles/DarkRP/garrysmod/addons
PAUSE[/code]
Enjoy! Let me know if there's any issues.
[url]https://github.com/BillyOnWiiUIn144p/WorkshopDLKiller[/url]
Sorry if this may sound dumb, but how exactly would this benefit servers with small amounts of addons?
Is it worth switching over to the direct legacy files and not getting automatic updates from the server?
I'm not really sure what the WorkshopDL is capped to, so if anyone knows, please let me know :)
Needless to say, this is a very neat tool and not bad at all for a starter project in C#.
[QUOTE=kpjVideo;50245129]Sorry if this may sound dumb, but how exactly would this benefit servers with small amounts of addons?
Is it worth switching over to the direct legacy files and not getting automatic updates from the server?
I'm not really sure what the WorkshopDL is capped to, so if anyone knows, please let me know :)
Needless to say, this is a very neat tool and not bad at all for a starter project in C#.[/QUOTE]
There are a lot of small advantages to using FastDL over WorkshopDL.
Pros:
1. WorkshopDL takes ages to mount on the client (and also needs mounting on the server, but that doesn't take long)
2. WorkshopDL downloads the [I]whole[/I] addon and not just textures, materials, models etc.
3. WorkshopDL subscribes the user, meaning they have to use your server's addons on singleplayer until they unsubscribe or disable the addon
4. Updates can delay the start up of your server and are also not that important for Workshop addons
5. Smaller servers can receive benefits from it as they are most likely going to run on a smaller budget which means it takes longer to start the server up. WorkshopDL can therefore obviously affect this.
6. Loading screens can more accurately track progress of the player joining
7. Instant editing access to the Workshop addons encase they need editing
Cons:
1. Valve's servers are global and international players may receive faster speeds by downloading from a closer location
2. Your webserver might not be as fast as Valve's servers
If anyone wants to expand on my points, feel free
I think Workshop downloads are much better. They usually download faster than any FastDL content will (both due to Valve's servers and the compression) and they allow for files to be modified without having to change their filename (users will not be stuck with outdated files).
If I'm connecting to a server and notice that I am downloading through FastDL, I'll probably switch server.
[quote]1. WorkshopDL takes ages to mount on the client (and also needs mounting on the server, but that doesn't take long)[/quote]
I think it's pretty fast since the previous update. Check it out yourself.
[quote]2. WorkshopDL downloads the whole addon and not just textures, materials, models etc.[/quote]
Yeah, but the Lua contents are often pretty small and there isn't much else you wouldn't want clients to have.
[quote]3. WorkshopDL subscribes the user, meaning they have to use your server's addons on singleplayer until they unsubscribe or disable the addon[/quote]
It doesn't _subscribe_ users, but the addon will be mounted until they restart the game. This should be fixed in a future update though.
[quote]6. Loading screens can more accurately track progress of the player joining[/quote]
At the moment the Workshop content progress bar is based on addon count and not size, that's true. But Workshop content also downloads and extracts in a separate thread. That means clients' games won't freeze while they are connecting. The downloads also run concurrently, meaning that three addons can be downloading/extracting at the same time.
You're right about it being difficult to modify a Workshop submission.
[editline]2nd May 2016[/editline]
Workshop addons count towards the same limit as FastDL files. That means you can have ~8000 files, ~8000 addons, or a mixture of both. You're much less likely to hit the limit with Workshop addons.
[QUOTE=Willox;50245387]:snip:[/QUOTE]
Thanks for all the insight Willox.
I wanted to try extracting/decompressing or whatever the .gmas that are downloaded from Valve's severs. So I opened Fiddler and started sniffing. I found the URL quite easily and opened it in my browser.
After a few seconds, it downloaded a file simply called "download" with no file extension (I've seen this before, MySQL etc.). I changed the file extension to .gma and the extractor/decompressor didn't work with it. I also tried others. You mentioned that Valve's files are compressed. Do these files have to be decompressed before they can be used as .gmas? Is it something open source like LZMA?
It takes me around a minute and 30 seconds to join, download 11 addons containing 2.05GB of content ranging from a map to hundreds of car models then load into the server. The whole thing also totals to 8,603 files which also includes the map's unpacked custom content which no longer has to be packed in.
Previous to this with FastDL it took roughly 15 minutes or more although much of that was probably because the webserver wouldn't serve me with upto 12MB/s. I don't even want to know what the full 2gb would take that rolls over the downloadables stringtable. I've heard some people have issues with valve content servers being slow but your webserver most likely doesn't have top notch routing to everyone either.
I think much of your issue might be with that you're downloading maybe hundreds of incredibly small workshop addons. I imagine downloading uncompressing then mounting a single 1kb addon is probably 100x slower than a 1kb bzip on a FastDL.
This however is an incredibly useful tool, being able to pull down entire collections and automatically extract them is something I've wanted to do a few times.
Thanks for the tool, Billy!
By the way, the file called download without an extension can be extracted using a tool called GWTool. If you want the tool, PM me or reply back on the thread.
[QUOTE=BillyOnWiiU;50245509]Thanks for all the insight Willox.
I wanted to try extracting/decompressing or whatever the .gmas that are downloaded from Valve's severs. So I opened Fiddler and started sniffing. I found the URL quite easily and opened it in my browser.
After a few seconds, it downloaded a file simply called "download" with no file extension (I've seen this before, MySQL etc.). I changed the file extension to .gma and the extractor/decompressor didn't work with it. I also tried others. You mentioned that Valve's files are compressed. Do these files have to be decompressed before they can be used as .gmas? Is it something open source like LZMA?[/QUOTE]
All legacy (read: all current) workshop addons stored on Valve servers are LZMA compressed. (by us, not Valve)
[QUOTE=BillyOnWiiU;50244786]At the moment, hundreds of shitty DarkRP servers still use the capped WorkshopDL method for distributing server content to their players.[/QUOTE]
You're referring to the size cap, correct? Or are you stating that there's a max DL speed from Workshop.
[QUOTE]At the moment, hundreds of shitty DarkRP servers still use the capped WorkshopDL method for distributing server content to their players.
You're referring to the size cap, correct? Or are you stating that there's a max DL speed from Workshop.[/QUOTE]
I think what Billy is trying to say is that WorkshopDL is more limited compared to FastDL. Since many servers use WorkshopDL, it sometimes can be slower than FastDL depending on the area you live in. WorkshopDL also has to be mounted which takes forever. So overall, WorkshopDL is slower than FastDL.
[QUOTE=brianm109;50285215]I think what Billy is trying to say is that WorkshopDL is more limited compared to FastDL. Since many servers use WorkshopDL, it sometimes can be slower than FastDL depending on the area you live in. WorkshopDL also has to be mounted which takes forever. So overall, WorkshopDL is slower than FastDL.[/QUOTE]
[QUOTE=code_thrax;50284696]You're referring to the size cap, correct? Or are you stating that there's a max DL speed from Workshop.[/QUOTE]
Actually Willox explained that that's old news and he's made a lot of improvements since then. Read above
[QUOTE=BillyOnWiiU;50285268]Actually Willox explained that that's old news and he's made a lot of improvements since then. Read above[/QUOTE]
I see, I understand now. I appreciate you pointing that out!
[QUOTE=brianm109;50285215]I think what Billy is trying to say is that WorkshopDL is more limited compared to FastDL. Since many servers use WorkshopDL, it sometimes can be slower than FastDL depending on the area you live in. WorkshopDL also has to be mounted which takes forever. So overall, WorkshopDL is slower than FastDL.[/QUOTE]
Slower only in situations where you have a significant amount of workshop addons and a low amount of content. Workshop will be multitudes faster at downloading a thousand or so files or hundreds of small files as the HTTP request to the FastDL webserver for every individual file would probably take almost as long as the download for a small file such as a vmt. Workshop is immeasurably useful in extending the 8000 file download limit.
There was a time as well that I would have killed for something like WorkshopDL back when too much data in the stringtables would crash clients who join and crash connected clients at random. Luckily that was all fixed long ago.
[QUOTE=Willox;50245387]If I'm connecting to a server and notice that I am downloading through FastDL, I'll probably switch server.[/QUOTE]
Damn, nigga
Sorry, you need to Log In to post a reply to this thread.
