"Getting Addon Info for #xxxxxxxxx" takes a long time

So this is pretty weird, my server was working fine, could join it super fast, but now when nothing is changed, I restarted it and getting addon info takes a long time now for me and only me, everyone else can join instantly, each addon info taking a second at most, while mine loops through the loading bar at least 3-4 times and it just takes much longer to connect than it should, any ideas on how I can fix it?

Delete some addons.
garrysmod/garrysmod/addons/xxx
delete unnecessary addons there, when you did that you might go to
garrysmod/garrysmod/data/xxx
and delete stuff you don’t need too.

yeah I kinda posted this thread a bit too fast, cleaned some of the old addons I don’t need, verified files and restarted everything, seems to be fine now (in case anyone else gets this)