Gmod crashes internet connection

So, ever since some time around December I’ve been having this issue in which starting up Gmod will cause my internet connection to crash and does not go back up until I reset my modem. It doesn’t matter if I play singleplayer or multiplayer, it always crashes after about 5 minutes. I’ve tried lowering the max pings/minute in the steam settings, I’ve tried disabling my router’s firewall protection and I’ve tried making a clean install by deleting my Gmod folder in steamapps and re downloading several times. This problem started randomly for no reason, the only possible cause I can think of is when I asked my ISP to open up to the ports in my router required for Xbox LIVE, but i’m not sure how that would affect it. Router is a Linksys E1500, i’m on an ADSL 2mb connection, and my connection is wired. Any help is appreciated in advance, please help!

EDIT: I forgot to mention other source games like TF2 and CSS don’t have this problem, it’s only Gmod that causes it.
EDIT: After browsing around a bit I discovered that using a VPN gets rid of the problem, I tried this and was able to play for an hour without any sort of problems.

Do you have any Lua/script errors popping?

Nope, that doesn’t seem to be the case.

Bump

Lower your video textures?

Been happening with me for months too. Click on server list then internet drops for 3-5 mins.

Bump. Please I really need some help with this, it’s been months now.