Can't connect to servers, people can connect to mine.

My friends and I can all play on my server just fine (although it doesnt display on the server list.). However, when I try to join any other GMod server, it just sits there connecting. After a while it says “Failed to connect after 4 tries.” and sends me back to the main menu. I’m running on wireless and I’ve forwarded the ports for GMod (Maybe thats the problem?).

Anyone have any suggestions? I’m all out of ideas.