Sometimes I can't connect to any Source game servers.

[jt_l33t]

I can connect to a servers but some times I can't connect anything and I have to wait a minute or two and press refresh a few times to find servers. This happens frequently and it's annoying. I know my ports are forwarded and that the firewall isn't a problem. Valve could not help me with issue either.

The server list just won't show me any servers for a while and if I try to connect to an IP that I know is a server (such as my friend's IP) I can't connect but once it shows me all the servers again I can connect again. I'm really puzzled as to what this is.