Can't join some servers

I have been trying to join a server with friends but it won’t let me in. It just brings me back to the menu. And I could join other server.
Anyone know why?