my server lags when players using voice chat

so when players using the voice chat the server get lags…
pls help me to improve my server
right know there are no more than 4 players in the server(friends that help me to test the server) and 5mbps upload speed
my server cfg:
//Server Settings
net_maxfilesize 81
sv_minrate 0
sv_maxrate 2500
decalfrequency 10
sv_maxupdaterate 66
sv_minupdaterate 10
//Execute ban files
exec banned_ip.cfg
exec banned_user.cfg

Be more specific, what gamemode? Are you using a custom voice chat? Did you buy this server or hosting it off of your own pc?

I had this issue with my old server, the only way I managed to fix it was by re-installing the server.

It fixed the issue for me, and please include what addons/gamemode your running as they may cause an issue.