High fps, but with a lot of fps drops


I’ve build a new pc and I get around 60-70 fps on medium/high graphic settings instead of the 15-20 fps on low settings I was used to on my old pc. The thing is that every now and then my fps drops to 40 or even 30 in a matter of seconds making it hard to play. I noticed that my GPU usage fluctuates a lot(goes down to 0 and back to around 100 every few seconds or so), could this be the cause of the fps drops? Is my pc at fault or is it Rust? I would really appreciate it if someone could help me out.

My specs:

Intelcore i5 4460 3.2Ghz
Amd radeon MSI R9 380 4gb gaming
MSI B85M-G43
corsair CL vengeance 2x4GB
Cooler Master G750M
Samsung 250GB SSD


Have you got the latest gpu drivers installed?

Yeah, i’ve downloaded the latest crimson driver and I must say that the drops are less noticable, but they are still annoying. Will I be better off if I trade in my R9 380 for a Geforce GTX 960 or something?