Rust Almost Consistent FPS drops to 30-50 only in certain areas?

So I noticed that I play at 60 fps pretty consistently (Not including the drops when loading in a new chunk of terrain, of course) but I noticed in certain little pockets outside and inside my base it can drop from 60 to anywhere from 50-30 in certain areas? I get a pretty consistent fps everywhere else.