Rust FPS Problem

kk so I play rust max settings i get 40+ normally but when I look in some places and go near some areas my fps drops to 12 Like I will look a certain way in an area and my fps will shit itselfs I kept looking in the same direction and changed all my settings to low and my fps didn’t move Is it because of all the buildings in that direction? and not do you guys know how to fix this.