slowly degrading fps

when I get in rust I start out with 66-70 fps and then over 5 mins my fps slowly degrades to 30 and then even 10-20. Is there any reason for this? I am using firefox nightly, updated my unity and updated my drivers.