Running low FPS while having a fine computer?

I figured out that having windowed mode fucks up my FPS. After using fullscreen I get around 15-25 more FPS.