Ladders being glitchy in Garry's Mod?

I’ve made some ladders using func_useableladder. Both the start and end are placed at the top and bottom of the ladder, but when getting on the ladder and holding space and W, you get off and back on the ladder really quickly. I have tested some other Garry’s Mod maps (ttt_clue), and they do not have this issue.

Here’s a video demonstrating the issue;

And here’s the map for download; ladder_test.bsp

What could be causing this issue? I tried moving the func_usableladder back a few units, as well as deleting the prop_static infront of it, both made no difference.

Could it also be the ladder shaft is not wide enough, so jumping off pushes me back on? (although I can jump off and not press W and I will fall down it)

I think you should just set it up like this:

That should work

I originally used the func_usableladder, as it allows the use of named dismount points. I have now switched to a func_ladder because it actually works.