So currently there is a lot of controversy concerning ladders and their implementation in game, I believe there is a place for ladders in rust however certain aspects of the ladders need to be changed.
Some of the recommendations I have concerning ladders are:
Ladders shouldn’t just be made out of wood and if they are should require a flat surface to place them on ladders should require more materials to make them these could include:
Horse Hair x100 this would need at least 5 horse kills. (ladders don’t have to be just made of wood the frame could have a ropey frame)
Metal Rods 100 metal per rod. (for inside the piece of wood you step on)
Rope ladders could be better for things like climbing rocks, wooden ladders could be better for straight walls.
Now regarding defense against ladders, why not implement buckets of fat which can be threw on your walls which simply make ladders slide off walls, it would require a lot of developing but in the end could offer some protection.
Ladders should not be erected straight away if anyone has ever played mount and blade the siege ladders are erected slowly this would make the raider vulnerable to attack from other players maybe the raider could make moaning sounds as the strain of lifting the ladder is too much.
these are just some ideas feel free to add to it.