I fear build harassment might be a very common thing in this game: People who just decide to build spikes in front of your door, add a wall in the middle of your house so you can’t get in or out, etc. I don’t see how such things makes the game more fun, more “realistic” or better, and it could potentially ruin the game. If you don’t think my suggestion is optimal, I still think it’s important, because house safety is overall very low.
How about a building gets a specified “owner”? Any piece constructed either as a part of a building, or within two tiles of the building, by anyone but the owner could have much less health (at least if destroyed by owner). This way any piece put in the house by other people could be destroyed much easier.
Who owns a building could be defined by whoever places the first door in it (and become owner-free when all of this user’s doors are destroyed. If someone wants to steal the house they would have to destroy all doors and insert at least one themselves.
I actually think the owner should be able to destroy all pieces quite easy. I have several times misplaced an item, and not being able to remove it, so I’ve had to just find a new location and start over. With ownership the owner could destroy pieces much easier, making it easier to remove misplaced items, expand, etc.