Entity:SetAngles() doesnt set the absolute angles

[SeveredSkullz]

Hello,
I am attempting to set angles of a prop to mimic the movements of the player's camera. All is well in every situation except for when the prop I am piloting comes in contact with a hitbox (entity / world / etc)
A bit hard to explain, but the prop seems to be maintaining some sort of angular velocity even though the angles are being set strictly to be the same angles of the player's eye angles.
For instance I can pilot my prop and hit a wall... For ease of explanation, say my prop starts rotating clockwise in relation to the players camera. If I pitch down on the camera, so does the prop. however it maintains that clockwise rotation relative to the camera.
function GM:FinishMove( ply, mv )
    local pilotedObject = ply:GetPilotedObject()
    if (pilotedObject != NULL) then
        local phys = pilotedObject:GetPhysicsObject()
        phys:SetVelocity(mv:GetVelocity());
        phys:SetAngles(mv:GetAngles());
        pilotedObject:SetAngles(mv:GetAngles()); -- Redundant set angles in attempt to CORRECTLY set angles.
        --pilotedObject:SetLocalAngularVelocity(Angle(0,0,0)) -- Doesnt work, nor does it work on the physobject
        ply:SetPos(pilotedObject:GetPos())
        return true
    end
end

Above is the snippet of code I am using to "Pilot" my prop. Any help would be appreciated.