SpawnIcon model error but not for localplayer

[Matsumoto]

I am stumped on finding a reason why the model would be error on the spawnicon but not for the local player. I have tried precaching the model but it still does it. I have provided two pictures, one that shows it being drawn and the other with an error sign (look at the voice chat avatars). It looks correctly for the localplayer with the model but for any other clients its an error. It is a custom model, not a hl2/css model.

https://steamuserimages-a.akamaihd.net/ugc/949599512599507909/6BE7EAC9383759001F933CF7C817DB8675ADC28D/

https://steamuserimages-a.akamaihd.net/ugc/955229012133739981/FB57E803053BA292D3D18E2458BADC5CD839D9BE/

panel.Avatar = vgui.Create( "SpawnIcon", panel )
panel.Avatar:SetSize( 32, 32 )
panel.Avatar:SetModel( ply:GetModel() )
panel.Avatar:Dock( LEFT )

function panel.Avatar:SetPlayer( p )

self:SetModel( p:GetModel() )

end