Only render a player if they are behind a wall

I have a feeling the solution might have to do with stencils, but for the life of me I cannot figure out how to accomplish it. Basically, I want all players to be rendered in front of obstacles, but if they would normally be visible, then they are invisible.

Draw the playermodel onto a stencil. Make your stencil set the pixels that Fail or FailZ, Selectively return true in PrePlayerDraw (don’t return true when you are drawing for the stencil, or for the following). Draw player onto screen while respecting the stencil.

From what I understand about how stencils work, I don’t need to set a Fail operation since I assume I’ll be using ALWAYS, which never reaches a failure state. Also, how do I draw a player onto the screen while respecting the stencil?

That’s only the case if you want to draw everything drawn onto the stencil. You use the operations if drawing it to the stencil follows a condition. Player:DrawModel()

Wait, will drawing a model while the stencil is active prevent it from being drawn?

Prevent what from being draw?

The playermodel.

Basically, if I were to use this code;

[lua]
hook.Add(“PrePlayerDraw”, “stencilwork”, function(ply)
ply:DrawModel()
return true
end)
[/lua]

Then the player would be drawn. But if I used this code, making the draw call while the stencil is enabled;

[lua]
hook.Add(“PrePlayerDraw”, “stencilwork”, function(ply)
render.SetStencilEnable(true)
ply:DrawModel()
render.SetStencilEnable(false)
return true
end)
[/lua]

Then the player wouldn’t be rendered?

Don’t call pl:DrawModel in PrePlayerDraw, you’re causing an infinite loop that will make you crash without an error. Do your work in “PostDrawOpaqueRenderables”, and control the return of the PrePlayerDraw via a local variable in the global scope, that you can change when needed.

So what you’re saying is, override the default rendering of players, but render them when it’s most appropriate anyway?

If I’m understanding you correctly, then what you want me to do is something like this;

[lua]
local shouldDraw = false

function renderModel(ply)
shouldDraw = true
ply:DrawModel()
shouldDraw = false
end

hook.Add(“PrePlayerDraw”, “preventplayerdraw”, function(ply)
return not shouldDraw
end)
[/lua]

And call the renderModel function in place of ply:DrawModel() while i’m tinkering with the stencils?

Yeah that’s how you would do it.

One last thing then; how would I render the player’s model with respect to the stencil?

The same way you would draw anything else with respect to the stencil, without the need for the Cam3D2D (I don’t have the code to hand, but you do)

Well, I know the code necessary to draw a material over the screen with respect to the stencil. I just don’t know how to do the same for a playermodel. Is it any similar?

It’s exactly the same, but not in a cam3D2D

Well, this is the final code I have;

[lua]
local shouldDraw = false

function renderModel(ply)
shouldDraw = true
ply:DrawModel()
shouldDraw = false
end

hook.Add(“PrePlayerDraw”, “preventplayerdraw”, function(ply)
return not shouldDraw
end)

hook.Add(“PostDrawOpaqueRenderables”, “stencilwork”, function()
for _, p in pairs(player.GetAll()) do
render.SetStencilEnable(true)

        render.ClearStencil()
        render.SetStencilReferenceValue(20)
        render.SetStencilCompareFunction(STENCIL_ALWAYS)
            render.SetStencilPassOperation(STENCIL_KEEP)
            render.SetStencilZFailOperation(STENCIL_REPLACE)
        
        render.SetBlend(0)
        renderModel(p)
        render.SetBlend(1)
        
        render.SetStencilCompareFunction(STENCIL_EQUAL)
        
        renderModel(p)
        
    render.SetStencilEnable(false)    
end

end)
[/lua]

However, it doesn’t work properly; the player is just invisible everywhere no matter what.

[editline]16th April 2015[/editline]

STENCIL_* enums work by the way, you can see them on the wiki’s enum page; http://wiki.garrysmod.com/page/Enums/STENCIL

Cause your renderModel code will always evaluate to false in PrePlayerDraw (!true == false)

Yes, but according to james, I’m supposed to render it manually with respect to the stencil, which I’m having trouble with.

[snip] Didn’t see the “not”


local IN_HOOK = false;
hook.Add("PrePlayerDraw", "", function(p)
	if(IN_HOOK) then return; end
	
	IN_HOOK = true;
	
	render.SetStencilEnable(true);
		
		render.ClearStencil();
		render.SetStencilReferenceValue(1);
		render.SetStencilCompareFunction(STENCIL_ALWAYS);
		render.SetStencilPassOperation(STENCIL_ZERO);
		render.SetStencilFailOperation(STENCIL_ZERO);
		render.SetStencilZFailOperation(STENCIL_REPLACE);
		render.SetBlend(0);
			p:DrawModel();
		render.SetBlend(1);
		
		render.SetStencilCompareFunction(STENCIL_EQUAL);
		
		cam.IgnoreZ(true);
		p:DrawModel();
		cam.IgnoreZ(false);
		
	render.SetStencilEnable(false);
	
	IN_HOOK = false;
	return true;
end)


this should work

render.IgnoreZ should be cam.IgnoreZ