A detailed Stencil tutorial

I don’t understand how this ties into drawing circles in a HUD, like how GTA IV is done. Can anyone explain that part?

You draw a circle, that is the mask for whatever else you want to draw, then you use stencils to test for if what you want to draw is inside that circle.
Then only draw what is inside the circle.

If you wanted to test for 1 and write 2, it’d be probably easier to use STENCIL_INCR as the stencil operation.
If you set the test mask to 1 and the write mask to 2 and then use 3 as your reference value, you’d be testing for 1 and writing 3, since the first bit will be left untouched.

Say I render a model though and some parts of it overlap, then it would write values >2 in places.

Increase only if the stencil value is equal to 1? Only STENCIL_REPLACE uses the reference value for writing, the others do not depend on it.

Ooh. Major understandment right there. Rhank you!

Since the stencil discussion is back, I was wondering if anyone was able to get the drawn texture from stencils to have alpha?

I tried drawing the silhouette of a model using a texture, but I needed it to be translucent. Using render.SetBlend() didn’t do anything and I couldn’t figure anything else out.

Oh yeah. A while back I saw someone made a properly transparent cloaking weapon for TTT. Tried doing the same but to no avail.

Sorry to bring this old thread up, but i can’t get this to work. I am trying to make a prop blue when its behind something, and red when its visible. This is my code:


hook.Add("PostDrawOpaqueRenderables","PlayerBorders",function()
	--stencil work is done in postdrawopaquerenderables, where surface doesn't work correctly
	--workaround via 3D2D
	local pos = LocalPlayer():EyePos()+LocalPlayer():EyeAngles():Forward() * 10
	local ang = LocalPlayer():EyeAngles()
	ang = Angle(ang.p + 90, ang.y, 0)
	for _, ent in pairs(ents.FindByClass("prop_physics")) do
		render.ClearStencil()
		render.SetStencilEnable(true)
			render.SetStencilWriteMask(255)
			render.SetStencilTestMask(255)
			render.SetStencilReferenceValue(15)
			render.SetStencilFailOperation(STENCILOPERATION_KEEP)
			render.SetStencilZFailOperation(STENCILOPERATION_REPLACE)
			render.SetStencilPassOperation(STENCILOPERATION_KEEP)
			render.SetStencilCompareFunction(STENCILCOMPARISONFUNCTION_ALWAYS)
			render.SetBlend(0)
			ent:DrawModel()
			render.SetBlend(1)
			render.SetStencilCompareFunction(STENCILCOMPARISONFUNCTION_EQUAL)
			cam.Start3D2D(pos, ang, 1)
					surface.SetDrawColor(100, 100, 255)
					surface.DrawRect(-ScrW(), -ScrH(), ScrW() * 2, ScrH() * 2)
			cam.End3D2D()
			ent:DrawModel()

			render.SetStencilReferenceValue(30)
			render.SetStencilFailOperation(STENCILOPERATION_KEEP)
			render.SetStencilZFailOperation(STENCILOPERATION_KEEP)
			render.SetStencilPassOperation(STENCILOPERATION_REPLACE)
			render.SetStencilCompareFunction(STENCILCOMPARISONFUNCTION_ALWAYS)
			render.SetBlend(0)
			ent:DrawModel()
			render.SetBlend(1)
			render.SetStencilCompareFunction(STENCILCOMPARISONFUNCTION_EQUAL)
			cam.Start3D2D(pos, ang, 1)
					surface.SetDrawColor(255, 100, 100)
					surface.DrawRect(-ScrW(), -ScrH(), ScrW() * 2, ScrH() * 2)
			cam.End3D2D()
			ent:SetRenderMode(RENDERMODE_TRANSALPHA)
			ent:SetColor(Color(255, 255, 255, 0))
			ent:DrawModel()
		render.SetStencilEnable(false)
	end
end)

And this is the outcome:
http://imgur.com/a/bjFsG
The only reason i dicided to use SetRenderMode, was so i could make the prop invis, but it just draws the full model, and not just only the scentil. Any advice?