Render Targets Alpha Issues

Garry's Mod coding
#1

I’m aware I’m not the first to post on this issue but it’s still an issue and it should be fixed. So for starters the threads I found on the issue:

http://webcache.googleusercontent.com/search?q=cache:w-J8sqokGEQJ:https://facepunch.com/showthread.php%3Ft%3D1277159+&cd=1&hl=en&ct=clnk&gl=us
http://forum.facepunch.com/showthread.php?t=1276498
http://forum.facepunch.com/showthread.php?t=1287345

I’m trying to render models as icons, but due to some additional effects I have planned I can’t just use spawn icons. Nevertheless, this issue is important for anyone who needs or wants to use rendertargets.

The issue with rendertargets is that their alpha behaves mysteriously, obeying render.Clear and render.ClearBuffersObeyStencil, but remaining unchanged by any other draw functions (at least the one’s I’ve tested so far). And furthermore, it behaves differently depending on the model used too.

Clearing the background to black and drawing a basket model:

Clearing the background to a semi-transparent black and drawing the basket model:

Sadly, it appears the the basket adopted the alpha values set by render.Clear, instead of overwriting them.

Now there’s a workaround using stencils and render.ClearBuffersObeyStencil, to essentially clear any area that was not drawn to.
The results:

That’s fine, I can work with that. After all, I’m generating icons which means this is rendered once, no performance impact.
But then I changed the model and tried rendering the AR2 world weapon model and…

Nothing… So I commented out all the stencil operations and:

Now, after tinkering around a bit I was able to make the AR2 render correctly by changing the Rendertarget’s image format to IMAGE_FORMAT_RGB888 so as to remove the alpha channel:

I’m really at a loss and I don’t have enough knowledge or experience with Garry’s mod or the Source Engine to even begin to understand what is going on. I’m hoping that someone with the right experience can help, even if not to solve the problem through Lua, find out what’s causing it so it can be fixed.




local icon_width = 128
local icon_height = 128
local icon_focus = Vector(2, 0, -4)
local icon_angle = Angle(30, 45, 0)
local icon_distance = 60
local icon_fov = 30
local icon_model = "models/props_junk/PlasticCrate01a.mdl"
--icon_model = "models/weapons/w_irifle.mdl"

local TEXTURE_FLAGS_CLAMP_S = 0x0004
local TEXTURE_FLAGS_CLAMP_T = 0x0008

local texture = GetRenderTargetEx("tex_icon"..CurTime(),
	icon_width,
	icon_height,
	RT_SIZE_NO_CHANGE,
	MATERIAL_RT_DEPTH_SEPARATE,
	bit.bor(TEXTURE_FLAGS_CLAMP_S, TEXTURE_FLAGS_CLAMP_T),
	CREATERENDERTARGETFLAGS_UNFILTERABLE_OK,
    --IMAGE_FORMAT_RGB888
    IMAGE_FORMAT_RGBA8888
)

local material = CreateMaterial("mat_icon"..CurTime(), "UnlitGeneric", {
	["$ignorez"] = 1,
	["$vertexcolor"] = 1,
	["$vertexalpha"] = 1,
	["$nolod"] = 1,
	["$basetexture"] = texture:GetName()
})

local campos = icon_focus - icon_angle:Forward()*icon_distance
local camang = icon_angle

local function Generate()
	
	local ent = ClientsideModel(icon_model, RENDERGROUP_BOTH)
	ent:SetNoDraw(true)
	
	local rendertarget_old = render.GetRenderTarget()
	render.SetRenderTarget(texture)
		
		render.ClearDepth()
		render.Clear(0, 0, 0, 255)
		
		cam.Start3D(campos, camang, icon_fov, 0, 0, icon_width, icon_height, 1, 4096)
			render.SuppressEngineLighting(true)
			
			render.SetLightingOrigin(ent:GetPos())
			render.ResetModelLighting(1, 1, 1)
			render.SetColorModulation(1, 1, 1)
			render.SetBlend(1)
			
			for i=0, 6 do
				render.SetModelLighting(i, 1, 1, 1)
			end
			
			--[[]]
			render.SetStencilEnable(true)
				
				render.ClearStencil()
				render.SetStencilEnable(true)
				
				render.SetStencilReferenceValue (1)
				render.SetStencilFailOperation  (STENCILOPERATION_KEEP)
				render.SetStencilZFailOperation (STENCILOPERATION_KEEP)
				render.SetStencilPassOperation  (STENCILOPERATION_REPLACE)
				render.SetStencilCompareFunction(STENCILCOMPARISONFUNCTION_ALWAYS)
				--[[]]
				ent:DrawModel()
				--[[]]
				render.SetStencilCompareFunction(STENCILCOMPARISONFUNCTION_NOTEQUAL)
				
				render.ClearBuffersObeyStencil(255, 255, 255, 0)
				
			render.SetStencilEnable(false)
			--[[]]
			render.SuppressEngineLighting(false)
		cam.End3D()
		
	render.SetRenderTarget(rendertarget_old)
	
	ent:Remove()
end

local generated
hook.Add("HUDPaint", "Icon", function()
	if not generated then
		generated = true
		Generate()
	end
	/*
	surface.SetDrawColor(255, 255, 255, 50)
	surface.DrawRect(0, 500, icon_width, icon_height)
	*/
	surface.SetMaterial(material)
	surface.SetDrawColor(255, 255, 255, 255)
	surface.DrawTexturedRect(500, 500, icon_width, icon_height)
end)
#2

So I was able to find 2 workarounds and just in case anyone else has the issue here they are.

The simple workaround is to use render.SetBlend(0.99) instead of render.SetBlend(1).

The other solution which is much more complex is to allocate another rendertarget and material, but without the alpha channel (IMAGE_FORMAT_RGB888). Draw onto this rendertarget the model, and then on the other rendertarget (the one with the alpha channel), enable stencil with pass operation replace and non-zero reference value, set the blend to 0, render the model again, change stencil pass operation to keep, draw the texture of the rendertarget without the alpha channel, and finally change the stencil compare to notequal and clearbuffersobeystencil. Ugly as hell but works.

Here’s the code for the second workaround in case the first doesn’t.




local icon_width = 128
local icon_height = 128
local icon_focus = Vector(2, 0, -4)
local icon_angle = Angle(30, 45, 0)
local icon_distance = 70
local icon_fov = 30
local icon_model = "models/props_junk/PlasticCrate01a.mdl"
icon_model = "models/weapons/w_irifle.mdl"

local TEXTURE_FLAGS_CLAMP_S = 0x0004
local TEXTURE_FLAGS_CLAMP_T = 0x0008

local texture = GetRenderTargetEx("tex_icon"..CurTime(),
	icon_width,
	icon_height,
	RT_SIZE_NO_CHANGE,
	MATERIAL_RT_DEPTH_SEPARATE,
	bit.bor(TEXTURE_FLAGS_CLAMP_S, TEXTURE_FLAGS_CLAMP_T),
	CREATERENDERTARGETFLAGS_UNFILTERABLE_OK,
    IMAGE_FORMAT_RGBA8888
)

local material = CreateMaterial("mat_icon"..CurTime(), "UnlitGeneric", {
	["$ignorez"] = 1,
	["$vertexcolor"] = 1,
	["$vertexalpha"] = 1,
	["$nolod"] = 1,
	["$basetexture"] = texture:GetName()
})

local tex_temp = GetRenderTargetEx("tex_icon_temp"..CurTime(),
	icon_width,
	icon_height,
	RT_SIZE_NO_CHANGE,
	MATERIAL_RT_DEPTH_SEPARATE,
	bit.bor(TEXTURE_FLAGS_CLAMP_S, TEXTURE_FLAGS_CLAMP_T),
	CREATERENDERTARGETFLAGS_UNFILTERABLE_OK,
    IMAGE_FORMAT_RGB888
)

local mat_temp = CreateMaterial("mat_icon_temp"..CurTime(), "UnlitGeneric", {
	["$ignorez"] = 1,
	["$vertexcolor"] = 1,
	["$vertexalpha"] = 1,
	["$nolod"] = 1,
	["$basetexture"] = tex_temp:GetName()
})

local campos = icon_focus - icon_angle:Forward()*icon_distance
local camang = icon_angle

local function draw_ent(ent, invis)
	cam.Start3D(campos, camang, icon_fov, 0, 0, icon_width, icon_height, 1, 4096)
		render.SuppressEngineLighting(true)
		
		render.SetLightingOrigin(ent:GetPos())
		render.ResetModelLighting(1, 1, 1)
		render.SetColorModulation(1, 1, 1)
		render.SetBlend(invis and 0 or 1)
		
		for i=0, 6 do
			render.SetModelLighting(i, 1, 1, 1)
		end
		
		ent:DrawModel()
	
		render.SuppressEngineLighting(false)
	cam.End3D()
end

local function Generate()
	
	local ent = ClientsideModel(icon_model, RENDERGROUP_BOTH)
	ent:SetNoDraw(true)

	local rendertarget_old = render.GetRenderTarget()
	render.SetRenderTarget(tex_temp)
		
		render.ClearDepth()
		render.Clear(0, 0, 0, 255)
		
		draw_ent(ent)
	
	render.SetRenderTarget(texture)
		
		render.ClearDepth()
		render.Clear(0, 0, 0, 255)
		
		
		render.SetStencilEnable(true)
			
			render.ClearStencil()
			
			render.SetStencilReferenceValue (1)
			render.SetStencilFailOperation  (STENCILOPERATION_KEEP)
			render.SetStencilZFailOperation (STENCILOPERATION_KEEP)
			render.SetStencilPassOperation  (STENCILOPERATION_REPLACE)
			render.SetStencilCompareFunction(STENCILCOMPARISONFUNCTION_ALWAYS)
			
			draw_ent(ent, true)
			
			local w, h = ScrW(), ScrH()
			render.SetViewPort(0, 0, icon_width, icon_height)
			cam.Start2D()
					
					render.SetStencilFailOperation  (STENCILOPERATION_KEEP)
					render.SetStencilZFailOperation (STENCILOPERATION_KEEP)
					render.SetStencilPassOperation  (STENCILOPERATION_KEEP)
					render.SetStencilCompareFunction(STENCILCOMPARISONFUNCTION_ALWAYS)
					
					surface.SetMaterial(mat_temp)
					surface.SetDrawColor(255, 255, 255, 255)
					surface.DrawTexturedRect(0, 0, icon_width, icon_height)
					
					render.SetStencilCompareFunction(STENCILCOMPARISONFUNCTION_NOTEQUAL)
					
					render.ClearBuffersObeyStencil(255, 255, 255, 0)
					
			cam.End2D()
			render.SetViewPort(0, 0, w, h)
			
		render.SetStencilEnable(false)
		
	render.SetRenderTarget(rendertarget_old)
	
	
	ent:Remove()
end

local generated
hook.Add("HUDPaint", "Icon", function()
	if not generated then
		generated = true
		Generate()
	end
	/*
	surface.SetDrawColor(255, 255, 255, 50)
	surface.DrawRect(0, 500, icon_width, icon_height)
	*/
	surface.SetMaterial(material)
	surface.SetDrawColor(255, 255, 255, 255)
	surface.DrawTexturedRect(500, 500, icon_width, icon_height)
end)