gm_preproc - Lua preprocessor

gm_preproc

gm_preproc hooks the loading of lua text and passes it to lua for preprocessing. It handles all lua from any files run, and anything by lua_run(_cl), RunString, SendLua, etc. (As a side bit, it is interesting to note that lua being run via SendLua can be identified by ‘name’ being ‘LuaCmd’). Another interesting use is being able to dump the lua that is loaded by the “hack” modules which just serve as delivery systems for lua files, since they usually send the lua over the network encrypted, but it has been decrypted by the time it gets to the preprocessing hook.



Hooks:
    Lua_Preprocess([string] file, [string] path, [string] lua)
        This hook is called whenever any lua is loaded. For some reason path is always an empty string. If a boolean
        value is returned, the lua will not be run and the specified boolean value will be returned to the original caller.
        If a string is returned, that string is run instead of the original string. If anything else is returned, the lua will
        be run unmodified.

Functions:
    RawRunString([string] file, [string] path, [string] lua)
        Runs lua text but bypasses the hook. Running lua with this function does not trip the Lua_Preprocess hook. This
        is needful when you want to run a string inside a Lua_Preprocess hook, which would cause an infinite loop.


Examples (after requiring the module):
[lua]
] lua_run_cl hook.Add(“Lua_Preprocess”, “fe”, function(file, path, lua)
if string.find(lua, “stopme”) then
return true
elseif string.find(lua, “changeme”) then
return “print(‘sup’)”
end
end)

] lua_run_cl print(‘hi’)
hi
] lua_run_cl print(‘stopme’)
] lua_run_cl print(‘changeme’)
sup
[/lua]

[lua]
] lua_run_cl hook.Add(“Lua_Preprocess”, “fe”, function(file, path, lua) print(lua) end)
] lua_run_cl RunString(‘print(“hi”)’)
RunString(‘print(“hi”)’)
print(“hi”)
hi
] lua_run_cl RawRunString("", “”, “print(‘hi’)”)
RawRunString("", “”, “print(‘hi’)”)
hi
– notice that print(“hi”) was printed in the first call but not the second, because RunString called the hook but RawRunString did not.
[/lua]

This next example is a simple preprocessor. It has three types of preprocessor directives:
[ul]
[li]$define x y[/li]Will replace all occurrences of $x with y.
Example use:
$define nope ‘yep’
$define blah $nope
print($blah)
prints ‘yep’.

[li]$macro m([x [, y [, …n]]]) z[/li]Will replace all occurrences of $m([x [, y [, …n]]]) with z. z can use the arguments passed to m by using the dollar sign and then the argument’s name.
Example use:
$macro PRINT(x, y) print($x, $y)
$PRINT(‘hi’, ‘lol’)

[li]$include “path/to/file”[/li]Will replace that line with the contents of “path/to/file”. Paths are relative to the script that is preparing to be run. This is not what lua’s built in ‘include’ function does, which just runs the lua file, this actually copies and pastes the text into the place of the $include, like C++'s #include.
[/ul]
[lua]
require(‘preproc’)

local MAX_RECURSIONS = 20

local function ErrMsg(file, line, msg)
ErrorNoHalt(’[’ … file … ‘:’ … line … '] ’ … msg … ’
')
end

local function startsWith(text, starts)
if (string.sub(text, 1, string.len(starts)) == starts) then
return true
end

return false

end

local function replaceSub(str, start, finish, replacement)
return string.sub(str, 1, start - 1) … replacement … string.sub(str, finish + 1, string.len(str))
end

local function countOccurrences(str, pattern)
local count = 0
local b, e = string.find(str, pattern)

while b and e do
    count = count + 1
    b, e = string.find(str, pattern, e)
end

return count

end

local function extractArgs(str)
local args = {}

local start = 1
local sub
for i = 2, str:len() do
    if str:sub(i, i) == ',' then
        if str:sub(i - 1, i - 1) ~= '\\' then
            sub = str:sub(start, i - 1):gsub('\\,', ','):Trim()
            table.insert(args, sub)
            start = i + 1
        end
    end
end

sub = str:sub(start):gsub('\\,', ','):Trim()
table.insert(args, sub)

return args

end

local function baseDir(path)
local sep = path:find(’/’) and ‘/’ or ‘\’
local split = sep:Explode(path)

table.remove(split)

return sep:Implode(split)

end

local substituteArgs

local function preprocess(file, path, text, dict, linenum)
local lines = string.Explode(’
', text)
local dict = dict or {}
local toremove = {} – remove these indices after we’re done

local changed = false

local savelinenum = linenum or 0

for i = 1, MAX_RECURSIONS do
    for k,v in pairs(lines) do
        if startsWith(string.Trim(v), '$define') then
            local word, replacement = string.match(v, '$define%s+(%w+)%s+(.*)')
            
            if word and replacement then
                dict[word] = {type = 'define', data = replacement}
            end

            if i == 1 then
                table.insert(toremove, k) -- remove the line or it will error because it's not valid lua
            end
        elseif startsWith(string.Trim(v), '$macro') then
            local name, args, body = string.match(v, '$macro%s+(%w+)%s*%((.*)%)%s+(.*)')

            if name and args and body then
                dict[name]= {type = 'macro', args = extractArgs(args), body = body}
            end

            if i == 1 then
                table.insert(toremove, k)
            end
        elseif startsWith(string.Trim(v), '$include') then
            local incfile = string.match(v, '$include%s*"(.+)"')
            local dir = baseDir(file)
            local path = dir .. '/' .. incfile

            local text = _G.file.Read("../" .. path)

            lines[k] = preprocess(incfile, '', text, dict)
        else
            -- first check for macros
            local start, fin, id, args = string.find(v, '$(%w+)%s*%((.*)%)')

            if start and fin and id and args then
                args = extractArgs(args)
                local entry = dict[id]

                if entry and entry.type == 'macro' then
                    if table.Count(entry.args) == table.Count(args) then
                        local data = substituteArgs(entry.body, args, entry.args, dict, file, k)
                        if type(data) == 'boolean' then
                            return data
                        end

                        v = replaceSub(v, start, fin, data)
                        lines[k] = v

                        changed = true
                    else
                        if entry then
                            ErrMsg(file, linenum or k, 'macro `' .. id .. '` expects ' .. table.Count(entry.args) .. ' arguments, not ' .. table.Count(args))

                            return false
                        end
                    end
                end
            end

            -- next check for defines
            local start, finish = string.find(v, '$%w+')
            
            if start and finish then
                local id = string.sub(v, start + 1, finish)
                local entry = dict[id] -- start + 1 because we want to omit the $

                if entry and entry.type == 'define' then
                    v = replaceSub(v, start, finish, entry.data)
                    lines[k] = v

                    changed = true
                else
                    if not entry then
                        ErrMsg(file, linenum or k, 'unknown preprocessor definition `' .. id .. '`')
                    else
                        ErrMsg(file, linenum or k, 'illegal use of macro as definition')
                    end

                    return false -- halt lua
                end

                start, finish = string.find(v, '$%w+')
            end
        end

        linenum = linenum + 1
    end

    linenum = savelinenum

    if changed then
        changed = false
    else
        break
    end
end

for k,v in pairs(toremove) do
    lines[v] = '' -- we don't do table.remove because we want to keep the line numbers intact for errors
end

return string.Implode('

', lines)
end

substituteArgs = function(str, args, replacements, curdict, filename, linenum)
local locals = {}

for k,v in pairs(replacements) do
    locals[v] = {type = 'define', data = args[k]}
end

local saves = {}

for k,v in pairs(locals) do
    saves[k] = curdict[k]
    curdict[k] = v
end

local ret = preprocess(filename, '', str, curdict, linenum)

for k,v in pairs(saves) do
    curdict[k] = v
end

return ret

end

hook.Add(‘Lua_Preprocess’, ‘preprocessor’, preprocess)
[/lua]

So that after running the above, this script:
[lua]
$macro ERRMSG(x) ErrorNoHalt($x … "
")
$define asdf 2
$define asdf ‘nope’
ERRMSG($asdf)
[/lua]
prints



nope

With the added newline.

These are just some quick examples, much more is possible,.

Please let me know if there are any bugs.

Download: Windows Binary | Linux Binary | Linux Makefile | Source
Thanks to Chris@ster for compiling the linux binary.

Updates:
[ul]
[li]Updated on 1/5/2011 to unreference ILuaObject*s properly, thanks to Chris@ster[/li][/ul]

Sooo, if we load this in the menu state, can I use RawRunString?
Would be nice since garry removed RunString from the menustate(in case it was there before).

Wow, nice work.

http://img5.imagebanana.com/img/9s9jx4ts/excellent.jpg

Excellent!

Oh hell yes this is awesome!

[editline]5th January 2011[/editline]

[lua]
function test()

using string

local b = byte("A")
print(b++,b,++b,char(b))

end

test()
[/lua]



] lua_openscript test.lua
Running script test.lua...
65	66	67	C


I love you yakahughes:3:

What about decrement operator, oh wait.

Wow…

I’m sorry but couldn’t this just as easily be achieved by overwriting include?

[lua]
local OldInclude = include

local LuaPaths = {
“lua/%s”,
#PATH#/%s”,
“lua/#PATH#/%s”
}

function include( s, b )
if( b ) then s = “…/” … s end

local WorkingDir = debug.getinfo( 2, "S" ).short_src:match( "(.+)/.*" );
local FileName = s:match( ".+/(.*)" )

for k, v in pairs( Paths ) do
    local Data = file.Read( "../" .. string.format( v:gsub( "#PATH#", WorkingDir ), s )
    if( Data ) then
        local Ret = hook.Call( "Lua_Preprocess", GAMEMODE, FileName, WorkingDir, Data )
        if( type( Ret ) == "boolean" ) then
            return Ret
        elseif( type( Ret ) == "string" ) then
            RunString( Ret )
        else
            RunString( Ret )
        end
        return 
    end
end

end[/lua]

Same could be done with RunString.

Some applications of this wouldn’t be so useful if you overwrote functions, and it’s best not to rely on being able to overwrite functions in the first place.

include doesn’t run many things, such as stuff run with lua_openscript, SendLua, or files in the autorun directory, or any of the files that aren’t run with include like SWEPS and whatnot. You can also include this in the menu environment for absolute control of everything, whereas you can’t do that with overwriting functions. Also, 35.

[editline]5th January 2011[/editline]

That would work fine, but you couldn’t use lua comments any more and it would mess up your syntax highlighting.

[lua]
$define _DECR 1
local a = 10
print(–a)
[/lua]

Linux binary uploaded, many thanks to Chris@ster.

So your going to have to explicitly disable – comments for scripts that want to use C style decrement? Why not just use a different character sequence?

I guess you could also do -=, +=, *=, /=, %=, and the lot of them as well, but without the comment problem. While a-- would be best, a -= 1 is still better than a = a - 1.

a~~? :v:

Brilliant.

[editline]5th January 2011[/editline]

Perhaps another good one would be :=, so that

a := sub(2, 3)

becomes

a = a:sub(2, 3)

I want to make stuff with this, but wouldn’t errors show the wrong line number? And how would you deal with this?

pcall the code then do the error handling yourself maybe?

Does this get called for lua run in other lua states? Like, if you hooked this in the menu environment, could you dump all lua loaded from a server?

-snip-