Vanillin [Testers Needed] - Use regular Lua binary modules with GMod (LPeg, LuaSocket, LuaMD5 sample

[Infectious]

After several months of work (twice), my latest work is ready...Vanillin. With Vanillin, you can build Lua binary modules that use the regular Lua C API, for GMod, [I]with no modifications to the source code needed[/I]. It provides implementations of every Lua C API function using the underlying GMod C API (and a lot of black magic round-aboutness and magical incantations). Whether you're rebuilding existing modules for GMod, or simply building new modules using a more consistent and easily-testable API, library writers are sure to find some use for this.

As it is, it's basically ready for release, and I've tried to clean it up as much as possible. I've given it some testing with LPeg, LuaSocket, and LuaMD5 (pictures below, builds of those libraries included) and I'd like testers who can both try out the features of those libraries as much as possible, but even moreso, I'd like testers who can try it with as many other libraries as possible in order to really make sure that Vanillin works widely. If you use Vanillin or one of the sample libraries included with it, reply here with how it works out! I'll try to get out an updated build within a day or two of learning about any new problems.

Basic knowledge of the build process will be necessary to use Vanillin.

[I]Edit: Should've updated this text months ago...the sources ARE currently included.[/I]

[U][B]Features[/B][/U]
No modification to sources! Vanillin takes the form of a static library, [I]lua51.lib[/I], which is a drop-in replacement for the normal Lua 5.1 DLL's import library [I]lua51.lib[/I]. Just re-link your program against that, give the files appropriate filenames to fit into GMod's library-naming scheme, and you're good to go!

Supports every function except the following:
[I]lua_newstate()[/I]
[I]lua_close()[/I]
[I]lua_atpanic()[/I]
[I]lua_checkstack()[/I] (kinda? read the notes)
[I]lua_dump()[/I]
[I]lua_getallocf()[/I]
[I]lua_setallocf()[/I]
As well, the thread-manipulation functions are not supported currently, but may be in the future.

Aside from the ones listed, every function is supported, and matches "proper" behavior as closely as possible! Regular and light userdata are supported correctly, function upvalues are supported correctly, execution stack info is supported correctly, pseudoindices are supported correctly, [B]everything [/B](but check the instructions that come with the package for more details).

[B][U]Demonstration[/U][/B]
[URL="http://i.imgur.com/iEPT8.png"]A test[/URL] using the [URL="http://www.inf.puc-rio.br/~roberto/lpeg"]LPeg[/URL] library that searches through every serverside Lua file, checking for any occurrence of a Garry syntax extension to Lua, noting the file, line, and type.
[URL="http://i.imgur.com/344xu.png"]A test[/URL] using the [URL="http://www.keplerproject.org/md5"]MD5[/URL] library created as part of the Kepler Project. It goes through every serverside Lua file and calculates a hash of it.
[URL="http://i.imgur.com/AZ79V.jpg"]A test[/URL] using both a GMod Lua and Vanilla Lua version of the [URL="http://w3.impa.br/~diego/software/luasocket"]LuaSocket[/URL] library (and my own LuaWinCon library) that sends basic info regarding the gamemode name, map name, and some basic info about every player and NPC on the server, every tick, to somewhere else on the internet.

[B][U]Download[/U][/B]
[URL="http://www.mediafire.com/?mzmc1yahnxc09nb"]Direct Download.[/URL]
Latest version: Full Release 1
Works with: Garry's Mod 13 Release ([B]BUT[/B], do read "Known Issues" below)
[B][U]
How To Use[/U][/B]
Instructions are included with the download.

[B][U]Known Issues[/U][/B]
Technically not a Vanillin issue, GMod 13's [I]require()[/I] function currently does not work with the Lua portions of many of the libraries you might try to get working with Vanillin. I've created a compatibility fix for this, which you can get [URL="http://www.facepunch.com/showthread.php?t=1226065"]here.[/URL]

[HR][/HR]
[B][U]Older Versions[/U][/B]
The first version I worked on was for GMod 12, and right as it was nearly finished, GMod 13 came out with a new API and I stopped working on it. It's about 99% finished though, if in rather rough state, and since I'm not working on it anymore, I'm offering up the GMod 12 version of Vanillin, the sample libraries built with it (again, LPeg, LuaMD5, and LuaSocket), and the source code. I'm not supporting this library anymore, so I won't make any updates to it, but feel free to ask questions about how it works.
[URL="http://www.mediafire.com/?vimje4iyf5a07f5"]Download GMod 12 Vanillin here.[/URL] [I]Edit: fixed the package, it was missing a few files.
[/I][I]Edit 2: And obviously, you cannot actually do anything with GMod 12 Vanillin anymore, it's just here for historical purposes.[/I]
[U]
[B]Older Demonstration[/B][/U]
These are just from the unsupported GMod 12 version:
[URL="http://i.imgur.com/ANYDn.png"]A simple test[/URL] of LPeg.
[URL="http://i.imgur.com/TYm75.png"]A simple test[/URL] of my LuaXInput library.
A simple test of the [URL="http://i.imgur.com/yv4Yr.jpg"]MD5[/URL] and [URL="http://i.imgur.com/oOla6.jpg"]DES56[/URL] libraries.
[URL="http://cloud.steampowered.com/ugc/578964650873313240/4B60C795B6668D7FE3638B831962B1DD7849FA23/"]Wired Socket Radio[/URL], running using its original LuaSocket for the first time in many years.

[Map in a box]

Neat, I remember making something like this. I only made it to port luaffi and didn't get it. If you could port that and release it, viola the most useful module in gmode.

[hakusaro]

I take it this means that you can use things like [url=https://github.com/nrk/redis-lua]lua-redis[/url] with it because of being able to import any old lua module? I'm impressed.

[Infectious]

[QUOTE=nicatronTg;37964385]I take it this means that you can use things like [url=https://github.com/nrk/redis-lua]lua-redis[/url] with it because of being able to import any old lua module? I'm impressed.[/QUOTE]

Well, I looked through the code for lua-redis, and its only dependency appears to be on LuaSocket (which I already provide a build of with Vanillin, for testing). As well, it doesn't seem to use the 'io' or 'os' libraries.
If you can make it fit into GMod's execrable require() system, or you're using your own custom-implemented one in your custom gamemode, then yeah, lua-redis should work quite nicely, though its example scripts or test suites might be another matter.

[Infectious]

Just noticed that the GMod 12 Vanillin package was missing a critical file from the sources, uploaded a fixed version.

[CapsAdmin]

Great work.

And luasocket!!

I've made a wrapper for luasocket if someone's interested. It's of course non blocking and intended to be easy to use.

[url]https://github.com/CapsAdmin/oohh/blob/master/mmyy/lua/includes/standard/libraries/luasocket.lua[/url]

The library it depends on is similar to glua so it should be easy to port. (I'll probably do it myself)

here's kinda what it looks like.

[lua]do -- UDP
local server = luasocket.Server("udp")
server:Host("10.0.0.1", 888)

function server:OnReceive(data, client) -- maybe make a dummy client ?
self:Send("hi", ip, port)
end

local client = luasocket.Client("udp")
client:Connect("10.0.0.1", 888)
client:Send("hello")
client:Send("hello")
client:Send("hello")
client:Send("hello")
end

do -- TCP
local server = luasocket.Server("tcp")
server:Host("10.0.0.1", 555)

function server:OnClientConnected(client)
printf("client connected with ip %s", client:GetIP())
end

function server:OnReceieve(data, client)
self:Send("hi", client:GetIP())
end

local client = luasocket.Client("tcp")
client:Connect("10.0.0.1", 555)
client:Send("hello\n")
client:Send("hello\n")
client:Send("hello\n")
client:Send("hello\n")

function client:OnReceieve(data)
print(data)
end
end[/lua]

You could also see the http.Get example in the source.

[Python1320]

LINUX

[Infectious]

[QUOTE=Python1320;37996592]LINUX[/QUOTE]

Why?

I asked every developer I knew and none of them even [i]knew[/i] anyone who'd ever run a GMod server on Linux (it does not even have a client to speak of). I have an intense suspicion with a fair bit of evidence to support it that Vanillin is superfluous on GMod-for-Linux because you can accomplish the same stuff with just a linker due to the nature of 'lua_shared.so' (which I'll explain in a bit). But I couldn't find [i]anyone[/i] who could possibly test it, and as neither myself nor my gamemode (which revolves mostly around a singleplayer campaign) would benefit from it in any way, why go through all the effort?

If you'd really like to try and get regular Lua binary modules working on Linux, I'll tell you what I've discovered so far, from my initial research into the problem for which Vanillin eventually became the solution. Do with the information what you will.

[b]I have not been able to confirm any of the following, so read the following with that in mind:[/b]
From what I've been able to tell, it looks like all of the Lua C API functions are exported from 'lua_shared.so' file (which you can locate in the GMod 12 and GMod 13 GCFs). Simply build your Lua file as normal and link it against that, and put a line in the linker script creating an exported symbol 'gmod13_open' which is merely an alias for the library's 'luaopen_<whatever>' symbol.

This solution will not work in GMod 12, as gmod_open() is not argument-compatible with luaopen_<whatever>(), unlike gmod13_open(), but then, GMod 12 will not be supported for much longer anyway, so who cares.

[lorde banana]

Linux is the only OS capable of running a GMod server that can defend itself against some of the endless attacks you will inevitably receive

[Python1320]

[QUOTE=Infectious;38010097]Why?

I asked every developer I knew and none of them even [i]knew[/i] anyone who'd ever run a GMod server on Linux (it does not even have a client to speak of). I have an intense suspicion with a fair bit of evidence to support it that Vanillin is superfluous on GMod-for-Linux because you can accomplish the same stuff with just a linker due to the nature of 'lua_shared.so' (which I'll explain in a bit). But I couldn't find [i]anyone[/i] who could possibly test it, and as neither myself nor my gamemode (which revolves mostly around a singleplayer campaign) would benefit from it in any way, why go through all the effort?

If you'd really like to try and get regular Lua binary modules working on Linux, I'll tell you what I've discovered so far, from my initial research into the problem for which Vanillin eventually became the solution. Do with the information what you will.

[b]I have not been able to confirm any of the following, so read the following with that in mind:[/b]
From what I've been able to tell, it looks like all of the Lua C API functions are exported from 'lua_shared.so' file (which you can locate in the GMod 12 and GMod 13 GCFs). Simply build your Lua file as normal and link it against that, and put a line in the linker script creating an exported symbol 'gmod13_open' which is merely an alias for the library's 'luaopen_<whatever>' symbol.

This solution will not work in GMod 12, as gmod_open() is not argument-compatible with luaopen_<whatever>(), unlike gmod13_open(), but then, GMod 12 will not be supported for much longer anyway, so who cares.[/QUOTE]

You are right, there's so much exported now my mind is about to explode.

It wasn't exactly the same in GMod12:

~/srcds/orangebox/garrysmod/bin> nm lua_shared.so  | fgrep lua_
nm: lua_shared.so: no symbols

GMod 13:

srcds  ~/srcds/orangebox/garrysmod/bin> nm lua_shared.so  | fgrep lua_
000707b0 T glua_bitwise
0004fb00 t lua_GetUserType
00066a20 T lua_atpanic
00067860 T lua_call
000682b0 T lua_checkstack
0005ae80 T lua_close
000672b0 T lua_concat
000675f0 T lua_cpcall
00067bd0 T lua_createtable
00067540 T lua_dump
00068060 T lua_equal
00067410 T lua_error
00067430 T lua_gc
00066f90 T lua_getallocf
00066ee0 T lua_getfenv
00067ce0 T lua_getfield
00058a50 T lua_gethook
00058a70 T lua_gethookcount
00058a60 T lua_gethookmask
00059880 T lua_getinfo
00059cc0 T lua_getlocal
00066e80 T lua_getmetatable
00058a80 T lua_getstack
00067d50 T lua_gettable
00066a40 T lua_gettop
00066fe0 T lua_getupvalue
000da680 R lua_ident
..snip..

Thanks for the heads up

[Infectious]

Alright, back at work on this after having some real-life stuff to deal with.
It seems GMod 13 Release Version has broken the require() system again, so I'm trying to work out a non-interfering fix for that so that the supplied demonstration modules will run correctly, if all goes well I'll try to have that up by the end of the weekend.
[I]
Update: I've got a fix, just need a bit more time to test it.
[/I][I]Update 2: Got the fix working, just need to wait on someone else to build the LuaSocket binaries and I'll upload the latest build of Vanillin (with sample libraries) as well as the fix to make require() work.
Update 3: Well, the require() fix is working flawlessly but the test scripts for LPeg and LuaSocket have revealed some unusual new problems with Full Userdata that have appeared since the game went live. It's looking like it might take a few days to track down, so the release I've been promising will have to wait that much longer.[/I]

[Infectious]

There, it's now updated to work with the new Full Userdata behavior. The test libraries have all been rebuilt as well. If Vanillin can go a week without anyone finding any errors in it, I will declare it release-ready and start releasing the source code along with it.

[B]IMPORTANT:[/B] LuaMD5 and LuaSocket both depend on a correctly-functioning require() function, and require() does not work anymore. An actual fix for this is both outside the scope of Vanillin, and impossible to reliably do from within Vanillin anyway. To get them to work (as well as any other funciton that relies on require()'s correct behavior), you will have to install a fix I've created, which you can get [URL="http://www.facepunch.com/showthread.php?t=1226065"]here.[/URL]

[Infectious]

Since I haven't found or heard of any bugs yet, I've uploaded the full-release version which includes all of the sources and build scripts (suitably cleaned-up for public viewing). As before, do remember to get [url="http://www.facepunch.com/showthread.php?t=1226065"]the [i]require()[/i] fix[/url], or many of the sample libraries will not work.