Minifer Testing

Hi everyone, as you may have seen in the WAYWO and some other places I have been working on a glua minifier. This minifier aims to minify a file to the smallest size it can; in other words this is not a obfuscator and should not be used as such.

I need any help I can get for breaking my minifier in the following ways:

  • internal errors (these will be reported internally)
  • parsing errors (these you might have to post on the thread)
  • unwanted code breakage (for example print “something” is removed in the output) (these will also have to be posted here)

No code is stored serverside besides ones that break the backend, and when it is stored it will be only for internal use for developing this further.

https://www.securegmod.com/minifier/

All processed code is free to use however you feel, as long as it’s not malicious to players of garry’s mod.

The site is still a WIP and is just a testing ground. Right now if you resize the window some things will break. That’s already known.

The following things have been fixed thanks to notcake:



Fixed self being considered global when it's inside of a method (fixes local variables named self breaking code generation)
Fixed number collapsing to infinity or nan turning into "Infinity" or "NaN"
Fixed left-side parenthesis missing after rebuilding code on certain expressions
Fixed logic expressions losing their meaning when chained with parenthesis

Added goto and label support


Other things fixed:



Fixed some internal variables not using Dictionary to store user input


I dumped some of my obfuscated code in it and it didn’t seem too happy. Now that I think about it, I’m not sure that code worked in the first place. Will throw more disgusting code at it later.

it still doesnt like 1mb lua files

Couldn’t get the website to work in Firefox, IE, or Pale [DEL]Meme[/DEL] Moon

It’s time to use a real browser then

It’s not working for me either in chrome. I think it’s something with the site. Not even the code already in there when the page loads works with it. I click submit and nothing happens.

The backend broke for some reason. I added a logging file for it now in case it happens again.

Thanks for the broken code and snide remarks Parakeet :slight_smile:

[editline]10th December 2016[/editline]

That file you sent me is rejected by the website software itself. Probably a setting I have to modify to fix it, but just try not to use 1mb files instead :slight_smile:

Is there a good reason to minify code other than a smaller file size? You’d think it would take longer to run code.

Gmod useing something called luaJIT (lua just-in-time [compiled]), as long as the minifed code gives the same binary as any non-minified code, it will be just as fast. Not only that, but since the file is shorter to parse (character-wise), it is (usually) also faster to load.

Smaller file size is the goal here, as well as the regex language to make it even smaller by discarding private members and identifiers.

Smaller file size will end up making it faster to download on clients and take up less disk space. However some tricks I use in this (such as indexing a table for longer indexes to be indexed) will slow the speed of execution by a tiny percent.

Thanks to parakeet the following is fixed:



Fixed javascript's regex not matching in the Combine function due to utf8


I don’t have access to gmod at the moment, but I’m testing via https://www.lua.org/cgi-bin/demo and thought up this weirdness:
[lua]local obj1, obj2 = {}, {}
obj1.DoSomething = function(arg)
print(“foo”)
return arg or obj1.DoSomething – Return itself
end
obj2.DoSomething = function(arg)
print(“bar”)
return arg or obj2.DoSomething – Return itself
end
local mt = {__add = function(a, b) return a end}
setmetatable(obj1, mt)
setmetatable(obj2, mt)

local somefunc = obj1.DoSomething;
(obj2 + obj1).DoSomething()

print(obj1.DoSomething == somefunc)[/lua]
Output:
bar
true

Remove the semicolon:
[lua]local obj1, obj2 = {}, {}
obj1.DoSomething = function(arg)
print(“foo”)
return arg or obj1.DoSomething – Return itself
end
obj2.DoSomething = function(arg)
print(“bar”)
return arg or obj2.DoSomething – Return itself
end
local mt = {__add = function(a, b) return a end}
setmetatable(obj1, mt)
setmetatable(obj2, mt)

local somefunc = obj1.DoSomething
(obj2 + obj1).DoSomething()

print(obj1.DoSomething == somefunc)[/lua]
Output:
foo
bar
false

With semicolon minified:
[lua]local
={=“DoSomething”}local
a,b={},{}a[.]=function(b)print"foo"return
b||a[.]end
b[.]=function(a)print"bar"return
a||b[.]end
local
c={__add=function(a,b)return
a
end}setmetatable(a,c)setmetatable(b,c)local
c=(a.).print(a[.]==c)[/lua]
Output (manually replace || with or):
foo
bar
false

This doesn’t match the original output.

Without semicolon minified:
[lua]local
={=“DoSomething”}local
a,b={},{}a[.]=function(b)print"foo"return
b||a[.]end
b[.]=function(a)print"bar"return
a||b[.]end
local
c={__add=function(a,b)return
a
end}setmetatable(a,c)setmetatable(b,c)local
c=(a.).print(a[.]==c)[/lua]
Output (manually replace || with or):
foo
bar
false

This is a rare case where the semicolon matters. The minifier DOES change the code slightly if it exists, but incorrectly so.

[editline]10th December 2016[/editline]

Also, does GLua accept this syntax?
[lua]local obj = {}
function obj"HUH"
print(“huh.”)
end

obj.HUH()[/lua]

Because the minifier generates it and lua demo does not accept it.

[lua]local obj1, obj2 = {}, {}
function obj1.DoSomething(arg)
print(“foo”)
end
function obj2.DoSomething(arg)
print(“bar”)
end

obj1.DoSomething()
obj2.DoSomething()[/lua] Becomes [lua]local
={=“DoSomething”}local
a,b={},{}function
a.print"foo"end
function
b.print"bar"end
a.b.[/lua]

and lua demo says:
input:4: ‘(’ expected near ‘[’

changes thanks to neatnit:



fix wrong generation of some code involving semicolons, fix double parenthesis
disallow CHANGE_BIG_INDICES to change function declaration identifiers


Other changes



remove console.log
remove another console.log
remove console.log... again


-snip-

I made a quick sublime plugin for this. My first one, too. There are no error checks because I’m lazy.

Place in C:\Users\YourName\AppData\Roaming\Sublime Text 3\Packages\User and create a key bind by going to Prefereces > Key Bindings and adding the entry below.


{ "keys": ["alt+o"], "command": "sgminify"},

Change alt+o to whatever you’d like it to be. Use it by highlighting the text you’d like to minify and pressing the bind.

I just checked, the semicolon is still ignored. Same code as before:
[lua]local obj1, obj2 = {}, {}
obj1.DoSomething = function(arg)
print(“foo”)
return arg or obj1.DoSomething – Return itself
end
obj2.DoSomething = function(arg)
print(“bar”)
return arg or obj2.DoSomething – Return itself
end
local mt = {__add = function(a, b) return a end}
setmetatable(obj1, mt)
setmetatable(obj2, mt)

local somefunc = obj1.DoSomething;
(obj2 + obj1).DoSomething()

print(obj1.DoSomething == somefunc)

– becomes:

local
={=“DoSomething”}local
a,b={},{}a[.]=function(b)print"foo"return
b||a[.]end
b[.]=function(a)print"bar"return
a||b[.]end
local
c={__add=function(a,b)return
a
end}setmetatable(a,c)setmetatable(b,c)local
c=a..print(a[.]==c) – WRONG

– last line should have been:
c=a[.];(b+a).print(a[.]==c)[/lua]

Still different output (on lua demo with || manually replaced with or)

Thanks to NeatNit:



remove matching in Combine when generating semicolons


Thanks to ShinyCow:



Fix ForGenericLoop generating ass


(to expand on this it generated the following)
[lua]
for,k,v
in
next
pairs_table
nil
do
SendValue(k)SendValue(v)end
return
num
end
[/lua]

Thanks to someone who put in a lua cheat:



Updated edge case in a few functions


[lua]local a = {}

a.LongIndex = a.LongIndex * a.LongIndex + a.LongIndex[/lua]
… Several iterations of minifying minified code later …
[lua]local
,a,b,c,d,e,f,g,h,i,j,k,l,m,n,o={},{},{},{},{},{},{},{},{},{},{},{},{},{},{=“LongIndex”},{}o[n.]=o[n.]*o[n.]+o[n.][/lua]
:v:

Edit: further attempts to break the minifier. I’m not too sure about this one.

The input code is actually bad syntax - it’s ill-defined. We declare and define two local variables with the same name in the same line:
[lua]local a, a = SomeGlobalFunction()
print(a)[/lua]
This can go one of two ways - a would be given either the first or the second return value of SomeGlobalFunction(). Obviously this is horrible coding and should never happen in the real world. However, I expect the minifier to preserve the idiocy. However, it instead produces:
[lua]local
_,a=SomeGlobalFunction()print(a)[/lua]
Now, the first and second a variables were given 2 different names, with the latter being the one used later on. This does match the behavior I’m getting when testing (in lua demo), but arguably, the minified code should have been: local , = SomeGlobalFunction().

this was fixed 2 days ago after you posted it! thanks!

since we are only targetting luajit and luajit behaves as such I think it won’t be changed. Thanks for the interesting thought though!

As always everyone, if you find something I’d love to fix it! I believe it’s nearing completion of the current features. I have some more features planned ahead such as collapsing all assignment statements and changing
[lua]
function Something:DoAThing()
self.DoAThing_2 = true
end
– to
Something.DoAThing=function(a)a.DoAThing_2=!!1
end
–instead of
function Something:DoAThing()self.DoAThing_2=!!1
end
[/lua]

This was pointed out to me by NeatNit that it can be shortened by >0 bytes if self is used inside the function at least once.