netStructures - net.Read/WriteTable replacement

[G4MB!T]

net.ReadTable and net.WriteTable are functions in the Garry's Mod net library tasked with networking a table of arbitrary data. Any Garry's Mod programmer worth their salt will know that these functions suck. They send more data than is really needed due to the fact that Lua and Garry's Mod cant know the structure of a table.

Enter [B]netStructures[/B]. netStructures introduces a way to send and receive tables of predefined structure in super friendly way.

Why would you want to predefine your table structures for networking? Strict, consistent structure in code is important whether its for networking or not. It can also help immensely with debugging. If the structure of the table you are sending is known, it means the type information that net.Read/Write table use can be omitted, thus only sending the data that is needed and reducing network data.

netStructures works by having the user "[I]register[/I]" a table structure. This structure can then be used with the [I]net.WriteStructure[/I] and [I]net.ReadStructure[/I] functions. These 2 functions are almost a copy/paste replacement for [I]net.Read/WriteTable[/I].

[B]Preamble: netStructures Type Enums[/B]
In order for netStructures to know which keys in your table correspond to which values, netStructures defines 12 global integral values for primitive types commonly used in networking:

STRUCTURE_STRING
STRUCTURE_ANGLE
STRUCTURE_VECTOR
STRUCTURE_COLOR
STRUCTURE_ENTITY
STRUCTURE_BIT
STRUCTURE_NUMBER -- Generic; floating point
STRUCTURE_INT8 -- Integer values between -128 and 127
STRUCTURE_INT16 -- Integer values between -32768 and 32767
STRUCTURE_INT32 -- Integer values between -2147483648 and 2147483647
STRUCTURE_UINT8 -- Integer values between 0 and 255
STRUCTURE_UINT16 -- Integer values between 0 and 65535
STRUCTURE_UINT32 -- Integer values between 0 and 4294967295

[B]Step 1: [/B] Registration
The first step to using netStructures is to register your table structure:

net.RegisterStructure("my_struct_weps", {
    name = STRUCTURE_STRING,
    ammo = STRUCTURE_NUMBER
})

The code above says to create a new structure called "my_struct_weps" with 2 fields: a [B]name[/B] [I]string[/I] and an [B]ammo[/B] [I]number[/I]. When this structure is used, netStructure will enforce these types on the values provided by the table.

[B]Step 2: [/B] Write the structure
Writing structures is almost identical to [I]net.WriteTable[/I]:

net.WriteStructure("my_struct_weps", {
    name = "pistol",
    ammo = 10
})

The first argument in [I]net.WriteStruct[/I] tells netStructs which structure to use. In this example we are using the structure we created before.

When [I]net.WriteStruct[/I] is called, it will make sure that the name you have provided is an actual struct, then it will make sure that the table you have provided matches the structure.

[B]Step 3: [/B] Reading the structure
Like writing, reading structures is almost identical to [I]net.ReadTable[/I]:

local wep = net.ReadStructure("my_struct_weps")

[I]wep[/I] will be a table containing [B]name = "pistol", ammo = 10[/B].

netStructures provides fixed-width integer types which are not wrapped. If you provide a value that is outside of the range specified in the structure, it will error.

netStructures allows fields to reference other structures:

net.RegisterStructure("my_struct", {
    name = STRUCTURE_STRING,
    weapon = "my_struct_weps"
})

The above code creates a new structure where [I]weapon[/I] refers to the structure we created earlier. This is known as a [B]Structure Reference[/B]. This new structure can be networked like so:

net.WriteStructure("my_struct", {
    name = "Gambit",
    weapon = {
        name = "pistol",
        ammo = 10
    }
})

netStructures also lets structures define sequential arrays. Arrays must be homogeneous and use the STRUCTURE_* enums and reference other structures.

net.RegisterStructure("my_struct", {
    name = STRUCTURE_STRING,
    weapons = {"my_struct_weps"} -- denotes that `items` is a sequential array of the "my_struct_weps" structure
})

net.WriteStructure("my_struct", {
    name = "Gambit",
    weapons = {
        {
            name = "pistol",
            ammo = 10
        }
    }
})

With the complete sample program, here are some results:

net.RegisterStructure("my_struct_weps", {
    name = STRUCTURE_STRING,
    ammo = STRUCTURE_NUMBER
})

net.RegisterStructure("my_struct", {
    name = STRUCTURE_STRING,
    items = {"my_struct_weps"} -- denotes that `items` is a sequential array of the "my_struct_weps" structure
})

if (SERVER) then
    util.AddNetworkString("my_struct_nws")
    util.AddNetworkString("my_struct_nws_tbl")
    
    function netStructureTest(ply)
        net.Start("my_struct_nws")
            net.WriteStructure("my_struct", {
                name = "Gambit",
                items = {
                    {
                        name = "pistol",
                        ammo = 10
                    }
                }
            })
        net.Send(ply)
        
        net.Start("my_struct_nws_tbl")
            net.WriteTable({
                name = "Gambit",
                items = {
                    {
                        name = "pistol",
                        ammo = 10
                    }
                }
            })
        net.Send(ply)
    end
else
    net.Receive("my_struct_nws", function(l)
        print("net.ReadStructure", l)
        PrintTable(net.ReadStructure("my_struct"))
    end)
    
    net.Receive("my_struct_nws_tbl", function(l)
        print("net.ReadTable", l)
        PrintTable(net.ReadTable())
    end)
end

net.ReadStructure	208
items:
		1:
				ammo	=	10
				name	=	pistol
name	=	Gambit
net.ReadTable	512
items:
		1:
				ammo	=	10
				name	=	pistol
name	=	Gambit

So we can see almost 2.5x improvement on size. Other tables and structures I have tested have yielded upwards of 3.5x so your mileage may vary.

It was suggested to allow the creation of custom "types" or at least have more control over the sending and receiving of types. I thought about it originally, however, I could not think of a valid use case where the primitives provided by netStructures would not suffice.

Source: [url]https://bitbucket.org/GamerGambit/netstructures[/url]

If you have any bugs, errors or problems please let me know.

[meharryp]

This is pretty cool, I got a 1.8x improvement, decreased my test table from 247kb to 138kb

[editline]a[/editline]

Might want to note that this was with a hack that let me write tables inside of a structure, since some tables I use don't have a consistent structure

[G4MB!T]

The idea behind this is to make your data consistent. There is no situation where writing arbitrary tables of data is better than knowing the structure before hand.

[conna]

Fantastic job, is this MIT licensed?

[G4MB!T]

It has no licence so it doesnt really matter. Do what you want.

[holon]

[QUOTE=G4MB!T;52625049]It has no licence so it doesnt really matter. Do what you want.[/QUOTE]

Would still license it, makes it easier to know where you stand without having to look too deeply throughout repo. MIT has no restrictions anyway.

[G4MB!T]

I'll put one up shortly.

[Zeh Matt]

[QUOTE=G4MB!T;52621914]The idea behind this is to make your data consistent. There is no situation where writing arbitrary tables of data is better than knowing the structure before hand.[/QUOTE]

There might by data that is conditional so there are reasons. I have a single message with a command byte, based on that more data might follow. I wouldn't want a static structure here.

[G4MB!T]

Ye thats definitely true. I guess its not a 100% replacement but I thought it was at least better.

[Luni]

[QUOTE=Zeh Matt;52637411]There might by data that is conditional so there are reasons. I have a single message with a command byte, based on that more data might follow. I wouldn't want a static structure here.[/QUOTE]

In that specific instance I think you could check the byte and read the appropriate structure, but I've had similar situations which take more work to refactor -- like networking items, where a bool in the middle tells you if it's in an inventory (read x and y bytes) or a world item (read entindex short).

Although, come to think of it, that might be a bad design. Hmm. :thinking:

I'm starting to realize how error-prone dynamic message formats are.

[G4MB!T]

netStructures doesn't allow for conditional data unfortunately. I could add it if there was a big enough demand and I would need to think about how it would work.

[G4MB!T]

I suppose it would work if you wrote separate structures based on some condition but it wouldnt work (at the moment) within a structure.

[G4MB!T]

Just an update for anyone still following this, the repo has been moved to https://github.com/GamerGambit/netStructures which now includes an MIT licence and a fix for worldspawn not being able to be sent. Not sure why you would want that but you can do it .
I would have edited the OP except I had to create a new account for the new forums?

[marZ]

Gambit still working on gmod stuff, holy shiet.

Otherwise good job with this!