rust_island_2013.sav.json format

[Gachl]

I am currently plugging the Rust world save json (save.friendly true, save.all) apart to generate heat maps and some other statistics. I'd like to share my findings and hope that others can use it or contribute to it.

Format:
[QUOTE]{
__instanceObject : array
__{
____object : SaveObject
____{
______id : int
______sortOrder : int
______
______structMaster : MasterObject
______structComponent : ComponentObject
______netInstance : NetObject
______coords : CoordObject
______ngcInstance : InstanceObject
______carriableTrans : CarriableObject
______takeDamage : HealthObject

______inventory : Array<InventoryObject>
______deployable : DeployableObject
______fireBarrel : FireObject
______lockable : LockableObject
____},
____object : SaveObject ...,
____object : SaveObject ...,
____...
__}
}[/QUOTE]

These are the subobjects
[QUOTE]MasterObject
{
__ID : int
__DecayDelay : double
__CreatorID : int
__OwnerID : int
}
ComponentObject
{
__ID : int
__MasterID : int
__MasterViewID : int
}
NetObject
{
__serverPrefab : int
__ownerPrefab : int
__proxyPrefab : int
__groupID : int
}
CoordObject
{
__pos : PosObject
__oldPos : PosObject
__rot : RotObject
__oldRot : RotObject
}
PosObject
{
__x : double
__y : double
__z : double
}
RotObject
{
__x : double
__y : double
__z : double
__w : double
}
InstanceObject
{
__ID : int
}
CarriableObject
{
__transCarrierID : int
}
HealthObject
{
__health : double
}
InventoryObject
{
__id : int
__slot : int
__count : int
__condition : int
__maxcondition : int
}
DeployableObject
{
__CreatorID : int
__OwnerID : int
}
FireObject
{
__OnFire : bool
}
LockableObject
{
__password : string
__locked : bool
__users : array<int>
}
[/QUOTE]

This is the list of possible ngcInstance.IDs (extracted from my server):
[QUOTE]-1655908709 (Could be [B]Large Wood Storage[/B])
-1635098332
-1613269705
-1507079735
-1370680922 (Either [B]Wood Door[/B] or [B]Metal Door[/B])
-1350838201
-1249869654
-1205619926 (Either [B]Camp fire[/B] or [B]Furnace[/B])
-1062486044
-257764790
-214976533 (Either [B]Camp fire[/B] or [B]Furnace[/B])
93560909 (Could be [B]Wood Storage Box[/B])
138513344
138609455
161925038 (Could be [B]Wood Pillar[/B])
212808836 (Either [B]Wood Door[/B] or [B]Metal Door[/B])
251358101 (Could be [B]Small Stash[/B])
331420370
444107540
543244287
617903599
872325649
1059720916
1118601794
1134498306
1217629512 (Could be [B]Wood Wall[/B])
1405599652
1512380814
1815597930
1843051753
2111870031[/QUOTE]
Still trying to figure out all the other IDs.

If it can burn (furnace, camp fire) it's got [B]fireBarrel[/B].
If it can hold items (furnace, fire, repair bench, crates etc) it's got [B]inventory[/B].
If it can be opened, closed and locked (doors, gates) it's got [B]lockable[/B].

I think components of a greater structure (important for decay) have structComponent but no structMaster
and the first component of a greater structure has structMaster but no structComponent.

I have absolutely no idea what netInstance holds.
I have also no idea what carriableTrans holds but it's part of the following IDs:
[QUOTE]1512380814
-214976533
-1613269705
-1205619926
212808836
-1655908709
1134498306
251358101
-1062486044
-1370680922
444107540
138513344
93560909
543244287
138609455[/QUOTE]

If you can, please provide more information such as ngc instance id to item name mappings, missing subobjects and better explaination for some of them.

[B]How to get this JSON[/B]
Enable [B]save.friendly true [/B]on your server, this will cause all map save processes to generate a rust_island_2013.sav.json file containing the complete map. To force the generation of this file, use [B]save.all[/B]. Note that depending on your server size, this file can reach huge sizes.
This resulting json file can directly be read by the [URL="http://php.net/json_decode"]decode_json[/URL] function in PHP and probably in any other language. Make sure you have enough RAM assigned to the thread because decoding tends to take up more than 32MB.

[B]Admin abuse[/B]
OMG YOU CAN READ ALL THEM LOCK CODES FROM THEM DOORS AND COMPILE THEM INTO A HANDY LIST FOR ADMERNS TO ABUSE OMG.
I know that. There's lots more bad things that can be done. That's too bad but this is part of what Rust offers to admins and I hope most of them will use this for good.

[B]So, what do I use that for?[/B]
Well, you could for example create [URL="http://www.hawnutor.org/image/zDWqxkc.png"]heat maps[/URL] on your vanilla ([URL="http://facepunch.com/showthread.php?t=1405343&p=45255727&viewfull=1#post45255727"]chocolate[/URL]) Rust server, maybe it can be used to indentify hackers (by finding Weapon Parts in Crates and the likes). I'm sure the Rust community is capable of finding some nice appliances.

[infusednz]

Interesting stuff.

We've been doing alot of coding, will be handy.

[Gachl]

Since Garry eliminated the need of GSPs for legacy, it's now much easier accessing and using avatar.bin and the json formatted save and use it for something.

I've used both recently to create a full 1:1 coordinate to pixel render of the Rust map with both structures and player movement. For that I save all avatars (save.avatars) every second and parse those of online players for their coordinates which are then stored in a database (7200 points per user, no idle locations, which is about 2 hours of movement) and use this data as overlay for all coordinates extracted from the json map. This is the result:
[URL=http://hawnutor.org/image/UXSSev2.png][highlight]Large image warning[/highlight][/URL]
[I](the original rendered map is 7836x6708 pixel, this is only a part, downscaled (about 1:1.3))[/I]
[IMG]http://hawnutor.org/image/2_nOHcb.png[/IMG]
[I](It's easy to figure out the internal structure of buildings thanks to player movement within the red area. This part is also unscaled, it's easy to see the distance between the individual coordinates which is always about 1 second (and a few ms))[/I]

For that I'd like to share this snippet for reading out the player coordinates in PHP and C#:
[QUOTE]$coordinates = unpack("cMagic1/cMagic2/cMagic3/fX/cXLim/fY/cYLim/fZ/cZLim", $avatar);[/QUOTE]
[QUOTE]float X = BitConverter.ToSingle(avatar, 3);
float Y = BitConverter.ToSingle(avatar, 8);
float Z = BitConverter.ToSingle(avatar, 13);[/QUOTE]
Note that there's one coordinate -7.84389e-29,-0.531344,1.84086e-18 that is something like the initial spawn coordinate, every time a player suicides it will return this coordinate. I have filtered it out or the map would be full of lines to this one point. The whole process from starting to save all avatars to having the coordinates parsed now takes an average 15ms instead of the 500ms it took using FTP to the GSP.

I'm still looking for practical applications, other than detecting speedhackers by checking their movement, as this map should not be used by players or they will find any base no matter how hidden it is. Tonight I will take a Rust map, vectorise it and scale it up to match the coordinates 1:1 to replace the black background with an actual map. So far I've only used existing satellite images, but they were all downscaled and would make the render very unprecise.

edit: there's also still room to upscale even more since one coordinate in rust does not equal one step, all coordinates can be scaled x10 to increase the resolution down so you can pinpoint single camp fires.
edit2: I think I have also figured out the format of items in a players inventory
[IMG]http://hawnutor.org/image/UcFanVy.png[/IMG]
still need to test this with single items and write down their IDs.

edit3: I tampered a bit with the rendering and also collected a lot more data.
[url=http://www.hawnutor.org/image/HUZFEtu.png][img]http://www.hawnutor.org/image/.HUZFEtu.png[/img]
[highlight]Large image warning[/highlight][/url]
[I]The shape of the street is clearly visible, I have walked it once in the center to adjust for vectoring. There are also a lot of straight death lines connecting place of death and respawn. The brighter they are the more often they were used.[/I]
[img]http://www.hawnutor.org/image/uYWk32r.png[/img]
[I]There's a lot of traffic at some places[/I]
[img]http://www.hawnutor.org/image/QHHTHBV.png[/img]
[I]A beginner's route. Don't go too far away from your home![/I]
[img]http://www.hawnutor.org/image/_7WqPe3.png[/img]
[I]Big Radtown, can be identified by the four buildings[/I]

edit4: having trouble with scaling coordinates up. I wanted to scale x10.0 to zoom in really close but even at x2.9 no program on my computer can open the generated map anymore, it's just too large.
I've come as far as x2.8 and then taking a quarter of the whole map and only saving that one which makes a 9417x10970 image. I'm gonna try and limit the coordinates being processed to a smaller area and not read out all coordinates that I can find.
[IMG]http://hawnutor.org/image/SXVhW2s.png[/IMG]
[I]x2.8 scale (1 coordinate = 2.8 pixels)[/I]
[img]http://www.hawnutor.org/image/AamvPUt.png[/img]
[I]It's pretty...[/I]

edit5: after a while an almost complete list of items has accumulated and I hope I wrote my parser right and the item IDs are correct...
[QUOTE]-2036007026 Armor
-2020047698 Armor
-2004692311 Armor
-1984234517 Armor
-1866144008 Armor
-1510165064 Armor
-1277514582 Armor
-1041505799 Armor
-860229637 Armor
-593373449 Armor
-576221312 Armor
-574503808 Armor
-504049760 Armor
-458513990 Armor
2966401 Armor
220731965 Armor
220731967 Armor
499082053 Armor
-2132749130 Blueprint
-2131104860 Blueprint
-2067619116 Blueprint
-2049533197 Blueprint
-2034007676 Blueprint
-2019392259 Blueprint
-2013798781 Blueprint
-1986092087 Blueprint
-1969291615 Blueprint
-1965573748 Blueprint
-1932005148 Blueprint
-1879206188 Blueprint
-1866015291 Blueprint
-1811957630 Blueprint
-1801010982 Blueprint
-1799775083 Blueprint
-1768760625 Blueprint
-1750088570 Blueprint
-1681028932 Blueprint
-1666394886 Blueprint
-1568285441 Blueprint
-1551115832 Blueprint
-1546666347 Blueprint
-1512603234 Blueprint
-1501128454 Blueprint
-1496282687 Blueprint
-1444511036 Blueprint
-1443430930 Blueprint
-1427775546 Blueprint
-1394433827 Blueprint
-1378835525 Blueprint
-1365314312 Blueprint
-1359438631 Blueprint
-1316307474 Blueprint
-1315590261 Blueprint
-1312380423 Blueprint
-1232086066 Blueprint
-1231033983 Blueprint
-1228551977 Blueprint
-1208626259 Blueprint
-1194804523 Blueprint
-1125000281 Blueprint
-1114913550 Blueprint
-1012942428 Blueprint
-1007057230 Blueprint
-958411370 Blueprint
-947795068 Blueprint
-942941209 Blueprint
-908619646 Blueprint
-891362172 Blueprint
-872895523 Blueprint
-841180015 Blueprint
-828002313 Blueprint
-778261042 Blueprint
-777738822 Blueprint
-755063358 Blueprint
-722294337 Blueprint
-691876205 Blueprint
-624966769 Blueprint
-622491190 Blueprint
-621762089 Blueprint
-588848900 Blueprint
-557189758 Blueprint
-507398774 Blueprint
-507345750 Blueprint
-493703493 Blueprint
-488316746 Blueprint
-439707649 Blueprint
-438781953 Blueprint
-393886788 Blueprint
-386750292 Blueprint
-354310774 Blueprint
-353573463 Blueprint
-339309948 Blueprint
-272777579 Blueprint
-234974567 Blueprint
-221137503 Blueprint
-173490957 Blueprint
-119675492 Blueprint
-90005810 Blueprint
-18486639 Blueprint
0 Blueprint
731103894 Blueprint
936435928 Blueprint
1203421142 Blueprint
1221499107 Blueprint
1604779232 Blueprint
2091379670 Blueprint
-2100309873 Item
-2098881320 Item
-2069637234 Item
-2069178372 Item
-2017091090 Item
-2003651683 Item
-1996712754 Item
-1986924116 Item
-1986859114 Item
-1983019389 Item
-1967264043 Item
-1930959642 Item
-1918048846 Item
-1913332835 Item
-1884102487 Item
-1862421344 Item
-1852785732 Item
-1795626574 Item
-1766408794 Item
-1746807044 Item
-1711560255 Item
-1702103375 Item
-1695905546 Item
-1685148246 Item
-1684411757 Item
-1683913004 Item
-1634625907 Item
-1596479757 Item
-1514489399 Item
-1494829402 Item
-1494748933 Item
-1444368987 Item
-1432622157 Item
-1394740489 Item
-1381237797 Item
-1376928607 Item
-1325484350 Item
-1281838103 Item
-1247681606 Item
-1246954244 Item
-1246716681 Item
-1229158473 Item
-1180446253 Item
-1178889567 Item
-1176517175 Item
-1164198487 Item
-1094145329 Item
-1091443008 Item
-1060650249 Item
-1045846842 Item
-1028946501 Item
-1007974928 Item
-997015124 Item
-943866449 Item
-907286022 Item
-877348917 Item
-808191036 Item
-759253071 Item
-726748233 Item
-669170752 Item
-608461659 Item
-608374622 Item
-605888005 Item
-589627460 Item
-524627283 Item
-511446578 Item
-510215031 Item
-491669843 Item
-470765857 Item
-406665993 Item
-325482606 Item
-325469290 Item
-291464241 Item
-286860331 Item
-252067363 Item
-237573966 Item
-219296538 Item
-205463157 Item
-202794508 Item
-107889023 Item
-40060431 Item
-23939862 Item
-8279363 Item
9540 Item
11586 Item
17727 Item
2442372 Item
499085344 Item
570144755 Item
672808670 Item
672808688 Item
672808751 Item
1042629920 Item
1083216561 Item
1273813928 Item
1810169290 Item
2005345604 Item[/QUOTE]
These are avatar.bin item IDs or blueprint IDs and I need to somehow assign them to their respective items... Oh my, that's gonna be fun fun fun. But first as a next step I will crosscheck those IDs with IDs generated from the JSON map to compare whether the IDs change (for example if you have a Large Wood Storage Blueprint (learned) and you craft one (item) and you place it (world) will it have the same item ID or will it change?

[arion]

Fascinating stuff. I would like to see the final outcome of your guys endeavours :)

[Zhahaman2001]

Great stuff Gachl!! Please post more!!! This is exactly what I was looking for, I am trying to read and write to the avatar file (with vb.net)....
If you can help me figure out how to start to tackle the blueprints learned...
The data that just singles isn't that hard to find out
[QUOTE]
Dim Health As Single = BitConverter.ToSingle(avatar, 42)
Dim Hunger As Single = BitConverter.ToSingle(avatar, 52)
Dim Radiation As Single = BitConverter.ToSingle(avatar, 57)
ect...
[/QUOTE]
but this is as far as I got for blueprints learned and I know I'm going about it the wrong way...
[QUOTE]
Dim Explosives As Boolean = BitConverter.ToString(avatar, 0).Contains("B6-E0-E4-E2")
[/QUOTE]

[Tripper3k]

How much do you guys earn from selling hacks?

[Zhahaman2001]

[QUOTE=Tripper3k;46094144]How much do you guys earn from selling hacks?[/QUOTE]
Tripper the information in this post is only useful to server admins, please be more careful with your accusations.

Gachl pretty please XD

[Gachl]

[QUOTE=Zhahaman2001;46092685]Dim Explosives As Boolean = BitConverter.ToString(avatar, 0).Contains("B6-E0-E4-E2")[/QUOTE]
Well you see, the problem is that I don't *know* whether or not I've got this right so I can't say that these IDs you will be getting are correct blueprints.
As far as I figured this out, every block and every value within a block has a prefixed tag kinda value of I think one byte which is

0x0A: 3 floats (position): 13 x, 21 y, 29 z
0x12: 4 floats (I *think* this is direction): 13 x, 21 y, 29 z, 37 angle?
0x1A: 10 floats: 13 hp, 21 ??, 29 food, 37 rads, 45 ??, 53 ??, 61 ??, 69 ??, 77 ??, 85 ??
0x22: 1 int: 8 blueprint id
wearable stuff has the same internal structure (the colourful hex sheet I posted before) and I think these ids:
0x2A: items
0x32: armor
0x3A: quick bar
(please note that all these values may be wrong, they are kinda reverse engineered serialized data)

My avatar parser I wrote in C# doesn't try to find specific fields but it rather reads the file and tries to figure out where to put data depending on the prefixes, so I read all data which helps me figuring out stuff that's incorrect or fields I'm missing. If you want to specifically search for blueprints you will probably get a couple of false matches when looking for 34 followed by 08 followed by a 32 bit integer blueprint ID but with a list of all possible (correct) blueprint IDs you will be able to filter those out.

[QUOTE=Zhahaman2001;46092685]and write to the avatar file (with vb.net)....[/QUOTE]
Writing without breaking? Tell me when that was successful and I may look into that too :D

[editline]28th September 2014[/editline]

I found the ID for explosives you were looking for
[img]http://www.hawnutor.org/image/zqxbvt4.png[/img]
but it's prefixed with (0x22) 0x06 so I don't know whether this is actually the id for explosives or what kind of field that is, there's a lot of stuff about these files I haven't fully understood yet.

[asphantix]

I absoluetly love your work, Gachl. And I'm really sad that I have no time right now for fun stuff like this. But I really would love to. Well, next year then :)

@Tripper3k
you should become a commedian, lol

[Gachl]

This map is super obsolete now so I can post it (click for full image)
[url=http://www.hawnutor.org/image/ZHPr8My.png][img]http://www.hawnutor.org/image/.ZHPr8My.png[/img][/url]
this is a quarter (the bottom left) of the complete rendering of the map with scale 1:2 (pixels to coordinates) or [URL="http://www.hawnutor.org/image/uSCnFwy.png"]click here[/URL] for the 1:3 (p:c) large cut [highlight]huge image warning[/highlight]

I'm glad to see some people interested in this and I hope we can do the same in experimental

[mrknifey]

it's very hard to make out the rad sites huh:) that's really cool gachl mate. interesting to see where people frequent other than the rad sites too.

[Gachl]

[QUOTE=mrknifey;46120608]it's very hard to make out the rad sites huh:) that's really cool gachl mate. interesting to see where people frequent other than the rad sites too.[/QUOTE]
If you follow the road they should be fairly easy to find:
[url=http://www.hawnutor.org/image/Ehrtt5z.png][img]http://www.hawnutor.org/image/.Ehrtt5z.png[/img][/url] (click to enlarge)
I once made a vector map of all places and mountains that fits these renderings but it looks ugly, I'm not very skilled with design :(

[B]edit:[/B]

[highlight]I won't be contributing to this forum anymore. I requested a perma. Goodbye.[/highlight]

[Zhahaman2001]

22 0B 08 BA A5 A4 D1 FF FF FF FF FF 01 - 556 Ammo
22 06 08 96 CD DF C6 07 - 9mm Pistol
22 06 08 8C 9F AF B8 07 - Blood Draw Kit
22 05 08 D6 BF BA 47 - Bolt Action Rifle
22 0B 08 FF 97 CA E5 FE FF FF FF FF 01 - Explosive Charge
22 06 08 B6 E0 E4 E2 03 - Explosives
22 06 08 D9 9C F8 AE 02 - F1 Grenade
22 06 08 DD AB F8 CB 06 - Flare
22 0B 08 B2 86 F9 C3 FA FF FF FF FF 01 - Flashlight Mod
22 06 08 A4 EB F9 80 07 - Holo sight
22 06 08 F7 AF A5 CE 07 - Kevlar Boots
22 06 08 81 E9 9F B6 05 - Kevlar Helmet
22 0B 08 A4 B9 9F C3 FE FF FF FF FF 01 - Kevlar Pants
22 06 08 8B AF 95 B1 02 - Kevlar Vest
22 0B 08 E4 F0 D7 8C FA FF FF FF FF 01 - Large Medkit
22 0B 08 F2 C4 8B BD F8 FF FF FF FF 01 - Large Wood Storage
22 0B 08 CE DB 8F B6 FC FF FF FF FF 01 - Laser Sight
22 05 08 D8 E0 D0 37 - Leather Boots
22 0B 08 86 C4 AF 97 FB FF FF FF FF 01 - Leather Helmet
22 06 08 FA D4 AC 9C 01 - Leather Pants
22 0B 08 AC A8 F2 E8 FB FF FF FF FF 01 - Leather Vest
22 06 08 CE AD 9C D1 05 - M4
22 06 08 D7 C8 C5 B6 06 - Metal Ceiling
22 06 08 F3 BC A8 F5 04 - Metal Doorway
22 06 08 D4 A2 C2 84 06 - Metal Foundation
22 0B 08 82 B8 FF 93 F8 FF FF FF FF 01 - Metal Pillar
22 06 08 DD A0 E2 AC 06 - Metal Ramp
22 06 08 D7 A9 F0 DA 04 - Metal Stairs
22 06 08 E7 DB CB C7 02 - Metal Wall
22 06 08 8A B5 C1 E1 04 - Metal Window
22 0B 08 99 92 FE F1 FD FF FF FF FF 01 - MP5A4
22 0B 08 F9 A9 C6 B1 FE FF FF FF FF 01 - P250
22 0B 08 FF B7 D8 E5 FA FF FF FF FF 01 - Paper
22 0B 08 A7 DB F1 BC FD FF FF FF FF 01 - Pick Axe
22 0B 08 BB AE 92 E2 FD FF FF FF FF 01 - Rad Suit Boots
22 0B 08 83 DE F7 87 F9 FF FF FF FF 01 - Rad Suit Helmet
22 06 08 F3 9A D6 85 02 - Rad Suit Pants
22 06 08 A1 FE 9E 8A 03 - Rad Suit Vest
22 06 08 FC E0 E6 DC 07 - Research Kit
22 06 08 B6 D4 E0 80 07 - Shotgun
22 06 08 C2 A5 FE D2 07 - Shotgun Shells
22 0B 08 A1 B5 D1 F2 FD FF FF FF FF 01 - Silencer
22 0B 08 F8 F4 9E AE FE FF FF FF FF 01 - Small Medkit

THANKS Gachl! Just got done with this blueprint list wanted to post it quick before I forgot. But I am not sure how to find out which is which for the first 45 starter crafts....

[iScripters]

This looks seriously awesome, hope to see much more of this!

[DeadRisen]

Neat, I think the item id's were unsigned ints though. He printed them out as signed in his list of IDs, which is why the id's were randomly positive and negative.