Issues compiling module on linux. (undefined symbol: g_pMemAlloc)

[mcd1992]

I've been working on a module to add some extra bindings to the CNavArea class. I've got it up and working just fine on windows, and linux kinda. On linux I can get the module to work just fine except for any functions that use valve's CUtlVector.

So basically if I try and use CUtlVector on linux `ldd -d` tells me.

linux-gate.so.1 (0xf7711000)
libtier0.so => not found
libvstdlib.so => not found
libstdc++.so.6 => /usr/lib32/libstdc++.so.6 (0xf75e3000)
libm.so.6 => /usr/lib32/libm.so.6 (0xf7597000)
libgcc_s.so.1 => /usr/lib32/libgcc_s.so.1 (0xf757b000)
libc.so.6 => /usr/lib32/libc.so.6 (0xf73b9000)
/usr/lib/ld-linux.so.2 (0xf7714000)
undefined symbol: g_pMemAlloc   (../lib/linux/libgmsv_fgt_navstuff_linux.so)

(Yes I see that libtier0 and libvstdlib are 'not found' but that's not the issue. The gmsv_example and every other module that work just fine say the same thing.)

And requiring the module with g_pMemAlloc symbol missing just gives a 'Couldn't load module library!'
Running gmod with LD_DEBUG=libs it confirms that somehow tier0's g_pMemAlloc symbol isn't getting linked.

Steam/SteamApps/common/GarrysMod/garrysmod/lua/bin/gmsv_fgt_navstuff_linux.dll: error: symbol lookup error: undefined symbol: g_pMemAlloc (fatal)

So has anyone else ran into this issue before? I'm compiling against the sdk2013's libtier0.so and everything works just fine otherwise.

The code is at [url]https://github.com/mcd1992/gmsv_fgt_navstuff[/url]

Any ideas appreciated.

[Willox]

[QUOTE=mcd1992;45951219]Yes I see that libtier0 and libvstdlib are 'not found' but that's not the issue.[/QUOTE]

I disagree. Your code is including nav_area.h, which includes tier0/memdbgon.h, which includes tier0/memalloc.h. The functions within this reference g_pMemAlloc which is a symbol within tier0.

Fix your linking issue to tier0 and you will hopefully be set.

The reason your other modules compile without being able to find tier0 is because you don't use any symbols from it in them :v:.

[mcd1992]

[QUOTE=Willox;45951812]I disagree. Your code is including nav_area.h, which includes tier0/memdbgon.h, which includes tier0/memalloc.h. The functions within this reference g_pMemAlloc which is a symbol within tier0.

Fix your linking issue to tier0 and you will hopefully be set.

The reason your other modules compile without being able to find tier0 is because you don't use any symbols from it in them :v:.[/QUOTE]

Hrmm. I'm linking against libtier0.so in my premake file though. Also I can still use functions like FOR_EACH_VEC which is defined in tier1 (which is being linked just the same as tier0). RandomInt which is in vstdlib, and linked the same as tier0. Ech, you're right CUtlVector is the only thing that uses tier0 in my code but I can't figure out why it's not linking properly when tier1 and vstdlib are linking just fine.

EDIT: Even static linking against GarrysMod/bin/libtier0.so fails...

EDITEDIT: Also if I remove my link to vstlib the Msg symbol is undefined and doesn't work. Link to vstdlib again and everything works but ldd stiil gives 'libvstdlib.so => not found' still. So I presume when the dll is loaded into gmod it finds libvstdlib dynamically, ldd is just clueless as to where it is on its own.
The more I look into this the more confused I get. Does anyone have a Makefile for some project that uses tier0 methods on linux they'd be willing to share?

[Willox]

FOR_EACH_VEC is a definition and not a symbol in tier1. I am not sure how your RandomInt call is succeeding, but I'll assume it is something Linux related as I don't know anything about compiling on Linux really.

What is your current error when trying to link against tier0?

[mcd1992]

[QUOTE=Willox;45952621]FOR_EACH_VEC is a definition and not a symbol in tier1. I am not sure how your RandomInt call is succeeding, but I'll assume it is something Linux related as I don't know anything about compiling on Linux really.

What is your current error when trying to link against tier0?[/QUOTE]

There isn't an error but there is a warning that I always get when I mess with the sdk.
I also get warnings for various things in the sdk that I have no control over. [url=https://gist.github.com/mcd1992/f3e5aa009231128b052e]warnings[/url]

Creating obj/Release
fgtnavarea.cpp
gmsv_fgt_navstuff.cpp
In file included from ../../../source-sdk-2013/mp/src/public/tier0/commonmacros.h:15:0,
                 from ../../../source-sdk-2013/mp/src/public/tier0/basetypes.h:11,
                 from ../../../source-sdk-2013/mp/src/public/tier0/dbg.h:15,
                 from ../../../source-sdk-2013/mp/src/game/server/cbase.h:33,
                 from ../src/gmsv_fgt_navstuff.cpp:12:
../../../source-sdk-2013/mp/src/public/tier0/platform.h:533:0: warning: "DLL_EXPORT" redefined [enabled by default]
../../gmod-module-base/include/GarrysMod/Lua/Interface.h:21:0: note: this is the location of the previous definition
Linking gmsv_fgt_navstuff_linux

`ldd -d output`
	linux-gate.so.1 (0xf76f2000)
	libvstdlib.so => not found
	libtier0.so => not found
	libstdc++.so.6 => /usr/lib32/libstdc++.so.6 (0xf75c3000)
	libm.so.6 => /usr/lib32/libm.so.6 (0xf7577000)
	libgcc_s.so.1 => /usr/lib32/libgcc_s.so.1 (0xf755b000)
	libc.so.6 => /usr/lib32/libc.so.6 (0xf7399000)
	/usr/lib/ld-linux.so.2 (0xf76f5000)
undefined symbol: g_pMemAlloc	(../lib/linux/libgmsv_fgt_navstuff_linux.so)

[Willox]

I'm gonna have to assume the reason it is "not found" is because it isn't in your libdirs or your libdirs are wrong. Double check / find a way to debug that with whatever compiler you have.

[mcd1992]

[QUOTE=Willox;45952785]I'm gonna have to assume the reason it is "not found" is because it isn't in your libdirs or your libdirs are wrong. Double check / find a way to debug that with whatever compiler you have.[/QUOTE]

That does fix the not found issue but not the undefined symbol issue.

[unknown@unknown-lap linux-gmake]$ echo $LD_LIBRARY_PATH
:/usr/lib32:/usr/lib:/usr/lib64:/usr/lib32:/usr/lib:/usr/lib64:/home/unknown/.steam/steam/SteamApps/common/GarrysMod/bin
[unknown@unknown-lap linux-gmake]$ ldd -d ../lib/linux/libgmsv_fgt_navstuff_linux.so 
	linux-gate.so.1 (0xf77a6000)
	libvstdlib.so => /home/unknown/.steam/steam/SteamApps/common/GarrysMod/bin/libvstdlib.so (0xf76b6000)
	libtier0.so => /home/unknown/.steam/steam/SteamApps/common/GarrysMod/bin/libtier0.so (0xf768c000)
	libstdc++.so.6 => /usr/lib32/libstdc++.so.6 (0xf7596000)
	libm.so.6 => /usr/lib32/libm.so.6 (0xf7549000)
	libgcc_s.so.1 => /usr/lib32/libgcc_s.so.1 (0xf752e000)
	libc.so.6 => /usr/lib32/libc.so.6 (0xf736c000)
	libdl.so.2 => /usr/lib32/libdl.so.2 (0xf7367000)
	libpthread.so.0 => /usr/lib32/libpthread.so.0 (0xf734a000)
	/usr/lib/ld-linux.so.2 (0xf77a9000)
	librt.so.1 => /usr/lib32/librt.so.1 (0xf7341000)
undefined symbol: g_pMemAlloc	(../lib/linux/libgmsv_fgt_navstuff_linux.so)

EDIT: Tried both. These are also after a recompile with the updated library path.

[unknown@unknown-lap linux-gmake]$ echo $LD_LIBRARY_PATH
:/usr/lib32:/usr/lib:/usr/lib64:/usr/lib32:/usr/lib:/usr/lib64:/home/unknown/Development/source-sdk-2013/mp/src/lib/public/linux32
[unknown@unknown-lap linux-gmake]$ ldd -d ../lib/linux/libgmsv_fgt_navstuff_linux.so 
	linux-gate.so.1 (0xf7706000)
	libvstdlib.so => /home/unknown/Development/source-sdk-2013/mp/src/lib/public/linux32/libvstdlib.so (0xf7616000)
	libtier0.so => /home/unknown/Development/source-sdk-2013/mp/src/lib/public/linux32/libtier0.so (0xf75ec000)
	libstdc++.so.6 => /usr/lib32/libstdc++.so.6 (0xf74f6000)
	libm.so.6 => /usr/lib32/libm.so.6 (0xf74a9000)
	libgcc_s.so.1 => /usr/lib32/libgcc_s.so.1 (0xf748e000)
	libc.so.6 => /usr/lib32/libc.so.6 (0xf72cc000)
	libdl.so.2 => /usr/lib32/libdl.so.2 (0xf72c7000)
	libpthread.so.0 => /usr/lib32/libpthread.so.0 (0xf72aa000)
	/usr/lib/ld-linux.so.2 (0xf7709000)
	librt.so.1 => /usr/lib32/librt.so.1 (0xf72a0000)
undefined symbol: g_pMemAlloc	(../lib/linux/libgmsv_fgt_navstuff_linux.so)

[>>oubliet]

You can't resolve an external symbol using libraries, you need to set g_pMemAlloc dynamically.

I don't think there's a source interface for IMemAlloc, so finding it would be difficult (you'd need to get
a signature for some place in memory where g_pMemAlloc is used, or get it with an offset from something).

If you do decide to find a reference to g_pMemAlloc in memory, you will also need to do this in some source file:

IMemAlloc* g_pMemAlloc;

But that's all a lot of work, it's probably best you just avoid using SDK headers that use g_pMemAlloc.

[mcd1992]

[QUOTE=>>oubliette<<;45952971]You can't resolve an external symbol using libraries, you need to set g_pMemAlloc dynamically.

I don't think there's a source interface for IMemAlloc, so finding it would be difficult (you'd need to get
a signature for some place in memory where g_pMemAlloc is used, or get it with an offset from something).

If you do decide to find a reference to g_pMemAlloc in memory, you will also need to do this in some source file:

IMemAlloc* g_pMemAlloc;

But that's all a lot of work, it's probably best you just avoid using SDK headers that use g_pMemAlloc.[/QUOTE]

I've already gone ahead and just remade any functions that use CUtilVector to use std::vector in my own class which inherits from the original instead. I'm just confused as to why this works just fine on windows and not linux.

[mcd1992]

Welp I've ran into more issues while trying to add new stuff. Does anyone know of an example makefile or something that I can see how to go about compiling a module with all of the source libraries? Or should I pretty much just compile my code with serverplugin_empty in the sdk. Seems like overkill to do it that way but I'm not sure.

[&gt;&gt;oubliet]

[QUOTE=mcd1992;45957199]Welp I've ran into more issues while trying to add new stuff. Does anyone know of an example makefile or something that I can see how to go about compiling a module with all of the source libraries? Or should I pretty much just compile my code with serverplugin_empty in the sdk. Seems like overkill to do it that way but I'm not sure.[/QUOTE]

That's a pretty vague question, what new problems are you running into?

[mcd1992]

[QUOTE=>>oubliette<<;45962206]That's a pretty vague question, what new problems are you running into?[/QUOTE]

The new problems are more missing symbols when trying to use CNavMesh. I need to compile alongside nav_area.cpp and nav_mesh.cpp which in turn ends up with me compiling the whole server/mathlib/tier1 code. Still working on just getting the server code to compile and link correctly (moving everything to premake4 instead of their vpc system). After that I hopefully can just add my code alongside the server code and it will have the neccisary symbols.

[Python1320]

I've compiled the old navmesh module on svn://svn.metastruct.org/gbins it might help with your problem, except I may have broken the windows build.

[mcd1992]

[QUOTE=Python1320;45963171]I've compiled the old navmesh module on svn://svn.metastruct.org/gbins it might help with your problem, except I may have broken the windows build.[/QUOTE]

The navigation directory looks like a module for the old nodegraph system? Still the premake files seem useful, digging around them now. Thanks for sharing!

[mcd1992]

Hrm after looking around there doesn't seem to be a simple way of getting access to the CNavMesh singleton. Looking through the sdk code I see a function called NavMeshFactory (in mp/src/game/server/nav_mesh_factory.cpp) which literally just returns the navmesh singleton :v:

It's called only once from what I see in game/server/gameinterface.cpp to initialize the 'TheNavMesh' extern. Anyone have any tips for sigscanning for NavMeshFactory or finding the 'TheNavMesh' extern? From what I'm thinking I should be able to get a reference to the NavMeshFactory() call it to get a CNavMesh object. Then typedef any methods of CNavMesh I need and call them successfully. Or am I an idiot and overlooking something?

[del](I have no clue what I'm doing anymore Robotboy/_Kilburn plz implement all navmesh bindings so I can stop breaking things. thx bb)[/del]

[&gt;&gt;oubliet]

[QUOTE=mcd1992;45967200]Hrm after looking around there doesn't seem to be a simple way of getting access to the CNavMesh singleton. Looking through the sdk code I see a function called NavMeshFactory (in mp/src/game/server/nav_mesh_factory.cpp) which literally just returns the navmesh singleton :v:

It's called only once from what I see in game/server/gameinterface.cpp to initialize the 'TheNavMesh' extern. Anyone have any tips for sigscanning for NavMeshFactory or finding the 'TheNavMesh' extern? From what I'm thinking I should be able to get a reference to the NavMeshFactory() call it to get a CNavMesh object. Then typedef any methods of CNavMesh I need and call them successfully. Or am I an idiot and overlooking something?

[del](I have no clue what I'm doing anymore Robotboy/_Kilburn plz implement all navmesh bindings so I can stop breaking things. thx bb)[/del][/QUOTE]

Here's a little explanation on how you can get global pointers from binaries:

To keep it simple, you'll need:

* IDA (Interactive Disassembler, it's not free) or OllyDbg (I'm using IDA for this)
* A sigmaking plugin for either, it makes things a lot simpler
* Some function that references the global and uses a string somewhere (the string part is not really required, however, if you don't have a function
that references the pointer and a string it will be a lot more difficult to find)
* The linux binary that contains the function.

The first thing you should do is look around the SDK and SourceLeak (if you're okay with downloading the latter) for a function that's in a source file
and references the global pointer we're looking for. For CNavMesh I'm using the console command 'nav_update_blocked'.

The part we're interested in looks like this:

[img]http://puu.sh/bxBPb/351e71cf5b.png[/img]

Now that we have a nice string to look for, we can open IDA and load server.so (this contains the function we're looking for).

It will take a while for the binary to fully load for the first time, when it's done loading it should have something like:

[img]http://puu.sh/bxCwc/a5dcb442bd.png[/img]

In the bottom left.

Now, we can search for the text we found in the SDK/Source leak by:

1) Going into text view mode (right click -> text view)
2) Scrolling to the top of the disassembly
3) And searching for the text:

[img]http://puu.sh/bxCHY/cfbf4c71ac.jpg[/img]

After the search completes, we'll find where that string was used in the disassembly:

[img]http://puu.sh/bxDeg/f6675070d1.png[/img]

Notice 'loc_B749D8' at the top of the image? That means this code can be jumped to from a somewhere else. That jump location is there because of the if statement:

if ( area->IsBlocked() )

Jumps for if statements are conditional (or at least at some point there is a condition, it doesn't have to be directly related to the jump itself).

If we click on 'loc_B749D8' and hit Ctrl+X, we'll find a list of places where this section of code is jumped to from, in this case there's only one (rightly so, if you look at code in C++, there's no reason why it should be ran more than once):

[img]http://puu.sh/bxEvm/5400ecc8ac.jpg[/img]

If we select the first xref and hit OK, IDA will show us the disassembly that jumps to that location, which is just above what we were looking at before:

[img]http://puu.sh/bxFR4/c5c9f7387a.png[/img]

Tada! we've found the pointers we were looking for, now we simply get a pointer to them with whatever plugin you're using, with mine (I sadly don't recall the name of it)
you select the instructions you want turned into a signature and hit Alt+F9. We're going to select these ones:

[img]http://puu.sh/bxGa0/19b9e9e500.png[/img]

To be safe, you should really grab a lot of the function and shorten the signature programmatically, but we're feeling brave today so we can go ahead and just
grab a nice chunk of it and not shorten it.

Notice how we select just at the top where our global pointer is first used? This is important, since as we want a pointer to it. The signature
that we're given looks like this:

mask:
x[B]????[/B]xxxx[B]????[/B]xxxx????x????xxxx????xxxxxxxxxxxxxx????xxxxxxxxxxxxxxxxxx

pattern:
\xa1\x00\x00\x00\x00\x89\x04\x24\xe8\x00\x00\x00\x00\x85\xc0\x0f\x84\x00\x00\x00\x00\xa1\x00\x00\x00\x00\x89\x04\x24\xe8\x00\x00\x00\x00\xc7\x44\x24\x04\x00\x00\x00\x00\x89\xc3\x89\x04\x24\xe8\x00\x00\x00\x00\x8b\x03\xc7\x44\x24\x08\x00\x00\x00\x00\xc7\x44\x24\x04\xfe\xff\xff\xff

As you can see, the mask ignores the 4 byte pointer to TheNavMesh (highlighted in bold) since as it can change at runtime. These are the 4 bytes we need.

So to wrap up, with the signature all we must do is resolve it, and 0x1 and deference twice. There's a bunch of public code you can use to resolve a signature, you can just google around for that.

DWORD ptr = **(DWORD**) (((DWORD*)resolve_my_signature()) + ((DWORD*)0x1))

This might be a bit hard to understand, I didn't have a lot of time to write this. Hope it helps.

[mcd1992]

Hah since I made my last post I was reading through some of the metamod stuff and started figuring it out. I haven't tested anything as I have to head to work in a bit but my method that I think will work is to use g_ICvar->FindCommand( "nav_use_place" ) and get it's m_fnCommandCallbackV1 member which contains the code TheNavMesh->PrintAllPlaces() in nav_mesh.cpp. And digging around in x64_dbg I can see that the TheNavMesh gets pushed into ecx in this function.

Sorry for any typos or if that doesn't make any sense, should of left for work 2 minutes ago.

[mcd1992]

Re-reading what I wrote I can see where I noticed I was running late and just started blabbering.

Thanks for the detailed write up oub, that actually answers a few of the questions I hadn't taken the time to look up yet. Also your writeup lets me know I was on the right track about this. I went through the source myself looking for const strings inside functions that also had the TheNavMesh global and found a different one though. CommandNavUsePlace is what I decided to look for and more importantly as I was finding the function in x64_dbg I saw that it was also a concommand callback function. After trying some stuff I found out that doing,

#define private public // Call the police, I don't give a fuck.
ConCommand *test = g_ICvar->FindCommand( "nav_use_place" );
Msg( "test: 0x%x", test->m_fnCommandCallbackV1 );

works exactly as I hoped. This will set test to the callback function that is used as the second argument in the ConCommand constructor. So now (theoretically, haven't tested this yet) I should be able to sigscan just this function and find

// nav_mesh.cpp:2347 TheNavMesh->PrintAllPlaces()
mov ecx,dword ptr ds:[63A4C4C8] // This is the TheNavMesh global
pop ebp // Not sure why we pop ebp here
jmp <server.CNavMesh_PrintAllPlaces>
// There's a lot more to this function that I can sigscan for if I get false positives.

So this way I don't have to sigscan the whole server.so/dll just this one function. Thanks again everyone hopefully I can get everything figured out this weekend and actually get to use the module instead of figuring out how to make it.

EDIT: Eyyyyy I've got it working. Just need to re-implement the functions in nav_mesh.cpp and I'm good to go. Thanks again everyone!

[borzh62]

[QUOTE=Willox;45951812]I disagree. Your code is including nav_area.h, which includes tier0/memdbgon.h, which includes tier0/memalloc.h. The functions within this reference g_pMemAlloc which is a symbol within tier0.

Fix your linking issue to tier0 and you will hopefully be set.

The reason your other modules compile without being able to find tier0 is because you don't use any symbols from it in them :v:.[/QUOTE]

It is ok to have library "not found" in Linux because it WILL be found in LD_LIBRARY_PATH.
It is a bad practice to hardcode your lib path dependency unless it is a system lib (read /usr/lib).

BTW I have same issue with g_pMemAlloc in my source server plugin. With new hl2dm and tf2 update console command completion stopped working.
Defining NO_MALLOC_OVERRIDE does help with linking issues, but I still got crash.

Source is calling

virtual int CommandCompletionCallback( const char *pPartial, CUtlVector< CUtlString > &commands )

so I create and fill commands with normal allocation, but after engine calls this method, it just crashes.
I assume that it tries to g_pMemAlloc->free() what I had allocated, but there is no such memory in his memory tables or whatever.