Using Detours on functions within engine-files

[aVoN]

Hello,
I've lately ran into the problem where I need to overwrite default functions in the engine (to be specific: virtual methods of IFileSystem) using [url=http://research.microsoft.com/en-us/projects/detours/]MS Detours[/url].

Now I'm quite unsure if this is safe (concerning VAC).

I have detoured functions in GMod before (in lua_shared.dll) and this is now months ago (VAC wasn't triggered for sure). But this wasn't an engine-function. Now I have to overwrite/hook engine functions.

[b]Background[/b]: I'm writing on a module (gm_mount2) which shall allow mounting vpk-files (the one, Left 4 Dead uses) using [url=http://nemesis.thewavelength.net/index.php?p=35]Nemesis HLLib[/url]. Reading from VPK's is easy but now I have to tell GMod "if you search this file, look it up in the vpks"

[tehbiga]

There are already a few modules that use detours. I [i]believe[/i] it is safe.

[|FlapJack|]

Detours are safe - for now. VAC has (Not yet) been enabled. My guess is, when we get the EP3 engine upgrade, we will also have working VAC.

[yakahughes]

Any idea when the Ep 3 update is coming?

[|FlapJack|]

2011 at the earliest.

[aVoN]

Can someone send me links to modules which use Detours?
In that case, I don't mean any SE2-bypass scripts. I already figured out, detouring functions in lua_shared (for instance: lua_Newstate) does not trigger VAC.
I really need to be sure, that someone already used detours on engine functions and didn't got banned.

Sidenote: I have been told by a person, he has been using Cheat Engine in GMod and didn't got banned but then he accidentally ran it (on another program) while he alt-tabbed TF2 to the desktop - And got banned.
So apparently, VAC (according to the stories I heared) doesn't ban in GMod (and allows VACed people to play in GMod).
Still I need some more evidence (links to modules, which use Detours)

[|FlapJack|]

gm_luaerror
gm_enginespew
gm_forceconvar
gm_gatekeeper
gm_gmodeworld (Not sure)

There's a few more, I think.

[aVoN]

Thanks.

luaerror does detour lua_shared functions - Not engine-function :(
enginespew doesn't detour at all
forceconvar is unknown to me (any links?)
gatekeeper is serverside and therefore irrelevant (and it uses "VTable hooking" - most probably using detours)
gmodworld doesn't use detours

Still I'm quite convinced using detours right now.

[Spacetech]

Checkout Azuisleet's gm_transmittools
[url]http://gmodmodules.googlecode.com/svn/trunk/gm_transmittools/gm_transmittools/[/url]

[pollyzoid]

[url=http://mauritsdebruin.com/data/garrysmod/lua/modules/gm_luaerror.zip]gm_luaerror[/url], [url=http://www.facepunch.com/showthread.php?t=859872]thread[/url]
[url=http://mauritsdebruin.com/data/garrysmod/lua/modules/gm_guardian.zip]gm_guardian[/url]

You can find lots of modules [url=http://mauritsdebruin.com/data/index.php?dir=garrysmod/lua/modules/]here[/url]

[Gbps]

If someone would be kind enough to explain exactly how to use detours in gmod modules, the general population could have a greater chance of developing module which fill in the holes in the current Lua binding. I think we would all appreciate it if someone would step forward. :smile:

[yakahughes]

It's a dark art and the people who know how to do it don't want anyone else to know because they then become less revered.

[Jinto]

[QUOTE=yakahughes;19930108]It's a dark art and the people who know how to do it don't want anyone else to know because they then become less revered.[/QUOTE]

You could, you know, research it.

[url]http://research.microsoft.com/en-us/projects/detours/[/url]
[url]http://software.intel.com/en-us/articles/intercepting-system-api-calls/[/url]

[|FlapJack|]

[QUOTE=Gbps;19930040]If someone would be kind enough to explain exactly how to use detours in gmod modules, the general population could have a greater chance of developing module which fill in the holes in the current Lua binding. I think we would all appreciate it if someone would step forward. :smile:[/QUOTE]

Along with what Jinto said, just take a read of a detour header file packaged with one of the modules that uses it.

[aVoN]

The easiest way is, downloading detours how Jinto suggested, and using their wiki.

Also, in the Detours Express version, there are neat examples on how to detour funtions, but you won't come around writing dirty hacks, if you have to detour stuff like virtual member functions (Thank's to jinto once more for you help!).

I may write a blog-entry about this.

Back to topic:
It really seems now, Detours is wideley used. Even [b]clientside[/b] and on [b]engine functions[/b] so it is (for now) safe to be use for my actions. I hope I get my module working - I want to mount L4D!

[AzuiSleet]

Detours are 100% safe in GMod, VAC isn't enabled. However, you might be making this harder than it has to be, just get the real ISteamFileSystem interface and replace the vtable wtih your own (which means you implement the ISteamFileSystem class). It should literally be *classptr = yourvtable;

[aVoN]

Oh I didn't know you can replace vtables that easy. Nice tip, thank you!

[haza55]

[QUOTE=aVoN;19938305]Oh I didn't know you can replace vtables that easy. Nice tip, thank you![/QUOTE]

The vtable pointer is always the first pointer in a object with virtual members.

class Bob
{
public:
 int a;
 int b;
 int c;

 virtual void Awesome();
}

Would be stored as

struct Bob
{
 void *vtable;
 int a;
 int b;
 int c;
}

[aVoN]

Thank's haza55.
I got a method (Jinto told me) how to retrieve the vtabl and convert it's contents to valid pointers I can store anywhere.

So I guess it's easy now: Get the vtable and backup + replace the pointer of my target-method by my own method. No detours necessary but same effect. :)

[aVoN]

Just a quick question again:
Overwriting the vtable's entries doesn't work directly, right? There is some type of memory-protection how it seems (Well, you can disable it)

If so, I better use detours: Easier to handle than coding my own vtable-hooking-system.

[Gbps]

Sounds like you should go with detours to me.

[aVoN]

Yes, I will now.
It's easy to disable the memory protection but I still tried to find a way which is safe concerning VAC (even if it's not banning in GMod).

[haza55]

size_t count = 32; // How many virtual members are in this class.
void **vtable = (void **)*obj;
DWORD protect = NULL;
VirtualProtect((void *)vtable , sizeof(size_t) * count, PAGE_READWRITE, &protect);

vtable[15] = AnnotherFunctionWithTheSameCallingConvention;

VirtualProtect((void *)vtable , sizeof(size_t) * count, protect, &protect);

[aVoN]

[QUOTE=haza55;19968739]

size_t count = 32; // How many virtual members are in this class.
void **vtable = (void **)*obj;
DWORD protect = NULL;
VirtualProtect((void *)vtable , sizeof(size_t) * count, PAGE_READWRITE, &protect);

vtable[15] = AnnotherFunctionWithTheSameCallingConvention;

VirtualProtect((void *)vtable , sizeof(size_t) * count, protect, &protect);

[/QUOTE]

Yes, that's what I tried, where I recognized there is memory protection I have to circumvent. But since VirtualProtect is also recognized by VAC (which is off in GMod), it makes no sense using it instead of detours, because Detours is easier to use.

[haza55]

[QUOTE=aVoN;19973186]Yes, that's what I tried, where I recognized there is memory protection I have to circumvent. But since VirtualProtect is also recognized by VAC (which is off in GMod), it makes no sense using it instead of detours, because Detours is easier to use.[/QUOTE]

Pretty sure detours uses VirtualProtect to write to the old function.

[aVoN]

Thank you for the information!
Still I'll go with detours. I already wrote my virtual-method detouring macros.

[cpp]#ifndef VIRTUALMEMBER_H
#define VIRTUALMEMBER_H

#include "detours.h"

//############## Converts a virtual member-function's pointer to a member-function pointer. Thank you Jinto for helping me out with this @Jinto, aVoN
template<typename T>
union VirtualPointer_t{
size_t* m_pointer;
T m_function;
};

//############## Sets up entries in your Detour-Class @aVoN
#define VDETOUR_SETUP(_type_,_class_,_method_,...)\
typedef _type_ (T##_method_)(##__VA_ARGS__);\
typedef _type_ (_class_::*T__##_method_)(##__VA_ARGS__);\
T##_method_ _method_;\
static T__##_method_ __##_method_;

//############## Creates your hook @aVoN
#define VDETOUR_HOOK(_type_,_class_,_method_,...)\
_class_::T__##_method_ _class_::__##_method_ = NULL;\
_type_ _class_::_method_(##__VA_ARGS__)

//############## Starts detouring your method, giving the object as argument @aVoN
#define VDETOUR_FROM_OBJECT(_object_,_vtableindex_,_class_,_method_)\
{\
size_t** vtable = *reinterpret_cast<size_t***>(_object_);\
VirtualPointer_t<_class_::T__##_method_> wrapper;\
wrapper.m_pointer = vtable[_vtableindex_];\
_class_::__##_method_ = reinterpret_cast<_class_::T__##_method_>(wrapper.m_function);\
}\
DetourAttach(&(PVOID&) _class_::__##_method_,(PVOID)(&(PVOID&) _class_::_method_));

//############## Same as above but for passing classes (Creates momentary overhead - but frees memory again - Does not work if no non-generic constructor is available) @aVoN
#define VDETOUR_FROM_CLASS(_target_class_,_vtableindex_,_class_,_method_)\
{\
_target_class_* object = new _target_class_;\
VDETOUR_FROM_OBJECT(object,_vtableindex_,_class_,_method_);\
delete object;\
}

//############## Removes/Detaches a hook @aVoN
#define VDETOUR_REMOVE(_class_,_method_)\
DetourDetach(&(PVOID&) _class_::__##_method_,(PVOID)(&(PVOID&) _class_::_method_));

/*
############## Example usage!
//Let's assume you want to detour the virtual method "int Read(bool b)" from IFileSystem.
class IFileSystem{
public:
virtual int Read(bool b); //VTable index: 0
virtual void Write(char c); //VTable index: 1
bool Delete(int d); //This has NO VTable index
virtual void Read(char e); //VTable index: 2

//Overloaded functions are getting added in opposite order!
virtual Size(int a);//VTable index: 5!!!!!
virtual Size(bool b);//VTable index: 4!!!!!
virtual Size(char c);//VTable index: 3!!!!!

//VTable indexing is straight forward ... (Just with multiple-inherited classes, it's not that easy though)
};

//Then this class can (but hasn't be necessarily) initialized (You see why in code later - Two ways of detouring using VDETOUR_FROM_OBJECT or VDETOUR_FROM_CLASS)
IFileSystem* FILESYSTEM = new IFileSystem;

//To prepare detouring, create a class like this
class CDetour{
public:
VDETOUR_SETUP(int,CDetour,Read,bool b);
//The above macro does simply this
// int Read(bool b); //This is your hook
// static int (CDetour::*__Read)(bool b);
}

//Then define your actual hook "int CDetour::Read(bool b)" with this
VDETOUR_HOOK(int,CDetour,Read,bool b){ //This defines CDetour::Read(bool b);
(this->*__Read)(b); //With this, you can call the original method again
}

//Later in code: Start detouring. Make sure, you use the correct VTable index (here, it is 0)
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());

//Now if your class already has an initialized object or has no generic constructor, use the already initialized object
VDETOUR_FROM_OBJECT(FILESYSTEM,0,CDetour,Read);
//Or if you want to, simply supply the classname with this method below (but be warned: This macro initializes the class and creates an object for the time it detours. It will destroy the object later)
VDETOUR_FROM_CLASS(IFileSystem,0,CDetour,Read);

DetourTransactionCommit();

//Unloading hooks

//Just run

DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());

VDETOUR_REMOVE(CDetour,Read);

DetourTransactionCommit();

*/

#endif //VIRTUALMEMBER_H[/cpp]

[haza55]

[QUOTE=aVoN;19974500]Thank you for the information!
Still I'll go with detours. I already wrote my virtual-method detouring macros.

[cpp]#ifndef VIRTUALMEMBER_H
#define VIRTUALMEMBER_H

#include "detours.h"

//############## Converts a virtual member-function's pointer to a member-function pointer. Thank you Jinto for helping me out with this @Jinto, aVoN
template<typename T>
union VirtualPointer_t{
size_t* m_pointer;
T m_function;
};

//############## Sets up entries in your Detour-Class @aVoN
#define VDETOUR_SETUP(_type_,_class_,_method_,...)\
typedef _type_ (T##_method_)(##__VA_ARGS__);\
typedef _type_ (_class_::*T__##_method_)(##__VA_ARGS__);\
T##_method_ _method_;\
static T__##_method_ __##_method_;

//############## Creates your hook @aVoN
#define VDETOUR_HOOK(_type_,_class_,_method_,...)\
_class_::T__##_method_ _class_::__##_method_ = NULL;\
_type_ _class_::_method_(##__VA_ARGS__)

//############## Starts detouring your method, giving the object as argument @aVoN
#define VDETOUR_FROM_OBJECT(_object_,_vtableindex_,_class_,_method_)\
{\
size_t** vtable = *reinterpret_cast<size_t***>(_object_);\
VirtualPointer_t<_class_::T__##_method_> wrapper;\
wrapper.m_pointer = vtable[_vtableindex_];\
_class_::__##_method_ = reinterpret_cast<_class_::T__##_method_>(wrapper.m_function);\
}\
DetourAttach(&(PVOID&) _class_::__##_method_,(PVOID)(&(PVOID&) _class_::_method_));

//############## Same as above but for passing classes (Creates momentary overhead - but frees memory again - Does not work if no non-generic constructor is available) @aVoN
#define VDETOUR_FROM_CLASS(_target_class_,_vtableindex_,_class_,_method_)\
{\
_target_class_* object = new _target_class_;\
VDETOUR_FROM_OBJECT(object,_vtableindex_,_class_,_method_);\
delete object;\
}

//############## Removes/Detaches a hook @aVoN
#define VDETOUR_REMOVE(_class_,_method_)\
DetourDetach(&(PVOID&) _class_::__##_method_,(PVOID)(&(PVOID&) _class_::_method_));

/*
############## Example usage!
//Let's assume you want to detour the virtual method "int Read(bool b)" from IFileSystem.
class IFileSystem{
public:
virtual int Read(bool b); //VTable index: 0
virtual void Write(char c); //VTable index: 1
bool Delete(int d); //This has NO VTable index
virtual void Read(char e); //VTable index: 2

//Overloaded functions are getting added in opposite order!
virtual Size(int a);//VTable index: 5!!!!!
virtual Size(bool b);//VTable index: 4!!!!!
virtual Size(char c);//VTable index: 3!!!!!

//VTable indexing is straight forward ... (Just with multiple-inherited classes, it's not that easy though)
};

//Then this class can (but hasn't be necessarily) initialized (You see why in code later - Two ways of detouring using VDETOUR_FROM_OBJECT or VDETOUR_FROM_CLASS)
IFileSystem* FILESYSTEM = new IFileSystem;

//To prepare detouring, create a class like this
class CDetour{
public:
VDETOUR_SETUP(int,CDetour,Read,bool b);
//The above macro does simply this
// int Read(bool b); //This is your hook
// static int (CDetour::*__Read)(bool b);
}

//Then define your actual hook "int CDetour::Read(bool b)" with this
VDETOUR_HOOK(int,CDetour,Read,bool b){ //This defines CDetour::Read(bool b);
(this->*__Read)(b); //With this, you can call the original method again
}

//Later in code: Start detouring. Make sure, you use the correct VTable index (here, it is 0)
DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());

//Now if your class already has an initialized object or has no generic constructor, use the already initialized object
VDETOUR_FROM_OBJECT(FILESYSTEM,0,CDetour,Read);
//Or if you want to, simply supply the classname with this method below (but be warned: This macro initializes the class and creates an object for the time it detours. It will destroy the object later)
VDETOUR_FROM_CLASS(IFileSystem,0,CDetour,Read);

DetourTransactionCommit();

//Unloading hooks

//Just run

DetourTransactionBegin();
DetourUpdateThread(GetCurrentThread());

VDETOUR_REMOVE(CDetour,Read);

DetourTransactionCommit();

*/

#endif //VIRTUALMEMBER_H[/cpp][/QUOTE]

Very nice. Are you using detours version 2.1? I am soo bloody used to 1.5's interface.

[aVoN]

Yes, it's 2.1 Express.
Actually, it's not harder to use than 1.5. How far I have seen, trampoline hasn't be defined like it was in 1.5. Just use a function with the calling-convention you want to detour.

[Grocel]

[QUOTE=aVoN;19958096]Yes, I will now.
It's easy to disable the memory protection but I still tried to find a way which is safe concerning VAC (even if it's not banning in GMod).[/QUOTE]

I know not much about DLL's or c++ but, I thing there is a point of memory protection...

[aVoN]

Sure there is a point. But that's out of interests here.

All I need to do is hooking default stuff up and I decided to take detours rather than writing my own "VTable-replacement-hook".