How to create a trampoline function for hook

Question

I\'m interested in hooking and I decided to see if I could hook some functions. I wasn\'t interested in using a library like detours because I want to have the experience of doi

Answer 1

If you want your hook to be safe when called by multiple threads, you don't want to be constantly unhooking and rehooking the original API.

A trampoline is simply a bit of code you generate that replicates the functionality of the first few bytes of the original API (which you overwrote with your jump), then jumps into the API after the bytes you overwrote.

Rather than unhooking the API, calling it and rehooking it you simply call the trampoline.

This is moderately complicated to do on x86 because you need (a fairly minimal) disassembler to find the instruction boundaries. You also need to check that the code you copy into your trampoline doesn't do anything relative to the instruction pointer (like a jmp, branch or call).

This is sufficient to make calls to the hook thread-safe, but you can't create the hook if multiple threads are using the API. For this, you need to hook the function with a two-byte near jump (which can be written atomically). Windows APIs are frequently preceded by a few NOPs (which can be overwritten with a far jump) to provide a target for this near jump.

Doing this on x64 is much more complicated. You can't simply patch the function with a 64-bit far jump (because there isn't one, and instructions to simulate it are often too long). And, depending on what your trampoline does, you may need to add it to the OS's stack unwind information.

I hope this isn't too general.

Answer 2

The defacto standard hooking tutorial is from jbremer and available here

Here is a simple x86 detour and trampoline hook based on this tutorial using Direct3D's EndScene() function as a example:

bool Detour32(char* src, char* dst, const intptr_t len)
{
    if (len < 5) return false;

    DWORD  curProtection;
    VirtualProtect(src, len, PAGE_EXECUTE_READWRITE, &curProtection);

    intptr_t  relativeAddress = (intptr_t)(dst - (intptr_t)src) - 5;

    *src = (char)'\xE9';
    *(intptr_t*)((intptr_t)src + 1) = relativeAddress;

    VirtualProtect(src, len, curProtection, &curProtection);
    return true;
}

char* TrampHook32(char* src, char* dst, const intptr_t len)
{
    // Make sure the length is greater than 5
    if (len < 5) return 0;

    // Create the gateway (len + 5 for the overwritten bytes + the jmp)
    void* gateway = VirtualAlloc(0, len + 5, MEM_COMMIT | MEM_RESERVE, PAGE_EXECUTE_READWRITE);

    //Write the stolen bytes into the gateway
    memcpy(gateway, src, len);

    // Get the gateway to destination addy
    intptr_t  gatewayRelativeAddr = ((intptr_t)src - (intptr_t)gateway) - 5;

    // Add the jmp opcode to the end of the gateway
    *(char*)((intptr_t)gateway + len) = 0xE9;

    // Add the address to the jmp
    *(intptr_t*)((intptr_t)gateway + len + 1) = gatewayRelativeAddr;

    // Perform the detour
    Detour32(src, dst, len);

    return (char*)gateway;
}

typedef HRESULT(APIENTRY* tEndScene)(LPDIRECT3DDEVICE9 pDevice);
tEndScene oEndScene = nullptr;

HRESULT APIENTRY hkEndScene(LPDIRECT3DDEVICE9 pDevice)
{
    //do stuff in here
    return oEndScene(pDevice);
}

//just an example
int main()
{
    oEndScene = (tEndScene)TrampHook32((char*)d3d9Device[42], (char*)hkEndScene, 7);
}