wacked
/
hook_tests

Introduction
============
This project aims to give a simple overview on how good various x64 hooking
engines (on windows) are. I'll try to write various functions, that are hard to
patch and then see how each hooking engine does.

I'll test:
* [EasyHook](https://easyhook.github.io/)
* [PolyHook](https://github.com/stevemk14ebr/PolyHook)
* [MinHook](https://www.codeproject.com/Articles/44326/MinHook-The-Minimalistic-x-x-API-Hooking-Libra)
* [Mhook](http://codefromthe70s.org/mhook24.aspx)

(I'd like to test detours, but I'm not willing to pay for it. So that isn't
tested :( )

There are multiple things that make hooking difficult. Maybe you want to patch
while the application is running -- in that case you might get race conditions,
as the application is executing your half finished hook. Maybe the software has
some self protection features (or other software on the system provides that,
e.g. Trustee Rapport)

Evaluating how the hooking engines stack up against that is not the goal here.
Neither are non-functional criteria, like how fast it is or how much memory it
needs for each hook. This is just about the challenges the function to be
hooked itself poses.

Namely:
* Are jumps relocated?
* What about RIP adressing?
* If there's a loop at the beginning / if it's a tail recurisve function, does
  the hooking engine handle it?
* How good is the dissassembler, how many instructions does it know?
* Can it hook already hooked functions?

At first I will give a short walk through of the architecture, then quickly go
over the test cases. After that come the results and an evaluation for each
engine.

I think I found a flaw in all of them; I'll publish a small POC which should at
least detect the existence of problematic code.

**A word of caution**: my results are worse than expected, so do assume I have
made a mistake in using the libraries. I went into this expecting that some
engines at least would try to detect e.g. the loops back into the first few
bytes. But none did? That's gotta be wrong.

**Another word of caution**: parts of this are rushed and/or ugly. Please
double check parts that seem suspicious. And I'd love to get patches, even for
the most trivial things -- spelling mistakes? Yes please.

Architecture
============
This project is made up of two parts. A .DLL with the test cases and an .exe
that hooks those, tests whether they still work and prints the results.

(I could have done it all in the .exe but this makes it trivial to (at some
point) force the function to be hooked and the target function to be further
apart than 2GB. Just set fixed image bases in the project settings and you're
done)

My main concern was automatically identifying whether the hook worked. I
consider a hook to work if: a) the original function can still execute
successfully *and* b) the hook was called.

The criteria a) is really similar to a unit test. Verify that a function
returns what is expected. So for a) the .exe just runs unit tests after all the
hooks have been applied. Each failing function is reported (or the program
crashes and I can look at the callstack) so I can correlate that with which
hooking engine I'm currently testing and see where those fail. I've used
Catch2 for the unit tests, because I wanted to try it anyway.

From the get-to it was clear that I wanted to test multiple hooking engines.
And they all needed to do the same steps in the same order -- so I implemented
a basic AbstractHookingEngine with a boolean for every test case and make a
child class for each engine. The children classes have to overwrite `hook_all`
and `unhook_all`. Inbetween the calls to that, the unit tests run.

Test case: Small
================
This is just a very small function; it is smaller than the hook code will be -
so how does the library react?
```ASM
_small:
	xor eax, eax
	ret
```

Test case: Branch
=================
Instead of the FASM code I'll show the disassembled version, so you can see the
instruction lengths & offsets.
```ASM
0026 | 48 83 E0 01 | and rax,1
002A | 74 17       | je test_cases.0043 ----+
002C | 48 31 C0    | xor rax,rax            |
002F | 90          | nop                    |
0030 | 90          | nop                    |
0031 | 90          | nop                    |
0032 | 90          | nop                    |
0033 | 90          | nop                    |
0034 | 90          | nop                    |
0035 | 90          | nop                    |
0036 | 90          | nop                    |
0037 | 90          | nop                    |
0038 | 90          | nop                    |
0039 | 90          | nop                    |
003A | 90          | nop                    |
003B | 90          | nop                    |
003C | 90          | nop                    |
003D | 90          | nop                    |
003E | 90          | nop                    |
003F | 90          | nop                    |
0040 | 90          | nop                    |
0041 | 90          | nop                    |
0042 | 90          | nop                    |
0043 | C3          | ret  <-----------------+
```

This function has a branch in the first 5 bytes. Hooking it detour-style isn't
possible without fixing that branch in the trampoline. The NOP sled is just so
the hooking engine can't cheat and just put the whole function into the
trampoline. Instead the jump in the trampoline needs to be modified so it jumps
back to the original destinations

(Preliminary) Results
=====================
+----------+-----+------+------------+---+------+----+-------+
|      Name|Small|Branch|RIP Relative|AVX|RDRAND|Loop|TailRec|
+----------+-----+------+------------+---+------+----+-------+
|  PolyHook|  X  |   X  |      X     | X |      |    |       |
|   MinHook|  X  |   X  |      X     |   |      |    |   X   |
|     MHook|     |      |      X     |   |      |    |       |
+----------+-----+------+------------+---+------+----+-------+