When VirtualAlloc Isn't VirtualAlloc

Mike S.

Sep 28

11 min read

Using AI-assisted reverse engineering to recover an EDR's decoy modules and colliding exports (and testing them with Crystal Palace)

AI is lowering the bar on reverse engineering work. Endpoint Detection and Response (EDR) components are now squarely in the crosshairs of wayward hackers. And it turns out you can find these components online, left exposed by universities and IT departments, or fromaccess to a host where the software is installed. I fed these components into an AI-driven reverse engineering harness using Ghidra to determine their capabilities.

The structure of modern sensors varies, but for most, there will be a kernel mode driver, a user mode service, an injectable user mode DLL, an ELAM driver, machine learning modules, and several supporting components to handle cloud comms, anti-tamper, and utility functions. Sometimes raw detection logic is included in an installer or can be recovered from disk and/or memory.

The rest of the article will focus on one example of extracting specific detection related functionality from these components and using Crystal Palaceto test them. To do this, I’ve found a vendor implementing Position Independent Code (PIC) traps and Process Environment Block (PEB) decoys through their user-mode injected DLL. This DLL should be injected into most processes on a monitored host, so it’s a great place to implement traps like this. These techniques are rather well-known ways to catch PIC injected into processes, so I’m not breaking any new ground here.

Why should we care? These traps are meant to be an early warning of malicious behavior on a system. Malware will often resolve the location of Windows APIs to abuse the functionality they provide. VirtualAlloc, for example, can be used to allocate memory for staging a follow-on payload. Before jumping into the code, here’s a quick background on traps being set.

Background

When a program is launched normally on Windows, the loader will check the program’s import table and load in the required DLLs. When PIC is injected into an already-running process, it does not have the benefit of using the Windows loader to fetch DLLs on its behalf. As a result, if the PIC wants to use a function exported by a particular DLL, it must first learn where that function exists in memory. One well-established way to do that is to check the Process Environment Block (PEB) for module entries (e.g. kernel32.dll) that export the target function (e.g. VirtualAlloc) that it wants to use. The code can then walk theexport list of the module to obtain the address that the function lives at.

As the PEB is walked, the injected code must have some way to know that it has found the correct module name and target function name. The pseudocode might look something like this:

for module in peb.Ldr.Modules: if module.name == "kernel32.dll": export_dir += module.exports for export in export_dir: if export.name == "VirtualAlloc": return export.address

This code would carry the literal strings “kernel32.dll” and “VirtualAlloc”, which a human analyst or EDR may pick up on immediately. To avoid that, malware authors began using small hashing algorithms to compute a hash for the target modules and functions and using those in the code instead. Before linking the final PIC, a malware author can calculate the hashes for the target module and export.

ror13hash("kernel32.dll") = 0x8FECD63F ror13hash("VirtualAlloc") = 0x91AFCA54

The pseudocode might look something like this:

def hash(name):

return ror13hash(name)

for module in peb.Ldr.Modules:

if hash(module.name) == 0x8FECD63F:

export_dir += module.exports

for export in export_dir:

if hash(export.name) == 0x91AFCA54:

return export.address

This allows the code to still check for the correct names, while not embedding the “kernel32.dll” and “VirtualAlloc” strings.The traps we’re looking at work by placing decoys directly into the path of malware walking the PEB. This is probably best summed up with this sentence from Exploiting Weak Shellcode Hashes to Thwart Module Discovery; or, Go Home, Malware, You’re Drunk!: “If we can generate a new module and/or function name that hashes to the same value that the malware is looking for, and if we ensure that the decoy module/function occurs before the real one in the EAT linked list, then any time that function is called we will know it is from malicious code.”

Process Environment Block (PEB) Trap

The PEB trap works by placing decoy modules in the PEB of the protected process. Claude found some of the decoy strings and references to a filename PEBtrap.cpp in the binary as shown below.

These fake modules use look-alike namesof the impersonated modules (e.g. Kernel32.dll becomes Kern3l32.dll). Anything that attempts to use them would ultimately be forwarded to the intended module, but would first pass through the EDR’s control, presumably firing an alert or flagging the process as suspicious on the way.

Position Independent Code (PIC) Trap

The PIC trap works by placing fake exports under modules like kernel32. Claude picked up on some of the decoy strings stored in the binary.

The names will be exported under kernel32.dll in the protected process. These names werespecifically chosen because they create hash collisions with the real target export names.We’ll explore this in the next section. As with the PEB trap, anything that attempts to use them would be forwarded to the intended export but first moves through the EDR’s control.

Testing It

I wanted to observe the actual behavior in a live process. In lieu of recreating the logic from scratch, I opted to have Claude build a harness that will load the vendor’s DLL into a controlled process to test against. I’m running unknown code here, so I moved over to a disposable VM with no internet access. Claude created a simple tool to start the new process, load in the vendor DLL, pause, arm the traps, and then pause again. This simulates a process running on a protected host with the EDR installed.

A note on the harness: It arms the traps manually. When loaded, DllMain performs a series of checks to determine if it is being driven by the agent. Those checks fail in our case, but we’re able to move to the addresses where the installer functions for the traps live and call them directly. This ensures that the vendor’s code is what is running to enable these traps, not ours. It’s possible that the traps are not in the exact same state as a production machine, but I think it works for what we’re testing.

I used the Crystal Palace project partly because it implements dynamic function resolution(dfr) out of the box and partly because I just wanted an excuse to use it. It is well-suited to developing the test cases we’ll show later. I created the following simple bit of PIC to resolve and print the location of the VirtualAlloc API after being injected into a process.

* Redistribution and use in source and binary forms, with or without modification, are

* permitted provided that the following conditions are met:

* 1. Redistributions of source code must retain the above copyright notice, this list of

* conditions and the following disclaimer.

* 2. Redistributions in binary form must reproduce the above copyright notice, this list of

* conditions and the following disclaimer in the documentation and/or other materials provided

* with the distribution.

* 3. Neither the name of the copyright holder nor the names of its contributors may be used to

* endorse or promote products derived from this software without specific prior written

* permission.

* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS”

* AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE

* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE

* ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE

* LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR

* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF

* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS

* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN

* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING

* IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF

* SUCH DAMAGE.

#include <windows.h>

#include "tcg.h"

WINBASEAPI LPVOID WINAPI KERNEL32$VirtualAlloc(LPVOID lpAddress, SIZE_T dwSize, DWORD flAllocationType, DWORD flProtect);

FARPROC resolve(DWORD modHash, DWORD funcHash) {

HANDLE hModule = findModuleByHash(modHash);

return findFunctionByHash(hModule, funcHash);

}

void go(void) {

dprintf("[ror13] 0x%p: KERNEL32$VirtualAlloc\n", KERNEL32$VirtualAlloc);

}

I used the following main.spec file. This configures dynamic function resolution to use ROR13 hashing. The dfr function lives in resolve_eat.c in the LibTCG.

x64:

load "bin/main.x64.o"

make pic +optimize +gofirst

# resolve hashes

dfr "resolve" "ror13"

# Merge in libtcg for resolver functions

mergelib "../tcg/libtcg/libtcg.x64.zip"

export

I then compiled and linked the shellcode into a file.

I used this simple remote shellcode injector to inject PIC into the “protected” process.

BOOL InjectShellcode(HANDLE hProcess, PBYTE pShellcode, SIZE_T sSize) {

PVOID pAddress = NULL;

SIZE_T sBytesWritten = 0;

DWORD dwOldProtection = 0;

pAddress = VirtualAllocEx(hProcess, NULL, sSize, MEM_COMMIT | MEM_RESERVE, PAGE_READWRITE);

if (!pAddress) {

printf("[!] VirtualAllocEx failed: %d\n", GetLastError());

return FALSE;

}

if (!WriteProcessMemory(hProcess, pAddress, pShellcode, sSize, &sBytesWritten) || sBytesWritten != sSize) {

printf("[!] WriteProcessMemory failed: %d\n", GetLastError());

return FALSE;

}

if (!VirtualProtectEx(hProcess, pAddress, sSize, PAGE_EXECUTE_READWRITE, &dwOldProtection)) {

printf("[!] VirtualProtectEx failed: %d\n", GetLastError());

return FALSE;

}

if (!CreateRemoteThread(hProcess, NULL, 0, (LPTHREAD_START_ROUTINE)pAddress, NULL, 0, NULL)) {

printf("[!] CreateRemoteThread failed: %d\n", GetLastError());

return FALSE;

}

printf("[-] CreateRemoteThread success\n");

return TRUE;

}

First, I stripped the binary Claude created to drive the DLL down to the essentials. It will start a new process, print the PID, check the number of modules in the PEB, the address space of kernel32, and the number of exports for kernel32. The traps are not armed at this point.

Note the start and end addresses for kernel32.dll (0x7ffe57700000 - 0x7ffe577bd000). Also note that there are currently 14 modules in the PEB and 1,647 functions exported by kernel32. Next, I injected the PIC into that process.

I’m using debugview to capture the output from the PIC.

The address for VirtualAlloc, 0x7FFE5771AB20, sits inside the kernel32.dll address space of 0x7FFE57700000 – 0x7FFE577BD000. Now let’s arm the traps and redo the injection.

We can see there is now a total of 16 modules (+2) in the PEB and 1,689 exports (+42) for kernel32.dll. I injected the PIC into the same process after arming the traps, and debugviewcaptured the output:

VirtualAlloc now resolves to the address 0x00007FFE577C06D5, which exists outside of the kernel32.dll address space. Let’s check what name is exported at that address.

Our PIC is matching on the string “8|H(58”. Since we’re using the default ROR13 hashing from dfr, this string must be creating a hash collision with the string VirtualAlloc. Let’s check this with https://github.com/ihack4falafel/ROR13HashGenerator.

You can see that the two strings create the same hash when using ROR13. This confirms the reverse engineering work and the theory behind the PIC trap. And lucky for us, Crystal Palace already provides a way to change the hashing algorithm used to resolve functions at https://tradecraftgarden.org/simpleapi.html. This allows us to replace the ROR13 hashing algorithm with SDBM, DJB2, or FNV1A.

Here’s the new main.spec file. The “pack” command takes in the provided algorithm name from the command line and the “patch” command ships it off to be printed by the PIC. This was completely unnecessary (and debug printing these strings actually adds back in the IoCs we’re avoiding by using dfr in the first place) but was fun to play around with. The “call”command pointing to the apihash.spec file is what actually implements what we care about.

x64:

load "bin/main.x64.o"

make pic +optimize +gofirst

echo "apihashing algo:" %algoname

pack $ALGO "z" %algoname

patch "__ALGONAME__" $ALGO

call "apihash.spec" %algoname

# Merge in libtcg for resolver functions

mergelib "../tcg/libtcg/libtcg.x64.zip"

export

I’m using the apihash.c and apihash.spec files verbatim from the site. The PIC now receivesthe algorithm choice from the spec file and includes it in the printed line.