Why So Syscalls? BOF Edition

Background: The Hook Problem

You know the drill. You craft the perfect BOF, compile it EDR shuts you down. Why? Because modern EDRs love to hook high-level Windows APIs if they’re historically known to be related to malware. Every OpenProcess, VirtualAlloc, and GetTokenInformation call gets intercepted, analyzed, and flagged if it smells even slightly malicious.

Enter direct syscalls. By bypassing userland API hooks and talking directly to the kernel through syscalls, we can slip past many EDR detection mechanisms.

Setting Up the Syscall Factory

Step 0: TrustedSec Format Conversion

To demonstrate this technique, we’re going to convert an existing BOF – SafeHarbor which I created previously, and rewrite some of the logic to use syscalls directly rather than the high level APIs. The goal is to maintain the functionality throughout without sacrificing anything.

First things first, we needed to convert the existing BOF to TrustedSec format. Why? Because SysWhispers3 and InlineWhispers3 play nice with TrustedSec’s inline assembly format, and fighting with Visual Studio’s compiler quirks isn’t my idea of fun.

The magic happens in bofdefs.h. I grabbed the header from TrustedSec’s situational awareness BOF repository and added a few missing definitions manually. Think of it as updating your API dictionary before the big syscall translation work.

Step 1: InlineWhispers3 Setup

Time to clone and configure our syscall generation toolkit. Fair warning – the original InlineWhispers3 repo has some bugs that’ll make you pull your hair out. I used radman404’s improved version instead:

git clone https://github.com/radman404/InlineWhispers3 && cd InlineWhispers3

cd SysWhispers3/ && python3 syswhispers.py -p all -a x64 -m jumper -o syscalls_all && cd ..

python3 InlineWhispers3.py --aio

Pro tip: Make sure your makefile includes the -masm=intelflag. I also ditched x86 support – InlineWhispers was not playing nice with it and debugging that is out of scope for this post.

Step 2: Choosing Your Syscall Targets

Not every API call deserves the syscall treatment. I went with gut instinct, experience, and a little help from our AI agents to identify the highest-value targets. Here’s my final list:

– OpenProcess → Sw3NtOpenProcess
– VirtualAlloc → Sw3NtAllocateVirtualMemory
– VirtualFree → Sw3NtFreeVirtualMemory
– VirtualQueryEx → Sw3NtQueryVirtualMemory
– OpenProcessToken → Sw3NtOpenProcessToken
– CloseHandle → Sw3NtClose
– EnumProcesses → Sw3NtQuerySystemInformation
– GetTokenInformation → Sw3NtQueryInformationToken

These are generally the bread and butter of process injection, memory manipulation, and token operations – exactly the kind of behavior that makes EDRs nervous. Some APIs like EnumProcesses required additional logic changes to handle the different data formats returned by NtQuerySystemInformation, but the OPSEC juice was not worth the squeeze for this one in a proof of concept demo.

Implementation: From High-Level to Low-Level

The Syscall Integration (high-level)

After running InlineWhispers3, you’ll have two new best files: syscalls.c and syscalls.h. The header contains your usual structs and declarations, while the C file houses the inline assembly magic that makes direct syscalls possible.

Your includes will look something like this:

#include <windows.h>
#include "beacon.h"
#include "<REDACTED_STUBS_C>"
#include "bofdefs.h"

The Great API Switcheroo

Now comes the fun part – systematically replacing high-level API calls with their syscall equivalents. Start with the easy wins that don’t require logic changes. For instance, swapping `NtClose` calls:

Before

/* MyFree(buffer);
NtClose(tokenHandle);
return result;
*/

After

MyFree(buffer);
Sw3NtClose(tokenHandle);
return result;
*/

Real-World Example: Token Juggling

As an example, here’s a complete function makeover – the `IsCurrentUserProcess` helper from SafeHarbor BOF.

Notice how we’ve gone full syscall mode:

bool IsCurrentUserProcess(HANDLE hProcess, const char* currentUser) {
    HANDLE tokenHandle = NULL;

    //  NTSTATUS status = Sw3NtOpenProcessToken(
        hProcess,
        TOKEN_QUERY,
        &tokenHandle
    );

    if (status != STATUS_SUCCESS || tokenHandle == NULL) {
        return false;
    }

    char* buffer = (char*)MyAlloc(256);
    if (!buffer) {
        Sw3NtClose(tokenHandle);
        return false;
    }

    ULONG returnLength = 0;
    // Placeholder: query token information via lower-level interface
    status = Sw3NtQueryInformationToken(
        tokenHandle,
        TokenUser,
        buffer,
        256,
        &;returnLength
    );

    bool result = (status == STATUS_SUCCESS);

    MyFree(buffer);
    Sw3NtClose(tokenHandle);
    return result;
}

Every API call now goes through the Sw3Nt syscall interface, bypassing userland hooks like a ninja. The function still does exactly what it did before, but now it’s speaking kernel directly. Our goal of switching over to direct syscalls without sacrificing functionality is achieved at this point.

The final implementation can be found on my Github for the full source code and project structure.

OPSEC Considerations

The Good News

Direct sys calls help you dodge many EDR hooks that monitor high-level API calls. You’re essentially taking the scenic route to kernel functionality, longer path but fewer checkpoints.

The Reality Check

• Kernel callbacks and mini filters.
• Behavioral analysis of sys call patterns.
• Hardware-based telemetry (looking at you, Intel CET).
• Stack walking and call origin verification.

Mitigation for the Blue Team

Defenders aren’t helpless against sys call shenanigans:

1. Implement kernel-level monitoring using ETW and kernel callbacks.
2. Monitor for unusual syscall patterns in your SIEM.
3. Deploy behavioral analysis that looks beyond individual API calls.

Conclusion

Converting BOFs to use direct syscalls via InlineWhispers3 gives you a solid edge against userland EDR hooks. The hardest part is which is generating compatible syscall stubs is already solved for you. While this won’t make you invisible to all detection mechanisms, it’s a valuable technique when standard APIs are getting you caught.

Remember: syscalls are just one tool in the evasion toolbox. Combine them with other techniques like ETW patching, AMSI bypasses, and good old-fashioned social engineering for maximum effectiveness.