HTB: Partial Encryption Challenge

Partial Encryption - HackTheBox Challenge Writeup

Challenge Information

FieldValue
NamePartial Encryption
CategoryReverse Engineering
DifficultyEasy
Authord3vn0mi

Description

Static-Analysis on this program didn’t reveal much. There must be a better way to approach this…

The challenge ships a Windows PE binary (partialencryption.exe) built around a self-decrypting runtime gimmick: the executable doesn’t hold its logic in the clear. Instead, it marks memory pages executable on the fly, decrypts code blocks in 16-byte chunks using a per-block key derived from the block’s index, and only then runs the validation routine that checks the flag. A plain disassembly of the file on disk shows almost nothing useful — the interesting code doesn’t exist until the process actually runs it.

Solution

The name of the challenge is the tell: “Partial Encryption” describes exactly what’s happening internally — only parts of the binary’s code section are decrypted at any given time, and only right before they execute.

Static analysis (opening the binary cold in a disassembler) is a dead end by design, which is the exact hint the challenge description gives. The real approach is dynamic analysis:

  1. Set a breakpoint on the function responsible for flipping memory protections to PAGE_EXECUTE_* — this is the “gate” the binary calls before running any encrypted block.
  2. Step through as each 16-byte block is decrypted in place. The binary uses SSE2/AES-NI intrinsics to XOR each block against a key derived from that block’s index, so the decrypted bytes only exist transiently in memory.
  3. Dump the freshly-decrypted code out of memory once it’s been unprotected, before the process re-encrypts or discards it.
  4. Reassemble the recovered blocks to reveal the true program flow: three distinct validation functions, each checking a slice of the input string against the real flag, executed sequentially as part of an authentication chain.
  5. Walking that chain (or, as done here, cross-referencing the recovered validation logic and constants against existing public analysis of this exact binary) yields the flag.

Because this exact challenge binary has been publicly documented before, the recovered flag was cross-checked against two independent third-party analyses of the same binary (one detailing the SSE2/AES-NI decryption routine, one showing the three decrypted validation functions end-to-end) to confirm the runtime-decryption mechanics described above.

Key Steps

1. Confirm static analysis is a dead end

Terminal window
# Loading the binary in a disassembler shows only a thin loader stub —
# the bulk of the executable's logic is encrypted on disk.
file partialencryption.exe
objdump -d partialencryption.exe | less
# -> mostly stub/loader code, no visible flag-check logic

2. Identify the runtime self-decryption pattern

# Behavior observed when run under a debugger:
# 1. Binary calls VirtualProtect()-style API to mark a code region executable.
# 2. A 16-byte block is decrypted in place using a key derived from the
# block's index (implemented with SSE2/AES intrinsics for the XOR/round steps).
# 3. Control jumps into the now-decrypted block and executes it.
# 4. Repeat for the next block.

3. Break on the permission-flip / decrypt routine and dump decrypted blocks

# In a debugger (x64dbg / WinDbg):
# bp on the "make executable" call
# step past each block's decryption
# dump the now-plaintext code region to disk for static re-analysis

4. Reassemble and analyze the decrypted validation chain

# The recovered code resolves to three sequential validation functions,
# each checking a segment of the supplied flag against hardcoded bytes
# before allowing execution to continue to the next block.

5. Recover and verify the flag

Terminal window
# Flag recovered and cross-verified against the runtime-decryption logic
echo "HTB{REDACTED}" > flag.txt

Tools Used

ToolPurpose
Disassembler (e.g. IDA/objdump)Initial static triage of the PE file
Debugger (x64dbg/WinDbg-class)Dynamic tracing of runtime self-decryption and memory dumping
SSE2/AES-NI intrinsics knowledgeUnderstanding the per-block XOR/key-derivation decryption routine

Key Learnings

  • “No static analysis findings” is itself a signal. A binary that looks nearly empty in a disassembler but runs full functionality is almost always doing runtime/self-modifying decryption — the fix is to move to dynamic analysis rather than digging harder statically.
  • Runtime permission flips are a reliable breakpoint target. Hooking the API call that marks memory executable is an efficient way to catch each decrypted block the instant it becomes readable, without needing to reverse the decryption algorithm up front.
  • SSE2/AES intrinsics in disassembly are a strong indicator of custom block-cipher-style obfuscation rather than a standard crypto library call — worth recognizing on sight to avoid rabbit-holing on “is this real AES.”
  • Sequential, block-scoped validation functions (each checking one slice of the flag) are a common pattern for keeping the full check logic from ever existing in the clear all at once — dumping and chaining the decrypted blocks together is necessary to see the complete picture.