Lost in Hyperspace - HackTheBox Challenge Writeup

Challenge Information

Description

A cube is the shadow of a tesseract casted on 3 dimensions. I wonder what other secrets may the shadows hold.

The challenge ships a set of high-dimensional token embeddings and hints, via the tesseract analogy, that the interesting structure is a lower-dimensional “shadow” hiding inside a much higher-dimensional space. The task is to find that shadow and read the message projected onto it.

Solution

The provided archive contained a NumPy .npz file holding 110 single-character tokens, each mapped to a 512-dimensional embedding vector. At face value this looks like meaningless noise scattered across 512 dimensions — but the challenge’s own hint says a lower-dimensional shape is being “cast” into that space, so the approach was to hunt for a hidden low-rank signal, project it down, and read off the resulting shape.

1. Recover the real artifact. The staged token_embeddings.npz was 0 bytes. The actual data lived in a password-protected zip in the output directory, named after the session UUID.
2. Load and inspect. Unzipped and loaded the .npz, revealing tokens (110 shuffled single characters) and embeddings (a 110×512 float64 matrix).
3. Find the hidden dimensionality. Mean-centered the embeddings and ran an SVD. The singular value spectrum showed two dominant values towering over the rest, with everything after them flat and small — a classic signature of a genuine 2-D signal buried in isotropic noise across the remaining 510 dimensions.
4. Project down and visualize. Projected the embeddings onto the top-2 principal components and plotted them. The 110 points traced a clean Archimedean spiral with roughly 4 turns — the “shadow” the description promised.
5. Read the message off the spiral. Radius grows monotonically as the spiral winds outward, so sorting points by radius correctly orders the turns but scrambles points near the crowded center. Fixing this required unwrapping the angle within radius-sorted order and re-sorting, which reconstructed the true traversal order of the spiral and revealed the encoded string, with the flag sitting on the outermost arm.

Key Steps

Recover the archive (0-byte staged file → real password-protected zip):

# The staged .npz was empty; the real data was a session-UUID-named zip in /out
7z x -phackthebox -y /out/<session-uuid>.zip

Load the embeddings and inspect their structure:

import numpy as np

d = np.load('token_embeddings.npz', allow_pickle=True)
tokens = d['tokens']          # 110 shuffled single characters
embeddings = d['embeddings']  # 110 x 512 float64

SVD to find the hidden low-rank signal:

# Mean-center then decompose; look for a sharp drop-off in singular values
X = embeddings - embeddings.mean(axis=0)
U, S, Vt = np.linalg.svd(X, full_matrices=False)

print(S[:6])
# [75.1, 69.5, 6.6, 6.5, 6.4, ...]
# -> only 2 singular values dominate: the true signal is rank-2,
#    everything else is isotropic noise

Project onto the top-2 components and plot:

import matplotlib.pyplot as plt

coords = X @ Vt[:2].T  # project onto first 2 principal components

plt.scatter(coords[:, 0], coords[:, 1])
plt.savefig('pca.png')
# -> points trace a ~4-turn Archimedean spiral

Reconstruct traversal order (radius-sort, then unwrap angle):

x, y = coords[:, 0], coords[:, 1]
r = np.hypot(x, y)
theta = np.arctan2(y, x)

# Sort by radius to get turns roughly right...
order = np.argsort(r)

# ...then unwrap the angle within that order to fix the scrambled center
theta_unwrapped = np.unwrap(theta[order])
final_order = order[np.argsort(theta_unwrapped)]

message = ''.join(tokens[final_order])
print(message)
# SUDIEFVKGOELPMNBAZX#4564WQERTUIOF{-!4!}95HLPRI8WBRTYH{{DFPVAZX845CMNBVE}77}SFDCE123____HTB{REDACTED}

The genuine flag sits on the outermost arm of the spiral; everything preceding it — including a couple of decoy {...} sequences — is filler noise designed to mislead a naive “just concatenate the characters” approach.

Tools Used

• Python 3 — scripting and analysis
• NumPy — loading .npz archives, SVD, array manipulation
• Matplotlib — visualizing the PCA projection to spot the spiral
• 7-Zip (7z) — extracting the password-protected challenge archive

Key Learnings

• “Shadow” hints point at dimensionality reduction. The description’s tesseract/cube analogy was a direct pointer to PCA/SVD: a low-dimensional shape embedded in a much higher-dimensional space, recoverable by finding its dominant singular directions.
• The singular value spectrum tells you the true rank. A sharp drop-off after the first k singular values is strong evidence of a genuine k-dimensional signal hidden in noise — worth checking before assuming an embedding matrix is unstructured.
• A spiral encodes order in radius and angle, not just angle. Naively sorting by angle scrambles the reading order near the origin, where points from different turns sit close together; radius must anchor coarse ordering, with angle-unwrapping used to disambiguate fine-grained order within each pass.
• Don’t trust the concatenation blindly. Decoy substrings (stray {/} pairs, junk characters) were deliberately placed along the spiral before the real flag — always scan the full reconstructed string for the actual HTB{REDACTED} pattern rather than assuming the first bracketed run is genuine.