glassbox

Grab vLLM's attention.

glassbox is a vLLM plugin for extracting model internals during inference and turning them into compact, structured signals for downstream reliability systems. These signals include spectral features and flow-based features that provide a routing-oriented view of model behavior.

The main use case is online or offline analysis of failure modes in LLM generation: hallucination detection, task drift detection, uncertainty quantification, routing analysis, and other forms of model-behavior monitoring.

The primary implementation is a custom vLLM attention backend in glassbox/backends/. There is also an experimental torch.compile / FX instrumentation path in glassbox/passes/, but the custom backend is the working path.

Why This Exists

Transformer internals can reveal a great deal about model behavior. They are useful for monitoring, debugging, and failure analysis, but most of the underlying objects are too large to inspect directly in a practical inference setting.

Raw activations and full attention matrices are expensive to retain, and modern attention systems are specifically engineered to avoid materializing the full L x L object during efficient inference. Because of that, many tools for inspecting transformer internals live in research harnesses around HuggingFace models rather than in production-grade inference stacks.

That creates a gap between interpretability results in papers and practical deployment in real systems. glassbox is built to close that gap by efficiently extracting compact signals of how attention is behaving:

• Is one mode dominating, or are multiple modes active?
• Is routing bottlenecked through a narrow channel?
• Is behavior becoming more asymmetric or circulatory over time?
• Do certain layers or heads shift sharply when the model starts to drift or hallucinate?

What It Extracts

At configurable intervals during inference, glassbox computes features from different stages of the attention computation. Each signal runs on the attention matrix its mathematics requires — see docs/operator-choice.md for the operator rationale and causal-masking caveats.

Signal	Operator	Key outputs	Source
spectral	pre-softmax scores S = QKᵀ	sv1, sv_ratio, sv_entropy (score-geometry / rank)	—
routing	degree-normalized M = D_Q^{-1/2} A D_K^{-1/2}	SVD spectrum, phi_hat (Cheeger conductance), asym_index (normalized asymmetry)	Dahlem et al. (upcoming)
asymmetry	row-stochastic P	G, Gamma, C — Hodge gradient/curl split G²=Γ²+C²	Dahlem et al. (upcoming)
cyclic	pre-softmax S (sign tournament)	T_cyc — count of non-transitive (cyclic) triangles	Dahlem et al. (upcoming)
magnetic	pre-softmax S (magnetic Laplacian L_φ = D − A⊙e^{iθ})	frustration (λ₁; 0 ⟺ balanced orientation), phase_curl / phase_curl_w (streamable frustration energy)	Dahlem et al. (upcoming)
tracker	raw post-softmax A = softmax(QKᵀ/√d)	SVD spectrum, sigma2, commutator_norm	AttentionTracker (arXiv:2411.00348)
selfattn	attention diagonal diag(A)	attn_diag_logmean, eigvals	LLM-Check (NeurIPS 2024)
laplacian	in-degree graph Laplacian L = D_in − A	eigvals (top-k)	LapEigvals (EMNLP 2025, arXiv:2502.17598)

Only spectral is enabled by default. Enable any signal via GlassboxConfig (e.g. GlassboxConfig(asymmetry={"enabled": True})) or the --signal CLI flag.

Each observation is emitted as an SVDSnapshot (glassbox/results.py) — see Output Format.

Spectral signal — scores matrix features (pre-softmax scores)

These come from the singular values of the pre-softmax scores matrix S = QK^T.

Feature	Formula	Meaning
sv1	σ₁(S)	Leading singular value of the scores matrix. Strength of the dominant attention mode
sv_ratio	σ₁(S) / σ₂(S)	Spectral sharpness. High values suggest near-rank-1 structure; low values suggest multiple competing modes
sv_entropy	-Σ pᵢ log pᵢ, with pᵢ = σᵢ / Σⱼ σⱼ	Entropy of the normalized singular-value distribution. Measures how concentrated or diffuse the spectrum is

Routing signal — degree-normalized matrix features (post-softmax attention) — Dahlem et al. (upcoming)

These come from the singular values and asymmetry analysis of the normalized operator M.

Feature	Formula	Meaning
sv1	σ₁(M)	Leading singular value of M. Dominance of the top routing mode
sv_ratio	σ₁(M) / σ₂(M)	Separation between the top routing mode and the rest
sv_entropy	-Σ pᵢ log pᵢ, with pᵢ = σᵢ / Σⱼ σⱼ	Entropy of the normalized singular-value distribution. Spread of routing mass across modes
sigma2	σ₂(M)	Second singular value of M. Raw spectral-gap measure and persistence of non-dominant routing structure
phi_hat	1 - σ₂(M)	Conductance-like bottleneck score. High φ̂ means attention concentrates through a single dominant mode; low φ̂ means multiple competing routing paths
asym_index	‖M_asym‖_F / ‖M‖_F	Normalized asymmetry index, where M_asym = (M - Mᵀ) / 2.

The routing metrics live in glassbox/hodge.py. The feature schemas are defined in glassbox/results.py.

Asymmetry signal — Hodge gradient/curl split on row-stochastic attention (Dahlem et al., upcoming)

The directed-routing asymmetry and its Hodge decomposition, computed on the row-stochastic post-softmax attention P (not the degree-normalized M — the normalization is an asymmetric scaling that distorts the antisymmetric structure; see operator-choice). A = (P - Pᵀ) / 2.

Feature	Formula	Meaning
G	‖P_asym‖_F / ‖P‖_F	Total asymmetry. Fraction of P's energy in the antisymmetric part
Gamma	‖A_grad‖_F / ‖P‖_F, with ‖A_grad‖² = 2‖r‖²/L, r = A_asym·1	Gradient (hierarchical) coefficient — exact via the row-sum identity in every tier
C	‖A_curl‖_F / ‖P‖_F (the Pythagorean residual, G² = Γ² + C²)	Curl (circulatory) coefficient — the divergence-free part

Matrix-free above the threshold via a direct Hutchinson estimator on ‖P_asym z‖² (Route B, non-negative); exact materialized below. Supports an exact incremental (delta-token) mode and block-diagonal streaming. Implementation: glassbox/diagnostics/asymmetry.py.

Cyclic signal — cyclic-triangle count on the pre-softmax tournament (Dahlem et al., upcoming)

Counts non-transitive (3-cycle) preference triangles in the sign tournament ω(S)_ij = sgn(qᵢ·kⱼ - qⱼ·kᵢ) on the unmasked pre-softmax scores S. Vacuous on causal post-softmax attention (lower-triangular ⇒ transitive ⇒ T_cyc = 0), so it must run on the raw scores.

Feature	Formula	Meaning
T_cyc	C(n,3) - Σᵢ C(sᵢ, 2) (Kendall identity, sᵢ = out-degree)	Number of cyclic triples {i,j,k} — how non-transitive the head's directional preferences are

Exact integer count in O(n) state (no SVD); supports exact one-token-at-a-time streaming. Implementation: glassbox/diagnostics/cyclic_triangles.py.

Magnetic signal — magnetic-Laplacian frustration on the pre-softmax tournament (Dahlem et al., upcoming)

The spectral partner of cyclic: the frustration of the Hermitian magnetic Laplacian L_φ = D - A⊙e^{iθ} on the unmasked pre-softmax scores, with magnitude W = (|S_ij| + |S_ji|)/2 and phase θ = arctan((S_ij - S_ji)/(S_ij + S_ji)).

Feature	Formula	Meaning
frustration (λ₁)	smallest eigenvalue of L_φ (≥ 0)	0 ⟺ balanced orientation (a coherent global ranking exists); > 0 ⟺ cyclic preference loops that cannot be gauged away
phase_curl	‖θ‖² − 2‖θ·1‖²/L (Hodge curl energy of θ)	Streamable frustration energy — eigensolver-free, 0 ⟺ balanced; the same row-sum identity as the asymmetry curl
phase_curl_w	Σ W_ij θ_ij² − 2 Σ b_i²/d_i (magnitude-weighted curl)	The faithful streamable λ₁ proxy — downweights weak/near-symmetric edges; the one to monitor in real time

Dense Hermitian eigendecomposition below the threshold; complex-Hermitian Lanczos above. In incremental mode the phase-curls stream exactly (delta-token fold, no eigensolve; frustration is left None). Gauge-invariant, so it is unchanged by degree normalization. Implementation: glassbox/diagnostics/magnetic.py; full guide: docs/magnetic-laplacian.md.

Tracker signal — AttentionTracker features (raw post-softmax attention) — arXiv:2411.00348

These come from the raw post-softmax attention matrix A = softmax(QK^T / sqrt(d)), without degree normalization. Based on the AttentionTracker paper, which uses these features for mechanistic classification of failure modes (prompt injection vs hallucination).

Currently implements the span-independent features only. Span-aware features (focus_score, cut_flow) require instruction/data span boundaries and are planned for a future release.

Feature	Formula	Meaning
sigma2	σ₂(A)	Second singular value of the raw attention matrix. Persistence of non-dominant attention structure
sigma2_asym	σ₂(A_asym)	Second singular value of A_asym = (A - Aᵀ) / 2. Whether the irreversible (asymmetric) component has multiple significant modes
commutator_norm	‖[A_sym, A_asym]‖_F / ‖A‖_F	Coupling between symmetric and antisymmetric parts of attention, where [X, Y] = XY - YX

The computation reuses the same two-tier approach as the degree-normalized operator: materialized for L <= threshold, matrix-free (blocked-streaming matvecs) above. The matrix-free path reuses existing matvec infrastructure by treating A as the special case of M where D_K^{-1/2} = I.

The feature computation lives in glassbox/attention_tracker.py.

SelfAttn signal — attention diagonal features (LLM-Check, NeurIPS 2024 + LapEigvals, EMNLP 2025)

These come from the diagonal of the post-softmax attention matrix A = softmax(QK^T / sqrt(d)).

Feature	Formula	Meaning
attn_diag_logmean	mean_i(log(A[i,i]))	Mean log self-attention weight. Higher values indicate stronger self-attention, which correlates with model confidence and factuality
eigvals	topk(diag(A))	Top-k diagonal values of A (descending). For causal attention these are A's eigenvalues; used as a baseline in LapEigvals

The matrix-free path avoids materializing A by computing log(A[i,i]) = s_ii - logsumexp_i where s_ii = Q[i]·K[i]/sqrt(d) (O(Ld)) and logsumexp is computed blockwise.

Implementation: glassbox/attention_diagonal.py. References: LLM-Check, LapEigvals.

Two additional LLM-Check extractors require signals the attention backend cannot see:

• LLMCheckLogitsExtractor — entropy and perplexity from output logits. Needs the final lm_head output, not Q/K.
• HiddenStateCovarianceExtractor — SVD of centered covariance of hidden states. Needs per-layer hidden states before projection.

These will require either torch.compile passes (see glassbox/passes/) or vLLM observation hooks.

Laplacian signal — eigenvalue features (LapEigvals, EMNLP 2025) — arXiv:2502.17598

These come from the in-degree graph Laplacian of the attention matrix: L = D_in - A, where D_in[i,i] = Σ_j A[j,i] (column sums of A). Treating attention as a weighted directed graph, the Laplacian diagonal captures how much attention each token receives from other tokens (in-degree minus self-attention).

For causal (lower-triangular) attention matrices, L is also triangular, so its eigenvalues are its diagonal entries — no eigendecomposition needed. The features are the top-k largest diagonal values.

Feature	Formula	Meaning
eigvals	topk(diag(D_in - A))	Top-k Laplacian diagonal values (descending). Profile of how attention in-degree is distributed across tokens

The two-tier approach reuses existing blocked-streaming infrastructure: apply_AT_blocked for column sums and compute_logsumexp_blocked for the attention diagonal. No SVD or eigendecomposition is performed.

The paper also constructs a multi-layer graph where layers are connected by vertical edges weighted by the next layer's self-attention diagonal. This gives richer features but requires cross-layer aggregation; the current implementation covers the single-layer case. The multi-layer variant is planned as a post-processing step over emitted per-layer features.

Implementation: glassbox/laplacian_eigvals.py. Reference: LapEigvals.

More features coming soon

Span-aware AttentionTracker features (raw post-softmax attention)

The AttentionTracker paper also defines focus_score (attention mass from data tokens to instruction span) and cut_flow (net directed attention flow between instruction and data regions). These require instruction/data span boundaries, which need a mechanism to pass into the backend (config, per-request metadata, or marker-token detection).

Transport features from the Degree-normalized matrix (post-softmax attention, value-weighted routing)

These move beyond pure attention geometry and start incorporating what is actually being transported through the head.

Curl spectrum features from the Degree-normalized matrix (post-softmax attention, per-dimension value analysis)

These summarize how curl-like behavior is distributed across important value dimensions.

LayerNorm-weighted features

These modulate routing features by the effective LayerNorm gain, with the goal of emphasizing heads and layers whose routed signal is more strongly amplified by the surrounding network.

How It Integrates With vLLM

The package registers itself through vLLM's plugin entrypoint and exposes a CUSTOM attention backend:

• Entry point: glassbox.vllm_plugin:register_svd_backend
• Backend: glassbox.backends.svd_backend.SVDTritonAttentionBackend

At runtime, the backend:

1. Calls the normal Triton attention implementation unchanged.
2. Captures and accumulates Q slices across prefill and decode for the active sequence, subject to the configured buffer policy.
3. Extracts K from vLLM's paged KV cache when a snapshot is due.
4. Dispatches to Diagnostic.reduce() for each enabled signal, optionally calling Diagnostic.accumulate() to merge local results into a running global state.
5. Emits snapshots via the handler pipeline (JSONL, OTel, or custom).

This lets you observe attention structure during real generation rather than in a separate offline re-run.

The Diagnostic Protocol

Every signal implements the Diagnostic protocol (glassbox/diagnostic.py) — it is the single dispatch path for all signal computation, batch or streaming. On each fire, the backend hands the diagnostic a window of per-head (Q, K) data and calls:

Method	Purpose
reduce(Qh, Kh, L, prior_state=...)	Compute the window's scalar features (the detect readout)
witness(Qh, Kh, L)	Per-token localization vector (the localize readout; optional, emit_witness=True)
accumulate(local, state)	Persist per-signal state across fires; the backend threads it back into the next reduce() as prior_state

reduce() is always exact for the window it sees — the SVD, Hodge decomposition, or Laplacian eigenvalues of the windowed attention submatrix. Whether statistics accumulated across windows are meaningful depends on each signal's algebra; the streaming modes below make that explicit.

Q-buffer windowing

For long or unbounded sequences, retaining the full Q buffer is impractical (O(L·H·d) per layer). The window each reduce() sees is controlled by q_buffer_max_tokens and q_buffer_mode:

• sliding (default) — keep the last W tokens: append one, trim the oldest, each decode step. Signals fire on their own interval, so consecutive observations overlap by W − interval tokens. The natural mode for continuous monitoring.
• tumbling — accumulate W tokens, fire all enabled signals, flush. The window size is the cadence (per-signal interval is ignored). Consecutive windows are disjoint — the window independence that additive local→global accumulation requires.
• q_buffer_max_tokens: 0 — unbounded: the buffer grows with the sequence and every fire sees the full prefix.

q_buffer_max_tokens: 512
q_buffer_mode: sliding   # or "tumbling"

Streaming modes

By default each fire reports its own window, and accumulate() keeps only the latest result. Signals whose algebra permits it expose streaming flags on their config:

• streaming: true (block-diagonal global) — sum additive sufficient statistics over disjoint windows (e.g. the asymmetry signal's ‖P_asym‖² / ‖P‖² Frobenius partials). Sound only over tumbling windows.
• incremental: true (exact full-operator) — fold only the delta tokens since the last fire into running state, reproducing the full-sequence statistic exactly (asymmetry G/Γ/C; the cyclic count). Requires the unbounded buffer.

A global streaming statistic is only correct if the statistic's algebra permits it (additive vs spectral/combinatorial) and the window matches. See docs/streaming-modes.md for the per-diagnostic matrix; GlassboxConfig rejects unsound mode↔window combinations at construction.

How We Avoid Materializing The Full L x L Matrix

The key design goal is to avoid building full score or attention matrices whenever sequence length grows.

1. Matrix-free multiplies for S = QK^T

For any vector v:

• Sv = Q(K^T v)
• S^T u = K(Q^T u)

That means applying S or S^T only requires two thin L x d multiplies instead of constructing an L x L matrix. In code, this is implemented by:

• matvec_S() in glassbox/svd.py
• matvec_ST() in glassbox/svd.py

Both the randomized SVD and Lanczos implementations consume only these matvecs.

2. Randomized SVD and Lanczos operate on operators, not matrices

glassbox/svd.py provides:

• randomized_svd()
• svd_via_lanczos() (Golub–Kahan–Lanczos bidiagonalization)
• hermitian_lanczos() (real- or complex-Hermitian eigenpairs; used by the magnetic signal)

All operate on matvec callables, so they never require the full matrix to exist in memory.

3. Blocked row streaming for post-softmax attention

For the degree-normalized operator, some quantities depend on softmaxed attention. When sequence length is large, glassbox uses blocked streaming:

• apply_A_blocked() / apply_AT_blocked()
• compute_dk_blocked() / compute_logsumexp_blocked() / compute_M_fro_norm_blocked()
• matvec_M_blocked() / matvec_MT_blocked()
• matvec_Masym_blocked() / matvec_Msym_blocked() / matvec_commutator_blocked() (the Hodge/asymmetry operators)

These compute the effect of A, M, or their symmetric/antisymmetric parts in row blocks, keeping memory bounded by the block size instead of L^2.

4. Two-tier execution for the normalized operator

For shorter sequences, glassbox will materialize the normalized operator because it is simple and exact. For longer sequences, it switches to the matrix-free path.

In practice:

• if L <= threshold, use a materialized path
• if L > threshold, use blocked matrix-free operators

Both tiers produce equivalent results within numerical tolerance. The materialized path is faster for short sequences (~1.8× at L=256 on A10G) and the matrix-free path wins for long ones; the default 512 sits near the crossover. Setting threshold=0 forces the matrix-free path for every sequence (L > 0 is always true), guaranteeing no full L×L matrix is ever materialized — useful for memory-constrained deployments or streaming with large windows, at the cost of the small-L speedup.

That behavior is implemented in each threshold-based diagnostic's reduce() method (routing, asymmetry, magnetic, tracker, selfattn, laplacian).

5. On-demand entry lookup for transpose access

Some asymmetry computations need entries of Mᵀ (e.g. M[j,i] while streaming row block i). Instead of building all of M, the matrix-free path computes only the needed entries on demand with get_M_entries_batch() in glassbox/svd.py.

Signal Emission Architecture

Extracted signals flow through a pluggable handler system with two tiers, designed for two downstream use cases: offline training of detection models and real-time inference-time detection.

                    ┌─────────────────────┐
                    │   Attention Backend  │
                    │   (SVDSnapshot)      │
                    └────────┬────────────┘
                             │
                 ┌───────────┴───────────┐
                 │                       │
          Tier 2: Full stream     Tier 1: Real-time
          (offline / bulk)        (inference-time)
                 │                       │
         ┌───────┴───────┐          OtelHandler
         │               │         glassbox.* spans
    JsonlHandler    Custom handler       │
    .jsonl file    (Kafka, Redis,   OTel Collector
                    webhook, etc.)       │
                                   Jaeger / detector
                                   / alerting

Tier 2 — Full feature stream (training / bulk analysis): JsonlHandler writes every snapshot as a JSON line. Custom handlers can implement the SnapshotHandler protocol to forward snapshots to Kafka, Redis Streams, or any other sink.

Tier 1 — OTel integration (real-time detection): OtelHandler emits each snapshot as a short-lived OpenTelemetry span with glassbox.* attributes (signal type, layer, head, step, and all derived features). It piggybacks on vLLM's global TracerProvider — when vLLM is started with --otlp-traces-endpoint, Glassbox spans flow through the same collector (Jaeger, Tempo, Datadog, etc.) with zero additional configuration. The heads, interval, and signal selection should be configured to match what your trained detection model expects.

Multiple handlers can be active simultaneously (e.g. JSONL for archival + OTel for real-time). When neither output nor otel is configured, a LoggingHandler logs snapshots to stderr.

Custom handlers

Any object with handle(snapshot) and close() methods satisfies the SnapshotHandler protocol:

from glassbox import SnapshotHandler, SVDSnapshot
from glassbox.backends.svd_backend import SVDTritonAttentionImpl

class MyHandler:
    def handle(self, snapshot: SVDSnapshot) -> None:
        # forward to Kafka, Redis, a classifier, etc.
        ...
    def close(self) -> None:
        ...

# Register before engine creation
SVDTritonAttentionImpl._handlers.append(MyHandler())

Output Format

The backend emits one JSON object per observation. Each row contains:

• signal: which signal produced the row (spectral, routing, asymmetry, cyclic, magnetic, tracker, selfattn, or laplacian)
• request_id
• layer and layer_idx
• head
• step
• L
• singular_values
• tier: materialized or matrix_free for signals that use the two-tier approach
• features: derived metrics for that observation
• witness: per-token localization vector, when emit_witness is on

This format is designed to feed downstream systems directly. You can:

• train hallucination detectors on snapshot features
• compare truthful vs hallucinated generations
• monitor drift across prompts or tasks
• aggregate by head, layer, request, or dataset

The glassbox-extract CLI can also write a wide Parquet file.

Example Downstream Uses

• Hallucination detection: compare spectral and routing signatures between factual and hallucinated responses
• Task drift detection: identify layer/head regimes that diverge when the model loses the task
• Uncertainty quantification: use spectral concentration and routing asymmetry as auxiliary confidence signals
• Failure mode analysis: inspect how internal structure changes across prompts, models, or checkpoints

Installation

pip install -e .

This pulls the declared dependencies (vllm>=0.15.1, pydantic-settings[yaml], click). If you already have a working vLLM environment (e.g. a GPU image with a prebuilt vLLM), install into that environment and your existing vLLM is used as-is.

For local development:

pip install -e .[dev]

For OpenTelemetry support (already present when running inside vLLM):

pip install -e .[otel]

Running Glassbox

There are three ways to run Glassbox, depending on your use case:

Mode	Use case	Typical emission
vllm serve	Production / inference-time detection	OTel spans via glassbox.yaml
glassbox-run	Development / single-prompt testing	OTel (--otel) or JSONL (--output)
glassbox-extract	Offline feature extraction for training detection models	JSONL + Parquet

vllm serve — production inference

vllm serve facebook/opt-125m --attention-backend CUSTOM --enforce-eager

Glassbox registers itself via the vllm.general_plugins entry point — vLLM loads it automatically. Configure via a glassbox.yaml in the working directory:

spectral:
  enabled: true
  interval: 32
  heads: [0]
emit:
  otel: true

When vLLM is started with --otlp-traces-endpoint, Glassbox spans flow through the same OTel collector with zero additional configuration. The heads, interval, and signal selection should match what your trained detection model expects.

glassbox-run — single-prompt testing

glassbox-run \
  --model facebook/opt-125m \
  --signal spectral \
  --interval 16 \
  --rank 4 \
  --heads 0 \
  --otel \
  --prompt "The future of artificial intelligence is"

Launches vLLM with attention_backend="CUSTOM" and enforce_eager=True. Supports the config surface as flags — --signal, --interval, --rank, --method, --heads, --threshold, --block-size, --otel, --output, --q-buffer-max-tokens, --q-buffer-mode (see glassbox-run --help). The --output flag writes JSONL and is available for debugging/archival, but the typical runner use case is --otel. Settings can also be loaded from a glassbox.yaml in the working directory.

glassbox-extract — offline feature extraction

glassbox-extract \
  --model Qwen/Qwen2-7B-Instruct \
  --dataset halueval_hallucination \
  --max-samples 200 \
  --signal spectral,routing,asymmetry,cyclic,magnetic,tracker,selfattn,laplacian \
  --parquet

Runs two-phase prefill (question-only + full prompt+response) on labeled datasets and produces:

• samples.jsonl — per-request sample metadata
• svd_features.jsonl — snapshot features
• features.parquet — optional wide Parquet for downstream training (with --parquet)

Output goes to experiments/results/{timestamp}/ by default, or override with --outdir. Also supports --otel (though the typical use case for extraction is JSONL or parquet).