Jun-Hyun Bae, Wonyong Jo, Jaehyup Lee, Heechul Jung
Kyungpook National University
Presentation
Abstract
Text-to-image diffusion models utilize cross-attention to integrate textual information into the visual latent space, yet the transformation from text embeddings to latent features remains largely unexplored. We provide a mechanistic analysis of the output-value (OV) circuits within cross-attention layers through spectral analysis via singular value decomposition. Our analysis demonstrates that semantic concepts are encoded in low-dimensional subspaces spanned by singular vectors in OV circuits across cross-attention heads. To verify this, we intervene on concept-related components in the diffusion process, demonstrating that intervention on identified spectral components affects conceptual changes. We further validate these findings by examining visual outputs of isolated subspaces and their alignment with text embedding space. Through this mechanistic understanding, we demonstrate that simply nullifying these spectral components can achieve targeted concept removal with performance comparable to existing methods while providing interpretability. Our work reveals how cross-attention layers encode semantic concepts in spectral subspaces of OV circuits, providing mechanistic insights and enabling precise concept manipulation without retraining.
Overview
In Stable Diffusion, Van Gogh, Monet, and Picasso share cross-attention heads but have high contributions along different singular vectors within them. This paper decomposes the OV transformations inside each head to analyze these differences.
We apply SVD to the OV matrices and compare prompt pairs such as “a mountain” and “a mountain in Van Gogh style”. Scaling components whose responses differ strongly between the prompts changes the style of generated images. When only style-related OV components are retained, characteristic brushstrokes and forms remain even as scene details disappear. The same analysis is also applied to removing nudity-related components.

Spectral isolation of semantic concepts. For each concept, the top row removes the concept-related components and the bottom row keeps only those components (top-k%). Style concepts (Van Gogh, Picasso) reduce to stylistic patterns, such as brushstrokes and cubist shapes, separated from scene content, whereas the content concept (nudity) reconstructs complete human forms. This suggests that the model employs qualitatively different encoding strategies depending on concept type.
Method
The output that cross-attention adds to the residual stream factorizes into two circuits. The QK circuit computes attention weights that determine which text tokens each spatial position attends to, while the OV circuit \(\mathbf{W}_{\text{OV}} = \mathbf{W}_{\text{V}}\mathbf{W}_{\text{O}}\) transforms the attended tokens’ semantic content into visual features. Attention patterns change throughout denoising, but the OV transformation is a fixed linear map, which makes the text-to-visual translation particularly amenable to static analysis.
Text embeddings are known to organize semantic information along intrinsic axes, so we hypothesize that \(\mathbf{W}_{\text{OV}}\), which linearly maps this space into visual features, learns subspaces aligned with these axes. We decompose each head’s OV matrix via SVD:
\[\mathbf{W}_{\text{OV}}^{(h)} = \sum_{i=1}^{r} \sigma_i^{(h)} \mathbf{u}_i^{(h)} (\mathbf{v}_i^{(h)})^\top\]Each rank-1 term (spectral component) is an independent text-to-visual pathway that maps a direction \(\mathbf{u}_i\) in text embedding space to a direction \(\mathbf{v}_i\) in the head’s output space with strength \(\sigma_i\). Semantic concepts like “Van Gogh style” or “nudity” concentrate in a small subset of these components.
Measuring Concept Contribution
To locate a concept, we compare prompt pairs that differ only in that concept (e.g., “a mountain” and “a mountain in Van Gogh style”). We project each prompt’s mean-pooled token embedding \(\bar{\mathbf{c}}\) onto head \(h\)’s singular vectors and weight it by the singular values: \(\mathbf{s}^{(h)}(\bar{\mathbf{c}}) = \bar{\mathbf{c}}\,\mathbf{U}^{(h)}\mathbf{\Sigma}^{(h)}\). Components with a large difference between prompts are candidates for carrying the concept. Actual spatial head outputs also depend on token-wise attention weights; this pooled score serves to identify candidate components.
\[\Delta\mathbf{s}^{(h)} = \mathbf{s}^{(h)}(\bar{\mathbf{c}}_{\text{concept}}) - \mathbf{s}^{(h)}(\bar{\mathbf{c}}_{\text{base}}), \qquad [\Delta\mathbf{s}^{(h)}]_i = \sigma_i^{(h)} \langle \bar{\mathbf{c}}_{\text{concept}} - \bar{\mathbf{c}}_{\text{base}},\, \mathbf{u}_i^{(h)} \rangle\]The norm \(\|\Delta\mathbf{s}^{(h)}\|_2\) is the head’s concept contribution, and the magnitude of each element, \(|[\Delta\mathbf{s}^{(h)}]_i|\), is the contribution of an individual spectral component.
At the head level, scaling only the outputs of the 20 heads with the highest concept contribution (about 10% of the 195 heads in SD v2.1) is enough to modulate the concept’s intensity.

Scaling the outputs of the top-20 high-contribution heads for the "Van Gogh" concept (about 10% of all 195 heads) by a factor $\alpha$.
From Heads to Subspaces
Head-level analysis shows which heads contribute to a concept, but not how the concept is organized within each head. Given polysemanticity, where a single component encodes multiple distinct features, the question is whether a concept uses an entire head or is localized to specific spectral components within it. To find out, we rank all components across all heads by their contribution \(|[\Delta\mathbf{s}^{(h)}]_i|\), take the top \(k\)% as the concept’s spectral signature \(\mathcal{S}_c\), and scale only these components:
\[\widetilde{\mathbf{W}}_{\text{OV}}^{(h)} = \mathbf{W}_{\text{OV}}^{(h)} + (\alpha - 1) \sum_{i:\,(h,i) \in \mathcal{S}_c} \sigma_i^{(h)} \mathbf{u}_i^{(h)} (\mathbf{v}_i^{(h)})^\top\]\(\alpha = 0\) removes the selected components (Spectral Nullification), \(\alpha = 1\) keeps the original model, and \(\alpha > 1\) increases their contribution. In qualitative experiments on SD v2.1, removing the top 15–20% by concept contribution suppressed the target concept in the illustrated examples. Removing more than 30% could also degrade overall generation quality.
The figure below compares scaling high-contribution heads with scaling selected components within heads. The selections are 10% of all heads and 10% of all components, respectively, so the two percentages have different denominators. The comparison shows how finer-grained interventions change the way a concept can be controlled.

Spectral modulation (top; the top 10% of all components by concept contribution) and head-level modulation (bottom; the top 10% of heads by contribution), with varying α.
The figure below takes the heads that score high for all three artistic styles (Van Gogh, Monet, and Picasso) and shows the contribution of each singular vector within them. Even within the same head, the three concepts activate different singular vectors, and the high-contribution singular vectors do not necessarily correspond to the largest singular values (lowest indices). The concepts thus share heads yet exhibit their own activation patterns, partially overlapping but distinguishable.

Contribution of each singular vector within heads that score high for all three artistic styles. Different concepts activate different singular vectors within the same head.
Results
Concept Removal Benchmark
We evaluate Spectral Nullification (SN) for nudity removal on SD v1.4 across five adversarial prompt benchmarks — Ring-A-Bell (K16, K38, K77), I2P, MMA, P4D, and UnLearnDiffAtk. Following the observation above, SN nullifies the top 20% of concept-related spectral components, and the metric is the Attack Success Rate measured with NudeNet (ASR; lower is better).
Attack Success Rate (%, ↓) on nudity-related content across adversarial benchmarks. Best per column in bold, second-best underlined. Method types are color-coded — gray: training-based, blue: closed-form, green: inference-time, deep blue: spectral.
| Method | Ring-A-Bell | I2P | MMA | P4D | UnLearn | ||
|---|---|---|---|---|---|---|---|
| K16 | K38 | K77 | |||||
| SD v1.4 | 97.89 | 94.74 | 87.37 | 25.03 | 68.10 | 69.76 | 50.70 |
| ESD | 76.84 | 78.95 | 74.74 | 13.04 | 24.80 | 50.24 | 26.06 |
| CA | 88.42 | 88.42 | 84.21 | 19.30 | 58.50 | 63.41 | 44.37 |
| MACE | 89.47 | 95.79 | 93.68 | 25.56 | 66.00 | 68.29 | 50.70 |
| SDID | 95.79 | 91.58 | 84.21 | 23.12 | 62.00 | 66.83 | 48.59 |
| UCE | 22.11 | 18.95 | 21.05 | 8.06 | 41.00 | 38.05 | 21.13 |
| RECE | 10.53 | 9.47 | 7.37 | 4.24 | 25.00 | 21.46 | 9.15 |
| SLD-Medium | 68.42 | 60.00 | 50.53 | 8.38 | 48.70 | 43.90 | 23.94 |
| SLD-Strong | 18.95 | 10.53 | 6.32 | 2.33 | 7.70 | 11.71 | 7.04 |
| SAFREE | 65.26 | 55.79 | 45.26 | 6.26 | 29.90 | 38.54 | 14.79 |
| SN (Ours) | 41.05 | 35.79 | 30.53 | 4.24 | 17.60 | 18.54 | 8.45 |
SN ties with RECE for second on I2P (4.24%) and ranks second on MMA, P4D, and UnLearnDiffAtk. It has lower ASR than the training-based methods in this table (ESD, CA, MACE, SDID) on every benchmark, but higher ASR than UCE, RECE, and SLD-Strong on Ring-A-Bell. Performance therefore varies with the attack prompts.
SN removes the identified concept-related components from the OV matrices without additional training. The trade-off between removal and text–image alignment is evaluated with CLIP score on 1,000 COCO captions. CLIP score measures semantic alignment and does not cover every aspect of visual quality.

Concept removal performance (P4D ASR) versus text–image alignment (CLIP score). SN achieves a competitive balance without retraining. SLD-Strong has the lowest ASR but also a lower CLIP score.
Verifying the Semantics of Spectral Subspaces
We verify that the identified spectral subspaces genuinely capture the semantics of their target concepts through two approaches.
Text space alignment: We reconstruct the text-embedding difference between concept and base prompts using the input directions of concept-specific components, then compute cosine similarity with all 49,408 CLIP vocabulary tokens. For the top two heads contributing to nudity, one head’s top 16 tokens include “nudes”, “nsfw”, and “naked”; the other includes “topless”, “lingerie”, “erotica”, and “nude”. The lists differ, but both contain concept-related tokens. Unrelated tokens also appear, showing that the concept is not perfectly isolated.

The 16 tokens with the highest cosine similarity to vectors reconstructed from nudity-related spectral components, for each of the top-2 heads. Highlighted tokens are explicitly nudity-related.
Representations before and after intervention (t-SNE), and overlap between concepts (Jaccard): Removing concept-related components increases overlap between the base/concept prompt clusters in a t-SNE of head outputs. This visualization complements the generation experiments in which direct interventions change the images; t-SNE alone does not establish causality. Jaccard similarity shows that related concepts such as Van Gogh and Monet share more components while retaining distinguishable spectral signatures.

(a) t-SNE of head outputs before and after removing the top 10% of components by concept contribution. Base/concept clusters overlap more after removal. (b) Jaccard similarity between concepts. Related concepts overlap while retaining distinguishable signatures.
Qualitative
In the I2P adversarial examples below, SN suppresses the target content while preserving scene composition. As the quantitative table shows, removal does not succeed on every prompt.

Generated images for I2P adversarial prompts. From left: SD v1.4 (original), CA, RECE, SLD-Medium, SLD-Strong, SAFREE, and SN (ours).