1. Bibliographic Information

1.1. Title

A Training-Free Style-Personalization via SVD-Based Feature Decomposition

1.2. Authors

Kyoungmin Lee, Jihun Park, Jongmin Gim, Wonhyeok Choi, Kyumin Hwang, Jaeyeul Kim, and Sunghoon Im. The authors are affiliated with DGIST (Daegu Gyeongbuk Institute of Science and Technology), Republic of Korea.

1.3. Journal/Conference

This paper is a preprint, indicated by its presence on arXiv. The abstract mentions competitive results compared to fine-tuned baselines, suggesting it is a research paper intended for publication in a major computer vision or machine learning conference (e.g., CVPR, ICCV, NeurIPS, ICLR) or journal.

1.4. Publication Year

The paper was published on arXiv on 2025-07-06T17:42:11.000Z, making its publication year 2025.

1.5. Abstract

The paper introduces a novel training-free framework for style-personalized image generation that operates during inference using a scale-wise autoregressive model. The method aims to generate stylized images guided by a single reference style while maintaining semantic consistency and preventing content leakage. Through a detailed step-wise analysis of the generation process, the authors identify that the dominant singular values of the internal feature at a pivotal step encode style-related components. Building on this insight, they propose two lightweight control modules: Principal Feature Blending (PFB), which precisely modulates style using SVD-based feature reconstruction, and Structural Attention Correction (SAC), which stabilizes structural consistency through content-guided attention correction across finer stages. Without any additional training, the method achieves competitive style and prompt fidelity compared to fine-tuned baselines, with faster inference and greater deployment flexibility.

1.6. Original Source Link

https://arxiv.org/abs/2507.04482v2

1.7. PDF Link

https://arxiv.org/pdf/2507.04482v2.pdf

2. Executive Summary

2.1. Background & Motivation

The rapid advancement of Text-to-Image (T2I) models has democratized visual content creation, but users are increasingly seeking personalized generation that reflects specific visual identities or artistic styles. Existing solutions, primarily based on diffusion models, often rely on fine-tuning for each new style. This fine-tuning approach poses significant scalability challenges due to high computational costs and long inference times, making them unsuitable for real-time or interactive applications. Furthermore, while diffusion models produce high-quality results, their iterative denoising process is inherently slow.

The core problem this paper aims to solve is the lack of efficient, flexible, and high-fidelity style-personalized image generation methods that do not require extensive training for every new style. The paper identifies a gap where current methods either sacrifice efficiency for quality (diffusion models with fine-tuning) or struggle with scalability.

The paper's innovative idea is to develop a training-free framework that leverages the efficiency of scale-wise autoregressive models (which are faster than diffusion models) and a deep understanding of their internal generation process. By identifying a specific stage where style information is predominantly encoded and using Singular Value Decomposition (SVD) to manipulate this information, the authors propose a novel way to achieve style personalization without re-training the base model.

2.2. Main Contributions / Findings

The primary contributions of this paper are:

• Training-Free Inference Framework: The introduction of a novel framework for style-personalized image generation that operates during inference from a single style reference, achieving competitive results with significantly faster inference times compared to existing methods.
• Detailed Step-wise Analysis: A thorough analysis of the internal mechanisms of a scale-wise autoregressive model (Infinity [16]), pinpointing a key generation step (specifically, the processing that leads to the third feature, $F_3$) that governs both content and style attributes.
• SVD-Based Style Component Extraction: The observation and demonstration that style-related components can be effectively extracted from the identified key feature ($F_3$) through SVD-based analysis of its dominant singular values. This insight forms the basis for precise style manipulation.
• Lightweight Control Modules: The proposal of two complementary and lightweight control modules:

  • Principal Feature Blending (PFB): Designed for precise modulation and injection of style, leveraging the SVD insight.

  • Structural Attention Correction (SAC): Implemented to stabilize structural coherence and semantic consistency throughout the generation process, especially in later refinement stages.

    The key findings demonstrate that this training-free approach achieves a strong balance between prompt fidelity (how well the generated image matches the text description) and style fidelity (how well it matches the reference style), outperforming fine-tuned baselines in terms of speed and deployment flexibility while maintaining competitive quality.

3. Prerequisite Knowledge & Related Work

3.1. Foundational Concepts

To understand this paper, a foundational grasp of several machine learning and computer vision concepts is necessary:

• Text-to-Image (T2I) Models: These are generative AI models that create images from natural language descriptions (text prompts). They have revolutionized content creation by allowing users to specify visual concepts, styles, and details through text. Examples include DALL-E, Midjourney, Stable Diffusion, and the Infinity model used in this paper.

• Generative Models (GANs, Diffusion Models, Autoregressive Models):

  • Generative Adversarial Networks (GANs): Comprise a generator (creates fake data) and a discriminator (tells real from fake). They learn to produce realistic data through adversarial training.
  • Diffusion Models: A class of generative models that learn to reverse a diffusion process. They start with random noise and iteratively denoise it, guided by a text prompt, to produce a high-quality image. They are known for high image quality but can be slow due to many iterative steps.
  • Autoregressive (AR) Models: These models generate data sequentially, predicting each part (e.g., pixel, token, or feature scale) based on previously generated parts. Scale-wise autoregressive models (like Infinity) generate images by predicting features at progressively finer scales, which can be much faster than diffusion models.

• Transformers and Attention Mechanisms:

  • Transformers: A neural network architecture introduced for sequence-to-sequence tasks (like language translation), now widely adopted in vision. They rely heavily on attention mechanisms.
  • Self-Attention: Allows a model to weigh the importance of different parts of the input sequence when processing a specific part. In image generation, it helps understand relationships between different regions of an image.
  • Cross-Attention: Enables a model to weigh the importance of elements from a different sequence (e.g., text prompt) when processing the current sequence (e.g., image features). It's crucial for guiding image generation with text.

• Singular Value Decomposition (SVD): A fundamental matrix factorization technique. For a matrix $A$, SVD decomposes it into three other matrices: $A = U \Sigma V^\top$.

  • $U$: Left singular vectors (orthogonal matrix).
  • $\Sigma$: A diagonal matrix containing singular values ($\sigma_1, \sigma_2, \dots, \sigma_r$) in descending order. These values represent the "strength" or "importance" of each corresponding singular vector pair. Larger singular values correspond to more dominant features in the data.
  • $V^\top$: Right singular vectors (orthogonal matrix).
  • In the context of this paper, SVD is used to decompose image features, where dominant singular values are hypothesized to encode crucial style components. By manipulating these dominant components, specific attributes (like style) can be isolated or transferred.

• CLIP (Contrastive Language-Image Pre-training): A neural network trained by OpenAI on a vast dataset of image-text pairs. It learns to associate images with their textual descriptions. CLIP embeddings can be used to measure the similarity between a text prompt and an image, or between two images, which is essential for evaluating prompt fidelity and style fidelity.

• Feature Maps: In Convolutional Neural Networks (CNNs) or Transformers, intermediate layers output feature maps which are multi-dimensional arrays representing learned patterns or characteristics in the input data (e.g., edges, textures, objects). Manipulating these feature maps is a common technique for style transfer or image editing.

3.2. Previous Works

The paper contextualizes its work within three main areas: Neural Style Transfer, Text-to-Image Generation, and Personalized Image Generation.

3.2.1. Neural Style Transfer (NST)

Neural Style Transfer (NST) aims to render the content of one image in the style of another.

• Early NST [14]: Pioneered by Gatys et al., this method used pre-trained Convolutional Neural Networks (CNNs) (like VGGNet) to separate content features and style features and then iteratively optimize a generated image to match both. While groundbreaking, it was computationally expensive, requiring optimization for each image.
• Efficient NST (AdaIN, WCT):
  • Adaptive Instance Normalization (AdaIN) [20]: Proposed by Huang and Belongie, AdaIN significantly sped up style transfer by aligning the mean and variance of content features to those of style features in a single forward pass.
  • Whitening and Coloring Transform (WCT) [28, 33]: Advanced NST by aligning the full covariance structure of features, leading to more detailed and higher-quality stylization.
• Attention-based NST: With the rise of attention mechanisms [11, 55], subsequent models have utilized attention to achieve remarkable stylization results [10, 15, 19, 32, 35, 62], allowing for more granular control over feature interactions.
• Text-driven NST: The emergence of vision-language models like CLIP [40] enabled text-driven style transfer [2, 24, 36], where styles could be described by natural language prompts, expanding beyond image-based references.

3.2.2. Text-to-Image (T2I) Generation

T2I generation has seen rapid progress, largely fueled by large-scale image-text datasets [4, 7, 30, 46].

• Diffusion-based Models [26, 38, 41, 42, 45]: Currently the dominant paradigm, known for producing exceptionally high-quality and diverse images. They achieve this by iteratively denoising a noisy image over many steps. However, this iterative process leads to high inference latency, making them slow for real-time applications. They are also successful in downstream tasks like style transfer and image editing [3, 9, 17, 21, 25, 39, 52, 61].
• GAN-based Approaches [22]: Earlier methods that used Generative Adversarial Networks for T2I synthesis.
• Visual Autoregressive (AR) Models [6, 16, 50]: Evolved from next-token prediction [12, 54] to masked token prediction [5, 6, 23] and more recently to next-scale prediction [51]. The next-scale prediction paradigm (e.g., Infinity [16], Switti [56, 57]) significantly accelerates inference while maintaining strong visual fidelity, positioning them as promising alternatives to diffusion models for efficiency. The paper specifically leverages Infinity [16].
  • Infinity Architecture [16]: This is the base model for the paper's framework. It uses a pre-trained text encoder ($\mathcal{E}_T$, based on Flan-T5 [8]), an autoregressive transformer ($\mathcal{M}$) for scale-wise feature prediction, and a decoder ($\mathcal{D}$) to reconstruct the final image. It iteratively predicts residual feature maps ($R_s$) at different scales, accumulates them ($F_s = \sum \mathsf{up}(\mathbf{R}_i)$), and then decodes the final accumulated feature ($F_S$) into an image. The prediction step involves self-attention ($M_{SA}$) and cross-attention ($M_{CA}$) mechanisms.

3.2.3. Personalized Image Generation

This field focuses on adapting pre-trained T2I models to generate images that adhere to specific user concepts.

• Content-oriented Methods [27, 29, 43, 58]: Aim to preserve specific objects or identities from a small set of reference images. Examples include DreamBooth [43] and PhotoMaker [29], which often involve fine-tuning parts of a pre-trained model or injecting learned embeddings.
• Style-oriented Methods [1, 13, 18, 37, 44, 47, 49, 65]: Extend the personalization concept to consistently control visual style across generations. These methods typically employ diffusion-based models and often require fine-tuning (e.g., DreamStyler [1], StyleDrop [49], DB-LoRA [44], B-LoRA [13], ZipLoRA [47], FineStyle [65]) or complex style adapters (StyleAligned [18], IP-Adapter [63]).
  • The paper explicitly contrasts its approach with these methods, highlighting that they are either diffusion-based (and thus slow) or fine-tuning-based (and thus computationally expensive and non-scalable for many styles).

3.3. Technological Evolution

The evolution of generative AI has moved from per-image optimization (early NST) to faster feed-forward networks (AdaIN, WCT), then to large-scale T2I models (GANs, Diffusion, AR). Within T2I, the trend is towards personalization—adapting models to user-specific content or style. This personalization initially relied on fine-tuning (e.g., DreamBooth, LoRA-based methods) or specialized architectural additions (IP-Adapter, StyleAligned) often built on computationally intensive diffusion models. This paper represents a crucial step in this evolution by:

1. Moving away from diffusion models to more efficient scale-wise autoregressive models.
2. Eliminating the need for fine-tuning or substantial architectural changes, making the style personalization process entirely training-free during inference.
3. Leveraging a deep internal analysis of the base model rather than external adaptations.

3.4. Differentiation Analysis

Compared to the main methods in related work, this paper's approach offers several key differentiators and innovations:

• Training-Free vs. Fine-tuning: Most existing personalized generation methods (e.g., DreamBooth, DB-LoRA, B-LoRA, DreamStyler, StyleDrop) require fine-tuning the base model for each new style, which is computationally expensive and limits scalability. This paper's method is entirely training-free, operating solely at inference time, offering significant flexibility and cost savings.

• Efficiency (AR vs. Diffusion): The method is built upon a scale-wise autoregressive model (Infinity), which provides significantly faster inference compared to the diffusion-based models commonly used by other state-of-the-art methods. This makes it suitable for real-time and interactive applications.

• Single Reference Image: It achieves effective style personalization from just a single reference style image, streamlining the user experience.

• SVD-Based Feature Decomposition: The core innovation lies in a detailed internal analysis of the autoregressive model to identify a key feature ($F_3$) whose dominant singular values are responsible for encoding style. This SVD-based decomposition allows for precise, interpretable, and targeted style modulation without external training. Other methods might use complex attention mechanisms or learn embeddings, but not this direct SVD manipulation.

• Lightweight Modules: Principal Feature Blending (PFB) and Structural Attention Correction (SAC) are lightweight modules that intervene at specific, identified stages of the generation process. They do not involve modifying the base model's weights or requiring extensive computations, making them highly efficient. In contrast, methods like StyleAligned or IP-Adapter might inject learned adapters or use shared attention mechanisms that, while training-free per inference, require pre-trained components that were themselves trained.

  In essence, this paper provides a faster, more flexible, and more interpretable approach to style personalization by deeply understanding and precisely manipulating the internal mechanics of efficient generative models, rather than relying on external adaptations or slow iterative processes.

4. Methodology

The paper proposes a training-free framework for style-personalized image generation that operates during inference using a scale-wise autoregressive model. The core idea is to identify and manipulate style-related components within the model's internal features using Singular Value Decomposition (SVD), while ensuring content and structural consistency.

4.1. Principles

The core idea behind the method is based on two key principles derived from an in-depth analysis of the Infinity scale-wise autoregressive model:

1. Localized Style Control: A specific intermediate feature ($F_3$) generated at an early stage (step 3) of the autoregressive process plays a pivotal role in shaping both the content and style of the final image. This step is a critical point for style intervention.

2. SVD for Style-Content Disentanglement: Within this pivotal feature ($F_3$), the dominant singular values and their corresponding components primarily encode style-related characteristics with minimal influence on content-related information. This insight allows for SVD to be used as a tool to precisely extract and inject style.

   Building on these principles, the method employs a dual-stream generation architecture (a content path and a generation path) and introduces two lightweight modules: Principal Feature Blending (PFB) for style modulation and Structural Attention Correction (SAC) for maintaining structural consistency.

4.2. Core Methodology In-depth (Layer by Layer)

4.2.1. Preliminary: Infinity Architecture

The paper utilizes Infinity [16], a scale-wise autoregressive Text-to-Image (T2I) model.

• Components: Infinity consists of a pre-trained text encoder ($\mathcal{E}_T$, based on Flan-T5 [8]), an autoregressive transformer ($\mathcal{M}$), and a decoder ($\mathcal{D}$).
• Generation Process: During inference, the transformer $\mathcal{M}$ iteratively predicts quantized residual feature maps ($R_s$) for each generation step $s \in \mathbf{S} = \{1, 2, \dots, S\}$. This prediction is conditioned on the input text prompt $T$ (encoded by $\mathcal{E}_T$) and the previously generated accumulated feature $F_{s-1}$.
• Prediction Step: The prediction is defined by the equation: $$R _ { s } = \mathcal { M } ( F _ { s - 1 } , \mathcal { E } _ { T } ( T ) ) \\ = M _ { C A } ( M _ { S A } ( Q _ { s - 1 } , K _ { s - 1 } , V _ { s - 1 } ) , \mathcal { E } _ { T } ( T ) ) , \forall s \in \mathbf { S }$$
  • $R_s$: The $s$-th scale quantized residual feature map being predicted.
  • $\mathcal{M}$: The autoregressive transformer network.
  • $F_{s-1}$: The accumulated feature from the previous step, serving as input to the transformer.
  • $\mathcal{E}_T(T)$: The text embedding of the input text prompt $T$.
  • $M_{SA}(\cdot)$: The self-attention mechanism within the transformer, processing queries ($Q_{s-1}$), keys ($K_{s-1}$), and values ($V_{s-1}$) derived from $F_{s-1}$.
  • $M_{CA}(\cdot)$: The cross-attention mechanism, integrating the text embedding $\mathcal{E}_T(T)$ with the output of self-attention.
  • $Q_{s-1}, K_{s-1}, V_{s-1}$: Query, Key, and Value matrices at step s-1, projected from feature $F_{s-1}$.
• Feature Accumulation: Each predicted residual $R_s$ is upsampled to the target resolution ($H \times W$) and accumulated to form the input for the next step, $F_s$: $$F _ { s } = \sum _ { i = 1 } ^ { s } \mathsf { u p } _ { H \times W } ( \mathbf { R } _ { i } ) , \quad R _ { s } \in \mathbb { R } ^ { c \times h _ { s } \times w _ { s } }$$
  • $F_s$: The accumulated feature at step $s$.
  • $\mathsf{up}_{H \times W}(\cdot)$: A bilinear upsampling function that resizes $R_i$ to the final image resolution.
  • $R_i$: The residual feature map at step $i$, with channel $c$ and spatial dimensions $h_s \times w_s$.
• Final Image Decoding: The final image $I$ is produced by decoding the accumulated feature $F_S$ at the last generation step: $$I = { \mathcal { D } } ( F _ { S } )$$
  • $\mathcal{D}$: The decoder network.

4.2.2. Analysis of Scale-wise AR Model

Before introducing their modules, the authors conduct a detailed analysis to understand the Infinity model's behavior.

4.2.2.1. Step-wise Analysis

The authors investigate which generation step critically influences content and style.

• Method: They create two sets of prompt pairs:
  • Content pair set (\mathbf{T}^{con}): Two object-centric prompts (e.g., "A photo of a donut", "A photo of a truck").
  • Style pair set (\mathbf{T}^{sty}): Two prompts for the same object with different colors (e.g., "A photo of a red truck", "A photo of a green truck"). For each pair, they generate an image by replacing the original prompt $T$ with an alternative prompt $\hat{T}$ only at a specific generation step $s \in \mathbf{S}$, while keeping other steps fixed to $T$.
• Assessment: They measure CLIP similarity between the alternative prompt $\hat{T}$ and the resulting modified image. A substantial change in the image's attributes (content or style) when $\hat{T}$ is injected at step $s$ indicates its importance.
• Finding: As shown in Figure 2 (bottom row), step\hat{s} = 2 consistently produces the highest `CLIP similarity` with the alternative prompt across both content and style prompt pairs. This suggests that the processing at step 2, which results in the `third feature (`F_3`)`, is crucial for shaping both `content` and `style`. The following figure (Figure 2 from the original paper) shows the step-wise prompt injection analysis:  *该图像是示意图，展示了基于SVD特征分解的风格个性化图像生成过程，包括不同生成步骤下的玩具熊和甜点图像。图中的生成步骤展示了风格相似度和内容相似度的变化，最后一行为数值统计图，反映不同生成步骤下的相似度变化。* #### 4.2.2.2. Key Step Feature Analysis Building on the finding that $F_3$ is pivotal, the authors hypothesize that its `principal components` encode `stylistic attributes`. * **Method:** They construct 100 prompt pairs $(T, \hat{T})$ where each differs in *both* object category and color (e.g., "A photo of a red truck" vs. "A photo of a purple cat"). For each prompt $T$, they apply `Singular Value Decomposition (SVD)` to its third feature $F_3$: $F_3 = U \Sigma V^\top$. They then create a modified diagonal matrix $\Sigma'$ by zeroing out all `singular values` except the largest one ($\sigma_1$). Using this, they reconstruct the `dominant singular component` as $F_3^{svd} = U \Sigma' V^\top$. The `residual component` is $F_3^{res} = F_3 - F_3^{svd}$. For each prompt pair, they generate three types of outputs: 1. `Baseline output (`\hat{I}`)`: Generated using the original prompt $\hat{T}$ without feature manipulation. 2. `Full replacement output (`\hat{I}^{rep}`)`: The entire feature $\hat{F}_3$ (from $\hat{T}$) is replaced with $F_3$ (from $T$). That is, $\hat{F}_3 \gets F_3$. 3. `SVD-guided output (`\hat{I}^{svd}`)`: Only the `dominant singular component` of $\hat{F}_3$ is replaced with that of $F_3$, while preserving $\hat{F}_3$'s residual components. That is, $\hat{F}_3 \gets F_3^{svd} + \hat{F}_3^{res}$. * **Assessment:** They evaluate `CLIP similarity` for both `object` and `color` attributes. * **Finding:** As shown in Figure 3, the `full replacement output` ($\hat{I}^{rep}$) significantly increases `CLIP similarity` to *both* the object and color of the substituted prompt $T$. In contrast, the `SVD-guided output` ($\hat{I}^{svd}$) shows a pronounced increase primarily in `color-related CLIP similarity`, with minimal change in `object-related similarity`. * **Conclusion:** This strongly suggests that the `first principal component` (derived from the dominant singular value) of the `third feature (`F_3`)` predominantly captures `style-related characteristics` (like color, texture) with limited influence on `content-related information` (like object identity or structure). The following figure (Figure 3 from the original paper) shows the key step feature analysis:  *该图像是图表，展示了在基准模型、全替换模型与SVD引导输出之间的内容和风格相似度比较。通过一组提示对 `extbf{T}` 的分析，结果显示了对应的相似度，进一步揭示了 SVD 引导的特征替换对风格的影响。* ### 4.2.3. Overall Pipeline The proposed method aims to generate a style-personalized image $I^{gen}$ by injecting a `principal style feature` while preserving `semantic consistency` and suppressing `content leakage`. The system employs a `dual-stream generation architecture` (Figure 4): * **Content Path:** This path operates as the standard inference branch of the pre-trained `Infinity` model without modification. It follows the iterative update rule and produces a sequence of `content features` $\{F_s^{con}\}_{s=1}^S$. Its role is to provide `structurally stable` and `semantically aligned guidance`. * **Generation Path:** This path also follows the same update formula, producing $\{F_s^{gen}\}_{s=1}^S$. It is modulated by the proposed `style-blending mechanisms` and synthesizes the final stylized output. * **Shared Prompt:** Both paths are conditioned on the same text prompt $T$ (e.g., "<content> in <style>"). This prevents `semantic mismatch` and ensures consistent `structural communication`. Two complementary modules intervene in the `generation path`: 1. **Principal Feature Blending (PFB) (Sec. 5.2):** Selectively injects `principal style representations` from a `style reference image` ($I^{sty}$) into the `generation features` at the critical `step`s=3.

2. Structural Attention Correction (SAC) (Sec. 5.3): Applied across subsequent fine stages ($\mathbf{S}_{fine} = \{3, 4, \dots, S\}$) to incorporate signals from the content path, stabilizing structural alignment and maintaining semantic consistency.

   The following figure (Figure 4 from the original paper) shows the overall pipeline of the model:

   该图像是示意图，展示了作者提出的风格个性化图像生成模型的整体流程。图中包括文本编码器、图像编码器、变换器以及解码器，标示了风格提取、主特征融合（Principal Feature Blending）和结构注意力校正（Structural Attention Correction）的步骤。在生成过程中，通过相同的文本提示处理内容和生成路径，并在第3阶段应用主特征融合以提取主导特征并稳定结构一致性。

4.2.4. Principal Feature Blending (PFB)

PFB is designed to effectively incorporate style information while suppressing irrelevant cues from the reference image.

• Style Feature Extraction: The process begins by extracting style features $F_s^{sty}$ from the style reference image $I^{sty}$ using a pre-trained multi-scale image encoder $\mathcal{E}_I$ (from the baseline Infinity model): $$\{ F _ { 1 } ^ { \mathrm { s t y } } , F _ { 2 } ^ { \mathrm { s t y } } , \cdot \cdot \cdot , F _ { S } ^ { \mathrm { s t y } } \} = \mathcal { E } _ { I } ( I ^ { \mathrm { s t y } } )$$ Based on the analyses in Section 4, $F_3^{sty}$ is identified as the primary carrier of style information.
• Style Extractor Function ($\Phi$): This function prioritizes the dominant singular components of the style feature $F_3^{sty}$ while smoothly incorporating residual style representations. It applies exponential reweighting to the singular values based on their spectral order, gradually reducing the impact of lower components. Given a feature matrix $\boldsymbol{F}$, its SVD is $\boldsymbol{F} = \mathbf{U} \boldsymbol{\Sigma} \mathbf{V}^\intercal$. The style extractor function $\Phi$ is defined as: $$\Phi ( \boldsymbol { F } ) \triangleq \mathbf { U } \mathbf { W } \boldsymbol { \Sigma } \mathbf { V } ^ { \intercal } \\ \mathrm { w h e r e } \textit { F } = \mathbf { U } \boldsymbol { \Sigma } \mathbf { V } ^ { \intercal } \\ \mathbf { W } = \mathrm { diag } ( \exp ^ { - 0 \cdot \alpha } , \exp ^ { - 1 \cdot \alpha } , \dots , \exp ^ { - ( r - 1 ) \cdot \alpha } )$$
  • $\boldsymbol{F}$: The input feature matrix (in this case, $F_3^{sty}$ or $F_3^{gen}$ for its own SVD).
  • $\mathbf{U}, \boldsymbol{\Sigma}, \mathbf{V}^\intercal$: The matrices from the Singular Value Decomposition of $\boldsymbol{F}$. $\boldsymbol{\Sigma}$ is a diagonal matrix containing the singular values $\sigma_0, \sigma_1, \dots, \sigma_{r-1}$.
  • $\mathbf{W}$: A diagonal weighting matrix where each diagonal element $\exp^{-i \cdot \alpha}$ reweights the $i$-th singular value.
  • $r$: The rank of the feature matrix, representing the number of non-zero singular values.
  • $\alpha$: A positive exponential decay rate that controls how quickly the influence of lower-ranked singular components diminishes. A larger $\alpha$ means faster decay, giving more weight to the most dominant singular values.
• Feature Blending: The refined style from $\Phi(F_3^{sty})$ is blended into the generation path's feature $F_3^{gen}$ at step 3. The generation feature $F_3^{gen}$ is updated as follows: $$F _ { 3 } ^ { \mathrm { g e n } } \gets \hat { F } _ { 3 } ^ { \mathrm { g e n } } , \qquad \\ \hat { F } _ { 3 } ^ { \mathrm { g e n } } = \Phi ( F _ { 3 } ^ { \mathrm { s t y } } ) + \big ( F _ { 3 } ^ { \mathrm { g e n } } - \Phi ( F _ { 3 } ^ { \mathrm { g e n } } ) \big )$$
  • $\hat{F}_3^{gen}$: The new, style-blended generation feature for step 3.
  • $\Phi(F_3^{sty})$: The style-prioritized component extracted from the reference style image.
  • $(F_3^{gen} - \Phi(F_3^{gen}))$: Represents the residual components of the generation feature itself, i.e., what remains after its own style-dominant components are removed. This formulation ensures that the dominant style component of $F_3^{sty}$ is injected, while the non-dominant (content-related) components of $F_3^{gen}$ are preserved. This prevents content leakage from the style reference while integrating its style.

4.2.5. Structural Attention Correction (SAC)

While PFB effectively injects style, it can sometimes disturb the structural coherence of the generated image. SAC is introduced to counteract this by leveraging the attention map of the content path as a structural prior.

• Motivation: Inspired by how self-attention in diffusion models preserves spatial relationships, SAC aims to align the attention map of the generation path with that of the content path.
• Application: SAC is applied to all subsequent steps after PFB, specifically in the fine stages denoted as $\mathbf{S}_{fine} = \{3, 4, \dots, S\}$.
• Mechanism: It directly injects the query and key representations from the content path into the generation path at each step $s \in \mathbf{S}_{fine}$: $$Q _ { s } ^ { \mathrm { g e n } } \gets Q _ { s } ^ { \mathrm { c o n } } , \quad K _ { s } ^ { \mathrm { g e n } } \gets K _ { s } ^ { \mathrm { c o n } } , \\ Q _ { s } ^ { \mathrm { c o n } } = W _ { Q } F _ { s } ^ { \mathrm { c o n } } , \quad K _ { s } ^ { \mathrm { c o n } } = W _ { K } F _ { s } ^ { \mathrm { c o n } }$$
  • $Q_s^{gen}, K_s^{gen}$: The query and key for the generation path at step $s$.
  • $Q_s^{con}, K_s^{con}$: The query and key derived from the content feature $F_s^{con}$ at step $s$.
  • $W_Q, W_K$: Linear projection matrices that transform the input features ($F_s^{con}$) into their respective query and key representations within the self-attention layers. By directly replacing the queries and keys of the generation path with those from the content path, SAC ensures that the attention mechanisms in the generation path prioritize structural relationships learned by the content path, thereby stabilizing the generated image's structure.

4.2.6. Dual-path style-personalized image generation (Algorithm 1)

The supplementary material provides Algorithm 1 which details the dual-stream generation process.

The following table (Algorithm 1 from the original paper) shows the dual-path style-personalized image generation:

Algorithm 1 Dual-path style-personalized image generation




Input: text prompt $T$, style image $I^{sty}$, number of steps $S$, exponential decay rate $\alpha$


Output: Generated image $I^{gen}$


1: Initialize $F_0^{con} \gets \langle SOS \rangle$, $F_0^{gen} \gets \langle SOS \rangle$


2: Extract style features `\{F_s^{sty}\}_{s=1}^S = \mathcal{E}_I(I^{sty})`


3: Compute text embeddings `e_T = \mathcal{E}_T(T)`


4: for $s = 1, \dots, S$ do


5:
  Compute content path residual: $R_s^{con} = \mathcal{M}(F_{s-1}^{con}, e_T)$


6:
  Compute generation path residual: $R_s^{gen} = \mathcal{M}(F_{s-1}^{gen}, e_T)$


7:
  Update content path feature: $F_s^{con} = \sum_{i=1}^s \mathsf{up}_{H \times W}(R_i^{con})$


8:
  Update generation path feature: $F_s^{gen} = \sum_{i=1}^s \mathsf{up}_{H \times W}(R_i^{gen})$


9:
  if $s = 3$ then


10:
    // Principal Feature Blending (PFB)


11:
    $\hat{F}_3^{gen} = \Phi(F_3^{sty}) + (F_3^{gen} - \Phi(F_3^{gen}))$


12:
    $F_3^{gen} \gets \hat{F}_3^{gen}$


13:
  end if


14:
  if $s \in \{3, \dots, S\}$ then


15:
    // Structural Attention Correction (SAC)


16:
    $Q_s^{con} = W_Q F_s^{con}$


17:
    $K_s^{con} = W_K F_s^{con}$


18:
    $Q_s^{gen} \gets Q_s^{con}$


19:
    $K_s^{gen} \gets K_s^{con}$


20:
  end if


21: end for


22: $I^{gen} = \mathcal{D}(F_S^{gen})$


23: return $I^{gen}$

Explanation of Algorithm 1:

1. Initialization: Both content path and generation path features ($F_0^{con}$, $F_0^{gen}$) are initialized with a start-of-sequence (\langle SOS \rangle) token.
2. Style Feature Extraction: Multi-scale style features $\{F_s^{sty}\}_{s=1}^S$ are extracted from the reference style image $I^{sty}$ using the image encoder $\mathcal{E}_I$.
3. Text Embedding: The text prompt $T$ is encoded into text embeddings $e_T$ using the text encoder $\mathcal{E}_T$.
4. Iterative Generation Loop: The process iterates through each generation step $s$ from 1 to $S$.
   • Residual Computation: For each step, the autoregressive transformer $\mathcal{M}$ computes residual feature maps for both the content path ($R_s^{con}$) and the generation path ($R_s^{gen}$), conditioned on their respective previous accumulated features and the text embeddings.
   • Feature Accumulation: The accumulated features for both paths ($F_s^{con}$, $F_s^{gen}$) are updated by summing the upsampled residual maps up to the current step.
   • Principal Feature Blending (PFB) (at $s=3$): If the current step is $s=3$, the Principal Feature Blending mechanism is applied.
     • The generation feature $F_3^{gen}$ is modified ($\hat{F}_3^{gen}$) by blending the style-prioritized component $\Phi(F_3^{sty})$ from the reference style image with the content-preserving residual $(F_3^{gen} - \Phi(F_3^{gen}))$ of the current generation feature.
     • $F_3^{gen}$ is then updated with this blended feature.
   • Structural Attention Correction (SAC) (for $s \ge 3$): If the current step $s$ is 3 or greater (i.e., in the fine stages), Structural Attention Correction is applied.
     • The query ($Q_s^{con}$) and key ($K_s^{con}$) for the content path are computed from its accumulated feature $F_s^{con}$ using linear projection matrices $W_Q$ and $W_K$.
     • The query ($Q_s^{gen}$) and key ($K_s^{gen}$) of the generation path are then replaced with those from the content path.
5. Final Image Decoding: After all steps, the final accumulated feature from the generation path ($F_S^{gen}$) is passed through the decoder $\mathcal{D}$ to produce the final generated image $I^{gen}$.

5. Experimental Setup

5.1. Datasets

The authors follow the evaluation protocol established by FineStyle [65].

• Benchmark Dataset: A filtered subset of prompts from Parti [64] is used.
  • Characteristics: This subset contains 190 prompts. Each prompt describes a subject along with its superclass to reduce semantic ambiguity.
  • Example: "A cat, animals, in watercolor painting style". This structure helps the model understand both the content (cat) and the broader category (animals) while specifying the desired style.
• Style References: For style guidance, 10 representative styles are selected from the FineStyle evaluation set. These styles are used as reference style images. The specific styles are listed in the supplementary material (Figure 10). For example, "A photo of a black teddy bear" and "A photo of a white teddy bear" illustrate how styles are applied.
• Reasoning: These datasets and styles are chosen because they provide a standardized and diverse benchmark for evaluating style-personalized image generation models, allowing for fair comparison with state-of-the-art methods.

5.2. Evaluation Metrics

The paper uses CLIP-based metrics to quantitatively evaluate the generated images, following the FineStyle protocol.

• $S_{txt}$ (CLIP Text Score):

  1. Conceptual Definition: This metric quantifies prompt fidelity, measuring how well the generated image semantically aligns with its corresponding input text prompt. A higher score indicates that the image accurately represents the content and concepts described in the text.
  2. Mathematical Formula: The paper refers to CLIP similarity without providing an explicit formula. Generally, CLIP similarity between a text embedding $e_T$ and an image embedding $e_I$ is computed as the cosine similarity: $ S_{\mathrm{txt}} = \mathrm{cosine_similarity}(e_T, e_I) = \frac{e_T \cdot e_I}{|e_T| |e_I|} $
  3. Symbol Explanation:
     • $e_T$: The CLIP embedding of the input text prompt.
     • $e_I$: The CLIP embedding of the generated image.
     • $\cdot$: Dot product.
     • $\|\cdot\|$: L2 norm (magnitude) of the vector.
     • $\mathrm{cosine\_similarity}(\cdot, \cdot)$: Measures the cosine of the angle between two vectors, indicating their directional similarity.

• $S_{img}$ (CLIP Image Score):

  1. Conceptual Definition: This metric assesses style fidelity, measuring the visual similarity between the generated image and the reference style image. A higher score suggests that the generated image successfully adopts the artistic style of the reference.
  2. Mathematical Formula: Similar to $S_{txt}$, this is also typically computed as CLIP cosine similarity between two image embeddings: $ S_{\mathrm{img}} = \mathrm{cosine_similarity}(e_{I_{gen}}, e_{I_{ref}}) = \frac{e_{I_{gen}} \cdot e_{I_{ref}}}{|e_{I_{gen}}| |e_{I_{ref}}|} $
  3. Symbol Explanation:
     • $e_{I_{gen}}$: The CLIP embedding of the generated image.
     • $e_{I_{ref}}$: The CLIP embedding of the reference style image. The paper cautions that a high $S_{img}$ alone might be misleading, as it could result from content leakage (where structural details from the reference image are unintentionally copied) or mode collapse (where the model merely mimics the reference image too closely), rather than true style transfer.

• $S_{harmonic}$ (Harmonic Score):

  1. Conceptual Definition: To provide a more balanced evaluation that jointly reflects both prompt fidelity and style fidelity, the harmonic mean of $S_{txt}$ and $S_{img}$ is used. This metric penalizes models that perform well on one aspect but poorly on the other, favoring models that achieve a good balance.
  2. Mathematical Formula: $$S _ { \mathrm { h a r m o n i c } } = { \frac { 2 S _ { \mathrm { t x t } } S _ { \mathrm { i m g } } } { S _ { \mathrm { t x t } } + S _ { \mathrm { i m g } } } }$$
  3. Symbol Explanation:
     • $S_{txt}$: The CLIP Text Score (prompt fidelity).
     • $S_{img}$: The CLIP Image Score (style fidelity).

5.3. Baselines

The paper compares its method against eight state-of-the-art style-personalized image generation models. These baselines are chosen for their prominence and diverse approaches within the field.

• Tuning-based Approaches (require fine-tuning per style):
  • DreamStyler [1]
  • DreamBooth-LoRA (DB-LoRA) [44]
  • B-LoRA [13]
  • StyleDrop [49] These methods typically involve training a new model instance or adapter (like LoRA) for each specific style, incurring high computational costs and long training times.
• Training-free or Pre-trained Approaches (operate without per-style optimization):
  • StyleAligned [18]
  • IP-Adapter [63]
  • CSGO [60]
  • StyleAR [59] These models either use pre-trained style adapters or rely on direct inference-time conditioning.

All baselines are run using publicly released implementations and their default hyperparameters to ensure fair and reproducible evaluation. The comparison also considers inference time as a critical factor, especially since most baselines are diffusion-based and thus slower. The paper aims to demonstrate that its training-free, scale-wise autoregressive approach achieves competitive performance while being significantly faster.

6. Results & Analysis

6.1. Core Results Analysis

The paper presents both quantitative and qualitative comparisons to validate its training-free style-personalization framework.

6.1.1. Quantitative Comparison with State-of-the-Art

The following are the results from Table 1 of the original paper:

Analysis of Table 1 (Qualitative comparison with state-of-the-art style-personalized image generation models):

• Inference Time: Our method stands out significantly in inference time, achieving 3.58 seconds, which is up to $195 \times$ faster than the closest competitor (StyleAR at 70.0 seconds). Most other models, especially the diffusion-based ones, take tens to hundreds of seconds, highlighting a major practical advantage for real-time applications.
• Style Fidelity ($S_{img}$): StyleAligned and IP-Adapter achieve the highest $S_{img}$ scores (0.707 and 0.697 respectively). However, the paper notes that high $S_{img}$ can be misleading due to content leakage or mode collapse, where the model merely copies the reference rather than transferring its style. Our method achieves a competitive $S_{img}$ of 0.630, which is robustly balanced with prompt fidelity.
• Prompt Fidelity ($S_{txt}$): DB-LoRA and B-LoRA show high $S_{txt}$ scores (0.354 and 0.355), indicating good semantic alignment. Our method is very competitive here with 0.334, surpassing several baselines.
• Harmonic Score ($S_{harmonic}$): This metric provides a balanced view. Our method achieves the highest $S_{harmonic}$ score of 0.437, indicating the best trade-off between prompt fidelity and style fidelity among all compared models. This demonstrates that our method successfully addresses the challenge of achieving both without sacrificing one for the other.
• Training-free vs. Fine-tuning: The table implicitly shows that training-based methods (like DB-LoRA, B-LoRA, DreamStyler, StyleDrop) are generally slower. Our method is training-free, offering deployment flexibility without the overhead of per-style fine-tuning.

6.1.2. Qualitative Comparison with State-of-the-Art

The following are the results from Figure 5 of the original paper:

Analysis of Figure 5 (Qualitative comparison with state-of-the-art style-personalized image generation models):

• Content Leakage: Models with high $S_{img}$ scores, such as StyleAligned and IP-Adapter, frequently exhibit content leakage. For instance, in the first row, a "photo of an eggplant" generated by StyleAligned clearly shows structural elements from the style reference (a building), rather than just transferring the style. This validates the paper's caution about $S_{img}$ not being a sole indicator of quality.
• Prompt Fidelity vs. Style Fidelity Trade-off:
  • DB-LoRA and B-LoRA tend to preserve the semantics of the input prompt better ($S_{txt}$), but sometimes show weaker style fidelity. Their outputs might not fully reflect the reference style. For example, DB-LoRA in the "photo of an eggplant" example retains the eggplant shape but the style is not as pronounced as in our method.
  • Our method demonstrates a good balance. It reliably preserves the intended content while faithfully transferring the reference style, without content leakage. For example, in the "photo of an eggplant" example, our method generates an eggplant with the desired artistic style without incorporating structural details from the reference image.
• Overall: The qualitative results visually reinforce the quantitative findings: our method provides a strong balance between prompt fidelity and style fidelity while being significantly faster, making it a practical solution for high-quality, real-time style-personalized generation.

6.2. Ablation Studies / Parameter Analysis

6.2.1. Ablation Study on Proposed Components

The authors perform an ablation study to verify the contribution of each proposed component (Principal Feature Blending (PFB) and Structural Attention Correction (SAC)).

The following are the results from Table 2 of the original paper: <div class="table-wrapper"><table> <thead> <tr> <th></th> <th>$S_{txt} \uparrow$</th> <th>$S_{img} \uparrow$</th> <th>$S_{harmonic} \uparrow$</th> </tr> </thead> <tbody> <tr> <td>(a) Baseline</td> <td><b>0.354</b></td> <td>0.292</td> <td>0.319</td> </tr> <tr> <td>(b) Baseline + REP</td> <td>0.260</td> <td><b>0.686</b></td> <td>0.377</td> </tr> <tr> <td>(c) Baseline + PFB (SVD-guided)</td> <td>0.323</td> <td>0.640</td> <td>0.429</td> </tr> <tr> <td>(d) Baseline + PFB (SVD-guided) + SAC (full model)</td> <td><u>0.334</u></td> <td><u>0.630</u></td> <td><b>0.437</b></td> </tr> </tbody> </table></div> Analysis of Table 2 (Ablation study on our proposed method):

• (a) Baseline: The unmodified Infinity model. It achieves the highest prompt fidelity ($S_{txt}=0.354$) but has very low style fidelity ($S_{img}=0.292$), as it has no explicit style modulation. Its harmonic score is low (0.319).

• (b) Baseline + REP (Direct Replacement): This variant directly replaces the entire third feature (F_3) of the generation path with the $F_3$ from the style reference. It yields the highest style fidelity ($S_{img}=0.686$), but with a severe drop in prompt fidelity ($S_{txt}=0.260$). This indicates significant content leakage from the style reference, where the content of the reference image is copied instead of just its style.

• (c) Baseline + PFB (SVD-guided): This variant integrates Principal Feature Blending using SVD-guided modulation. It shows a much better trade-off: prompt fidelity ($S_{txt}=0.323$) is significantly improved compared to REP, while style fidelity ($S_{img}=0.640$) remains high. The harmonic score (0.429) is a substantial improvement over REP, confirming that SVD effectively isolates style.

• (d) Baseline + PFB (SVD-guided) + SAC (Full Model): The full proposed model, combining PFB and Structural Attention Correction. It achieves the highest harmonic score (0.437). While its $S_{txt}$ and $S_{img}$ are slightly lower than the best individual scores, the overall balance is optimal. SAC stabilizes the structural coherence without significantly degrading style or prompt fidelity, demonstrating the synergistic effect of both modules.

  The following figure (Figure 6 from the original paper) shows the qualitative ablation study on the proposed method:

  Analysis of Figure 6 (Qualitative ablation study on proposed method):

• (a) Baseline: Generates images with correct content but lacks any stylistic resemblance to the reference.

• (b) Direct Replacement (REP): Enforces strong style transfer but also introduces undesirable content elements from the reference. For example, if the reference is a painting of a landscape, direct replacement might inject parts of that landscape into the generated object. This leads to clear prompt mismatch.

• (c) PFB (SVD-guided): Successfully captures the intended style while retaining the target content. The generated images show the correct object in the desired style. However, the prompt adherence might still be slightly weaker than the full model.

• (d) Full Model (PFB + SAC): Preserves the style of the reference and simultaneously generates images that align closely with the prompt. This variant achieves the most balanced and desirable output, visually confirming the quantitative improvements observed in Table 2.

6.2.2. User Study

A user study was conducted with 30 participants to evaluate prompt fidelity and style fidelity subjectively.

The following are the results from Table 4 of the original paper: $$<div class="table-wrapper"><table> <thead> <tr> <th>Model</th> <th>Prompt Fidelity ↑</th> <th>Style Fidelity ↑</th> </tr> </thead> <tbody> <tr> <td>StyleAligned [18]</td> <td>4.3<td>30.7</tr> <tr> <td>IP-Adapter [63]</td> <td>5.0<td>23.3</tr> <tr> <td>DB-LoRA [44]</td> <td>26.7<td>8.3</tr> <tr> <td>B-LoRA [13]</td> <td>28.7<td>5.7</tr> <tr> <td>Ours</td> <td><b>35.3<td><b>32.0</tr> </tbody> </table></div>$$ Analysis of Table 4 (User study preference results):

• Our method achieves the highest preference for both prompt fidelity ($35.3\%$) and style fidelity ($32.0\%$).

• StyleAligned and IP-Adapter (baselines with high $S_{img}$ quantitatively) indeed show high style fidelity in user preference but very low prompt fidelity, confirming content leakage or prompt mismatch.

• DB-LoRA and B-LoRA (baselines with high $S_{txt}$ quantitatively) show strong prompt fidelity but very low style fidelity, indicating they struggle to transfer the style effectively.

• The user study results strongly corroborate the quantitative findings, demonstrating that our method provides the most preferred balance for users, validating its practical effectiveness.

  The following figure (Figure 11 from the original paper) shows an example of the user study interface:

  该图像是示意图，展示了一个选择图像与参考文本匹配的任务，其中参考文本为"一辆皮卡车"。图中显示了五个选项，供用户选择与参考图像风格最匹配的图像。

6.2.3. Additional Results for Key Step Feature Analysis (Varying $k$)

The supplementary material expands on the Key Step Feature Analysis by varying the number of preserved singular values, $k$.

• Method: The experiment uses the same prompt setup and intervention protocol as in the main paper, constructing 100 mixed prompt pairs. For each prompt, SVD is performed on $F_3$, and truncated variants $F_3^{(k)}$ are reconstructed, retaining only the top-$k$ singular values. $$F _ { 3 } ^ { ( k ) } = U \Sigma ^ { ( k ) } V ^ { \top }$$
  • $F_3^{(k)}$: The reconstructed feature retaining only the top-$k$ singular values.
  • $\Sigma^{(k)}$: Diagonal matrix with only the largest $k$ singular values preserved, others zeroed out. SVD-guided outputs are generated by replacing the corresponding portion of $\hat{F}_3$: $$\hat { F } _ { 3 } \gets F _ { 3 } ^ { ( k ) } + \hat { F } _ { 3 } ^ { \mathrm { r e s } ( k ) }$$
  • $\hat{F}_3^{res(k)} = \hat{F}_3 - \hat{F}_3^{(k)}$: Preserves the residual components of the original feature. They evaluate for $k \in \{1, 2, 4, 8, 16, 32\}$ and measure object-related and color-related CLIP similarity.

The following figure (Figure 7 from the original paper) shows the qualitative results of SVD-guided feature replacement with varying top-$k$ singular values:

Analysis of Figure 7 (Content and Style Similarity with varying $k$):

• Color-related Similarity (Style): This sharply increases at $k=1$ and then largely saturates. This demonstrates that the most dominant singular direction (the first principal component) primarily captures style (color and texture).

• Object-related Similarity (Content): This increases gradually as $k$ grows. This indicates that higher-rank components (beyond just the first) are needed to encode structural and content information.

• Conclusion: This further supports the main finding that the first principal component of $F_3$ predominantly encodes style, justifying the exponential reweighting design in PFB.

  The following figure (Figure 8 from the original paper) shows the qualitative results of SVD-guided feature replacement with varying $k$:

  Analysis of Figure 8 (Qualitative results of SVD-guided feature replacement with varying $k$):

• At $k=1$, the output successfully transfers texture and color from the reference while largely preserving the object shape (e.g., the bunny remains a bunny, but its fur texture and color change).

• As $k$ increases (e.g., $k=16, 32$), the generated image starts to pick up more structural details from the reference, altering the geometry and potentially the object identity. This visually confirms that higher-rank components carry more content-related information.

6.2.4. Analysis of the Exponential Decay Rate ($\alpha$)

An ablation study is performed on the exponential decay rate $\alpha$ in Principal Feature Blending (PFB).

The following are the results from Table 3 of the original paper: <div class="table-wrapper"><table> <thead> <tr> <th>alpha ($\alpha$)</th> <th>$S_{txt} \uparrow$</th> <th>$S_{img} \uparrow$</th> <th>$S_{harmonic} \uparrow$</th> </tr> </thead> <tbody> <tr> <td>0.2</td> <td>0.323</td> <td>0.640</td> <td>0.429</td> </tr> <tr> <td>0.6</td> <td>0.331</td> <td>0.631</td> <td>0.434</td> </tr> <tr> <td>1.0 (ours)</td> <td>0.334</td> <td>0.630</td> <td><b>0.437</b></td> </tr> <tr> <td>2.0</td> <td>0.334</td> <td>0.624</td> <td>0.435</td> </tr> <tr> <td>5.0</td> <td><b>0.335</b></td> <td>0.621</td> <td>0.435</td> </tr> </tbody> </table></div> Analysis of Table 3:

• The method shows robustness across different values of $\alpha$.

• As $\alpha$ decreases (e.g., from 5.0 to 0.2), the prompt fidelity ($S_{txt}$) slightly decreases, while style fidelity ($S_{img}$) slightly increases. This is because a smaller $\alpha$ means slower decay, giving more influence to higher-rank singular components which carry more content-related information from the style reference, thus increasing the risk of content leakage and lowering prompt fidelity.

• The chosen $\alpha = 1.0$ (ours) provides the most balanced performance, yielding the highest harmonic score (0.437). This confirms the hypothesis that the dominant singular value encodes style, and carefully weighting components is crucial.

  The following figure (Figure 9 from the original paper) shows the visualization of exponential decay rates $\alpha$ with respect to the singular value index $i$:

  该图像是一个示意图，展示了与奇异值索引 i ext{ (i o ext{0, 1, ext{...}, 30})} 相关的指数衰减率 $ext{exp}(- ext{α}_i)$ 的可视化。不同的曲线代表不同的衰减率 $ext{α}$ 值，分别为 0.2, 0.6, 1.0, 2.0 和 5.0。 Analysis of Figure 9: This visualization clearly shows how different $\alpha$ values control the exponential decay of weights. A smaller $\alpha$ leads to a flatter curve, meaning higher-indexed (less dominant) singular values retain more weight. A larger $\alpha$ leads to a steeper curve, giving overwhelming importance to the first few dominant singular values and rapidly diminishing the influence of others.

6.2.5. Generalization to Other Scale-wise Autoregressive Models

The paper demonstrates the model-agnostic nature of its method by applying it to other scale-wise autoregressive models.

• Infinity-8B: A larger variant of the base model. The method produces consistent and stable stylization, showing robustness to architectural scaling.

• Switti [56]: A distinct scale-wise autoregressive T2I model with different architecture. The plug-and-play modules (PFB and SAC) function reliably without modification, generating coherent, style-personalized outputs.

  The following figure (Figure 12 from the original paper) shows the qualitative results of applying our method to other scale-wise autoregressive models:

  该图像是插图，展示了将我们的方法应用于不同的规模自回归模型的定性结果。图中对比了“Infinity-8B”模型与结合我们方法后的生成效果，包括多组图像及其对应的风格参考，展示了风格个性化的能力。每组图片展示了不同对象的风格呈现，底部的图像则引入了“Switti”模型的对比结果。 Analysis of Figure 12: The images demonstrate that the method successfully transfers styles to images generated by Infinity-8B (Top row) and Switti (Bottom row), indicating its portability and generality across different scale-wise AR architectures.

6.2.6. Additional Qualitative Results

The paper includes additional qualitative results (Figure 13) to further showcase the model's ability to transfer style while avoiding content leakage. It also demonstrates style-aligned image generation using only a style prompt (no reference image needed) by leveraging a dedicated style pathway in the same batch (Figure 14).

The following figure (Figure 13 from the original paper) shows various style-personalized results of the model:

该图像是多种风格个性化结果的示意图，展示了不同风格参考下生成的图像，包括动物、植物及场景等各类主题，每个单元格展示了相应的风格转换效果。 Analysis of Figure 13: This figure displays a diverse set of examples, confirming that the model can faithfully transfer various styles (e.g., different artistic media, color palettes, textures) across a wide range of content, effectively avoiding content leakage and mode collapse.

The following figure (Figure 14 from the original paper) shows style-aligned image generation results:

该图像是对比实例，其中展示了在不同风格下生成的图像，包括采用我们的框架生成的样本和使用 StyleAligned、AlignedGen 方法生成的样本。每一行展示了不同类型（如龙、钟表、宇航员等）的图像，并与对应的描述相结合，标示出了各个艺术风格。 Analysis of Figure 14: This figure demonstrates the model's capability for style-aligned image generation where the style is guided by text (e.g., "fantasy art style") rather than an image. It shows competitive performance against StyleAligned and AlignedGen, highlighting the method's versatility in text-guided as well as image-guided style scenarios.

6.3. Implementation Details

• Base Model: Pre-trained Infinity 2B model [16], with all parameters frozen.
• Steps: Scale-wise prediction across 12 steps.
• Codebook: Baseline uses a codebook of size $2^{32}$, with quantized feature maps of resolution $64 \times 64 \times 32$.
• PFB Parameter: Exponential decay rate $\alpha$ for Principal Feature Blending is set to 1.0.
• Intervention Stages:
  • PFB is applied at $s=3$.
  • SAC is applied at fine stages ($\mathbf{S}_{fine} = \{3, 4, \dots, S\}$).
• Hardware: Generating a $1024 \times 1024$ image takes approximately 3.58 seconds on a single NVIDIA A6000 GPU.

7. Conclusion & Reflections

7.1. Conclusion Summary

This paper introduces a highly efficient and flexible training-free framework for style-personalized image generation. By conducting a meticulous step-wise analysis of a scale-wise autoregressive model (Infinity), the authors identified that the third feature (F_3) is crucial for shaping both content and style. Further SVD-based spectral analysis revealed that the dominant singular components of this feature primarily encode style-specific variations. Leveraging these insights, the paper proposed two lightweight modules: Principal Feature Blending (PFB) for precise style modulation and Structural Attention Correction (SAC) for maintaining structural consistency. The framework achieves competitive style fidelity and prompt fidelity compared to fine-tuned baselines, while significantly reducing inference time and offering greater deployment flexibility due to its training-free nature.

7.2. Limitations & Future Work

The authors acknowledge a key limitation:

• Handling Heterogeneous/Conflicting Styles: The current method may struggle when the style reference image contains heterogeneous or conflicting stylistic attributes (e.g., a mix of different artistic media or multiple visual motifs). Because the style extraction relies on dominant singular components of a single feature, the injected style might reflect a blended representation of multiple styles rather than isolating a specific, desired sub-style.

  They suggest several promising future research directions:

• Localized Style Decomposition: Developing mechanisms to decompose style at a more granular, spatially variant level.

• Spatially Variant Basis Representations: Exploring representations that can adapt to different stylistic needs across an image.

• User-Guided Selection: Incorporating user input to enable more fine-grained style control, allowing users to select or refine specific sub-styles from a complex reference.

• Application to other attributes: The analysis of feature decomposition (SVD) could potentially be extended to control other visual attributes beyond style.

7.3. Personal Insights & Critique

This paper presents a highly insightful and practical contribution to the field of generative AI.

• Innovation in Efficiency and Flexibility: The primary strength lies in its training-free and fast inference approach. In an era where large models require immense computational resources for fine-tuning, a method that achieves comparable quality without additional training offers significant value for real-world deployment, especially in interactive applications. The choice of a scale-wise autoregressive model over diffusion models is a clever move to prioritize speed.
• Elegant Use of SVD: The rigorous step-wise analysis and the identification of $F_3$ as a pivotal feature, followed by the precise use of SVD to disentangle style from content, is particularly elegant. SVD is a classical linear algebra technique, and its effective application here demonstrates that deep understanding of model internals can lead to powerful, interpretable control mechanisms. This contrasts with more black-box approaches that might rely on learning complex embeddings.
• Interpretability: The ability to trace style components to dominant singular values offers a degree of interpretability into how the model perceives and manipulates style, which is often lacking in complex neural networks.
• Model Agnostic Potential: The demonstrated generalization to Infinity-8B and Switti highlights the method's potential model-agnostic nature within the scale-wise AR family, which is a valuable characteristic for future adoption.
• Critique and Areas for Improvement:

  • Complexity of Style References: The identified limitation regarding heterogeneous style references is crucial. Real-world artistic styles are often layered and complex. Future work could explore more advanced feature decomposition methods (beyond simple SVD on a single feature) or attention-based style extraction that can isolate specific regions or components of a style.

  • Generalization Beyond Infinity: While it generalizes to other scale-wise AR models, its direct applicability to diffusion models (which are still dominant) might be limited due to architectural differences. Adapting these insights to diffusion models could further expand its impact.

  • User Control Granularity: While PFB and SAC offer control, the level of fine-grained control a user has over which aspect of style is transferred (e.g., color palette vs. brushstrokes) might be enhanced. This aligns with the authors' future work suggestions.

  • Quantitative Measure of Content Leakage: While $S_{img}$ and $S_{txt}$ indirectly capture content leakage (by comparing the drop in $S_{txt}$ when $S_{img}$ is high), a more direct, dedicated metric for content leakage could further strengthen the evaluation.

    Overall, this paper provides a robust and innovative framework that pushes the boundaries of training-free style personalization, making significant strides in balancing efficiency, flexibility, and output quality.