1. Bibliographic Information

1.1. Title

Breaking the Bottleneck: User-Specific Optimization and Real-Time Inference Integration for Sequential Recommendation

1.2. Authors

The authors of this paper are Wenjia Xie, Hao Wang, Minghao Fang, Ruize Yu, Defu Lian, and Enhong Chen from the University of Science and Technology of China (USTC) & State Key Laboratory of Cognitive Intelligence, and Wei Guo and Yong Liu from Huawei Noah's Ark Lab in Singapore. The affiliations indicate a collaboration between a top Chinese university in AI and a leading industrial research lab, suggesting a blend of rigorous academic research and a focus on practical, scalable solutions.

1.3. Journal/Conference

The paper was published in the Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (KDD '25). KDD is a premier, A*-rated international conference in the fields of data mining, knowledge discovery, and machine learning. Publication at KDD signifies a high level of innovation, technical soundness, and potential impact on the field.

1.4. Publication Year

2025

1.5. Abstract

The abstract introduces sequential recommendation (SR) as a commercially valuable field where performance gains from increasing model size (scaling laws) eventually plateau on fixed datasets. The authors identify a key limitation in existing models: they learn a single, unified pattern of user behavior and apply it to all users, neglecting individual interaction habits. To address this, they first propose PCRec-simple, a post-training plugin that performs a one-time optimization for each user's sequence using Kullback-Leibler (KL) divergence, demonstrating that this user-specific approach can break the performance bottleneck. Building on this, they introduce PCRec, a more advanced model that integrates this optimization directly into the forward pass. This end-to-end approach solves practical issues like robustness, parallelism, and optimization stability. The paper concludes that PCRec significantly outperforms state-of-the-art methods in experiments on real-world datasets.

1.6. Original Source Link

The paper provides a relative link: /files/papers/69146991e3c2400e9ee5eda4/paper.pdf. The publication status is accepted for the KDD 2025 conference.

---

2. Executive Summary

2.1. Background & Motivation

• Core Problem: In sequential recommendation (SR), simply making models larger (increasing parameters) does not always lead to better performance, especially when the amount of training data is fixed. This phenomenon, where performance hits a ceiling or even degrades, is referred to as the "performance bottleneck" of scaling laws.
• Importance and Gap: SR systems are crucial for e-commerce, streaming, and content platforms. The current state-of-the-art models, often inspired by large language models (LLMs), are trained to find a unified behavioral pattern across all users. This "one-size-fits-all" approach is the central gap identified by the authors. They argue that this method overlooks the rich, idiosyncratic information present in individual user sequences. For instance, some users explore diverse interests rapidly, while others have stable, long-term preferences. A single model struggles to capture this full spectrum of behaviors.
• Innovative Idea: The paper's core idea is to shift from a static, universally applied model to one that can dynamically specialize itself for each user at inference time. Instead of just using the trained model to predict, they propose performing an additional, fast optimization step using the user's own historical interaction sequence. This "single optimization" allows the model to fine-tune its parameters to better reflect that specific user's unique behavioral patterns, thereby unlocking performance gains that scaling alone cannot achieve.

2.2. Main Contributions / Findings

The paper makes the following primary contributions:

1. Identifies and Addresses a New Problem: It is the first work to systematically investigate user-specific optimization as a means to overcome the performance bottlenecks of scaling laws in sequential recommendation. It pioneers the idea of mining extra information from individual sequences when model scaling on fixed data yields diminishing returns.

2. Proposes PCRec-simple: A straightforward yet effective plugin that validates the core hypothesis. By performing a single optimization step on each user's sequence after the main training is complete, it demonstrates that this approach can indeed improve the performance of existing state-of-the-art SR models.

3. Proposes PCRec: An advanced, end-to-end model that integrates the user-specific optimization directly into the model's forward propagation process. This novel design solves critical practical challenges of the simpler approach, ensuring:

   • Robustness: Uses a learnable learning rate to balance general knowledge with user-specific insights.
   • Parallelism: The optimization is embedded in matrix operations, allowing for efficient batch processing like standard deep learning models.
   • Stability: A layer-by-layer error control mechanism, derived from variational inference, allows for stable updates even in very deep networks.

4. Achieves State-of-the-Art Performance: Extensive experiments show that PCRec significantly outperforms existing top-tier SR models across multiple datasets and settings, proving the effectiveness of its real-time inference and user-specific optimization approach.

   ---

3. Prerequisite Knowledge & Related Work

3.1. Foundational Concepts

To fully understand this paper, one must be familiar with the following concepts:

• Sequential Recommendation (SR): SR is a subfield of recommender systems that focuses on predicting a user's next interaction (e.g., next item to buy, watch, or click) based on their chronologically ordered history of previous interactions. Unlike traditional collaborative filtering, SR models the temporal dynamics and sequential patterns in user behavior. For example, given a user's history [item A, item B, item C], the goal is to predict item D.
• Transformer Architecture: Originally developed for natural language processing, the Transformer is a neural network architecture that relies heavily on the self-attention mechanism. It processes all items in a sequence simultaneously, allowing it to capture long-range dependencies more effectively than recurrent neural networks (RNNs). The core self-attention mechanism calculates a weighted sum of all item representations in the sequence, where the weights (attention scores) signify the importance of other items for understanding the current item. The standard formula is: $ \mathrm{Attention}(Q, K, V) = \mathrm{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V $
  • $Q$ (Query), $K$ (Key), and $V$ (Value) are linear projections of the input item embeddings. The dot product $QK^T$ computes a similarity score between each pair of items.
  • $\sqrt{d_k}$ is a scaling factor to stabilize gradients.
  • softmax converts the scores into probabilities (attention weights).
  • The final output is a weighted average of the Value vectors.
• Scaling Laws: A concept popularized by research on LLMs, scaling laws describe the predictable relationship between a model's performance and factors like its size (number of parameters), the amount of training data, and the computational budget. Typically, as these factors increase, the model's loss decreases in a power-law fashion. This paper investigates the scenario where this law breaks down, i.e., when increasing model size on a fixed dataset no longer improves performance.
• Kullback-Leibler (KL) Divergence: A measure from information theory that quantifies how one probability distribution differs from a reference probability distribution. It is asymmetric and non-negative. For discrete distributions $P$ and $Q$, it is defined as: $ D_{\mathrm{KL}}(P | Q) = \sum_{x \in \mathcal{X}} P(x) \log\left(\frac{P(x)}{Q(x)}\right) $ In this paper, KL divergence is used to measure the difference between a model's predicted distribution and a target distribution, forming the basis of the optimization objective.
• Variational Inference: A technique from Bayesian statistics used to approximate complex (often intractable) posterior probability distributions. The core idea is to define a simpler, parameterized family of distributions (the variational distribution) and then find the member of that family that is closest to the true posterior. "Closeness" is typically measured by minimizing the KL divergence, which is equivalent to maximizing a lower bound on the log-likelihood of the data, known as the Evidence Lower Bound (ELBO). This paper uses variational inference to derive a tractable loss function for its single-sequence optimization.

3.2. Previous Works

The paper positions itself within the context of several lines of research:

• Mainstream SR Models: The field has evolved from Markov Chains to deep learning models. Key architectures include:
  • GRU4Rec: An early deep learning model using Gated Recurrent Units (GRUs), a type of RNN, to capture sequential patterns.
  • SASRec: A seminal work that first applied the Transformer's causal self-attention mechanism to SR, becoming a dominant architecture.
  • BERT4Rec: Inspired by BERT, it uses a bidirectional Transformer and a "Cloze" (masking) objective to learn richer item representations.
• Scaling in Recommender Systems: Inspired by LLMs, researchers have explored building very large recommendation models.
  • HSTU: A model specifically designed for scalability, reaching trillions of parameters. It introduces architectural modifications like removing softmax from attention to improve efficiency and performance at scale. This model represents the "bottleneck" that the current paper aims to break.
  • Wukong: Another scalable model that uses stacked factorization machines to capture high-order feature interactions, primarily for click-through rate prediction.
• LLM-Enhanced Recommendation: This category involves using pre-trained LLMs to improve recommendations.
  • Semantic Enhancement: Models like UniSRec use LLMs to extract rich semantic embeddings from item text (e.g., product descriptions), which are then used by the SR model.
  • User Understanding: Models like InstructRec and LFM use LLMs to interpret user profiles or instructions in natural language to provide more personalized recommendations. These differ from PCRec as they rely on external textual information and pre-trained LLMs, whereas PCRec derives personalization solely from the interaction sequence itself.
• Test-Time Training (TTT):
  • TTT4Rec: This model also updates parameters during inference. However, its motivation is different: it aims to adapt the model to new user interactions that occur after deployment, helping it cope with data distribution shifts. In contrast, PCRec focuses on mining deeper patterns from the user's existing historical sequence available at inference time. The paper also notes that TTT4Rec's update mechanism is less parallelizable than PCRec's.

3.3. Technological Evolution

The technological timeline in SR has progressed from simpler statistical models to increasingly complex and large deep learning architectures:

1. Early Methods: Markov Chains and Matrix Factorization captured simple, often first-order, transitions between items.
2. RNN-based Era: Models like GRU4Rec introduced deep learning, using RNNs to model the sequence as a hidden state that evolves over time.
3. Transformer Era: SASRec demonstrated that attention mechanisms could capture more complex and long-range dependencies more effectively, making Transformers the de-facto standard.
4. Scaling Era: Inspired by LLMs, models like HSTU focused on architectural and engineering innovations to build massive-scale recommendation models, pushing the limits of performance through parameter growth.
5. The Proposed PCRec: This paper marks a potential new direction. Instead of continuing to scale the "unified" model, it proposes a complementary approach: inference-time personalization. It argues that once scaling hits its limit, the next frontier for improvement lies in specializing the model for individual users in real-time.

3.4. Differentiation Analysis

• vs. Standard SR Models: The fundamental difference is that standard models (e.g., SASRec, HSTU) are static at inference time—they use the same set of learned weights for every user. PCRec is dynamic; it performs a rapid, user-specific optimization to temporarily adapt its state for each inference request.

• vs. Scaling Law Models (HSTU): While HSTU seeks performance gains by making the model bigger, PCRec seeks gains by making the model smarter and more adaptive at an individual level. It is designed to work on top of or in conjunction with scalable architectures, breaking the performance ceiling that HSTU eventually hits.

• vs. TTT4Rec: The key distinction is the source and purpose of the update. TTT4Rec adapts to new data to stay current. PCRec "reflects" on the user's existing past to better understand their unique habits. Furthermore, PCRec's design allows for full parallelism, a significant advantage in practical systems.

  ---

4. Methodology

4.1. Principles

The core principle of the paper is single optimization: leveraging a user's entire interaction sequence to perform a tailored, one-time optimization of the model's state just before making a recommendation for that user. This allows the model to capture unique behavioral patterns that are missed by a general model trained on all users.

The theoretical foundation for this approach is variational inference. Since directly maximizing the probability of the next item is intractable, the authors reframe the problem as minimizing the KL divergence between an approximate posterior distribution (which the model learns) and the true posterior. This leads to a tractable optimization objective that can be used as a loss function for the single optimization step.

4.2. Core Methodology In-depth (Layer by Layer)

4.2.1. Base Model Architecture

The paper first defines a scalable SR architecture that serves as the foundation for both PCRec-simple and PCRec. This architecture is Transformer-based but with specific modifications for scalability.

Step 1: Input Embedding Given a user's interaction sequence $S_u^{1:n} = \{v_1, v_2, \dots, v_n\}$, the model first converts it into a sequence of embedding vectors. This is done by summing item embeddings with positional encodings. $ E = \left{ e _ { 1 } + p _ { 1 } , e _ { 2 } + p _ { 2 } , \cdot \cdot \cdot , e _ { n } + p _ { n } \right} \in \mathbb { R } ^ { n \times d } $

• $e_i$: The embedding vector for the $i$-th item in the sequence.
• $p_i$: A learnable positional encoding vector for the $i$-th position.
• $d$: The dimension of the embeddings.

Step 2: Scalable Attention Block The core of the model consists of stacked layers of a modified attention block. For an input $E$, the block computes Query ($Q$), Key ($K$), Value ($V$), and a gating vector ($U$) using linear transformations followed by an activation function $\phi$. $ { U , V , Q , K } = { \phi ( W ^ { i } E + b ^ { i } ) \ | \ i \in { U , V , Q , K } } $

• $W^i$ and $b^i$: Learnable weight matrices and bias vectors for each of the four components.

  The output of the attention mechanism is then calculated as: $ Y = f \left( N o r m \left( \phi \left( { Q K } ^ { T } + r a b ^ { p , t } \right) V \right) \odot U \right) $

• Norm: Layer Normalization.

• $rab^{p,t}$: A relative attention bias that incorporates both positional ($p$) and temporal ($t$) information.

• $\odot$: Element-wise multiplication.

• $U$: A gating mechanism that controls information flow, which is crucial for stability in deep networks.

• $f$: A multi-layer perceptron (MLP) for further processing.

• Key Differences from Standard Attention:

  1. No Softmax: The authors omit the softmax function, following work like HSTU, as it can "dilute" engagement signals and is computationally expensive.
  2. Gating Mechanism: The gate $U$ helps regulate which information is passed to the next layer, preventing issues like vanishing/exploding gradients in very deep models.

4.2.2. Optimization Objective Derivation

The ultimate goal is to maximize the conditional probability $p(v_{n+1} | S_u^{1:n})$. Using Bayes' theorem, this is $p(S_u^{1:n}, v_{n+1}) / p(S_u^{1:n})$. The denominator is intractable to compute. The authors use variational inference to overcome this. They introduce an approximate posterior distribution $q(v_{n+1} | S_u^{1:n}; \phi)$ and aim to minimize the KL divergence between $q$ and the true posterior $p$: $ q ^ { * } \left( \boldsymbol { v } _ { n + 1 } \mid \boldsymbol { S } _ { u } ^ { 1 : n } ; \boldsymbol { \phi } \right) = \underset { \boldsymbol { \phi } } { \mathrm { argmin } } \ : D _ { \mathrm { KL } } \left[ q \left( \boldsymbol { v } _ { n + 1 } \mid \boldsymbol { S } _ { u } ^ { 1 : n } ; \boldsymbol { \phi } \right) \mid \mid p \left( \boldsymbol { v } _ { n + 1 } \mid \boldsymbol { S } _ { u } ^ { 1 : n } \right) \right] $ The paper shows that minimizing this KL divergence is equivalent to minimizing an upper bound, which they call $\mathcal{F}$: $ \mathcal{F} = D_{\mathrm{KL}}\left[q\left(\boldsymbol{v}{n+1} \mid \boldsymbol{s}{0} ; \boldsymbol{\phi}\right) | \hat{p}\left(\boldsymbol{s}{0}, \boldsymbol{v}{n+1}\right)\right] $

• $\boldsymbol{s}_0$: Abbreviation for the input sequence $S_u^{1:n}$.

• $\hat{p}(\boldsymbol{s}_0, \boldsymbol{v}_{n+1})$: The joint probability of the sequence and the next item.

  This $\mathcal{F}$ becomes the central, tractable objective function for single optimization.

4.2.3. PCRec-simple: Single Optimization After Training

PCRec-simple serves as a proof-of-concept. It separates the standard training from the user-specific optimization.

1. Model Training: The model is trained conventionally using interaction sequences from all users. The loss function is Sampled Softmax, which is efficient for large item sets. $ \mathcal { L } _ { t r a i n } = - \frac { 1 } { B } { \sum _ { i = 1 } ^ { B } } log ( \frac { exp ( \hat { y } _ { i } \cdot e _ { n + 1 } / \tau ) } { \sum _ { j \in \mathcal { V } _ { M } } exp ( \hat { y } _ { i } \cdot e _ { j } / \tau ) } ) $

• $B$: Batch size.
• $\hat{y}_i$: The model's final output vector for the $i$-th sequence.
• $e_{n+1}$: The embedding of the true next item (ground truth).
• $\mathcal{V}_M$: A set of negative samples (randomly chosen items).
• $\tau$: A temperature parameter.

2. Model Inference: During inference for a specific user, PCRec-simple performs a single gradient descent step on the model's parameters to minimize a loss derived from the objective $\mathcal{F}$. This loss is computed over the user's known history, effectively asking the model to "re-learn" this user's specific sequence. $ \mathcal { L } _ { i n f e r e n c e } = \frac { 1 } { n - 1 } \sum _ { i = 1 } ^ { n - 1 } D _ { \mathrm { KL } } \left[ q \left( v _ { i + 1 } ~ | ~ S _ { u } ^ { 1 : i } ; \phi \right) ~ | | ~ \mathcal { P } \left( S _ { u } ^ { 1 : i } , v _ { i + 1 } \right) \right] $ After this single update, the modified model is used to predict the next item. Crucially, the parameter updates are temporary and are discarded before processing the next user's sequence. This approach confirmed the hypothesis but suffered from being slow (not parallelizable), sensitive to the learning rate, and unstable for deep models.

The architecture of PCRec is shown in Figure 3 from the original paper.

该图像是示意图，展示了PCRec模型的架构。图中包含多个层次，如嵌入层、线性层、注意力机制、层归一化与多层感知器（MLP）等，以及预测层和输出概率。各层之间通过门控机制和矩阵运算相连，体现了信息传递及优化过程。重点强调了利用注意力机制和预测误差进行用户交互序列的分析与优化。

4.2.4. PCRec: Single Optimization in Forward Process

PCRec is the main contribution, integrating the optimization into the forward pass to solve the issues of PCRec-simple.

1. Layer-by-Layer Error Control: Instead of defining a single loss at the final layer and backpropagating, PCRec approximates the objective $\mathcal{F}$ as a sum of local prediction errors at each layer. This is inspired by predictive coding theory. The model is viewed as a hierarchical generative model where each layer's state $s_l$ is predicted from the layer above, $s_{l+1}$. The distribution is assumed to be Gaussian: $ p \left( s _ { l } \mid s _ { l + 1 } \right) = N \left( s _ { l } ; \theta _ { l + 1 } f \left( s _ { l + 1 } \right) , I \right) $

• $s_l$: The latent state (activations) at layer $l$.

• $f$: A non-linear function (e.g., Sigmoid, Softplus).

• $\theta_{l+1}$: A weight matrix that generates a prediction of $s_l$ from $s_{l+1}$. This matrix learns to fuse information across the sequence, as illustrated in Figure 4.

• $I$: Identity covariance matrix.

  The process of fusing information via the weight matrix $\theta$ is depicted below.

  该图像是公式与示意图结合的展示，描述了如何通过加权求和来融合用户交互序列的信息。具体公式为 $\mu_{l,j} = \sum_{k=1}^{n} \theta_{l+1,i}^k \cdot e_{l+1,k,j}$，其中 $n$ 表示序列长度，$d$ 为嵌入维度，图中阐述了这一融合过程的矩阵形式。

With this assumption, the global objective $\mathcal{F}$ can be approximated as the sum of squared prediction errors at each layer: $ \mathcal { F } \approx \sum _ { l = 0 } ^ { L - 1 } \Vert \varepsilon _ { l } \Vert ^ { 2 } : = \mathcal { E } $

• $\varepsilon_l = s_l - \mu_l$: The prediction error at layer $l$, where $\mu_l = \theta_{l+1}f(s_{l+1})$ is the prediction of $s_l$.

  This decomposition allows for local updates, making the optimization stable even in deep networks, as visualized in Figure 5.

  

  2. Local Gradient Updates during Forward Pass: The optimization is performed via gradient descent on the local error $\mathcal{E}$ and integrated into the forward pass. This happens in two stages for each layer:

• Activation Update (Inference): The neural activations $s_l$ are updated to minimize the local error. $ \Delta \mathsf { s } _ { l } = - \gamma \cdot \frac { \partial \mathcal { E } } { \partial \mathsf { s } _ { l } } = \left{ \begin{array} { l l } { \gamma \cdot \left( \theta _ { l } f ^ { \prime } \left( \boldsymbol { s } _ { l } \right) \cdot \varepsilon _ { l - 1 } \right) , } & { l = L ; } \ { \gamma \cdot \left( - \varepsilon _ { l } + \theta _ { l } f ^ { \prime } \left( \boldsymbol { s } _ { l } \right) \cdot \varepsilon _ { l - 1 } \right) , } & { 0 < l < L ; } \ { \gamma \cdot \left( - \varepsilon _ { l } \right) , } & { l = 0 . } \end{array} \right. $

  • $\gamma$: The learning rate for this update, which is learnable. This is key to balancing general and user-specific information. It is computed dynamically for each layer: $ \gamma = \sigma ( W _ { l r } E _ { l } + b _ { l r } ) $ where $W_{lr}, b_{lr}$ are trainable parameters and $\sigma$ is the sigmoid function.

• Weight Update (Learning): After updating the activations, the weights $\theta_{l+1}$ that make the predictions are updated to further reduce the error. $ \Delta \theta _ { l + 1 } = - \alpha \left( \partial \mathcal { E } / \partial \theta _ { l + 1 } \right) = \alpha \left( \varepsilon _ { l - 1 } \cdot f ( s _ { l + 1 } ) ^ { \top } \right) $

  • $\alpha$: A fixed learning rate hyperparameter.

    By embedding these update rules directly into the forward pass as matrix operations, PCRec can be trained end-to-end and can process batches of sequences in parallel, solving all the practical issues of PCRec-simple.

---

5. Experimental Setup

5.1. Datasets

The experiments were conducted on three widely-used public datasets from different domains to ensure the generalizability of the results.

• MovieLens-1M & MovieLens-20M: These datasets contain movie ratings from users of the MovieLens website. They are benchmarks in recommendation research, with MovieLens-20M being significantly larger and sparser than MovieLens-1M.

• Amazon Beauty: This is a subset of the Amazon product review dataset, containing user reviews for "Beauty" category products. It is characterized by having much shorter user interaction sequences on average compared to MovieLens.

  The following are the statistics from Table 1 of the original paper:

  Datasets	#Users	#Items	#Inters.	Avg. n
  MovieLens-1M	6,040	3,706	1,000,209	165.6
  MovieLens-20M	138,493	26,744	20,000,263	144.4
  Amazon Beauty	22,363	12,101	198,502	8.53

These datasets were chosen because they represent different scales and data characteristics (e.g., sequence length, density), which effectively tests the robustness of the proposed methods.

5.2. Evaluation Metrics

The paper uses two standard ranking metrics to evaluate the performance of the models. For both metrics, a higher value indicates better performance.

• Hit Rate (HR@K):

  1. Conceptual Definition: This metric measures the percentage of users for whom the correct next item is found within the top-K recommended items. It answers the simple question: "Was the right item in the top-K list?".
  2. Mathematical Formula: $ \text{HR@K} = \frac{1}{|\mathcal{U}|} \sum_{u \in \mathcal{U}} \mathbb{I}(\text{rank}_{u} \le K) $
  3. Symbol Explanation:
     • $|\mathcal{U}|$: The total number of users in the test set.
     • $\text{rank}_u$: The rank of the ground-truth item in the sorted recommendation list for user $u$.
     • $\mathbb{I}(\cdot)$: The indicator function, which is 1 if the condition inside is true, and 0 otherwise.

• Normalized Discounted Cumulative Gain (NDCG@K):

  1. Conceptual Definition: NDCG@K also evaluates whether the correct item is in the top-K list, but it additionally considers the position of the item. It assigns a higher score if the correct item is ranked higher (e.g., at position 1 vs. position 10). The score is normalized by the ideal score, so it always falls between 0 and 1.
  2. Mathematical Formula: First, Discounted Cumulative Gain (DCG) is defined as: $ \text{DCG@K} = \sum_{i=1}^{K} \frac{rel_i}{\log_2(i+1)} $ Then, NDCG is the DCG divided by the Ideal DCG (IDCG), which is the DCG of a perfect ranking. $ \text{NDCG@K} = \frac{\text{DCG@K}}{\text{IDCG@K}} $
  3. Symbol Explanation:
     • $K$: The number of items in the recommendation list.
     • $i$: The rank position in the list.
     • $rel_i$: The relevance of the item at position $i$. In this paper's setting (predicting a single next item), $rel_i$ is 1 if the item at rank $i$ is the ground-truth item, and 0 otherwise.
     • $\text{IDCG@K}$: The maximum possible DCG@K score, achieved if the ground-truth item is ranked at position 1.

5.3. Baselines

The proposed models were compared against a comprehensive set of representative baselines, covering various architectures.

• GRU4Rec: A classic RNN-based model for session-based recommendation.

• BERT4Rec: A powerful SR model inspired by BERT, using a bidirectional Transformer.

• SASRec: A widely-used baseline that pioneered the use of causal self-attention for SR.

• Mamba4Rec: A recent model that uses State Space Models (SSMs) as an alternative to Transformers for efficient sequence modeling.

• TTT4Rec: A model that performs Test-Time Training to adapt to new user interactions during inference.

• HSTU: A state-of-the-art scalable Transformer-based model designed for very large parameter counts, representing the performance ceiling of the "scaling law" approach.

• LLaMa: The authors adapted the LLaMa architecture, a prominent LLM, for the SR task to see how a general-purpose powerful sequence model performs.

  These baselines were chosen to represent the evolution of SR models and include the current state-of-the-art, ensuring a rigorous and fair comparison.

---

6. Results & Analysis

6.1. Core Results Analysis

The main experimental results are presented in Table 2, which compares PCRec-simple and PCRec with all baselines across three datasets, using both base and large model sizes.

The following are the results from Table 2 of the original paper:

Key Findings:

1. Baseline Performance and Scaling: Among the baselines, HSTU-Large consistently performs the best, especially on the larger MovieLens datasets. This confirms its strong scalability and establishes it as a powerful state-of-the-art model. In contrast, SASRec-Large shows significant performance degradation, highlighting that not all architectures benefit from simply increasing depth.
2. PCRec-simple Validates the Hypothesis: PCRec-simple-Large shows slight improvements over HSTU-Large on MovieLens-1M, confirming that even a simple, one-time optimization on individual sequences can extract additional useful information and push past the scaling bottleneck. However, its gains are marginal and not consistent across all settings, reflecting its inherent instability.
3. PCRec Achieves Superior Performance: The main model, PCRec, and its large version, PCRec-Large, consistently and significantly outperform all baselines across all three datasets. The improvements over the strongest baseline (HSTU-Large) are statistically significant (indicated by *) in most cases. This demonstrates that by integrating the optimization into the forward pass in a robust and parallelizable manner, the model can effectively balance general patterns and individual user behaviors to achieve a new level of performance.

6.2. Ablation Studies / Parameter Analysis

6.2.1. Impact of the Nonlinear Function $f$

Table 3 investigates how the choice of the nonlinear function $f$ in the predictive model affects PCRec's performance.

The following are the results from Table 3 of the original paper:

| Function | Movielens-1M | Amazon Beauty | :--- | :--- | :--- | :--- | :--- | | NDCG@10 | HR@10 | NDCG@10 | HR@10 | - | 0.1642 | 0.3051 | 0.0689 | 0.0967 | Sigmoid | 0.1730 | 0.3042 | 0.0710 | 0.1028 | Tanh | 0.1688 | 0.3064 | 0.0688 | 0.0989 | Softplus | 0.1728 | 0.3126 | 0.0727 | 0.1012 | ReLU | 0.1684 | 0.3073 | 0.0698 | 0.0996

Analysis: Softplus and Sigmoid yield the best results. The authors attribute this to their smoothness, which provides more stable derivatives during the local gradient updates. Using no activation function at all (identity mapping, denoted by -) performs the worst, as it limits the model's expressive power in predicting layer states. This study shows that the specific form of the generative assumption within PCRec is an important design choice.

6.2.2. Impact of Learning Rates $α$ and $β$

Figure 6 shows the sensitivity of PCRec-simple to its learning rate $\beta$ and PCRec to its weight-update learning rate $\alpha$.

Analysis:

• PCRec-simple (left plot): The performance is highly sensitive to the choice of $\beta$. A learning rate that is too high (e.g., $10^{-3}$) causes severe overfitting, degrading performance. A rate that is too low (e.g., $10^{-8}$) has no effect. This highlights the fragility of the post-training optimization approach.
• PCRec (right plot): The model is much more robust to the choice of $\alpha$. Performance is consistently high across a wide range of values. This stability is attributed to two factors: (1) the main inference is driven by activation updates, which use a learnable learning rate $\gamma$ for self-adaptation, and (2) the weights $\theta_l$ are part of the internal predictive mechanism, not the final output layer, making their updates less volatile.

6.2.3. Efficiency Analysis

The paper analyzes the computational complexity and runtime of the models.

The following are the results from Table 4 of the original paper:

(Note: The complexity formulas and some data for PCRec-simple were incomplete in the provided text. Based on the description, PCRec-simple's training time is similar to the base model, but its inference time is much higher due to lack of parallelism.)

Analysis:

• PCRec-simple is inefficient at inference time because it processes each sequence individually, breaking the batch parallelism of standard deep learning frameworks.
• PCRec integrates the optimization into parallelizable matrix operations. While this adds some computational overhead, leading to a slight increase in training time compared to baselines, its inference time remains efficient and practical for real-world deployment. The trade-off of slightly longer training for significantly better accuracy is well-justified.

6.2.4. Robustness Analysis

The paper examines model performance on different subsets of data to test robustness.

• Head and Long-tail Items (Figure 7):

  该图像是图表，展示了在 MovieLens-1M 和 Amazon beauty 数据集中，三种推荐模型（SASRec, Mamba4Rec, HSTU）与新模型 PCRec 在头部项和长尾项上的性能对比。图中显示了不同模型在指标 NDCG@10 上的表现差异，表明 PCRec 在两种数据集上均优于其他模型。

  The results show that PCRec outperforms baselines on both popular (head) and unpopular (long-tail) items. Notably, the performance gain is more pronounced for long-tail items. This suggests that by focusing on an individual user's sequence, PCRec can better understand their preferences for niche items, reducing its reliance on the global popularity signals that dominate standard models.

• Different Sequence Lengths (Figure 8):

  该图像是一个柱状图，展示了不同长度序列下，几种推荐模型的性能比较，包括SASRec、Mamba4Rec、HSTU、PCRec-simple和PCRec。图中表示的NDCG@10值显示，PCRec在各长度序列上均表现优越，明显超过其他模型。

  As expected, all models perform better on longer sequences, which contain more information. PCRec-simple shows little to no improvement on short sequences, where its single optimization step can easily lead to overfitting. In contrast, PCRec demonstrates consistent and significant improvements across all sequence lengths, from very short to very long, highlighting its robustness and the effectiveness of its integrated, balanced optimization strategy.

---

7. Conclusion & Reflections

7.1. Conclusion Summary

This paper successfully identifies and addresses a critical limitation of current sequential recommendation models: the performance bottleneck encountered when scaling model size on a fixed dataset. The authors convincingly argue that this bottleneck arises because existing models learn a single, unified pattern of user behavior, ignoring valuable information within individual user interaction sequences.

The primary contributions are the two proposed models:

• PCRec-simple serves as an effective proof-of-concept, demonstrating that a one-time, user-specific optimization can indeed break the performance ceiling.

• PCRec provides a practical, robust, and end-to-end solution by integrating this optimization into the model's forward pass. This novel design solves key challenges related to stability, parallelism, and robustness, making it suitable for large-scale, real-world systems.

  Through extensive experiments, the paper shows that PCRec significantly outperforms state-of-the-art models, proving that real-time, user-specific inference is a powerful and promising direction for the future of recommendation systems.

7.2. Limitations & Future Work

The authors suggest several directions for future research:

• Generalization: They plan to develop a more general framework that can apply the principle of single optimization to a wider variety of sequential recommendation architectures beyond the specific Transformer-like model used in the paper.
• Enhanced Scaling: They aim to explore whether the layer-wise update mechanism can be further improved to not only adapt to deep models but also enhance the model's intrinsic scaling capabilities. This includes investigating the possibility of achieving "scaling during inference" by updating hidden states, which could lead to even more powerful and dynamic models.

7.3. Personal Insights & Critique

• Inspirations:

  • A New Personalization Paradigm: The paper's core idea of "inference-time personalization" is highly innovative. It represents a paradigm shift from building ever-larger static models to creating models that can dynamically adapt. This concept is broadly applicable to many other machine learning domains beyond recommendation, such as personalized dialogue systems, user modeling, and any task where individual-level patterns are important.
  • Bridging ML and Cognitive Science: The connection drawn to "predictive coding" theory in neuroscience is fascinating. This theory posits that the brain constantly makes predictions about sensory input and updates its internal models based on prediction errors. PCRec's layer-wise error minimization mechanism mirrors this principle. This link suggests that exploring brain-inspired computational models could be a fruitful avenue for building more intelligent and efficient AI systems.
  • Practicality: The design of PCRec thoughtfully addresses real-world engineering constraints like parallelism and robustness, making it not just a theoretical curiosity but a potentially deployable solution.

• Potential Issues and Areas for Improvement:

  • Underlying Assumptions: The theoretical derivation of the layer-wise error update relies on a Gaussian assumption for the latent variables and a Laplace approximation. While empirically effective, the validity of these assumptions for modeling complex, discrete user-item interactions could be further explored. Alternative distributional assumptions might yield different or improved results.
  • Computational Overhead: Although PCRec is parallelizable, its forward pass is inherently more complex than that of a standard Transformer. In latency-critical applications (e.g., real-time bidding), this additional computation, however small, might need careful consideration and optimization.
  • Qualitative Understanding: The paper demonstrates that PCRec works but provides limited insight into what it learns during the single optimization. A qualitative analysis would be valuable. For example, for a user with diverse interests, how do the updates differ from those for a user with narrow preferences? Visualizing the changes in activations or attention patterns could provide deeper understanding.
  • Interaction with Data Scaling: The paper's premise is breaking the bottleneck on a fixed data scale. An important open question is how this method's benefits evolve as the dataset size also scales. It is possible that with a sufficiently large and diverse dataset, the "unified" model might become powerful enough that the marginal gains from user-specific optimization diminish. Investigating this trade-off would be a compelling next step.