Xuran Meng and Yi Li's contribution to the Discussion of "On optimal linear prediction" by I. Helland.
Congratulations to Professor Helland on the elegant development of optimal partial least squares (PLS) regression, which reconceptualizes model reduction through the geometry of group actions and Hilbert spaces. Within this framework, PLS identifies a lower-dimensional subspace of functions that is provably optimal among all m $$ m $$ -dimensional reductions under specified covariance or spectral conditions. This rigorous notion of optimality naturally raises the question: can an analogous concept be established for domain adaptation (Ganin et al., 2016; He et al., 2024; Luo et al., 2025)., a technique that has recently attracted significant attention in modern machine learning? To make the context of domain adaptation concrete, consider a simple example. Suppose we want to classify images of cats and dogs. In the source domain S $$ S $$ , we observe clean images; in the target domain T $$ T $$ , the same cats and dogs appear, but all images are rotated by arbitrary angles. A classifier trained on S $$ S $$ may fail badly in T $$ T $$ , because the marginal distributions of pixels, say, f S $$ {f}_S $$ and f T $$ {f}_T $$ , differ drastically. Yet the domains share invariant predictive features, such as those extracted by convolutional pooling layers that ignore rotations. If we can find a representation map h : 𝒳 → 𝒵 that captures these invariants, then prediction becomes possible across domains. This rotation example highlights the central challenge of domain adaptation: although f S $$ {f}_S $$ and f T $$ {f}_T $$ differ, a representation h $$ h $$ may capture outcome-relevant invariants across domains. The key question is whether one can rigorously define an optimal representation, balancing predictive accuracy with invariance, analogous to how PLS identifies optimal subspaces in regression. Moreover, what does it mean for a representation h $$ h $$ to be “optimal” for domain adaptation. This matters because, without a formal notion of optimality, existing methods such as adversarial training can only propose candidate solutions without offering a principled benchmark. The challenge arises from the fact that the distributions of x $$ \mathbf{x} $$ may differ arbitrarily across domains, making direct transfer of models infeasible and requiring careful extraction of predictive features that remain invariant. Helland's symmetry-based framework provides precisely the right tools for this task: group actions, Hilbert spaces, and invariance measures that together define optimality in rigorous geometric terms. In practice, adversarial training (Ganin et al., 2016) enforces h # f S ≈ h # f T $$ {h}_{\#}{f}_S\approx {h}_{\#}{f}_T $$ by jointly training a domain discriminator to distinguish between h # f S $$ {h}_{\#}{f}_S $$ and h # f T $$ {h}_{\#}{f}_T $$ and a representation map h $$ h $$ to both confuse the discriminator and minimize prediction loss on the source domain; however, optimality was not addressed. Our formulation (1), grounded in Helland's geometry, provides the missing rigor by defining “optimal” invariant representations as solutions to constrained optimization problems, in the same way that PLS characterizes optimal subspaces. In sum, the challenge in domain adaptation is that source and target domains differ in their raw covariates x $$ \mathbf{x} $$ , making direct transfer infeasible and necessitating extraction of invariant predictive features z $$ \mathbf{z} $$ . By embedding this problem into Helland's symmetry-based framework, we obtain a principled notion of optimality, that is, minimizing prediction risk subject to distributional invariance. This new formulation replaces heuristic criteria with a precise benchmark for optimality, clarifying why some representations are preferable and enabling fair comparison across methods. This also demonstrates the novel applicability of Helland's framework to modern machine learning. The article was supported by National Institutes of Health under grants R01CA249096 and R01CA269398. The authors declare no conflicts of interest. Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.
Read more
10