[upstream rENA 0.3.1] ena.rotate.by.hena.regression does not deflate the y axis

[HUDongpin]

This is an issue in rENA 0.3.1, not in ena-python. It is tracked here because it explains a deliberate difference in ena-python's output. Reported as an observation, not an accusation — it may be fixed upstream or intended, in which case please correct me. Full write-up: docs/rena-upstream-issues.md

Summary

Given both x_var and y_var, ena.rotate.by.hena.regression returns two rotation axes that are strongly collinear, so the resulting "2D" projection is close to one-dimensional. Its sibling ena.rotate.by.generalized returns orthogonal axes on identical data.

Severity: high — it affects returned coordinates.

Reproduce

Tested with rENA 0.3.1, R 4.4.2, aarch64-apple-darwin20.

library(rENA)
rows <- data.frame(
  unit  = rep(paste0("U", 1:6), each = 4),
  conv  = rep(rep(c("c1", "c2"), each = 2), times = 6),
  grp   = rep(c("g1", "g2"), each = 12),
  score = rep(c(10, 20, 30, 40, 50, 60), each = 4),
  A = c(1,0,1,0, 0,1,1,1, 1,1,0,1, 0,0,1,0, 1,0,0,1, 0,1,1,0),
  B = c(0,1,1,1, 1,1,0,0, 0,1,1,0, 1,0,0,1, 0,1,1,0, 1,0,0,1),
  C = c(1,1,0,0, 1,0,1,0, 1,0,0,1, 0,1,1,1, 1,1,0,0, 0,0,1,1),
  D = c(0,0,1,1, 0,1,0,1, 0,1,1,0, 1,1,0,0, 0,0,1,1, 1,1,0,0),
  stringsAsFactors = FALSE)
acc <- ena.accumulate.data(
  units = rows[, "unit", drop = FALSE], conversation = rows[, "conv", drop = FALSE],
  metadata = rows[, c("grp", "score")], codes = rows[, c("A","B","C","D")],
  window.size.back = 2)
set <- ena.make.set(acc, dimensions = 2)

cosine <- function(m, i = 1, j = 2) {
  a <- m[, i]; b <- m[, j]
  sum(a * b) / (sqrt(sum(a * a)) * sqrt(sum(b * b)))
}
rot <- function(r) as.matrix(r$rotation)

reg <- ena.rotate.by.hena.regression(set, list(x_var = "V ~ score", y_var = "V ~ grp"))
gen <- ena.rotate.by.generalized(set, list(
  x_var = set$meta.data[, "score", drop = FALSE],
  y_var = set$meta.data[, "grp", drop = FALSE]))

cosine(rot(reg))   #> [1] 0.8116353       <- x and y are 81% collinear
cosine(rot(gen))   #> [1] -3.330669e-16   <- same data, orthogonal

The y axis is exactly what you get by regressing y on the undeflated points:

y_alone <- ena.rotate.by.hena.regression(set, list(x_var = "V ~ grp"))
all.equal(abs(rot(reg)[, 2]), abs(rot(y_alone)[, 1]))   #> TRUE

How collinear depends on the data — 0.97 occurs on other datasets. That they are collinear at all is the point.

Root cause

ena.rotate.by.regression.R:91 sets V <- defA to regress y on the x-deflated points. That assignment never takes effect. The call at line 99 is:

v2_res <- with.ena.matrix(enaset$model$points.for.projection, {
  lm(formula(params$y_var));
});

and with.ena.matrix (ena.rotate.by.regression.2.R:18-29) binds its own V:

with.ena.matrix <- function(data, expr, ...) {
  dot_args <- list(...);
  V <- NULL;
  if (length(dot_args) > 0 && !is.null(dot_args$V)) {
    V <- dot_args$V;          # only when a V = argument is passed
  } else {
    V <- as.matrix(data);     # otherwise: the raw points.for.projection
  }

No V = argument is passed, so V is rebound to the undeflated points and the caller's V <- defA is shadowed. The commented-out line immediately above the call (:98) still shows the argument that would have made it work:

# v2 <- with(enaset$model$points.for.projection, NULL, formula = y, V = V);

This reads as a refactor that dropped V = V rather than a design decision: the V <- defA line is left as dead code, with.ena.matrix still supports V =, and the sibling ena.rotate.by.generalized deflates before its y-axis gmr() call on the same data.

The later deflation at line 130 (defA <- defA - defA %*% v2 %*% t(v2)) does use v2, so the SVD columns completing the basis are deflated by both axes. Only v2 itself comes from undeflated data.

Suggested fix

with.ena.matrix(enaset$model$points.for.projection, { lm(formula(params$y_var)) }, V = defA)

What ena-python does

Deflates, giving orthogonal axes; the x axis matches rENA exactly. Documented in pyena.rotation.rotate_by_regression. The test test_regression_xy_axes_are_orthogonal_unlike_rena asserts rENA's axes are still collinear, so it fails if rENA starts deflating — the signal for ena-python to match it again.

[HUDongpin]

Reported upstream: rENA #49 (2026-07-16).

The report there pins its source references to rENA main @ a09a4b20, verified byte-identical to the released 0.3.1 this was found against — so it applies to their current tree, not just the CRAN build.

This issue stays open here as ena-python's record of the divergence; it can close when the upstream one resolves.