screen_interactions

SuperGLM.screen_interactions(

X: object,

y: NDArray,

sample_weight: NDArray | None = None,

*,

offset: NDArray | None = None,

candidates: list[tuple[str, str]] | None = None,

edf0: float | tuple[float, ...] = (2.0, 4.0, 8.0, 16.0),

max_cells: int = 5000000,

screen_bins: int = 256,

phi: float | None = None,

) → DataFrame

Rank candidate pair interactions by PSST.

PSST (Penalized Smooth Score Test) asks, for each candidate pair of fitted features, how much of the model’s leftover working signal the interaction the pair would actually refit as could absorb at a fixed screening complexity, after profiling out what the pair’s own main effects already explain. The kind column names that refit target — ti for spline x spline, spline_cat for a spline crossed with a factor, cat_cat for two factors, numeric_cat for a per-level numeric slope and numeric_numeric for a product of two numerics — and z normalizes each kind against its own noise floor, so the one sorted table ranks them together, though not on equal terms and not simply by df: the measured null maxima span 3.98 to 7.53 across kinds. The heaviest tails sit at low probe df, yet “neither kind is monotone in df”, and the headline maxima read as “Gaussian-driven rather than as something every family reproduces”. Compare like with like before spending a refit (see the measured floors in the screening guide). A spline-mode OrderedCategorical margin screens as a spline on its mapped level scores, so its pairs carry the spline kinds. A kind whose block carries no penalty (cat_cat, numeric_cat, numeric_numeric) has no bandwidth to scan and is evaluated at a single rung: edf0 then reports the block’s achieved rank with lambda0 0. edf0 is a probe bandwidth; the default ladder evaluates each pair at several budgets and ranks by the best noise-normalized score z (the statistic is scaled by the fit’s Pearson dispersion estimate first, so the noise floor stays honest beyond unit-dispersion families), so smooth and high-frequency interactions are both visible. One O(n) cell pass per pair; no refits. offset and sample_weight both default to the values the model was fitted with (weights only when the fit’s were non-unit); inheriting requires X/y to be the training data — pass both arrays explicitly to screen a holdout, subsample, or reordered frame. The dispersion used is attached as table.attrs["phi"] and can be overridden via phi=.

Fitted mains with no screenable margin — Polynomial, RandomEffect, step-mode OrderedCategorical and any OrderedCategorical carrying specials= — are excluded from the sweep and reported in table.attrs["deferred_features"], a {feature: reason} mapping that is empty when everything fitted was screened. Naming one of them in candidates raises with the same reason.

The returned frame is sorted by z descending; rank by z — statistic/edf0/lambda0 describe each pair’s winning rung, so statistic is not comparable across rows. Pairs whose unique-value grid exceeds max_cells are quantile-binned to screen_bins support points per margin and flagged approx=True; pairs within budget are always computed exactly. max_cells bounds allocation AND time: the probe block’s dimension k enters every rung as a (k, k) decomposition — a pivoted QR of the pair’s design factor with one SVD beside it — so per-pair time grows as k^3 where the allocations grow as k^2. A probe column collinear with the overlap span is routine rather than exceptional (one empty cat_cat cell or one singleton level does it) and is settled by a rank cut on that factor; the Cholesky that used to run here, and the pseudo-inverse it used to fall back to, went with the assembled Grams in issue #257, which changed neither the cost class nor any ceiling below. A cubic-work gate caps k at the FLOOR of (1000 * max_cells)^(1/3) for an unpenalized block and of (500 * max_cells)^(1/3) for a penalized one, whose ladder can bisect: 1709 and 1357 at the default. Take the floor — at max_cells 1e4 the penalized root is 170.998 and the cap is 170, not 171. That cap is NOT a constant — max_cells sets it and nothing else does. Penalized ceilings: 1357 at 5e6, 793 at 1e6, 368 at 1e5, 292 at 5e4, 170 at 1e4, with an allocation gate k <= 2 * sqrt(max_cells) binding instead below max_cells ~3906 (the crossover is ~15625 for an unpenalized block, so a wide cat_cat reaches the allocation gate at a max_cells four times larger). Family, link, phi, penalty order, basis kind, prior weights and the edf0 ladder move none of it; only the block’s own width does, and for spline_cat that width is k_spline times the factor’s CONTRAST count — the levels that carry a column, not the declared universe, since a level with no effective training rows is pinned to base and contributes none. (Per-pair times are deliberately not quoted here; the module’s own comments carry the measured ones.)

Which kernel scores a pair follows from that. The dense path gets first refusal and keeps every pair it can score exactly. A spline_cat pair it cannot is RETRIED through the arrow kernel — linear in the level count, with no block-dimension ceiling — rather than refused, at any of three exits. First, the block over the cap above: at the default a library-default Spline() margin crosses it at 105 contrasts, so from L = 106 UNPINNED levels (ps(15) from 77, ps(20) from 61) — a declared levels= universe can be far larger and still route dense, since only the unpinned levels widen the block. Second, the pair’s dense support intermediate over budget where the arrow kernel’s transposed one still fits, which fires with the dense block far BELOW the cap (measured at k = 231: a 22-level factor against a ps(8) margin with 46,119 distinct values). Third, the exhaustion of the binning fallback. Only spline_cat has that retry: ti and cat_cat are gridded and dense-only, so for them the cap IS the refusal. The two numeric kinds are neither — they enter through the z-moment kernels rather than the gridded one, and numeric_numeric contracts to a five-column design whatever the supports, so it never meets the cap at all. Every kind is reduced to a triangular FACTOR of its own weighted design rather than to a curvature matrix (issue #257); the caps above are dimensional and are unchanged by that.

A NaN row is not by itself a routing signal. A spline_cat pair reaches one either because the arrow gates or its ladder refused after a handoff, or because the ladder that ran — on EITHER path — resolved no direction at any rung, which is reported as no test rather than as an infinite z.

A categorical margin never bins — its support is the fitted level set — and a numeric margin never grids at all: it enters its probe linearly, so a numeric_cat/numeric_numeric row that carries a statistic is exact. Those kinds have no binning fallback, so their only degradation is refusal: a numeric_cat pair whose factor is too wide for the pair’s blocks to fit max_cells — or to be solved inside it, which binds first and admits 1710 UNPINNED levels (1709 contrasts) at the default, counted the same way as the spline_cat threshold above — gets a NaN row, and raising max_cells computes it exactly. Screening always probes exact bases, so a pair whose confirmatory refit would discretize LOSSILY is flagged approx=True to make that support-discretization gap — measured at ~3.5% on signal pairs, the same class as the quantile fallback — visible in the output. The flag applies the refit’s own gate, which differs by kind: a ti refit bins its marginals only when BOTH parents resolve to fit-time discretization, a spline_cat refit bins its spline margin whenever that ONE parent does, and the cat_cat/numeric_cat/numeric_numeric refits bin nothing. Lossless binning returns the exact support and stays approx=False. Pairs already fitted as an interaction of any class are excluded from the sweep. The statistic is a ranking device, not a calibrated p-value: confirm the top-ranked pairs by refitting them as the interaction their kind names.