Inference

Term inference

TermInference dataclass

Per-term inference result.

Holds the fitted curve (or levels/slope), uncertainty measures, and metadata for a single model term. Returned by SuperGLM.term_inference().

to_dataframe()

Convert to a tidy DataFrame for plotting or export.

InteractionInference dataclass

Per-interaction inference result (lighter than TermInference).

SplineMetadata dataclass

Knot and basis metadata for a spline term.

REML

REMLResult dataclass

Result of REML smoothing parameter estimation.

Cleanup histories, when present, are accepted post-update snapshots from each outer REML iteration.

Profile estimation

NBProfileResult dataclass

Result of NB theta parameter estimation.

ci(alpha=0.05)

Profile likelihood confidence interval for theta.

Requires that the result was produced by estimate_nb_theta. Results are cached so repeated calls (e.g. from summary()) are free.

profile_plot(*, alpha=0.05, n_points=100, ax=None)

Profile deviance plot for NB2 theta.

Shows the profile deviance curve with the MLE, confidence interval bounds, and chi-squared cutoff. Cheap — each evaluation is O(n) with no refitting.

Parameters:

Name	Type	Description	Default
alpha	float	Significance level for CI (default 0.05).	0.05
n_points	int	Number of grid points for the curve.	100
ax	matplotlib Axes	Axes to plot on. If None, creates a new figure.	None

Returns:

Type	Description
Figure

TweedieProfileResult dataclass

Result of Tweedie power parameter estimation.

Attributes:

Name	Type	Description
p_hat	float	Estimated power parameter.
phi_hat	float	Estimated dispersion at p_hat.
nll	float	Mean negative log-likelihood at (p_hat, phi_hat).
n_evaluations	int	Completed distinct fixed-p records in the immutable search snapshot. Later CI or plotting probes do not change this value.
n_total_evaluations	int	Dynamic count of all completed distinct fixed-p records, including successful post-search CI or plotting probes.
n_post_search_evaluations	int	Number of completed distinct records added after the search snapshot.
converged	bool	Whether the search converged.
method	str	Search method used ("brent", "grid", etc.).
phi_method	str	How phi was profiled ("pearson" or "mle").
search_trace	DataFrame	Per-evaluation record with columns: step, p, phi, nll, n_iter, fit_converged, source.
saddlepoint_fraction	float	Fraction of positive density evaluations that used the saddlepoint approximation at the final (p_hat, phi_hat).
density_method	{'exact', 'hybrid_exact_saddlepoint', 'saddlepoint'} or None	Density evaluation method in the immutable winning record.
density_exact	bool or None	Whether every winning-record density term was evaluated exactly.
density_warning_severity	{'none', 'label', 'warning', 'high'}	Highest density or saddle-qualified near-power diagnostic severity.
near_power_boundary	bool	Whether saddlepoint terms were used at p <= 1.08 or p >= 1.98.
outer_converged	bool	Whether the requested outer search itself converged.
outer_message	str	Diagnostic message returned by the outer optimizer.
outer_boundary	{'lower', 'upper'} or None	Configured search endpoint selected as the winning record, if any.
fit_converged	bool	Whether the winning fixed-p fit converged, including both REML and final solver convergence for fit_mode="fit_reml".
objective_finite	bool	Whether the winning profiled objective is finite and valid.
phi_converged	bool	Whether the winning inner dispersion profile converged.

cache property

Deprecated: use search_trace instead.

Returns a dict mapping p → nll reconstructed from the search trace for backward compatibility.

n_total_evaluations property

Completed distinct fixed-p records, including later CI/plot probes.

n_post_search_evaluations property

Completed distinct fixed-p records added after the search snapshot.

__post_init__()

Derive new density fields for legacy positional construction.

__setstate__(state)

Restore compatibility fields absent from legacy pickle state.

ci(alpha=0.05)

Profile likelihood confidence interval for Tweedie p.

Requires that the result was produced by estimate_tweedie_p. Results are cached so repeated calls (e.g. from summary()) are free. The interval targets the nearest detected connected LR component. Its finite max-gap scan can miss a narrower unsampled LR island.

ci_details(alpha=0.05)

Return immutable endpoint status and evaluation evidence for ci.

trace_plot(*, ax=None)

Plot cached Tweedie p-search evaluations without fitting new models.

profile_plot(*, alpha=0.05, n_points=50, ax=None)

Profile-objective plot for Tweedie power parameter p.

Evaluates the profile objective on a dense grid for the curve, and overlays the search evaluation points from search_trace. MLE profiles use likelihood-ratio wording; Pearson profiles use neutral objective and interval wording.

Parameters:

Name	Type	Description	Default
alpha	float	Significance level for CI (default 0.05).	0.05
n_points	int	Number of grid points for the smooth curve.	50
ax	matplotlib Axes	Axes to plot on. If None, creates a new figure.	None

Returns:

Type	Description
Figure

estimate_nb_theta(model, X, y, sample_weight=None, offset=None, *, theta_bounds=(0.1, 50.0), xatol=0.01, maxiter=30, verbose=False, trace_callback=None)

Estimate NB2 theta via alternating GLM fit + Newton (MASS::glm.nb).

Algorithm: 1. Build design matrix once, calibrate lambda. 2. Alternate: fit GLM at current theta (PIRLS with warm starts) → update theta via Newton on the profile score (MASS::theta.ml). 3. Converge when |theta_new - theta_old| < xatol (~3-5 iterations).

Parameters:

Name	Type	Description	Default
model	SuperGLM	A configured but unfitted model with features already added. Must have a NegativeBinomial family (e.g. families.nb2(theta=1.0)).	required
X	pandas or eager Polars DataFrame	Feature matrix.	required
y	array - like	Response variable (counts).	required
sample_weight	array - like	Frequency weights (sample_weight). Must be frequency weights, not variance weights — theta estimation assumes each observation's log-likelihood contribution is scaled by its sample_weight.	None
offset	array - like	Offset added to the linear predictor.	None
theta_bounds	tuple	Bounds for theta, default (0.1, 50.0).	(0.1, 50.0)
xatol	float	Convergence tolerance on theta (absolute).	0.01
maxiter	int	Maximum outer iterations (GLM fits).	30
verbose	bool	Print progress.	False

Returns:

Type	Description
NBProfileResult

estimate_tweedie_p(model, X, y, sample_weight=None, offset=None, *, p_bounds=(1.05, 1.95), xatol=0.001, maxiter=30, verbose=False, fit_mode='fit', phi_method='mle', method='auto', n_grid=20, grid=None, n_grid_coarse=10, optimizer='L-BFGS-B', trace_callback=None, trace_iterations=False)

Estimate the Tweedie power parameter via profile likelihood.

Builds the design matrix once and searches over candidate p values, fitting the GLM at each candidate with warm starts.

Parameters:

Name	Type	Description	Default
model	SuperGLM	A configured but unfitted model with features already added. Must have a Tweedie family (e.g. families.tweedie(p=1.5)).	required
X	pandas or eager Polars DataFrame	Feature matrix.	required
y	array - like	Response variable.	required
sample_weight	array - like	Finite, strictly positive EDM prior weights, not replication or frequency weights. The Tweedie variance convention is Var(Y_i | x_i) = phi * mu_i**p / w_i. Remove zero-weight rows consistently from X, y, sample_weight, and offset before calling this function.	None
offset	array - like	Offset added to the linear predictor.	None
p_bounds	tuple	Bounds for p search, default (1.05, 1.95).	(1.05, 1.95)
xatol	float	Tolerance for scalar optimisers (Brent).	0.001
maxiter	int	Maximum iterations for the optimiser.	30
verbose	bool	Print progress.	False
fit_mode	('fit', 'fit_reml')	Fitting regime for each candidate p evaluation.	"fit"
phi_method	('pearson', 'mle')	How to profile out phi at each candidate p. "mle" is the default; "pearson" is an explicit faster plug-in that does not support likelihood-ratio confidence intervals.	"pearson"
method	('auto', 'joint_ml', 'brent', 'grid', 'grid_refine', 'profile_opt', 'integrated')	Search strategy. "auto" (default) uses the safeguarded exact joint ML solver for ordinary MLE profiles and Brent otherwise. "joint_ml" explicitly requests that fast path with diagnosed Brent fallback, including when configured bounds extend outside [1.05, 1.95]. "brent" uses bounded scalar optimisation. "grid" does exhaustive grid search. "grid_refine" does a coarse grid + local Brent refinement. "profile_opt" uses a general-purpose optimizer (L-BFGS-B or Powell) on logit-transformed p.	"auto"
n_grid	int	Number of grid points for method="grid" (default 20).	20
grid	array - like	Explicit p grid for method="grid". Overrides n_grid.	None
n_grid_coarse	int	Number of coarse grid points for method="grid_refine" (default 10).	10
optimizer	str	Optimizer backend for method="profile_opt". One of "L-BFGS-B" (default) or "Powell".	'L-BFGS-B'
trace_iterations	bool	If True, include the nested fit learning curve for each candidate p evaluation in search_trace["fit_trace"].	False

Returns:

Type	Description
TweedieProfileResult

estimate_phi(y, mu, p, *, weights=None, df_resid=None)

Weighted Pearson estimate of dispersion parameter phi.

phi_hat = sum(w * (y - mu)^2 / mu^p) / denom

Under the prior-weight convention used here, Var(Y_i) = phi * mu_i^p / w_i. Therefore E[w_i * (Y_i - mu_i)^2 / mu_i^p] = phi for each observation, and the natural denominator is the residual observation count rather than the sum of weights.

where denom = df_resid if provided, else n_obs (i.e. no df correction).

For the prior-weight convention used by sample_weight in SuperGLM, callers should pass the residual observation count df_resid = n_obs - edf.