Regression Modeling

Pulse exposes regression through a compact, composable surface. Three operators, two orthogonal modifiers, and one upstream feature transform together cover every textbook regression variant. This chapter is the human-facing counterpart to skills/regression-modeling.md; agents should fetch the skill via pulse_skills_get rather than read this page.

Overview

Two spec-level modifiers compose with any of the three:

• Resample ∈ {jackknife, bootstrap} — replaces analytical SE / p-values with resample-based estimates. Forces buffered.
• Selection ∈ {forward, backward, stepwise} — drives AIC- or BIC-based greedy subset search. Requires Criterion. Forces buffered.

One upstream feature operator (FEAT_POLY) extends the linear core to polynomial regression. Per-row attributes (ATTR_REG_FITTED, ATTR_REG_RESIDUAL, ATTR_REG_LEVERAGE) attach per-record diagnostics in the output row stream.

The 13 textbook names → Pulse specs

The Indeed regression taxonomy double-counts (Simple ≡ Linear univariate, Multiple ≡ Multiple Linear) and treats orthogonal wrappers (Jackknife, Stepwise) as families. Pulse does not. The table below maps each textbook name onto the corresponding Pulse spec and links to a runnable example file under examples/regression/.

Streamability matrix

pulse_predict reports per-request streamability on PredictResult.Streamable, mirroring the runtime gate.

Operator reference

REG_OLS

Ordinary least squares with optional regularization.

Modifier compatibility: Resample and Selection may be combined; Selection runs first, Resample re-fits the selected subset.

Error codes: PROCESSING_REGRESSION_RANK_DEFICIENT, PROCESSING_REGRESSION_SINGULAR_GRAM, PROCESSING_REGRESSION_NO_CONVERGE, PROCESSING_REGRESSION_INSUFFICIENT_DATA, PROCESSING_REGRESSION_APPROXIMATE_SE (warning, l1/elasticnet without resample), PROCESSING_REGRESSION_REGULARIZED_SELECTION (warning, penalty + selection), PROCESSING_CONFIG.

REG_GLM

Generalized linear model via iteratively-reweighted least squares.

Always buffered. Setting penalty / alpha / l1_ratio on a REG_GLM spec is rejected with PROCESSING_CONFIG; regularized GLM is reserved for a later phase.

Error codes: PROCESSING_REGRESSION_INVALID_FAMILY, PROCESSING_REGRESSION_INVALID_LINK, PROCESSING_REGRESSION_NO_CONVERGE, PROCESSING_REGRESSION_INSUFFICIENT_DATA, PROCESSING_CONFIG.

REG_BAYES_LINEAR

Bayesian linear regression with a conjugate Normal-Inverse-Gamma prior.

Modifier compatibility: Resample and Selection are rejected for REG_BAYES_LINEAR at spec validation — the posterior already conveys uncertainty via credible intervals, and stepwise feature selection on a Bayesian model is a posterior-based question the conjugate-NIG engine doesn’t support.

Setting penalty / alpha / l1_ratio / family / link on a Bayes spec is rejected with PROCESSING_CONFIG.

Error codes: PROCESSING_REGRESSION_RANK_DEFICIENT, PROCESSING_REGRESSION_INSUFFICIENT_DATA, PROCESSING_CONFIG.

Modifiers

Resample

Layered on top of any base operator (except REG_BAYES_LINEAR).

For l1 / elasticnet OLS, setting Resample is the rigorous answer for standard errors: it suppresses the PROCESSING_REGRESSION_APPROXIMATE_SE warning (the SEs are now resample-based, not plug-in over the active set).

Selection

Layered on top of any base operator (except REG_BAYES_LINEAR).

Requires Criterion ∈ {"aic", "bic"}.

• AIC = -2·logL + 2·k. Lighter penalty; may retain weak predictors at moderate n.
• BIC = -2·logL + log(n)·k. Heavier per-parameter penalty; rejects noise predictors more reliably at moderate n.

SelectedFeatures lists the chosen subset; Coefficients drops non-selected predictors entirely (absence ≠ zero — selection’s contract is stronger). Selection may be combined with Resample: Selection picks the active subset, then Resample replaces SE / p-values on the selected model.

Compositional patterns

Polynomial regression — FEAT_POLY + REG_OLS

Polynomial regression is linear in the coefficients; the non-linearity lives in the feature space. Use FEAT_POLY upstream to materialize x_2, x_3, …, x_<degree> derived columns, then list them alongside the original x in predictors:

{
  "features": [
    {"type": "FEAT_POLY", "field": "x", "label": "x", "params": {"degree": 3}}
  ],
  "regressions": [
    {"type": "REG_OLS", "name": "polyfit", "target": "y",
     "predictors": ["x", "x_2", "x_3"]}
  ]
}

Degree is gated at [2, 10]. Numerical stability is the caller’s responsibility: x^10 overflows f64 once |x| clears a few hundred, and the Gram matrix conditions poorly long before that. Centre or standardize predictors before requesting FEAT_POLY.

Ecological regression — group → regress

“Ecological regression” is a regression fit on aggregated group-level statistics — per-precinct means, per-county sums, per-region rates — rather than individual-level rows. Use pulse_compose with two slots: slot 1 produces per-group means via GROUP_* + AGG_AVERAGE, slot 2 fits REG_OLS over the aggregate output (or, in practice, over a pre-aggregated .pulse file).

The two slots are intentionally independent; Pulse does not pipe slot-1 results into slot-2 as cohort input. Either (a) materialize slot 1’s aggregate as its own .pulse cohort upstream, or (b) treat slot 1 as the audit trail (per-group means visible in the composed response) and run slot 2 over a pre-aggregated fixture.

Caution — the ecological fallacy. A significant group-level slope does not imply an individual-level association. Robinson (1950) showed that ecological correlations and individual correlations can take opposite signs in the same data: a per-state regression of literacy on race might suggest a strong relationship that vanishes (or reverses) at the per-person level. Aggregation collapses within-group variation, leaving only between-group structure that frequently encodes confounders.

When ecological regression is the right tool: aggregate-only data (census output, public-health summary tables); genuinely group-level research questions (“do counties with higher median income have higher turnout?”). When it is the wrong tool: individual-level claims; causal claims. Annotate consumer-facing prose with this caveat; Pulse cannot enforce it.

Robinson, W.S. (1950). “Ecological Correlations and the Behavior of Individuals.” American Sociological Review 15(3): 351–357.

Per-row regression attributes

Three attribute operators emit per-record diagnostics from a fitted regression onto the row stream.

Each attribute references a sibling regression spec by regression_name. See skills/attribute-composition.md for the parameter table.

Error codes

Look up full prose via pulse_errors_lookup or pulse errors lookup CODE.

Worked examples

Every Indeed name has a runnable JSON file under examples/regression/. Fetch via pulse_examples_get or read directly:

• 01_ecological_fallacy.json — per-region aggregation + ecological caveat (#12).
• 02_simple_linear.json — univariate OLS (#1, #3).
• 03_multiple_linear.json — multivariate OLS (#2, #4).
• 04_logistic.json — binary classification (#5).
• 05_ridge.json — l2 penalty (#6).
• 06_lasso.json — l1 penalty (#7).
• 07_polynomial.json — FEAT_POLY + OLS (#8).
• 08_bayesian_linear.json — conjugate NIG (#9).
• 09_jackknife.json — leave-one-out resampling (#10).
• 10_elasticnet.json — combined l1 / l2 penalty (#11).
• 11_stepwise.json — BIC-driven stepwise selection (#13).