Regenerating Goldens

Audience: Pulse internals contributors after a legitimate change to a descriptor / manifest generator.

Golden files live in descriptor/testdata/. Each file ends with a // golden-hash: <sha256> line; TestGoldensNotHandEdited verifies the hash against the file’s body. Hand-editing a golden file flips the hash and fails the gate.

When regeneration is justified

You should regenerate after:

• Adding or removing a registered operator (the manifest enumerates every entry).
• Changing the capability shape for an existing operator.
• Renaming an error code (the manifest carries the canonical list).
• Updating the format_version (rare — additive changes do not bump the version).
• Any other change that legitimately changes the deterministic manifest output.

If you cannot articulate which deterministic output changed, the golden update is probably wrong and the underlying generator is emitting non-deterministic content (e.g. iterating a map without sorting). Fix that first.

The regenerate flow

go test ./descriptor/ -run 'Test.*Golden' -update

The -update flag asks the test runner to rewrite the goldens with the current generator output. The generator stamps a fresh // golden-hash: <sha256> line at the end of every regenerated file.

Then verify the gate accepts the new hashes:

go test ./descriptor/ -run TestGoldensNotHandEdited

If the gate still fails after -update, the cause is usually one of:

• The hash trailer is missing from a golden you added by hand — the generator only stamps files it owns.
• An extra blank line or comment was added by editor configuration before the final hash line — the hash is computed over the body before the trailer.
• The generator emits map iteration without a sort — non-determinism itself.

Per-feature-profile goldens

Beside the full-registry manifest.json and payload-schema.json, each fixture feature profile under descriptor/testdata/profiles/ (minimal, survey-crosstab, empty) pins its instance’s self-description: manifest.<fixture>.json (the --json envelope around Pulse.Manifest) and payload-schema.<fixture>.json (Pulse.PayloadSchema byte for byte), written by TestProfileManifestGolden and TestProfilePayloadSchemaGolden. Both match Test.*Golden, so the regenerate command above rewrites them too, and they sit at the top of testdata/ where TestGoldensNotHandEdited checks their hashes. The profile inputs themselves live in the profiles/ subdirectory, which the gate skips because they are hand-written.

Regenerate them whenever a change moves what an instance offers or how it describes it: a feature added to or removed from the feature table, a change to a fixture profile, a capability block or schema root that gating omits, or prose the hidden-name scrub redacts. The manifest goldens also carry the skill list, so a skill-pack edit churns them like manifest.json.

The generated Analysis Guide

The book’s “Analysis Guide” part — every page under docs/src/guide/ plus the docs/src/SUMMARY.md entries between the <!-- docgen:summary begin --> and <!-- docgen:summary end --> lines — is generated by pulse docs export from the default instance: the Purpose and Interpretation registries, the glossary and intents, every operator’s ComponentSchema, the embedded skills, and the Components intro in internal/docgen/components_intro.md. It is committed so the published book needs no Go toolchain, and TestDocsGeneratedCurrent (root package) fails the moment a source edit lands without it.

Regenerate after any edit to those sources:

make docs

make docs runs make docs-generate — go run ./cmd/pulse docs export --out docs/src/guide with PULSE_LABEL_TABLES_DIR, PULSE_RANGE_TABLES_DIR, PULSE_TEMPLATES_DIR and PULSE_FEATURE_PROFILE cleared (so a local .env cannot change the output), then go run ./internal/tools/docsummary, which splices the export’s SUMMARY.md into the book’s (title line dropped, paths prefixed guide/) — and then mdbook build docs. On a current tree it changes nothing. go test . -run TestDocsGeneratedCurrent -update rewrites the same files without building the book.

Commit everything the export wrote, the .pulse-docs-export marker included: it lists the files the export owns, so the next export deletes a page that no longer renders. Never hand-edit docs/src/guide/ — the next make docs overwrites it and the gate fails until it does. Edit the source and regenerate.

Never hand-edit a golden

The gate exists to catch the case where a contributor edits a golden to make a test pass instead of fixing the underlying drift in the generator. If a golden diff surprises you in code review, that is a red flag — ask the contributor to show the generator change that justifies the new hash.

The 0x01 format golden is a compatibility fixture, not a snapshot

encoding/testdata/format_v1.pulse is a synthetic 0x01 cohort that every build must still read — and that the current writer must still reproduce byte-for-byte for the same schema and rows. It exists because old cohorts stay readable forever. A failure there is a backward-compatibility break, not a stale golden: do not run go test ./internal/encoding/ -run TestFormatV1Golden -update to make it pass. The -update flag exists only to create the file; to cover more of the 0x01 surface, add a second fixture rather than rewriting this one, so the bytes an earlier binary wrote stay under test.

Shard-archive fixtures are generator-pinned snapshots

testdata/sharding/*.pulse are produced by internal/shardfixtures (go run ./testdata/sharding/build_fixtures.go writes them) and TestShardFixtures_MatchGenerator fails when the committed bytes and the generator disagree. Unlike the 0x01 golden these are snapshots of the CURRENT writer: they once went stale across the nullable-flag schema-block change (a declared clean break) and stopped opening, with nothing noticing. Regenerate with the command above or go test ./internal/shardfixtures/ -update, then git add -f them (*.pulse is gitignored). The two JSON envelopes beside them are pinned to live CLI output by TestShardingSnapshots_MatchCLIOutput (go test ./internal/cli/ -run TestShardingSnapshots -update).

R reference goldens are an external oracle, not a snapshot

internal/processing/testdata/reference/*.json hold R’s values for the shared distribution primitives (Student t, chi-square, the noncentral chi-square CDF, F, normal CDF and quantile, the limiting Kolmogorov survival, the studentized range). They are written by scripts/reference/gen_reference.R and carry the R and package versions that produced them; make reference runs the script (needs Rscript plus the jsonlite package) and appends the // golden-hash: footer that the processing package’s TestGoldensNotHandEdited checks. CI never runs R.

The TestReferenceOracle_* tests in internal/processing and internal/statdist (the Student-t family’s one implementation) hold every primitive to a relative 1e-10 of those values. A failure there means the Go primitive is wrong, not that the golden is stale: fix the primitive, never regenerate to match it. Regenerate only to change the grid (edit the script, run make reference, commit script and goldens together). The studentized-range and Student-t quantile references are refined in R beyond the stock ptukey() / qtukey() / qt() because those are less accurate than the oracle needs; the script’s header explains why.

Multivariate goldens and their fixtures

The same make reference run also sources scripts/reference/gen_multivariate.R, which writes the matrix-operator oracle as mv_*.json beside the primitives: rank correlation (Spearman, Kendall τ-b), partial correlation (ppcor::pcor, corpcor::cor2pcor), reliability (psych::alpha, ω from psych::fa(nfactors = 1, fm = "minres")), the one-factor minres solve, PCA (eigen / prcomp) with psych::KMO and psych::cortest.bartlett, collinearity (car::vif, perturb::colldiag uncentered-with-intercept and centered), vcov(lm) / vcov(glm) (binomial, poisson) and Matrix::nearPD(corr = TRUE). Each case is unweighted and, where the method admits it, frequency- and probability-weighted under Pulse’s one weighting rule (stats::cov.wt, frequency asserted equal to the rep()-expanded rows). Eigenvectors carry the linalg sign convention.

Extra packages, PINNED — the script refuses any other version, so a CRAN update cannot move a golden silently: psych, car, ppcor, perturb, corpcor and Matrix (exact versions and the install line are in the script header).

Fixtures live in internal/processing/testdata/reference/fixtures/ as a CSV (citation header, %.17g cells, empty = null) plus a .pulse twin that make reference writes with go run ./internal/tools/refcohorts: the vendored attitude (Chatterjee & Price 1977) and mtcars (Henderson & Velleman 1981) from R’s datasets package, each with two seeded weight columns appended, and seeded synthetic fixtures likert_ties, nulls (listwise vs pairwise), weighted (zero-weight rows), nonpsd_pairwise, heywood, reversed_items and two_factor (a proper two-factor fit under every weight kind, so weighted Bartlett factor scores are pinned on real values). No GPL-package data. mv_fixtures.json records each fixture’s source, citation, seed and CSV md5; the internal/tools/refcohorts tests fail when a CSV drifts from that record or a .pulse drifts from its CSV.

The R toolchain

make reference refuses any R but 4.6.1 (2026-06-24) and any package version but the pinned one, so every package below must be installed at exactly that version. Every script header repeats its own pins; this is the whole list in one place.

ICSNP, MVTests and MVN do NOT build on R 4.6.1; nothing needs them. The scripts compute what they would (the unequal-covariance Hotelling test, Mardia’s statistics) as closed forms in base R, each cross-checked as its comment says. The weighted-reference generator (internal/service/testdata/weight_reference/) needs only base R plus Hmisc 5.3.0, installed into a private library; its README covers it, along with its Python pins and the --check mode.

Install the pins into the user library (remotes::install_version fetches the exact version even after CRAN moves on):

Rscript -e 'r <- "https://cloud.r-project.org"; l <- Sys.getenv("R_LIBS_USER");
  dir.create(l, recursive = TRUE, showWarnings = FALSE)
  install.packages("remotes", lib = l, repos = r)
  pins <- c(psych = "2.6.9", car = "3.1-5", ppcor = "1.1", corpcor = "1.6.10",
            gdata = "3.0.1", Hotelling = "1.0-8", biotools = "4.3",
            GPArotation = "2026.8-2", sandwich = "3.1-3", lmtest = "0.9-40")
  for (p in names(pins)) remotes::install_version(p, pins[[p]], lib = l, repos = r, upgrade = "never")
  install.packages(paste0(r, "/src/contrib/Archive/perturb/perturb_2.10.tar.gz"),
                   lib = l, repos = NULL, type = "source")'

Homebrew Fortran workaround. Packages with Fortran code (Hmisc, and any dependency that compiles Fortran) fail to link on Homebrew R 4.6.1 with library 'emutls_w' not found: R’s recorded FLIBS names a gcc target directory that does not exist (R CMD config FLIBS reports aarch64-apple-darwin27; Homebrew gcc 16 installs aarch64-apple-darwin23). Write a TEMPORARY Makevars naming the real directory and point R_MAKEVARS_USER at it for the install only — never commit it, and keep it out of ~/.R/Makevars:

GCC=$(brew --prefix gcc)/lib/gcc/current
TRIPLE=$(ls "$GCC/gcc"); VER=$(ls "$GCC/gcc/$TRIPLE")
printf 'FLIBS = -L%s/gcc/%s/%s -L%s/gcc -L%s -lemutls_w -lheapt_w -lgfortran -lquadmath\n' \
  "$GCC" "$TRIPLE" "$VER" "$GCC" "$GCC" > /tmp/Makevars.pulse
R_MAKEVARS_USER=/tmp/Makevars.pulse Rscript -e '...the install line...'

A rerun of make reference on this toolchain is byte-identical (no clock, no unseeded RNG): git status must show only the goldens your change meant to move.