Field Types

Audience: anyone designing a cohort schema, decoding a .pulse file by hand, or trying to understand which type to pick for a column.

Every field type has a fixed type byte, a fixed (or bit-packed) byte size, and well-defined semantics. The registry is encoding/field_type.go — the FieldType const block gives the byte values (iota order), ByteSize() the widths and String() the names below. For the live list at runtime, read cohort_types out of pulse manifest --json rather than counting rows in this page.

LLM agents using MCP: see the cohort-schema-design skill via pulse_skills_get for the same matrix plus “which type to pick”, and type-<name> (e.g. type-set-u256) for one type in depth.

Nullability is orthogonal to type

There are no nullable_* types. Any field of any type may be marked nullable (encoding.Field.Nullable), which sets the schema’s per-field nullable flag byte and enrols the field in the per-record null bitmap described in Record Layout. Nullability never changes a field’s type byte, its width, or the operator the engine infers for it.

The catalog

Type bytes are the FieldType iota order. Note that it is not grouped by family — datetime (17) and the two wide set rungs (18, 19) were appended after the original block, and appending is the only way to add a type without invalidating every existing file.

Type families

Plain integers and floats

u8, u16, u32, u64, f32, f64. Standard little-endian encoding, full range, no in-band null sentinel. Mark the field nullable if the column carries missing values.

Bit-packed types

u4 (a 4-bit unsigned) and packed_bool (one bit) both report ByteSize() == 0 and share bytes with adjacent packed fields; the schema’s BitPosition locates them. ByteSize() == 0 is the reader’s signal that a field shares bytes — a non-zero ByteSize field never does. See Record Layout → Bit-packing.

Temporal types

date is a signed (two’s-complement int32) count of days since the Unix epoch. datetime is a signed (two’s-complement int64) count of seconds since the same epoch. Pre-1970 values are negative and supported end to end; encoding.DateDays / encoding.DateTimeSeconds interpret a raw ReadFieldValue word.

They are never interchangeable: swapping them rescales every value by 86,400. Everything downstream of the operator boundary speaks epoch days only, so a datetime column is day-truncated exactly once there (encoding.DateTimeToDay, toward the past, naive UTC).

Categoricals

categorical_u8, categorical_u16, categorical_u32 store a single dictionary ID per row and carry the string-to-ID mapping inline as a dictionary block immediately after the field’s schema entry. Pick the smallest variant that fits your cardinality (import inference does this for you). See Dictionary Blocks.

Sets (multi-select bitmasks)

set_u8, set_u16, set_u32, set_u64, set_u128, set_u256 carry the same inline dictionary block as a categorical, but the on-wire payload is a fixed-width bitmask: bit i set means dictionary entry i is selected. One row can select any number of labels, which is what makes these the landing type for an SPSS multiple-dichotomy battery.

• Inference picks the smallest rung that fits the option count; a column with more than 256 options has no set type at all and stays as its constituent columns.
• An empty mask is a valid value meaning “answered, selected nothing” — it is NOT null, and the two must stay distinct because “ticked none of these” and “skipped the question” give different denominators.
• The wide rungs (set_u128, set_u256) exceed the uint64 value API: ReadFieldValue / WriteFieldValue refuse them with ENCODING_TYPE_MISMATCH rather than truncating to the low 64 bits (FieldType.IsWideSet()). Wide masks travel as encoding.SetMask.

Decimal128

decimal128 is a 16-byte fixed-point decimal. Each field carries a per-field (precision, scale) pair written into the schema after the description; both top out at 38 (PULSE_DECIMAL_OVERFLOW, PULSE_DECIMAL_PRECISION_LOSS).

Use it for currency and any other column where IEEE-754 rounding is not acceptable. See the financial-cohorts skill for full semantics including banker’s rounding and divide-by-zero policy. decimal128 nulls ride the bitmap only — there is no in-band sentinel.

Unknown type bytes

The schema reader rejects unknown FieldType bytes at parse time with ENCODING_INVALID. This is the same fail-loud strategy as the header version check: a file written by a future binary that introduced a new type fails immediately at schema parse, not later during row decode where the corruption could go unnoticed.

What you can do with each type