Repository Go Modules

Nexus is one Go module — github.com/frankbardon/nexus — plus a small number of submodules under modules/, each with its own go.mod.

This page is the contract for that arrangement: where a submodule may live, how it resolves the core module, how the build and CI cover it, and how it is versioned. If you are writing an object-store backend, an exporter or anything else that needs a dependency Nexus core must not carry, start here.

Why submodules exist at all

The root module deliberately carries no vendor SDKs. Every LLM provider is raw net/http, storage is pure-Go modernc.org/sqlite, and the direct dependency count is a number the project defends.

A cloud object-store backend cannot honour that. The AWS and Google SDKs are large, transitively deep, and pull in their own HTTP, auth and retry machinery. Adding one to go.mod would impose it on every nexus build, including builds that will never touch a bucket.

So the seam is designed to be implemented from outside: pkg/engine/objectstore imports nothing beyond the standard library, and its conformance suite pkg/engine/objectstore/objectstoretest is exported. A backend lives in its own module with its own dependency graph, and an embedder who wants it blank-imports it into their own main. The root module’s dependency list does not move.

The one root-module exception: golang.org/x/oauth2/google

The rule above is a rule, not an absolute, and it has exactly one recorded exception. Read this before citing it as precedent — the grounds are narrow and most cloud dependencies do not meet them.

The Gemini provider’s Vertex AI mode has to obtain an access token. Under GKE Workload Identity there is no key file to read: the token comes from Google Application Default Credentials, which fall back to the instance metadata server. So plugins/providers/gemini depends on golang.org/x/oauth2/google, and that dependency is in the root go.mod.

It has to be in the root module, because the value of the feature is that the shipped bin/nexus authenticates on a keyless pod. Putting the credential source in modules/ would mean every operator running Nexus on GKE has to write and maintain their own main package — which is precisely the cost a submodule exists to avoid imposing on people who do not need the dependency, and precisely the wrong trade when the people who need it are the default deployment.

The grounds are size, measured rather than assumed. Two libraries can serve ADC. Built into a throwaway module and resolved with go mod tidy:

The 35-module option transitively pulls google.golang.org/grpc, google.golang.org/protobuf, three google.golang.org/genproto/* modules, five OpenTelemetry modules, s2a-go, go-pkcs11, enterprise-certificate-proxy, google.golang.org/api and gax-go — most of a gRPC stack, into a binary that speaks only net/http. It was rejected on that basis alone; it buys nothing this feature needs (its extras are the enterprise certificate proxy and S2A, neither of which applies to Vertex over HTTPS).

golang.org/x/oauth2/google resolves to golang.org/x/oauth2 and cloud.google.com/go/compute/metadata, and in this repository both were already in the build list before the change — x/oauth2 as an indirect requirement, compute/metadata reachable through gRPC via the OTLP trace exporter. Adopting it promoted golang.org/x/oauth2 from the indirect block to the direct one and added compute/metadata as a root; go list -m all is byte-for-byte identical either side of that commit. Net new modules in the build list: zero.

Two further points, so the precedent is not read wider than it is:

• It is a credential library, not a provider SDK. The “no vendor SDKs” rule is about the API surface: every Vertex call Nexus makes is still hand-written net/http against a URL it builds itself. oauth2/google mints a bearer token and does nothing else. cloud.google.com/go/aiplatform remains out of the question.
• The seam it feeds is still submodule-shaped. Outbound credential sources are registered by name, the way object-store backends are, so a heavier one — Azure, Vault, an in-house STS — belongs in modules/ and gets blank-imported by an embedder. Only the ADC source ships in the root module, and only because it is two modules wide and the default deployment needs it.

The bar for the next exception is that whole list: root-module residence is required for the stock binary to work at all, the dependency is credentials or similar plumbing rather than a service SDK, and the measured build-list cost is in single digits. Anything that fails one of those three goes under modules/.

Layout

nexus/
  go.mod                      # the root module: github.com/frankbardon/nexus
  pkg/ plugins/ cmd/ ...      # all of it root-module code, no exceptions
  modules/
    objectstore-s3/           # github.com/frankbardon/nexus/modules/objectstore-s3
      go.mod
    objectstore-gcs/          # github.com/frankbardon/nexus/modules/objectstore-gcs
      go.mod
    objectstore-seamcheck/    # github.com/frankbardon/nexus/modules/objectstore-seamcheck
      go.mod

That is every submodule that ships today, and the list is short enough to be worth naming rather than gesturing at:

make submodules prints the same list from the filesystem, and is the authoritative answer if this table ever falls behind.

Rules:

• Every non-root go.mod lives at modules/<name>/go.mod. One level, no nesting. make check-modules (run as part of make lint) fails the build if a go.mod appears anywhere else.
• Module path mirrors the directory: modules/<name> becomes github.com/frankbardon/nexus/modules/<name>. This is not a style preference — Go derives a submodule’s version tag from its directory, so path and directory cannot diverge.
• Names are flat and hyphenated, describing the seam then the implementation: objectstore-s3, objectstore-gcs. Grouping them as modules/objectstore/s3 was rejected because it makes modules/objectstore look like a module it is not, and lengthens every version tag for no gain.

Placing a backend beside the interface it implements — pkg/engine/objectstore/s3/ — reads better and was rejected anyway. pkg/ means “root-module code” with no exceptions worth remembering, and a nested go.mod under a directory everything else sweeps is exactly the invisible-package trap described below.

Why there is no go.work

A workspace is the obvious way to make a multi-module repo build as one, and it is deliberately not used here. go.work is in .gitignore.

A workspace merges every listed module into a single build list. The cloud SDKs required by the backend modules would then take part in version selection for the root module as well, and bin/nexus built by a contributor with a workspace active could resolve different transitive versions than the binary CI and the release build produce. Guaranteeing that the root module’s dependency graph is exactly what its go.mod says is the entire reason the backends were split out; a committed go.work would quietly give that back.

A workspace also does not buy what people assume. go test ./... does not span workspace modules — ./... still stops at the current module — so a go.work would not have removed the need for the Makefile to walk modules/.

Instead, each submodule carries a replace:

require github.com/frankbardon/nexus v0.18.2

replace github.com/frankbardon/nexus => ../..

The replace is scoped to that one module and affects nothing else in the tree. Local development and CI therefore always build the submodule against the working tree, which is what makes a submodule capable of failing when a change to the seam breaks it. Anyone who depends on the submodule from outside this repository ignores the replace — Go honours replace only in the main module — and gets the required version, which is why that line must always name a real published tag rather than a v0.0.0 placeholder.

go.work and go.work.sum are gitignored rather than merely absent, so a local workspace for editor or debugging convenience is fine. Just never commit one.

Build, test and lint coverage

The failure mode this plumbing exists to prevent: a separate module is invisible to every ./... pattern. go build ./..., go test ./..., go vet ./... and staticcheck ./... all stop at a nested go.mod without a word. A submodule that does not compile, or whose tests fail, reports as success forever.

The Makefile therefore discovers submodules by glob and sweeps each one:

GO_SUBMODULES := $(patsubst %/go.mod,%,$(wildcard modules/*/go.mod))

LINT_TAGS defaults to integration,evalrecord,minio,fakegcsserver — every build tag in the tree that gates Go files. The second pass exists because a file behind //go:build minio compiled only when someone ran that suite, so neither vet nor staticcheck had ever seen it on any commit, CI included. One combined list is applied everywhere rather than a per-module list: a tag matching no file in a module is a no-op, so minio costs nothing in the GCS module.

It does not cover wasip1, which is a GOOS constraint rather than a build tag — -tags will not reach plugins/tools/codeexec’s wasm files. make verify-yaegi-wasm is what exercises those.

Three deliberate exceptions:

• The emulator targets are not sweeps. Each runs one submodule’s build-tagged suite against an emulator the target starts and stops itself: test-objectstore-minio against MinIO via scripts/with-minio.sh, and test-objectstore-fake-gcs against fake-gcs-server via scripts/with-fake-gcs.sh. Two targets rather than one shared “emulator” target, because MinIO emulates S3 and fake-gcs-server emulates GCS — folding them together would mean one red step for two unrelated stores, and neither suite could be run on its own while working on its own backend. Everything above stays untagged, which is what keeps make test offline and secret-free even though it sweeps modules/.

  The two scripts are the same shape on purpose, end to end: a pinned emulator version built with go install <module>@<version> into a scratch GOBIN using the toolchain the repository already requires, a readiness wait, a port nothing else can be holding, an EXIT trap, and a NEXUS_TEST_*_REQUIRED variable that turns the suite’s no-emulator skip into a failure so a provisioned run cannot pass by skipping. Neither emulator suite needs a container runtime.

  MinIO used to come from its official container image. It is built from source because every official MinIO image and binary channel stopped serving anonymous machines, and chasing another registry would mean chasing the thing that had already failed twice. The pin is RELEASE.2025-09-07T16-13-09Z, which NEXUS_TEST_MINIO_VERSION overrides. MinIO is a large program, so the first run takes about a minute on a cold cache, and the module and build caches make later runs a few seconds. To skip the build, point NEXUS_TEST_MINIO_ENDPOINT at a MinIO you already run, and set NEXUS_TEST_MINIO_ACCESS_KEY / NEXUS_TEST_MINIO_SECRET_KEY if its credentials differ from the defaults. Each script records the reasoning and the alternatives that were rejected. with-minio.sh also records the channel history.

  Both suites are also where the kill-and-resume cycle is proven against a real store, which is why both modules/objectstore-s3/go.mod and modules/objectstore-gcs/go.mod carry indirect requirements — modernc.org/sqlite, gopkg.in/yaml.v3, klauspost/compress — that have nothing to do with either cloud. They come from the root module’s pkg/engine, which those tests import so they can drive a real engine against a real bucket; no non-test file in either module imports anything above pkg/engine/objectstore. The direction that matters is unchanged: the root module still does not require either one.

  The cycle itself is written once, in pkg/engine/objectstore/enginetest, and each module supplies only the four store-specific hooks it needs — register a factory, make an empty bucket, list the bucket, read one object. It is exported from the root module for the reason objectstoretest is: a backend may live in a module this repository never sees, and passing the interface conformance suite does not prove a session resumes from it. It is a separate package from objectstoretest because it imports pkg/engine, and pkg/engine’s own tests are package engine and import objectstoretest — so the two halves have to sit in different packages or neither builds.

• check-events stays root-only. scripts/check-event-versions.sh cds to the repository top level and inspects pkg/events/ alone. Event structs live in the root module and nowhere else, so running it per submodule would repeat the identical check while looking like it were checking something else.

• go run honnef.co/go/tools/cmd/staticcheck@$(STATICCHECK_VERSION) resolves independently of the current module, so the same pinned staticcheck runs inside a submodule without that module having to require it.

CI needs no submodule-specific job for the sweeps: .github/workflows/ci.yml runs make build, make test, make test-race, make vet and make lint, and those cover modules/ already. A build-tagged emulator suite is the one thing that does need a workflow edit, because no sweep runs it — the objectstore-minio and objectstore-fake-gcs jobs exist for that, one per emulator, and each runs the same make target a developer runs. That is the design — one command per concern, shared verbatim between CI and a developer’s terminal, so the two cannot drift into a state where CI skips something. Adding a module under modules/ requires no workflow edit; adding an emulator suite to it does.

Dependabot covers the submodules through a glob (directories: [/, /modules/*]) for the same reason.

Dependabot and submodule tidy

Every submodule’s replace github.com/frankbardon/nexus => ../.. means it inherits the root module’s requirements. When a root-module bump raises the minimum version of a dependency a submodule shares, that submodule’s go.mod/go.sum is stale too, and make build fails on the submodule sweep:

==> modules/objectstore-gcs: go build ./...
go: updates to go.mod needed; to update it:
	go mod tidy

(Reproduced by applying the modernc.org/sqlite 1.56.0 → 1.58.0 bump to main; go build ./... at the root stays green.) Dependabot’s gomod updater edits only the directory it was scheduled for, so every root bump it opens would arrive red.

.github/workflows/dependabot-tidy.yml fixes that. On a pull_request to main from dependabot[bot] on a dependabot/go_modules/* branch it:

1. checks out the PR’s head branch (not the merge ref, which cannot be pushed back), sets up Go 1.26 with GOTOOLCHAIN=local;
2. runs go mod tidy at the root, then in every directory make submodules lists;
3. if anything changed, commits it as github-actions[bot] (build(deps): go mod tidy root and modules/* after dependabot bump), pushes to the Dependabot branch, and runs gh workflow run ci.yml --ref <branch>; if nothing changed it logs that and exits 0.

Points worth knowing:

• It needs no secret. A Dependabot-triggered pull_request run gets a read-only GITHUB_TOKEN unless the workflow grants more; the job’s permissions: { contents: write, actions: write } is what lets it push and dispatch.
• CI is dispatched, not triggered. A push made with GITHUB_TOKEN does not fire pull_request or push workflows (GitHub’s recursion guard), so the tidy commit would otherwise get no CI run. workflow_dispatch is the documented exception, and ci.yml already declares it. Whether those dispatched runs show up in the PR’s checks list is validated post-merge with a live @dependabot recreate on #175.
• Dependabot stops rebasing the PR once another actor has pushed to its branch. If a tidied PR later conflicts with main, comment @dependabot recreate: Dependabot rebuilds the branch from scratch, its own push fires pull_request, and this workflow tidies the fresh branch again. That tradeoff was accepted over every root bump arriving red.
• Submodule-directory PRs usually tidy to a no-op. Nothing inherits from a submodule, so a /modules/* bump rarely leaves another module stale; the job still runs (the branch name does not say which directory it came from) and exits clean.
• github.head_ref reaches the shell only through env:, never interpolated into run: text, and a newer push to the same branch cancels an in-flight tidy through a per-branch concurrency group.

Versioning and tagging

The core module is released as a bare tag, vX.Y.Z, with a matching GitHub Release. Nothing about submodules changes that, because a bare vX.Y.Z tag never versions a submodule — Go requires the tag to be prefixed with the module’s directory.

The rules:

1. Submodule tags are not cut by default. Cutting a core release does not cut modules/* tags, and the release process does not have to know how many submodules exist. Inside the repository a submodule’s version is irrelevant anyway: make and CI always build the working tree through the replace.
2. A submodule tag is cut on demand, when someone needs to go get that module into a program built outside this repository. The tag format is fixed by Go: modules/<name>/vX.Y.Z, for example modules/objectstore-s3/v0.1.0.
3. Submodule versions are independent of the core version. They are not kept in step with vX.Y.Z and must not be assumed to match. A backend whose SDK needs a patch release should not have to wait for a core release, and a core release should not imply that every backend was re-tested.
4. When a submodule tag is cut, bump its require github.com/frankbardon/nexus line to the newest core tag first, in the same commit. That line is what external consumers actually resolve, and a stale one gives them a core module older than the seam the backend was written against. The replace stays where it is — consumers ignore it.
5. Compatibility is expressed by that require line, not by a naming convention. “Which Nexus does this backend work with” is answered by reading modules/<name>/go.mod, and by the fact that CI builds it against the current tree on every commit.

Adding a submodule

1. mkdir modules/<name> and write go.mod with module path github.com/frankbardon/nexus/modules/<name>, a require on the newest core tag, and replace github.com/frankbardon/nexus => ../...
2. Write the code. Depend on whatever you need — that is the point.
3. make build && make test && make lint. The glob picks the module up with no Makefile, CI or Dependabot edit.
4. Prove the coverage is real: break something in the new module on purpose and confirm make build and make test go red. If they stay green, the module is in the wrong place or the plumbing has regressed.

modules/objectstore-seamcheck is the worked example, and is also the permanent canary for step 4: it is not a backend and stores nothing, it exists to hold true the property that objectstore.Backend and objectstoretest.RunSuite are usable from a module that is not github.com/frankbardon/nexus.